Talk:Softdisk Library Format
Jump to navigation
Jump to search
Compression and exceptions
I had a quick look at the LZW compression used by SOFTLIB.EXE and I was wondering in which case the exception that is mentioned in the text occured. I was able to write a very quick implementation of an LZW decompressor (with a sliding window of 4 KB) that worked perfectly, so I'd like to test that code for those exceptions. Another thing I noticed is that the LZH compressed data doesn't appear to be based on the compressed data that the LZW compression produces. This seems to be a different compression scheme, using codeword lengths of 8 and 9 bits. --K1n9 Duk3 22:32, 2 January 2012 (GMT)
- Seems I was a bit of a twit, I have rewritten the page entirely given my recent pokings at this format. There are no exceptions, just a faulty windows calculator that gave me the wrong binary for a hex value. The 'LZH' I think is just regular LZW, but using a dictionary instead of a sliding window, much like Keen 1's graphics use. --Endian? What are you on about? 05:13, 18 April 2012 (GMT)
- So I've implemented this in Wombat, and I've found the following: type 1 compression is LZSS (the description which talks about 'flag bytes' on this page confirms that). Would it make sense to move the text that describes the algorithm to the LZSS page? It's quite a good writeup, but would make more sense there. Type 2 is LZH: LZ77+Huffman, 4k window, adaptive Huffman for codes, offsets use a pre-defined Huffman tree, read-ahead of 30 bytes. It's identical to LHA's -lh1- compression, except that -lh1- uses a 60 byte read-ahead - which makes sense, as the algorithms in both were written by Haruhiko Okumura. (Szevvy (talk) 21:40, 14 July 2021 (GMT))
- If it's definitely LZSS then it should be merged in with that description so we don't have multiple descriptions of what is ultimately the same thing! Worst case at least add a {{TODO|message}} to mention that it's LZSS to save future coders from reinventing the wheel! -- Malvineous (talk) 09:46, 4 August 2021 (GMT)
- There are already several articles with explanations about LZW. Some examples include Westwood LZW-12 and Westwood LZW-14. Both were renamed to "LZW" after they were identified as such. Unfortunately, there are loads of slightly different implementations of the LZW compression, too. I'm honestly not sure how to sort out that mess. -Nyerguds (talk) 11:00, 4 August 2021 (GMT)
- I would link all those pages to the standard LZW algorithm, then mention the differences on each format's page, which seems to be more or less what has been done with the Westwood ones. I'd also put links on the LZW page to all the different variants, again just like Westwood. The goal is for someone trying to make use of a file format to quickly understand how to do so. Having to implement an algorithm from scratch isn't great, but having to only slightly modify a standard already existing algorithm is much easier. The way the Westwood articles are, mentioning the variations against a standard format, I think is the best we can hope for and it gives people a good place to start from. The ultimate goal, if possible, would be for the main LZW page to describe a generic enough algorithm that would handle most of the variations. I have tried to do this in the gamecomp.js LZW implementation by having a bunch of options you can specify, where only the options are different between each game but the LZW code is the same. -- Malvineous (talk) 04:06, 6 August 2021 (GMT)
- All right, thanks for the heads up. The reason I mentioned the WW ones was mostly because, as I wrote in the article, they are a 100% textbook implementation without any modifications at all. I found an implementation of it online to use in my program, and it just worked right away (besides the fact the programmer used an ASCII-stream in that code, which filtered out all higher-value bytes. Dev derp there, lol) -Nyerguds (talk) 08:11, 6 August 2021 (GMT)
- I would link all those pages to the standard LZW algorithm, then mention the differences on each format's page, which seems to be more or less what has been done with the Westwood ones. I'd also put links on the LZW page to all the different variants, again just like Westwood. The goal is for someone trying to make use of a file format to quickly understand how to do so. Having to implement an algorithm from scratch isn't great, but having to only slightly modify a standard already existing algorithm is much easier. The way the Westwood articles are, mentioning the variations against a standard format, I think is the best we can hope for and it gives people a good place to start from. The ultimate goal, if possible, would be for the main LZW page to describe a generic enough algorithm that would handle most of the variations. I have tried to do this in the gamecomp.js LZW implementation by having a bunch of options you can specify, where only the options are different between each game but the LZW code is the same. -- Malvineous (talk) 04:06, 6 August 2021 (GMT)
- There are already several articles with explanations about LZW. Some examples include Westwood LZW-12 and Westwood LZW-14. Both were renamed to "LZW" after they were identified as such. Unfortunately, there are loads of slightly different implementations of the LZW compression, too. I'm honestly not sure how to sort out that mess. -Nyerguds (talk) 11:00, 4 August 2021 (GMT)
- If it's definitely LZSS then it should be merged in with that description so we don't have multiple descriptions of what is ultimately the same thing! Worst case at least add a {{TODO|message}} to mention that it's LZSS to save future coders from reinventing the wheel! -- Malvineous (talk) 09:46, 4 August 2021 (GMT)
- So I've implemented this in Wombat, and I've found the following: type 1 compression is LZSS (the description which talks about 'flag bytes' on this page confirms that). Would it make sense to move the text that describes the algorithm to the LZSS page? It's quite a good writeup, but would make more sense there. Type 2 is LZH: LZ77+Huffman, 4k window, adaptive Huffman for codes, offsets use a pre-defined Huffman tree, read-ahead of 30 bytes. It's identical to LHA's -lh1- compression, except that -lh1- uses a 60 byte read-ahead - which makes sense, as the algorithms in both were written by Haruhiko Okumura. (Szevvy (talk) 21:40, 14 July 2021 (GMT))
Broken link
Hello. Titlebuild's dropbox link is down: http://dl.dropbox.com/u/3940020/Titlebuild.zip Please fix or update it. Or tell me where to get Titlebuild. Szemigi (talk) 07:43, 6 September 2021 (GMT)
- Fixed link. Don't hesitate to check dead links in the Internet Archive Wayback Machine project. CTPAX-X Team (talk) 10:12, 6 September 2021 (GMT)