DAT Format (God of Thunder)
There is no known signature for this format, however reading in the entire FAT and checking that the filenames contain valid characters and that the offsets and file sizes are within range should correctly identify files.
The FAT is a fixed size, so the offset of the first file will always be 0x1700.
The file begins with 256 file entries, in the format below. All these file entries (i.e. the first 5888 bytes of the file) are encrypted (see below).
After decryption, each file entry is as follows:
|char filename||Filename (8 chars max), must be NULL-terminated|
|UINT32LE offset||File offset from start of file|
|UINT32LE size||Length of file stored here|
|UINT32LE decompressedSize||Length of file after decompression|
|UINT16LE flags||Flags (1 if file is compressed, 0 otherwise)|
The remaining unused file entries are all zeroes. Since the FAT is fixed at 256 entries, the offset of the first file will always be 5888 (0x1700).
The file content immediately follows the FAT, and is not encrypted (but may be compressed, see below).
The list of 256 file entries (the first 5,888 bytes in the file) are encrypted with an XOR cipher, at the byte level, starting with a value of 128 and incrementing at each byte (so the first byte in the file is XOR'd with 128, the second byte with 129, third byte with 130 and so on. After XOR'ing with 255 the value wraps and so the next byte is XOR'd with 0).
Since XOR ciphers are symmetrical, the same algorithm applies to both encrypt and decrypt the data.
The following Python 2.7 script will decrypt the header and output the result to a file:
# Setup our starting variables. Cypher = 128 # Cypher starts at 128. Header = "" # Header will store the decrypted header. HeaderSize = 5888 # The header in GOTRES.DAT is 5888 bytes. # Open the God of Thunder data file. with open("GOTRES.DAT", "rb") as Input: # Loop through the header which consists of the first 5888 bytes. for x in range(0, HeaderSize): # Read a byte. Byte = Input.read(1) # XOR the byte with the current cypher. Header += chr(ord(Byte) ^ Cypher) # Increment the cypher value, and set it back to 0 when it hits 256. Cypher += 1 if Cypher == 256: Cypher = 0 # Write the decrypted header to a file. with open("GOTHead.bin", "w") as Output: Output.write(Header)
If a file's flag indicates it is compressed, then it should be decompressed with the LZSS algorithm below. Each file's data begins with a UINT16LE value holding the decompressed size, in bytes, followed by another UINT16LE value of unknown purpose which should be ignored (the value is always 0x0001.) The rest of the data decompresses as follows:
- If the amount of data decompressed matches the target size, finish. Otherwise:
- Read a byte from the input data
For each bit in the previous byte, from the least significant to the most:
- If the bit is 1, copy a byte unchanged from the input data to the output
Otherwise the bit is zero:
Read a UINT16LE
- Add two to the upper (most significant) four bits, and treat this value as the LZSS "count"
- Take the lower 12 bits and treat the value as the LZSS "offset"
- Look back "offset" bytes into the newly decompressed data
- Copy "count" bytes from here to the end of the newly decompressed data. Take note that as each byte is copied to its destination, that new byte may later become a source byte in this same copy operation. For example, if "offset" is 1 (i.e. look back one byte) and the counter is 15, then the last byte will be copied 17 times (15 + 2 = 17). This is because as each byte is copied, it becomes the source byte for the next copy cycle.
- Read a UINT16LE
- Go back to step 1
If a generic LZSS algorithm is used, it must be configured as follows:
- Inverted flags (1 = literal, 0 = length/value pair)
- Length/distance pair is UINT16LE
- Length is 4-bits, in most significant bits of length/distance pair
- Distance is 12-bits, least significant bits of length/distance pair
- Add 0 to distance value, add 2 to length value
- Distance is relative to current window position
- Initial window offset is 0
Since both the file offset and size are stored, it is possible to hide data before and after each file. Without a corresponding file entry, this data would be invisible.
The following tools are able to work with files in this format.
|Name||Platform||Extract files?||Decompress on extract?||Create new?||Modify?||Compress on insert?||Access hidden data?||Edit metadata?||Notes|
This file format was reverse engineered by Grom PE. If you find this information helpful in a project you're working on, please give credit where credit is due. (A link back to this wiki would be nice too!)