DAT Format (God of Thunder)

DAT Format (God of Thunder)

Format type	Archive
Max files	256
File Allocation Table (FAT)	Beginning
Filenames?	Yes, 8 chars
Metadata?	None
Supports compression?	Yes
Supports encryption?	Yes
Supports subdirectories?	No
Hidden data?	Yes
Games	God of Thunder

The DAT format is used by God of Thunder to store game data in GOTRES.DAT. Most files are compressed, and the header where the filenames are stored is encrypted with a simple XOR cipher.

File format

Signature

There is no known signature for this format, however reading in the entire FAT and checking that the filenames contain valid characters and that the offsets and file sizes are within range should correctly identify files.

The FAT is a fixed size, so the offset of the first file will always be 0x1700.

File entry

The file begins with 256 file entries, in the format below. All these file entries (i.e. the first 5888 bytes of the file) are encrypted (see below).

After decryption, each file entry is as follows:

Data type	Description
char filename[9]	Filename (8 chars max), must be NULL-terminated
UINT32LE offset	File offset from start of file
UINT32LE size	Length of file stored here
UINT32LE decompressedSize	Length of file after decompression
UINT16LE flags	Flags (1 if file is compressed, 0 otherwise)

The remaining unused file entries are all zeroes. Since the FAT is fixed at 256 entries, the offset of the first file will always be 5888 (0x1700).

The file content immediately follows the FAT, and is not encrypted (but may be compressed, see below).

Encryption

The list of 256 file entries (the first 5,888 bytes in the file) are encrypted with an XOR cipher, at the byte level, starting with a value of 128 and incrementing at each byte (so the first byte in the file is XOR'd with 128, the second byte with 129, third byte with 130 and so on. After XOR'ing with 255 the value wraps and so the next byte is XOR'd with 0).

Since XOR ciphers are symmetrical, the same algorithm applies to both encrypt and decrypt the data.

The following Python 2.7 script will decrypt the header and output the result to a file:

# Setup our starting variables.
Cypher = 128                      # Cypher starts at 128.
Header = ""                       # Header will store the decrypted header.
HeaderSize = 5888                 # The header in GOTRES.DAT is 5888 bytes.

# Open the God of Thunder data file.
with open("GOTRES.DAT", "rb") as Input:
    # Loop through the header which consists of the first 5888 bytes.
    for x in range(0, HeaderSize):
        # Read a byte.
        Byte = Input.read(1)
        # XOR the byte with the current cypher.
        Header += chr(ord(Byte) ^ Cypher)

        # Increment the cypher value, and set it back to 0 when it hits 256.
        Cypher += 1
        if Cypher == 256:
            Cypher = 0

# Write the decrypted header to a file.
with open("GOTHead.bin", "w") as Output:
    Output.write(Header)

Compression

If a file's flag indicates it is compressed, then it should be decompressed with the LZSS algorithm below. Each file's data begins with a UINT16LE value holding the decompressed size, in bytes, followed by another UINT16LE value of unknown purpose which should be ignored (the value is always 0x0001.) The rest of the data decompresses as follows:

1. If the amount of data decompressed matches the target size, finish. Otherwise:
2. Read a byte from the input data
3. For each bit in the previous byte, from the least significant to the most:
   • If the bit is 1, copy a byte unchanged from the input data to the output
   • Otherwise the bit is zero:
     1. Read a UINT16LE
        • Add two to the upper (most significant) four bits, and treat this value as the LZSS "count"
        • Take the lower 12 bits and treat the value as the LZSS "offset"
     2. Look back "offset" bytes into the newly decompressed data
     3. Copy "count" bytes from here to the end of the newly decompressed data. Take note that as each byte is copied to its destination, that new byte may later become a source byte in this same copy operation. For example, if "offset" is 1 (i.e. look back one byte) and the counter is 15, then the last byte will be copied 17 times (15 + 2 = 17). This is because as each byte is copied, it becomes the source byte for the next copy cycle.
4. Go back to step 1

If a generic LZSS algorithm is used, it must be configured as follows:

• Byte-level
• Inverted flags (1 = literal, 0 = length/value pair)
• Length/distance pair is UINT16LE
• Length is 4-bits, in most significant bits of length/distance pair
• Distance is 12-bits, least significant bits of length/distance pair
• Add 0 to distance value, add 2 to length value
• Distance is relative to current window position
• Initial window offset is 0

Hidden data

Since both the file offset and size are stored, it is possible to hide data before and after each file. Without a corresponding file entry, this data would be invisible.

Tools

The following tools are able to work with files in this format.

Name	Platform	Extract files?	Decompress on extract?	Create new?	Modify?	Compress on insert?	Access hidden data?	Edit metadata?	Notes
Camoto	Linux/Windows	Yes	Yes	Yes	Yes	Yes	No	N/A
Camoto/gamearchive.js	Any	Yes	Yes	Yes	Yes	Yes	No	N/A
Wombat	Windows	Yes	Yes	No	No	N/A	No	N/A

Credits

This file format was reverse engineered by Grom PE. If you find this information helpful in a project you're working on, please give credit where credit is due. (A link back to this wiki would be nice too!)