ModdingWiki - User contributions [en-gb]

DAT Format (God of Thunder)

2014-08-04T16:34:18Z

Luis46coco: /* Compression */

{{Archive Infobox
| MaxFiles = 256
| FAT = Beginning
| Names = Yes, 8 chars
| Metadata = None
| Subdirectories = N
| Compressed = Y
| Encrypted = Y
| Hidden = Y
| Game1 = God of Thunder
}}
The '''DAT format''' is used by [[God of Thunder]] to store game data in <tt>GOTRES.DAT</tt>. Most files are compressed, and the header where the filenames are stored is encrypted with a simple [[XOR cipher]].

== File format ==

=== Signature ===

There is no known signature for this format, however reading in the entire FAT and checking that the filenames contain valid characters and that the offsets and file sizes are within range should correctly identify files.

=== File entry ===

The file begins with 256 file entries, in the format below. All these file entries (i.e. the first 5888 bytes of the file) are encrypted with the [[XOR cipher|XOR algorithm]], at the byte level, starting with 128 and incrementing at each byte (so the first byte in the file is XOR'd with 128, the second byte with 129, third byte with 130 and so on. After XOR'ing with 255 the value wraps and so the next byte is XOR'd with 0.)

The following Python 2.7 script will decrypt the header and output the result to a file:
<source lang="Python">
# Setup our starting variables.
Cypher = 128 # Cyper starts at 128.
Header = "" # Header will store the decrypted header.
HeaderSize = 5888 # The header in GOTRES.DAT is 5888 bytes.

# Open the God of Thunder data file.
with open("GOTRES.DAT", "rb") as Input:
# Loop through the header which consists of the first 5888 bytes.
for x in range(0, HeaderSize):
# Read a byte.
Byte = Input.read(1)
# XOR the byte with the current cypher.
Header += chr(ord(Byte) ^ Cypher)

# Increment the cypher value, and set it back to 0 when it hits 256.
Cypher += 1
if Cypher == 256:
Cypher = 0

# Write the decrypted header to a file.
with open("GOTHead.bin", "w") as Output:
Output.write(Header)
</source>

After XOR decryption, each file entry is as follows:

{|class="wikitable"
!Data type!!Description
|-
|[[char]] filename[9]||Filename (8 chars max), must be NULL-terminated
|-
|[[UINT32LE]] offset||File offset from start of file
|-
|[[UINT32LE]] size||Length of file stored here
|-
|[[UINT32LE]] decompressedSize||Length of file after decompression
|-
|[[UINT16LE]] flags||Flags (1 if file is compressed, 0 otherwise)
|}

The remaining unused file entries are all zeroes. Since the FAT is fixed at 256 entries, the offset of the first file will always be 5888 (0x1700).

== Compression ==

If a file's flag indicates it is compressed, then it should be decompressed with the LZSS algorithm below. Each file's data begins with a [[UINT16LE]] value holding the decompressed size, in bytes. skip the next 2 bytes Other [[UINT16LE]] value(??). The rest of the data decompresses as follows:

<ol>
<li>
If the amount of data decompressed matches the target size, finish. Otherwise:
</li>
<li>
Read a byte from the input data
</li>
<li>
For each bit in the previous byte, from the least significant to the most:
<ul>
<li>
If the bit is 1, copy a byte unchanged from the input data to the output
</li>
<li>
Otherwise the bit is zero:
<ol style="list-style-type: lower-alpha;">
<li>
Read a [[UINT16LE]]
<ul>
<li>
Add two to the upper (most significant) four bits, and treat this value as the LZSS "count"
</li>
<li>
Take the lower 12 bits and treat the value as the LZSS "offset"
</li>
</ul>
</li>
<li>
Look back "offset" bytes into the newly decompressed data
</li>
<li>
Copy "count" bytes from here to the end of the newly decompressed data
(explanatory:
remember that each byte copied recently, extending the data and can be copied again, this occurs, for example, when the offset is 1, back one byte, and the counter is 15 + 2 = 17, this copy 17 times the last copied byte).
</li>
</ol>
</li>
</ul>
</li>
<li>
Go back to step 1
</li>
</ol>

== Tools ==

{{BeginFileFormatTools|Type=group}}
{{FileFormatTool
| Name = [[Camoto]]
| Platform = Linux/Windows
| canExtract = Yes
| canDecompress = No
| canCreate = Yes
| canModify = Yes
| canCompress = No
| editHidden = No
| editMetadata = N/A
}}
{{FileFormatTool
| Name = [[Wombat]]
| Platform = Windows
| canExtract = Yes
| canDecompress = Yes
| canCreate = No
| canModify = No
| canCompress = N/A
| editHidden = No
| editMetadata = N/A
}}
{{EndFileFormatTools}}

== Credits ==

This file format was reverse engineered by [http://grompe.org.ru/ Grom PE]. If you find this information helpful in a project you're working on, please give credit where credit is due. (A link back to this wiki would be nice too!)

DAT Format (God of Thunder)

2014-08-04T16:33:23Z

Luis46coco: /* Compression */

{{Archive Infobox
| MaxFiles = 256
| FAT = Beginning
| Names = Yes, 8 chars
| Metadata = None
| Subdirectories = N
| Compressed = Y
| Encrypted = Y
| Hidden = Y
| Game1 = God of Thunder
}}
The '''DAT format''' is used by [[God of Thunder]] to store game data in <tt>GOTRES.DAT</tt>. Most files are compressed, and the header where the filenames are stored is encrypted with a simple [[XOR cipher]].

== File format ==

=== Signature ===

There is no known signature for this format, however reading in the entire FAT and checking that the filenames contain valid characters and that the offsets and file sizes are within range should correctly identify files.

=== File entry ===

The file begins with 256 file entries, in the format below. All these file entries (i.e. the first 5888 bytes of the file) are encrypted with the [[XOR cipher|XOR algorithm]], at the byte level, starting with 128 and incrementing at each byte (so the first byte in the file is XOR'd with 128, the second byte with 129, third byte with 130 and so on. After XOR'ing with 255 the value wraps and so the next byte is XOR'd with 0.)

The following Python 2.7 script will decrypt the header and output the result to a file:
<source lang="Python">
# Setup our starting variables.
Cypher = 128 # Cyper starts at 128.
Header = "" # Header will store the decrypted header.
HeaderSize = 5888 # The header in GOTRES.DAT is 5888 bytes.

# Open the God of Thunder data file.
with open("GOTRES.DAT", "rb") as Input:
# Loop through the header which consists of the first 5888 bytes.
for x in range(0, HeaderSize):
# Read a byte.
Byte = Input.read(1)
# XOR the byte with the current cypher.
Header += chr(ord(Byte) ^ Cypher)

# Increment the cypher value, and set it back to 0 when it hits 256.
Cypher += 1
if Cypher == 256:
Cypher = 0

# Write the decrypted header to a file.
with open("GOTHead.bin", "w") as Output:
Output.write(Header)
</source>

After XOR decryption, each file entry is as follows:

{|class="wikitable"
!Data type!!Description
|-
|[[char]] filename[9]||Filename (8 chars max), must be NULL-terminated
|-
|[[UINT32LE]] offset||File offset from start of file
|-
|[[UINT32LE]] size||Length of file stored here
|-
|[[UINT32LE]] decompressedSize||Length of file after decompression
|-
|[[UINT16LE]] flags||Flags (1 if file is compressed, 0 otherwise)
|}

The remaining unused file entries are all zeroes. Since the FAT is fixed at 256 entries, the offset of the first file will always be 5888 (0x1700).

== Compression ==

If a file's flag indicates it is compressed, then it should be decompressed with the LZSS algorithm below. Each file's data begins with a [[UINT16LE]] value holding the decompressed size, in bytes. skip the next 2 bytes Other [[UINT16LE]] value(??). The rest of the data decompresses as follows:

<ol>
<li>
If the amount of data decompressed matches the target size, finish. Otherwise:
</li>
<li>
Read a byte from the input data
</li>
<li>
For each bit in the previous byte, from the least significant to the most:
<ul>
<li>
If the bit is 1, copy a byte unchanged from the input data to the output
</li>
<li>
Otherwise the bit is zero:
<ol style="list-style-type: lower-alpha;">
<li>
Read a [[UINT16LE]]
<ul>
<li>
Add two to the upper (most significant) four bits, and treat this value as the LZSS "count"
</li>
<li>
Take the lower 12 bits and treat the value as the LZSS "offset"
</li>
</ul>
</li>
<li>
Look back "offset" bytes into the newly decompressed data
</li>
<li>
Copy "count" bytes from here to the end of the newly decompressed data
(explanatory:
remember that each byte copied recently, extending the data and can be copied again, this occurs, for example, when the offset is 1 ,back one byte, and the counter is 15 + 2 = 17, this copy 17 times the last copied byte).
</li>
</ol>
</li>
</ul>
</li>
<li>
Go back to step 1
</li>
</ol>

== Tools ==

{{BeginFileFormatTools|Type=group}}
{{FileFormatTool
| Name = [[Camoto]]
| Platform = Linux/Windows
| canExtract = Yes
| canDecompress = No
| canCreate = Yes
| canModify = Yes
| canCompress = No
| editHidden = No
| editMetadata = N/A
}}
{{FileFormatTool
| Name = [[Wombat]]
| Platform = Windows
| canExtract = Yes
| canDecompress = Yes
| canCreate = No
| canModify = No
| canCompress = N/A
| editHidden = No
| editMetadata = N/A
}}
{{EndFileFormatTools}}

== Credits ==

This file format was reverse engineered by [http://grompe.org.ru/ Grom PE]. If you find this information helpful in a project you're working on, please give credit where credit is due. (A link back to this wiki would be nice too!)

DAT Format (God of Thunder)

2014-08-04T15:05:11Z

Luis46coco: /* Compression */ error in header and error if offset+repeat > end the newly decompressed data

{{Archive Infobox
| MaxFiles = 256
| FAT = Beginning
| Names = Yes, 8 chars
| Metadata = None
| Subdirectories = N
| Compressed = Y
| Encrypted = Y
| Hidden = Y
| Game1 = God of Thunder
}}
The '''DAT format''' is used by [[God of Thunder]] to store game data in <tt>GOTRES.DAT</tt>. Most files are compressed, and the header where the filenames are stored is encrypted with a simple [[XOR cipher]].

== File format ==

=== Signature ===

There is no known signature for this format, however reading in the entire FAT and checking that the filenames contain valid characters and that the offsets and file sizes are within range should correctly identify files.

=== File entry ===

The file begins with 256 file entries, in the format below. All these file entries (i.e. the first 5888 bytes of the file) are encrypted with the [[XOR cipher|XOR algorithm]], at the byte level, starting with 128 and incrementing at each byte (so the first byte in the file is XOR'd with 128, the second byte with 129, third byte with 130 and so on. After XOR'ing with 255 the value wraps and so the next byte is XOR'd with 0.)

The following Python 2.7 script will decrypt the header and output the result to a file:
<source lang="Python">
# Setup our starting variables.
Cypher = 128 # Cyper starts at 128.
Header = "" # Header will store the decrypted header.
HeaderSize = 5888 # The header in GOTRES.DAT is 5888 bytes.

# Open the God of Thunder data file.
with open("GOTRES.DAT", "rb") as Input:
# Loop through the header which consists of the first 5888 bytes.
for x in range(0, HeaderSize):
# Read a byte.
Byte = Input.read(1)
# XOR the byte with the current cypher.
Header += chr(ord(Byte) ^ Cypher)

# Increment the cypher value, and set it back to 0 when it hits 256.
Cypher += 1
if Cypher == 256:
Cypher = 0

# Write the decrypted header to a file.
with open("GOTHead.bin", "w") as Output:
Output.write(Header)
</source>

After XOR decryption, each file entry is as follows:

{|class="wikitable"
!Data type!!Description
|-
|[[char]] filename[9]||Filename (8 chars max), must be NULL-terminated
|-
|[[UINT32LE]] offset||File offset from start of file
|-
|[[UINT32LE]] size||Length of file stored here
|-
|[[UINT32LE]] decompressedSize||Length of file after decompression
|-
|[[UINT16LE]] flags||Flags (1 if file is compressed, 0 otherwise)
|}

The remaining unused file entries are all zeroes. Since the FAT is fixed at 256 entries, the offset of the first file will always be 5888 (0x1700).

== Compression ==

If a file's flag indicates it is compressed, then it should be decompressed with the LZSS algorithm below. Each file's data begins with a [[UINT16LE]] value holding the decompressed size, in bytes. skip the next 2 bytes Other [[UINT16LE]] value(??). The rest of the data decompresses as follows:

<ol>
<li>
If the amount of data decompressed matches the target size, finish. Otherwise:
</li>
<li>
Read a byte from the input data
</li>
<li>
For each bit in the previous byte, from the least significant to the most:
<ul>
<li>
If the bit is 1, copy a byte unchanged from the input data to the output
</li>
<li>
Otherwise the bit is zero:
<ol style="list-style-type: lower-alpha;">
<li>
Read a [[UINT16LE]]
<ul>
<li>
Add two to the upper (most significant) four bits, and treat this value as the LZSS "count"
</li>
<li>
Take the lower 12 bits and treat the value as the LZSS "offset"
</li>
</ul>
</li>
<li>
Look back "offset" bytes into the newly decompressed data
</li>
<li>
Copy "count" bytes from here to the end of the newly decompressed data (if end the file repeat the same byte)
</li>
</ol>
</li>
</ul>
</li>
<li>
Go back to step 1
</li>
</ol>

== Tools ==

{{BeginFileFormatTools|Type=group}}
{{FileFormatTool
| Name = [[Camoto]]
| Platform = Linux/Windows
| canExtract = Yes
| canDecompress = No
| canCreate = Yes
| canModify = Yes
| canCompress = No
| editHidden = No
| editMetadata = N/A
}}
{{FileFormatTool
| Name = [[Wombat]]
| Platform = Windows
| canExtract = Yes
| canDecompress = Yes
| canCreate = No
| canModify = No
| canCompress = N/A
| editHidden = No
| editMetadata = N/A
}}
{{EndFileFormatTools}}

== Credits ==

This file format was reverse engineered by [http://grompe.org.ru/ Grom PE]. If you find this information helpful in a project you're working on, please give credit where credit is due. (A link back to this wiki would be nice too!)