DLT Format
| Format type | Archive |
|---|---|
| Max files | 65,535 |
| File Allocation Table (FAT) | Embedded |
| Filenames? | Yes, 8.3, encrypted |
| Metadata? | None |
| Supports compression? | Yes |
| Supports encryption? | Yes |
| Supports subdirectories? | No |
| Hidden data? | No |
| Games |
The DLT format is used to store most of the data for Stargunner.
File format
Signature
The first four characters of the file are "DAVE". Each compressed subfile begins with "PGBP", while uncompressed subfiles do not. It is probably not possible to store a file that has "PGBP" as its first four bytes unless it is first compressed.
Header
The file starts with a single field indicating the number of files present.
| Data type | Description |
|---|---|
| char signature[4] | "DAVE" |
| UINT16LE version | File version |
| UINT16LE numFiles | File count |
The file version must be 0x100 ("00 01") to work with .exe version 1.0b (the version in the full freeware release.)
File entry
After the header, the following structure is repeated numFiles times.
| Data type | Description |
|---|---|
| BYTE filename[32] | Encrypted filename (8.3, with a path, backslash as separator) |
| UINT32LE lastModified | Last modified date, as number of seconds since 1980-01-01 (no timezone, so local time wherever the archive was created) |
| UINT32LE size | File size |
| BYTE data[size] | File content, size bytes long |
! Does the filename need to be null terminated before encryption (31 chars max) or can it be the full 32 chars?
Filename encryption
The filenames are encrypted using an XOR cipher. The first character is in cleartext, then the second and subsequent characters are XOR'd with the previous character's cleartext value incremented by its offset from the start of the string:
for (int i = 1; i < 32; i++) name[i] ^= name[i - 1] + i;
Compression
Many of the files inside the DLT are compressed using a variant of byte-pair encoding. If a file begins with the four bytes "PGBP" then it is compressed.
The_coder has decompiled the decompression algorithm and produced C++ code which can extract and decompress the files. A cleaned up version of this, with only the decompression algorithm, follows:
//
// unstargun.cpp
//
// Stargunner decompression utility.
//
// Written by Adam Nielsen <malvineous@shikadi.net>. Based on The_coder's work
// for tombexcavator.
//
// Use: unstargun < infile > outfile
//
#include <iostream>
#include <cstring>
#include <cassert>
#include <stdint.h>
/// Chunk size used during compression. Each chunk expands to this amount of data.
#define CHUNK_SIZE 4096
/// Largest possible chunk of compressed data. (No compression + worst case dictionary size.)
#define CMP_CHUNK_SIZE (CHUNK_SIZE + 256)
/// Read 32-bit little-endian int, in an endian-neutral way
uint32_t get_u32()
{
uint8_t sig[4];
std::cin.read((char *)sig, 4);
return sig[0] | (sig[1] << 8) | (sig[2] << 16) | (sig[3] << 24);
}
/// Read 16-bit little-endian int, in an endian-neutral way
uint16_t get_u16()
{
uint8_t sig[2];
std::cin.read((char *)sig, 2);
return sig[0] | (sig[1] << 8);
}
/// Decompress a data chunk.
/**
* @param in
* Input data. First byte is the one immediately following the chunk length.
*
* @param expanded_size
* The size of the input chunk after decompression. The output buffer must
* be able to hold this many bytes.
*
* @param out
* Output buffer.
*/
unsigned int explode_chunk(const uint8_t* in, size_t expanded_size, uint8_t* out)
{
uint8_t tableA[256], tableB[256];
unsigned int inpos = 0;
unsigned int outpos = 0;
while (outpos < expanded_size) {
// Initialise the dictionary so that no bytes are codewords (or if you
// prefer, each byte expands to itself only.)
for (int i = 0; i < 256; i++) tableA[i] = i;
//
// Read in the dictionary
//
uint8_t code;
unsigned int tablepos = 0;
do {
code = in[inpos++];
// If the code has the high bit set, the lower 7 bits plus one is the
// number of codewords that will be skipped from the dictionary. (Those
// codewords were initialised to expand to themselves in the loop above.)
if (code > 127) {
tablepos += code - 127;
code = 0;
}
if (tablepos == 256) break;
// Read in the indicated number of codewords.
for (int i = 0; i <= code; i++) {
assert(tablepos < 256);
uint8_t data = in[inpos++];
tableA[tablepos] = data;
if (tablepos != data) {
// If this codeword didn't expand to itself, store the second byte
// of the expansion pair.
tableB[tablepos] = in[inpos++];
}
tablepos++;
}
} while (tablepos < 256);
// Read the length of the data encoded with this dictionary
int len = in[inpos++];
len |= in[inpos++] << 8;
//
// Decompress the data
//
int expbufpos = 0;
// This is the maximum number of bytes a single codeword can expand to.
uint8_t expbuf[32];
while (1) {
if (expbufpos) {
// There is data in the expansion buffer, use that
code = expbuf[--expbufpos];
} else {
// There is no data in the expansion buffer, use the input data
if (--len == -1) break; // no more input data
code = in[inpos++];
}
if (code == tableA[code]) {
// This byte is itself, write this to the output
out[outpos++] = code;
} else {
// This byte is actually a codeword, expand it into the expansion buffer
assert(expbufpos < (signed)sizeof(expbuf) - 2);
expbuf[expbufpos++] = tableB[code];
expbuf[expbufpos++] = tableA[code];
}
}
}
return outpos - expanded_size;
}
int main(void)
{
std::cerr << "Stargunner decompressor.\n"
"Written by Adam Nielsen <malvineous@shikadi.net>\n"
"Decompression algorithm decompiled by The_coder" << std::endl;
// Make sure the input data has a valid "PGBP" signature.
char sig[4];
std::cin.read(sig, 4);
if (strncmp(sig, "PGBP", 4) != 0) {
std::cerr << "ERROR: Input file is not a Stargunner compressed file."
<< std::endl;
return 1;
}
// Read the header
uint32_t outSize = get_u32();
std::cerr << "Writing " << outSize << " bytes" << std::flush;
uint8_t *chunk = new uint8_t[CMP_CHUNK_SIZE];
uint8_t *chunkOut = new uint8_t[CHUNK_SIZE];
while (!std::cin.eof()) {
long chunkSize = get_u16();
if (chunkSize > CMP_CHUNK_SIZE) {
std::cerr << "\nERROR: Compressed chunk is too large!"
<< std::endl;
return 2;
}
std::cerr << "." << std::flush;
std::cin.read((char *)chunk, chunkSize);
int lenOut;
if (outSize < CHUNK_SIZE) lenOut = outSize;
else lenOut = CHUNK_SIZE;
if (explode_chunk(chunk, lenOut, chunkOut)) {
std::cerr << "\nERROR: Failed to explode chunk!" << std::endl;
return 2;
}
std::cout.write((char *)chunkOut, lenOut);
outSize -= lenOut;
}
std::cerr << "done." << std::endl;
return 0;
}
Tools
The following tools are able to work with files in this format.
| Name | Platform | Extract files? | Decompress on extract? | Create new? | Modify? | Compress on insert? | Access hidden data? | Edit metadata? | Notes |
|---|---|---|---|---|---|---|---|---|---|
| Camoto | Linux/Windows | Yes | No | Yes | Yes | No | N/A | N/A | |
| Camoto/gamearchive.js | Any | Yes | Yes | Yes | Yes | Yes | N/A | N/A | |
| tombexcavator | Any/console | Yes | Yes | No | No | No | N/A | N/A |
See also
- Some early discussion about the compression algorithm
Credits
The filename encryption algorithm was reverse engineered by a user on the Xentax forum. The compression algorithm was reverse engineered by The_coder. If you find this information helpful in a project you're working on, please give credit where credit is due. (A link back to this wiki would be nice too!)