DAT Format (Shadowcaster)

Overview

Shadowcaster contains two forms of DAT archive. This page is about the game's WAD-like game data archives. Another page details the cutscene DAT format.

A Shadowcaster game-data archive has the following general structure:

uint16_t  NumRecords;    // the number of sub-files contained in this archive.
uint32_t  Directory;     // an offset from the start of this file to the beginning of the record list.

uint8_t DataRegion[...]; // raw sub-file data. there can be hidden sections here; a sub-file's start does not have to begin right after the end of the previous.

// within each sub-file record is the following format:
uint32_t Offset;    // offset to the start of the data, from the beginning of the file.
uint32_t Size;      // the sub-file's size in bytes.
uint16_t NameOfs;   // a pointer from the beginning of the directory to a null-terminated string. this can be zero.
uint16_t Flags;     // in the floppy version, this is always zero. in the CD version, a value of 1 indicates RLE compression.

Record Names

In this format, sub-file records do not have to have a name.

The underlying logic of this is that the game will often request a specific named lump, then refer to unnamed records using an ID offset from that. For example, if the lump named "view" is lump #0, then the game can refer to "view+5" to refer to lump #5, and therefore, lump #5 does not have to store its name. In such a case, NameOfs = 0.

By convention, SLADE and MapCaster notate the names of these lumps as ID offsets from the last named entry found.

Reading a DAT Archive

Adapted from MapCaster GML:

NumRecords = buffer_read(buf, buffer_u16);
Directory = buffer_read(buf, buffer_u32);

buffer_seek(buf, buffer_seek_start, Directory);
for(z = 0; z < NumRecords; z++) {
    Records[z] = ds_map_create();
    
    r = Records[z];
    r[? "Offset"] = buffer_read(buf, buffer_u32);
    r[? "Size"] = buffer_read(buf, buffer_u32);
    r[? "NameOffset"] = buffer_read(buf, buffer_u16);
    r[? "Flags"] = buffer_read(buf, buffer_u16);
    
    // name all lumps
    if(r[? "NameOffset"] != 0) {
        last_name = 0;
        prev_ofs = buffer_tell(buf);
        
        // seek to the name table
        buffer_seek(buf, buffer_seek_start, Directory + r[? "NameOffset"]);

        r[? "Name"] = ReadString(...); // read until 0x00 is encountered
        
        // go back to where we were before in reading the directory
        buffer_seek(buf, buffer_seek_start, prev_ofs);
    }
    else {  // if the lump has no name, treat it as being
            // named as an offset from the previous named lump
        last_name++;
        l = Records[z - last_name];
        r[? "Name"] = l[? "Name"] + "+" + string(last_name);
    }
}