GameMaps Format
| Format type | Map/level |
|---|---|
| Map type | 2D tile-based |
| Layer count | 3 |
| Viewport (pixels) | Varies by game |
| Games |
The GameMaps Format stores levels in a number of id Software games. The filenames and compression varies somewhat across different games but all files stored in this format were produced by the TED5 level editor.
There are three main varieties of the file format. The most basic version is just RLEW compressed, the other two are either carmackized or Huffman compressed on top of the RLEW compression. Each variation has its own file naming scheme and pattern of external/internal files.
There are two main components to the format. The game maps proper, which contain the actual level data, and the map headers, which contain both the location of each level's data within the game maps file, and the tile info for the game.
Variants
| Game | RLEW | Carmackization | Huffman | Head | Gamemaps | Notes |
|---|---|---|---|---|---|---|
| Bio Menace | Yes | No | No | maphead.bm[123] | maptemp.bm[123] | gamemaps.bm[123] and mapthead.bm[123] are present but not used |
| Blake Stone: Aliens of Gold | Yes | No | No | maphead.bs[16] | maptemp.bs[16] | |
| Blake Stone: Planet Strike | Yes | No | No | maphead.vsi | maptemp.vsi | |
| Catacomb 3-D | Yes | Yes | No | Inside .exe | gamemaps.c3d | |
| Catacomb Abyss | Yes | Yes | No | Inside .exe | gamemaps.abs | |
| Catacomb Apocalypse | Yes | Yes | No | Inside .exe | gamemaps.apc | |
| Catacomb Armageddon | Yes | Yes | No | Inside .exe | gamemaps.arm | |
| Commander Keen Dreams | Yes | No | Yes | Inside .exe | gamemaps.kdr (v1.00), kdreams.map (v1.01+) |
|
| Commander Keen 4-6 | Yes | Yes | No | Inside .exe | gamemaps.ck[456] | |
| Corridor 7 Alien Invasion | Yes | No | No | Inside .exe (cd, disk), maphead.dmo (demo) |
maptemp.co7 (cd, disk), maptemp.dmo (demo) |
Uses MAPTEMP.xxx (only RLEW compression) |
| Dangerous Dave 3 | Yes | No | Yes | Inside .exe | gamemaps.dav | |
| Dangerous Dave 4 | Yes | No | Yes | Inside .exe | gamemaps.dav | |
| Super 3-D Noah's Ark | Yes | Yes | No | maphead.n3d | gamemaps.n3d | |
| Operation Body Count | Yes | No | No | maphead.bc (cd, disk, demo 2), maphead.co7 (demo 1) |
maptemp.bc (cd, disk, demo 2), maptemp.co7 (demo 1) |
Uses MAPTEMP.xxx (only RLEW compression) |
| Rescue Rover 2 | Yes | Yes | No | Inside .exe | gamemaps.rr2 | |
| Spear of Destiny | Yes | Yes | No | maphead.sod | gamemaps.sod | |
| Wolfenstein 3-D (v1.0) | Yes | No | No | maphead.wl1 | maptemp.wl1 | |
| Wolfenstein 3-D (v1.1 and above) | Yes | Yes | No | maphead.wl[16] | gamemaps.wl[16] |
The games that use Huffman compression will have a MAPDICT (embedded in the game's main executable). Those missing the map dictionary are not Huffman encoded.
To work out whether Carmackization is used, read the first UINT16LE in a map plane (the field storing the decompressed size). If the value matches the expected size of the plane (i.e. width * height * 2), then only RLEW compression is used. If it doesn't match, check the second UINT16LE in the file (which will become the first UINT16LE once Carmackization has been removed). If this matches the expected plane size then the plane must first have Carmackization removed followed by removal of RLEW. The exception to this rule are plane sizes that have either 0xA7 or 0xA8 as the high byte (i.e. any value from 0xA700 to 0xA8FF), as those values would have been encoded differently by the Carmack compression algorithm.
Data structures
The map data is split into two files: The map header (MAPHEAD.xxx) and the main map file (usually MAPTEMP.xxx or GAMEMAPS.xxx). The map header may be embedded in the game's main executable.
MAPTEMP.xxx is the working format saved by TED5 when maps are being edited and can be directly accessed and edited by this utility, allowing changes to be made to the game. Note that if the MAPHEAD file exists as an external file, the game always reads it from a file named named MAPHEAD.xxx for both the GAMEMAPS.xxx and the MAPTEMP.xxx variation. TED5 also saves a MAPTHEAD.xx file when editing maps, but that file is only used by TED5 and usually omitted from any game release.
The maps were often compressed even further for the public release versions of the games. This more compressed version of the map file was saved as GAMEMAPS.xxx by TED5. Later releases would omit the additional compression layer and ship with the MAPTEMP.xxx file instead. One possible reason for this could be that the size of the games grew to the point where they would no longer fit onto and be playable from a single floppy disk, and therefore the additional compression was no longer worth the amount of time it took to compress the maps and re-compile the code.
Map headers (MAPHEAD)
| Offset | Type | Name | Description |
|---|---|---|---|
| 0 | UINT16LE | magic | Magic word signalling RLEW compression |
| 2 | INT32LE[100] | ptr | 100 pointers to start of level 0-99 data in the game maps file |
| 402 | ! Unknown | tileinfo | Optional tileinfo data |
The map header file (MAPHEAD) is of varying length and contains three main types of data.
- The first is the magic word or flag used for RLEW compression, which is almost always $ABCD in the original files. It could be changed to any other value if necessary.
- The second is 100 level pointers which give the location of the start of level data in the GAMEMAPS file, relative to the start of that file. A value less than 1 indicates no level (generally 0, but occasionally -1 (0xFFFFFFFF) is used). Using -1 should be preferred over 0, as the pre-Wolf3D version of the engine will treat any negative value as "a non existant map" [sic!] and quit with an appropriate error message, while 0 will cause it to try reading a level header at offset 0 (where the "TED5v1.0" string is usually stored), which will cause all kinds of errors as there is clearly no valid level header at that offset.
- The third is the tileinfo data, which contains tile properties for each tile used in level creation. (These are masked and unmasked and either 8x8, 16x16 or 32x32.)
Many programs treat the tileinfo as a separate file from the MAPHEAD and it is possible to modify a game in this manner. Indeed, some games such as Wolfenstein 3-D do not have any tileinfo data at all in the map header file (giving a total file length of 402 bytes.) However TED5 works with any tileinfo data in the MAPHEAD.
Map data (GAMEMAPS)
The GAMEMAPS file consists of the string "TED5v1.0" and a number of compressed chunks of varying length. Each level in the file will have from two to four chunks (usually four) depending on the game, with all levels in a given game having the same number of chunks. These are the level header and 1-3 planes (foreground, background and sprite/info.) The chunks are in no particular order and it is possible to read through the entire file decompressing chunks as they're found.
Chunks are ordered by the MAPHEAD file, which will point to the GAMEMAPS level header chunks which in turn contain pointers to the other GAMEMAPS chunks used by that level.
All level data is in the form of UINT16LE values (or in the case of pointers, UINT32LE.)
Level headers
The header for each level inside the GAMEMAPS file (which is pointed to by MAPHEAD) is 38 bytes long and never RLEW compressed. If the GAMEMAPS file is using Huffman compression, the level header itself is Huffman-compressed. In that case, the size of the compressed level header can be found in the MAPHEAD file. Since the uncompressed level header always has a size of 38 bytes, that size is not stored at the beginning of the compressed data. The header is sometimes followed by a signature "!ID!" string, which would give the header a total size of 42 bytes, but that string is not part of the actual level header.
The expanded size of each map plane is the size of a word (UINT16LE) times the width times the height. This is the final, expected size and can be precomputed even if there are two layers of compression.
Plane 0 is background using unmasked tiles, plane 1 is foreground and uses masked tiles, and plane 2 is sprite/info. Levels must contain a background plane and usually an infoplane.
For the compressed size of each level plane, a value of 0 indicates the plane does not exist. The offset value for unused planes appears to be set to a random number with the most significant byte set to 0xFF, making it either a negative number or an offset close to the 4 gigabyte range. The games only check the compressed size and ignore the offset if the size is 0.
Note that Wolf3D is hard-coded to always load the first two planes, even if the compressed size of a plane is 0. Wolf3D also ignores the width and height stored in the level header and just assumes the level is 64x64 tiles. This also applies to Blake Stone and probably any other game based on the Wolf3D code.
All versions of the "Id Software Caching Manager" ignore the expanded size value at the beginning of the RLEW data and just assume that the size is the same as the size of the plane (width * height * 2).
| Offset | Type | Name | Description |
|---|---|---|---|
| 0 | INT32LE | offPlane0 | Offset in GAMEMAPS to beginning of compressed plane 0 data (or <= 0 if plane is not present) |
| 4 | INT32LE | offPlane1 | Offset in GAMEMAPS to beginning of compressed plane 1 data (or <= 0 if plane is not present) |
| 8 | INT32LE | offPlane2 | Offset in GAMEMAPS to beginning of compressed plane 2 data (or <= 0 if plane is not present) |
| 12 | UINT16LE | lenPlane0 | Length of compressed plane 0 data (in bytes) |
| 14 | UINT16LE | lenPlane1 | Length of compressed plane 1 data (in bytes) |
| 16 | UINT16LE | lenPlane2 | Length of compressed plane 2 data (in bytes) |
| 18 | UINT16LE | width | Width of level (in tiles) |
| 20 | UINT16LE | height | Height of level (in tiles) |
| 22 | char[16] | name | Internal name for level (used only by editor, not displayed in-game. null-terminated) |
Note that for Wolfenstein 3D, a 4-byte signature string ("!ID!") will normally be present directly after the level name. The signature does not appear to be used anywhere, but is useful for distinguishing between v1.0 files (the signature string is missing), and files for v1.1 and later (includes the signature string).
Carmack compression
Carmack compression is the method used to compress later id Software games, when file size was still a concern. It is the most efficient and complex compression method and was created specifically to work with the 16-bit word structure of the GameMaps file. The compression is detailed on its own page.
Carmackized game maps files are external GAMEMAPS.xxx files and the map header is usually (but not always) stored internally in the executable. The map header must be extracted and the game maps decompressed before TED5 can access them. TED5 itself can produce carmackized files and external MAPHEAD.xxx files. Carmackization does not replace the RLEW compression used in uncompressed data, but compresses this data, that is, the data is doubly compressed.
Note that for Wolfenstein 3D v1.0, map files are not carmackized, only RLEW compression is applied.
Huffman compression
Huffman Compression was probably used by earlier versions of TED5 (but possibly not TED5 at all) before carmackization was introduced. It uses the same method to compress its data as is used by id Software games to compress their graphics and sounds. Again this compression method works with RLEW compressed data and has its own page.
Huffman compression is easily detected since it works on the bit level and thus disrupts the word structure of the game data. This is easily seen in a hex editor. Compressed data will not contain the string $00 $00 or indeed even $00 very often. (In contrast, even carmackized data contains both strings hundreds of times.)
There will be two internal files for this format: the map header and the Huffman dictionary (which is always the first dictionary in the executable.) The map header format is also slightly different, being 502 bytes long, the extra 100 bytes being the length of the compressed level headers in the game maps data, which occur immediately after the normal level header offsets and before the tileinfo. Each entry is one octet indicating the decompressed header length in bytes or zero if the level does not exist. (These can be ignored when decompressing since Huffman data can be read until the decompressed level header's fixed size is reached, but if they are omitted when writing the MAPHEAD, the game may experience a buffer overflow when reading the maps.)
Location of internal files
The GAMEMAPS file itself is always external, but in the case of compression, the MAPHEAD is stored internally in the main .exe file. Executables are themselves compressed with either with LZEXE or PKLite. Once the .exe has been decompressed it is a trivial task to find the MAPHEAD as it will start with the UINT16LE value $ABCD (i.e. the byte $CD followed by the byte $AB.) For level editing purposes only the first 402 (or 502) bytes of the file need to be extracted, though it is possible to read the MAPHEAD file to calculate its length.
The following table lists the offsets of the MAPHEAD file for various games, relative to the start of the decompressed game .exe file.
! TODO: Add all known versions of all games
| Game | Version | Location | Filename | Offset | Notes |
|---|---|---|---|---|---|
| Bio Menace | Freeware | External | MAPHEAD.BM[123] | - | |
| Blake Stone: Aliens of Gold | Shareware | External | MAPHEAD.BS1 | - | |
| Registered | External | MAPHEAD.BS6 | - | ||
| Blake Stone 2: Planet Strike | All | External | MAPHEAD.VSI | - | |
| Catacomb 3-D (3) | 1.00 | Internal | CAT3D.EXE | $1C570 | After UNLZEXE |
| Catacomb Abyss (4) | 1.13 | Internal | CATABYSS.EXE | $1C510 | After UNLZEXE |
| Catacomb Armageddon (5) | 1.01a | Internal | CATARMA.EXE | $1D900 | After UNLZEXE |
| Catacomb Apocalypse (6) | 1.00b | Internal | CATAPOC.EXE | $1DD50 | After UNLZEXE |
| Corridor 7 Alien Invasion | Demo | External | MAPHEAD.DMO | - | |
| CD | Internal | CORR7CD.EXE | $30D50 | File is not compressed | |
| Floppy | Internal | C7.EXE | $24BF0 | File is not compressed | |
| Keen 4 | Special Demo | ? | ? | ? | File is PKLite compressed |
| 1.4 EGA | Internal | KEEN4E.EXE | $24830 | After UNLZEXE | |
| Keen 5 | 1.4 EGA | Internal | KEEN5E.EXE | $25990 | After UNLZEXE |
| Keen 6 | 1.4 EGA | Internal | KEEN6.EXE | $25080 | After UNLZEXE |
| Commander Keen Dreams | 1.13 | Internal | ? | $1FA50 | After UNLZEXE |
| Super 3-D Noah's Ark | All | External | maphead.n3d | - | |
| Operation Body Count | All | External | MAPHEAD.BC | - | |
| Spear of Destiny | All | External | MAPHEAD.SOD | - | |
| Wolfenstein 3-D | Shareware | External | MAPHEAD.WL1 | - | |
| Registered | External | MAPHEAD.WL6 | - |
Utilities
- TED5 can edit the GAMEMAPS format of any games that use it. It is the original editor used to create these files.
Credits
This file format was reverse engineered by Andrew Durdin (adurdin). If you find this information helpful in a project you're working on, please give credit where credit is due. (A link back to this wiki would be nice too!)