ModdingWiki - User contributions [en-gb]

GameMaps Format

2009-02-22T21:49:24Z

CK Guy: Thanks, but it wasn't actually me who reverse engineered it. See the 'Plumbing the Depths of Keen' article.

The map data for [[Commander Keen 4-6]] is mainly stored in the <tt>GAMEMAPS.CKx</tt> file. This stores all the maps of the levels used in the game. The GAMEMAPS file is read and decompressed using data stored in the <tt>MAPHEAD.CKx</tt> file. The file's start is easily located as it always begins with the word $ABCD followed by several dwords and a lot of blank space. In the 1.4 versions of Keen 4-6 (Dumped executable) the file starts at the following:

Keen 4: $24830
Keen 5: $25990
Keen 6: $25080
(Keen D: $1FA50)

== Map headers ==

In Keen, as in most games, the <tt>MAPHEAD</tt> file is 402 bytes long, consisting of a word and 100 level dwords.

The first word is the flag for [[RLEW compression]], used to compress the <tt>GAMEMAPS</tt> file. By default this is $ABCD (This can be changed.). The remaining dwords point to the headers of each level in the <tt>GAMEMAPS</tt> file. (Nonexistent levels have a blank dword, so most of the file is empty space.)

== GAMEMAPS Format ==

The structure of the <tt>GAMEMAPS</tt> file is disordered due to the fact that the level headers and level data can be located in any place in any order. The usual structure consists of a variable string of TED settings ('TEDv1.0...') followed by the levels in no particular order. Each level usually consists of the level header followed by three planes of data, though not always.

The TED settings are unimportant and can usually be ignored. (Though in some cases TED may use them.)

=== Levels in GAMEMAPS ===

The header for each level that is pointed to by MAPHEAD is 42 bytes long:

0 4 Offset in GAMEMAPS to beginning of compressed plane 0 data
4 4 Offset in GAMEMAPS to beginning of compressed plane 1 data
8 4 Offset in GAMEMAPS to beginning of compressed plane 2 data
12 2 Length of compressed plane 0 data (in bytes)
14 2 Length of compressed plane 1 data (in bytes)
16 2 Length of compressed plane 2 data (ib bytes)
18 2 Width of level (in tiles)
20 2 Height of level (in tiles)
22 16 Internal name for level (used only by editor, not displayed in-game. null-terminated)
38 4 Signature "!ID!"
42 x Level data, three planes RLEW and Carmack compressed

The presence of the level name and the signature are not required by the Keen engine to play the level. Plane 0 is the 'background' plane, plane 1 is the 'foreground' plane, and plane 2 is the 'info' plane. Each is compressed separately, as described below.

=== Compression in GAMEMAPS ===

Each plane is compressed with two successive compression algorithms. The first is [[RLEW compression]], identical to that used in [[Commander Keen 1-3]], except that the magic word (Flag.) is different. (It is specified in the MAPHEAD file, rather than always being xFEFE.) The second is [[Carmack compression]].

The order of compression is RLEW followed by Carmack. One must also keep in mind that each compression algorithm outputs as its first word the length of the uncompressed data. So in the GAMEMAPS file for a particular plane of a particular level, the first word is the number of bytes in the RLEW-compressed data, and the first word of the RLEW-compressed data is the number of bytes in the uncompressed data.

== Notes ==

This format was reverse engineered by Andrew Durdin (adurdin).

The format is used by other games as well, including [[Bio Menace]], [[Catacomb 3D]], [[Dangerous Dave 3]], [[Dave Goes Nutz]], and [[Wolfenstein 3D]]

Carmack compression

2009-02-14T02:26:44Z

CK Guy: initial page on carmack compression

[[Carmack compression]] is used in the [[GameMaps Format|GAMEMAPS file]] in [[Commander Keen 4-6]] to further shrink the levels down beyond what [[RLEW compression]] can achieve. Its basic idea is somewhat like LZ (Lempel-Ziv) compression in that it contains pointers back to previous data.

As in RLEW compression, the first word in the Carmack compressed data is the number of bytes (not words) in the decompressed data. This is typically the number of bytes in the compressed RLEW data, as Carmack compression is performed after RLEW compression.

Carmack compression contains two types of references to previous data: near pointers and far pointers.

= Near Pointers =

Near pointers occupy three bytes in the compressed data. The first is the number of words in the referenced sequence, the second is the signal byte of xA7, and the third is the number of words to the start of the reference (counting backwards from the current location). As a concrete example, the three bytes x05 xA7 x0A mean 'repeat the 5 words starting 10 words ago'.

Notice that near pointers only let one refer to the last 255 words. To refer to sequences further back, one must use far pointers.

= Far Pointers =

Far pointers occupy four bytes in the compressed data. The first is, again, the number of words in the referenced sequence, the second is xA8, and the third and fourth are interpreted as a word - a 0-based pointer to the start of the reference (in words). As a concrete example, the four bytes x10 xA8 x01 x20 mean 'repeat the 16 words starting at word number 513'.

= Words with a high byte of xA7 or xA8 =

Words whose high byte is xA7 or xA8 would appear to be issue, as they would be confused with near or far pointers. These are handled by representing them as the three bytes: x00, xA7 or xA8 as appropriate, the low-order byte of the word.

Commander Keen 1-3

2009-02-14T01:39:05Z

CK Guy: repointed 'full screen planar ega' link to proper section name of 'raw ega data'

{{Stub}}
{{GamePage|Commander Keen 1}}

Commander Keen episodes 1-3 was a revolutionary series that launched [[ID Software]]. The games made use of a number of formats and novel features that were to be adapted to other games ID produced. Keen is notable for being the first sidescrollling EGA game on the PC, and for cementing ID's love for EGA graphics (Most previous games had CGA, EGA and VGA graphics options, all later games were EGA, or had a separate CGA version.)

In them, the hero, Commander Keen, must collect stolen parts of his spaceship hidden on Mars (1), stop a Vorticon ship blowing up earth (2) and finally storm Vorticon VI itself in search of the cause of all this trouble, the mysterious Grand Intellect.

== File formats ==

This section lists the major file formats used in the game.

* [[B800 Text]] - 4000-byte internal files are text screens shown at exit.
* [[Commander Keen 1-3 Level format]] - <tt>LEVELxx.CKy</tt> are game levels.
* [[Commander Keen EGA Header]] the EGA header is used to read graphics files
* [[Keen 1-3 Tileinfo format]] - information about the properties of tiles are stored internally in the executable
* [[Commander Keen 1-3 Score format]] <tt>SCORES.CKy</tt> stores the game's highscores
* [[Keen 1-3 Saved game format]] <tt>SAVEx.CKy</tt> store the game's save files. Keen is one of the few early ID games to allow this.
* [[Inverse Frequency Sound format]] - the sound file is stored internally and uses Apogees 'inverse frequency' sound format
* [[Raw EGA data#Planar EGA data|Full-screen planar EGA]] Most graphics are in this format, compressed or rearranged in various ways. For example, the 320x200 screen graphics are this compressed with [[Keen 1-3 RLE compression]], the <tt>EGALATCH</tt> AND <tt>EGASPRIT</tt> files use this too. Note that the screen's plane size is 8000 bytes, 192 bytes are just 'padding'

* [[Keen 1-3 RLE compression]] This is used to compress the ending and previews screens and is a modified form of RLE.
* [[Keen 1-3 LZW compression]] Used in episode 1 to compress the graphics files
* Text is contained in the executable, the executable is compressed with [[LZW compression]], and can be uncompressed with UNLZEXE.

== Notes ==

The so called 'Keen engine series' begins with this game. The two games based on this engine are [[Dangerous Dave 2]] and [[Shadow Knights]]

Most of the file formats here were reverse engineered by the Keen modding community.

Commander Keen is mentioned in [[Epic Megagames]] [[Jill of the Jungle]].

== Related Links ==
* [[Commander Keen Dreams]]
* [[Commander Keen 4-6]]
* The [[KeenWiki:Main Page|Commander Keen Wiki]]

[[Category:Apogee]]
[[Category:id Software]]
[[Category:Sidescroller]]

GameMaps Format

2009-02-14T01:35:52Z

CK Guy: Initial page on GameMaps format

The map data for [[Commander Keen 4-6]] is mainly stored in one file: GAMEMAPS.CKx, along with some header information in the MAPHEAD resource, located within the executable.

= MAPHEAD Header Resource =

In Commander Keen, the MAPHEAD resource is 402 bytes long, consisting conceptually of one word and 100 dwords.

The first word specifies the so-called 'magic word' used by the [[RLEW compression]] (see below). This is xABCD in all versions of episodes 4-6, but it can be changed.

Each following dword gives an index into the GAMEMAPS file. The first points to the header of level 0, the second to the header of level 1, and so on up to level 99. The pointer is x00000000 for any level which is not present in GAMEMAPS.

= GAMEMAPS File Format =

In the original Keen games, the first thing in the GAMEMAPS file is the string "TED5v1.0" followed by several junk bytes before that actual level data begins. The contents of this string and the junk bytes are not important, nor are their length.

After this come, in no particular order, the headers for the various levels, as well as the compressed plane data.

== Level Header in GAMEMAPS ==

The header for each level that is pointed to by MAPHEAD is 42 bytes long.

0 4 Offset in GAMEMAPS to beginning of compressed plane 0 data
4 4 Offset in GAMEMAPS to beginning of compressed plane 1 data
8 4 Offset in GAMEMAPS to beginning of compressed plane 2 data
12 2 Length of compressed plane 0 data (in bytes)
14 2 Length of compressed plane 1 data (in bytes)
16 2 Length of compressed plane 2 data (ib bytes)
18 2 Width of level (in tiles)
20 2 Height of level (in tiles)
22 16 Internal name for level (used only by editor, not displayed in-game. null-terminated)
38 4 Signature "!ID!"

The presence of the level name and the signature are not required by the Keen engine to play the level.

Plane 0 is the 'background' plane, plane 1 is the 'foreground' plane, and plane 2 is the 'info' plane. Each is compressed separately, as described below.

== One plane in GAMEMAPS ==

Each plane is compressed with two successive compression algorithms. The first is [[RLEW compression]], identical to that used in [[Commander Keen 1-3]], except that the magic word is different. (It is specified in the MAPHEAD file, rather than always being xFEFE.) The second is [[Carmack compression]].

When compressing, one must first use [[RLEW compression]], then [[Carmack compression]]. When one is decompressing, one must undo the [[Carmack compression]], then the [[RLEW compression]].

One must also keep in mind that each compression algorithm outputs as its first word the length of the uncompressed data. So in the GAMEMAPS file for a particular plane of a particular level, the first word is the number of bytes in the RLEW-compressed data, and the first word of the RLEW-compressed data is the number of bytes in the uncompressed data.