ModdingWiki - User contributions [en-gb]

GameMaps Format

2011-09-02T10:16:36Z

CarlKenner: /* Location of internal files */ Catacomb Apocalypse and Armageddon

The GameMaps format is how the levels in a number of [[Id Software]] games is stored. The filenames and compression varies somewhat across different games but all files stored in this format were produced by the program [[TED5]] or a close analogue.

There are three main varieties of the GameMaps format, uncompressed, [[Carmack compression||carmackized]], and [[Huffman Compression|huffman compressed]]. Each variation has its own file naming scheme and pattern of external\internal files.

There are two main components to the format, the game maps proper, which contain the game level data and the map headers, which contain both the locations of the start of level data in the game maps file and the tile info for the game.

== Structure of uncompressed data ==

The uncompressed format is used by several games including [[Bio Menace]] and [[Wolfenstein 3D]]. In this format map data is stored in <tt>MAPTEMP.xxx<tt> and map headers in <tt>MAPTHEAD.xxx</tt> this is the format saved by [[TED5]] when maps are being edited and can be directly accessed and edited by this utility, allowing changes to be made to the game.

Note that filenames differ and that Wolfenstein 3D uses <tt>GAMEMAPS.WLx</tt> and <tt>MAPHEAD.xxx</tt> files to store data. The main indicator of this format being used is that in the unaltered game the map header file is external.

Only the latest Id Software games lacked compression, when file size was no longer an issue.

=== Map headers ===

0 2 Magic word Magic word for [[RLEW compression]]
2 400 Level ptr 100 4-byte pointers to start of level 0-99 data in game maps file
402 ??? Tileinfo Tileinfo data

The map header file is of varying length and contains three main types of data. The first is the magic word or flag used for RLEW compression, it is always $ABCD. The second is 100 level pointers which give the location of the start of level data in the game maps file, relative to the start of that file. A value of 0 indicates no level. The third is the tileinfo data, tile properties for each tile used in level creation. (These are masked and unmasked and either 8x8, 16x16 or 32x32)

Many programs treat the tileinfo as a separate file from the maphead and it is possible to modify a game in this manner. Indeed, some games, such as Wolfenstein 3D do not have any tileinfo data at all in the map header file (Giving a total file length of 402 bytes.) However TED5 works with any tileinfo data in the maphead.

=== Game maps ===

The game maps file consists of the string 'TED5v1.0.' and a number of RLEW compressed chunks of varying length. Each level in GameMaps will have from two to four chunks (usually four), depending on game, with all levels in a given game having the same number of chunks. These are the level header and 1-3 planes (Foreground, background and sprite or info.) The chunks are in no particular order and it is possible to read through the entire file decompressing chunks as they're found.

Chunks are ordered by the map header file, which will point to the level header chunks which in turn contain pointers to the other chunks used by that level. It is possible the string at the start of the file is to avoid the possibility of a level header at the start of the file which would be read as not existing.

All data in game maps file is in the form of words (Or in the case of pointers, dwords.)

=== Level headers ===

The header for each level that is pointed to by MAPHEAD is 42 bytes long and RLEW compressed, though this is difficult to see since the data is so short and nonrepetitive. For the offsets to level planes a value of 0 indicates the plane does not exist.

Plane 1 is background using unmasked tiles, plane 2 is foreground and uses masked tiles. Plane 3 is sprite or info. Levels must contain c background plane and usually an infoplane.

0 4 Offset in GAMEMAPS to beginning of compressed plane 0 data
4 4 Offset in GAMEMAPS to beginning of compressed plane 1 data
8 4 Offset in GAMEMAPS to beginning of compressed plane 2 data
12 2 Length of compressed plane 0 data (in bytes)
14 2 Length of compressed plane 1 data (in bytes)
16 2 Length of compressed plane 2 data (in bytes)
18 2 Width of level (in tiles)
20 2 Height of level (in tiles)
22 16 Internal name for level (used only by editor, not displayed in-game. null-terminated)

== Carmack compression ==

[[Carmack compression]] is the method used to compress later Id Software games, when file size was an issue. It is the most efficient and complex compression method and was created specifically to work with the word structure of the game maps file. The compression is detailed on its [[Carmack compression|own page.]]

Carmackized game maps files are external <tt>GAMEMAPS.xxx</tt> files and the map header is stored internally in the executable. The map header must be extracted and the game maps decompressed before TED5 can access them. TED5 itself can produce carmackized files and external <tt>MAPHEAD.xxx</tt> files. Carmackization does not replace the RLEW compression used in uncompressed data, but compresses this data, that is, the data is doubly compressed.

== Huffman compression ==

Huffman compression was used by earlier versions of TED5 (possibly not TED5 at all.) before carmackization was created. It uses the same method to compress its data as is used by Id Software games to compress their graphics and sounds. Again this compression method works with RLEW compressed data and has its [[Huffman Compression|own page.]]

Huffman compression is easily detected since it works on the bit level and thus disrupts the word structure of the game data. This is easily seen in a hex editor. Compressed data will not contain the string $00 $00 or indeed even $00 very often (In contrast even carmackized data contains both strings hundreds of times.)

There will be two internal files for this format, the map header and the huffman dictionary (which is always the first dictionary in the executable.) The map header format is also slightly different, being 502 bytes long, the extra 100 bytes being the length of the compressed level headers in the game maps data (These can be ignored when decompressing since huffman data can be read until the decompressed level header's fixed size is reached.)

== Location of internal files ==

Game maps are always external, but in the case of compression the map header file is stored internally in the executable. Executables are themselves compressed wither with LZEXE or PKLite. Once decompressed it is a trivial task to find the map header file as it will start with the word $ABCD For level editing purposes only the first 402 (or 502) bytes of the file needs to be extracted, though it is possible to read the maphead file to calculate its length.

The following addresses are in the locations in the decompressed executables for the start of the Map header files. Values are different for different versions.

TODO: Add all known versions of all games.

External files
Biomenace
Blake Stone 2: Planet Strike
Corridor 7 Demo
Noahs Ark 3D
Operation Body Count
Spear of Destiny
Wolfenstein 3D

Internal files:
Catacomb 3-D (1.00): $1C570 from start of file CAT3D.EXE (after file is UNLZEXE)
Catacomb 4 Abyss (1.13): $1C510 from start of file CATABYSS.EXE (after file is UNLZEXE)
Catacomb 5 Armageddon (1.01a): $1D900 from start of file CATARMA.EXE (after file is UNLZEXE)
Catacomb 6 Apocalypse (1.00b): $1DD50 from start of file CATAPOC.EXE (after file is UNLZEXE)
Corridor 7 CD (): $30D50 from start of file CORR7CD.EXE (file is not compressed)
Corridor 7 Floppy (): $24BF0 from start of file C7.EXE (file is not compressed)
Keen 4 (Special Demo): (file is PKLite compressed)
Keen 4 (1.4): $24830
Keen 5 (1.4): $25990
Keen 6 (1.4): $25080
(Keen D (1.13): $1FA50)

== Utilities ==

[[TED5]] This program can edit the <tt>GAMEMAPS</tt> format of any games that use it. It is the original editor used to crate these files.

== Notes ==

This format was reverse engineered by Andrew Durdin (adurdin).

The format is used by other games as well, including [[Bio Menace]], [[Blake Stone]], [[Catacomb 3D]], [[Catacomb Abyss]], [[Corridor 7]], [[Dangerous Dave 3]], [[Dave Goes Nutz]], [[Noah's Ark 3D]], [[Operation Body Count]], [[Spear of Destiny]], and [[Wolfenstein 3D]].

[[Category:File Formats]]
[[Category:Map Files]]

EGAGraph Format

2011-09-02T04:35:44Z

CarlKenner: /* EGA Head (EGAHEAD.xxx) */ Catacomb Apocalypse

{{Stub}}

'''EGAGraph''', short for 'EGA graphics library' is the format used to store graphics, demos, fonts, game texts and more in many early [[:Category:Id Software|id Software]] games.

The basic format is that of a number of [[Huffman Compression]] compressed sub-files (Chunks) stored in a similar manner to the [[AudioT Format]]. Like this format it has three main files, <tt>EGAGRAPH</tt>, <tt>EGAHEAD</tt>and <tt>EGADICT</tt>.

It is notable that while the <tt>EGAHEAD</tt> and <tt>EGADICT</tt> are stored in the game executable, they need to be extracted for [[TED5]] to edit levels.

This article may contain errors as some formats have not been worked out completely.

= File formats =

== EGA Head (EGAHEAD.xxx) ==
There are two versions of this format. The first version, used in [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], is an array of 4-byte little-endian values. The second version, used in all later EGA games, is an array of 3-byte unsigned little-endian values.

This header file may be present as an external file (eg. [[Bio Menace]]), or it may be included in the EXE file (eg. [[Catacomb 3D]]). The last entry in the header is always the size of the EGAGRAPH.xxx file, and the first entry in the header is always 00 00 00. To find the end, you can search for filesize(EGAGRAPH.xxx) as a 3-byte (little-endian) value in the EXE. Then to find the start, go back 3 bytes at a time until you get to 00 000 00. Or use the fact that entries always get smaller as you go back, unless the entry is $FFFFFF.

In [[Catacomb 3D]] v1.00 <tt>EGAHEAD.C3D</tt> is at $1BFD0 from the start of the decompressed ([[unlzexe]]) EXE file, and is 1437 bytes long, or 479 3-byte entries (for 478 chunks). It is then followed by 3 zero bytes (padding to multiple of 16?) 00 00 00, then the <tt>MAPHEAD.C3D</tt>.

In [[Catacomb Abyss]] v1.13 <tt>EGAHEAD.ABS</tt> is at $1BDB0 from the start of the decompressed ([[unlzexe]]) EXE file, and 1881 bytes long and has 626 chunks.

In [[Catacomb Apocalypse]] v1.00b <tt>EGAHEAD.APC</tt> is at $1E270 from the start of the decompressed ([[unlzexe]]) EXE file, and 2049 bytes long and has 682 chunks.

This header file stores the offsets (relative to the start of the <tt>EGAGRAPH</tt> file) for each sub file. The format is trivial, in that the header file is simply an array of 3 or 4 byte variables. (Each variable is a "slot", as the game refers to files by index.) Most games use the more compact 3-byte variables, since the <tt>EGAGRAPH</tt> file is never larger than $FFFFFF. However games based on older engines, notably [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], use 4-byte vales as these are easier to deal with.

It is important to note that not every 'slot' will be in use. Tiles especially if blank (All black) will be considered 'empty' and not worth adding to the file. (A blank 16x16 tile takes up 12 bytes of space if compressed, and there can be hundreds.) There are two ways of dealing with this; by default the games set all empty slot headers to -1, but many programs compress the graphics anyway and use a normal header value for them.

The last offset in the file will be an offset to the end of the main <tt>EGAGRAPH</tt> file. (So this should be ignored when reading the file to prevent a zero-byte file appearing last.) The total size of the <tt>EGAHEAD</tt> divided by 3 (Or 4) will also give the number of 'slots' the game uses. This varies, but is usually about 10,000.

Unlike [[AudioT Format]] files, <tt>EGAGRAPH</tt> is always compressed. However the header and dictionary may be external (E.g [[Bio Menace]])

== Dictionary (EGADICT.xxx) ==

The same [[Huffman Compression]] scheme used elsewhere in the [[Commander Keen]] series is used. When reading compressed chunks out of the <tt>EGAGRAPH</tt> file, the first four bytes located at the <tt>EGAHEAD</tt> offset are a UINT32LE specifying the file's decompressed size (which is required for the decompression algorithm.) The compressed data then follows, EXCEPT for 16x16 masked and unmasked tiles. Since these all have a similar decompressed size (128 bytes for unmasked, 256 bytes for masked) it is a waste of space to include these (Since tiles make up the bulk of graphics slots.) The decompressed size is hard coded, as are the start and finish of the unmasked and masked tile slots.

Note that the <tt>EGADICT</tt> file is not often obvious, usually being embedded in the main exe file. It is possible to locate it by looking for the string <tt>$FD $01 $00 $00 $00 $00</tt>, which appears at the end of nearly all Huffman dictionaries. They are the last 6 bytes out of 1024 (256*2*2), so add 6 and subtract 1024 to get the start (pointer to the head node).

Executables usually contain two or more dictionaries, but the <tt>EGADICT</tt> is usually '''the second one''', except in the case of early games like Keen Dreams, where it is the first of THREE. (A simple check of whether it decompresses the data sensibly works.)

In [[Catacomb 3D]] <tt>EGADICT.C3D</tt> is at offset $24464 from the start of the decompressed CAT3D.EXE, which is the second Huffman dictionary and comes immediately after the first Huffman Dictionary.

In [[Catacomb Abyss]] v1.13 <tt>EGADICT.ABS</tt> is at offset $2734C from the start of the decompressed CATABYSS.EXE, which is the second Huffman dictionary.

In [[Catacomb Apocalypse]] v1.00b <tt>EGADICT.APC</tt> is at offset $26B24 from the start of the decompressed CATAPOC.EXE, which is the second Huffman dictionary.

== Main file (EGAGRAPH.xxx) ==

This file is simply an array of data files. Each file starts at the offset specified in the <tt>EGAHEAD</tt> file, and as there are no filenames each file is referred to by its index/slot number. Slots containing dummy values in the header are treated as if they don't exist in the <tt>EGAGRAPH</tt> file. Each file within the <tt>EGAGRAPH</tt> file is individually compressed. A file can be read by opening the <tt>EGAGRAPH</tt> file, seeking to the offset specified in the <tt>EGAHEAD</tt> file, reading a UINT32LE for the decompressed file size, and then decompressing the data from that point onwards using standard Huffman decompression techniques.

Note that in early games such as Keen Dreams, the <tt>EGAGRAPH</tt> file may have another name. (Such as <tt>KDREAM.EGA</tt> in that case.) in which case it is always the largest non-executable file.

=== Chunks ===

Each sub file or chunk is a separate graphic, ordered in the <tt>EGAGRAPH</tt> file in a specific way, much again like the [[AudioT Format]]. Chunks are labeled starting at 0. The game executable is hard coded for the start and finish of various chunk types. For example, Keen 4 has chunks 124-520 as sprites and will treat any chunk between these two numbers as a sprite. (Note that the game refers to chunk number, not sprite number.) The usual order of chunks is:

Picture table
Masked picture table
Sprite table
Fonts
Pictures (Unmasked bitmaps)
Masked pictures
8x8 unmasked tiles (Single chunk)
8x8 masked tiles (Single chunk)
16x16 unmasked tiles
16x16 masked tiles
32x32 unmasked tiles (Optional)
32x32 masked tiles (Optional)
Misc graphics (Optional)
Game texts
Demo files (Optional)
Misc data (Optional)

== Chunk formats ==

When compressed each chunk consists of a dword giving the decompressed size followed by the compressed data.

=== Picture table ===

Always present and always the first chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying picture chunks.

Divide the decompressed size by 4 to get numpics.

=== Masked picture table ===

Always present and always the second chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying masked picture chunks. This table is often quite small.

=== Sprite table ===

Always present and always the third chunk, the picture table is 16 * numspr bytes long, consisting of eight words for each sprite image (Width\8, Height, x offset, y offset and clipping rectangle left, top, right and bottom; all in that order.) used when displaying sprite chunks. This is what Modkeen exports as its notoriously hard to use xSPRITES.txt file.

=== Fonts ===

Always present and always starting at the fourth chunk. (Chunk 3!) Each chunk is a single monochrome font, containing all the entries from 0-255 for that font. Each 'letter' is stored separately and consecutively. A game will have at least one, and usually 3 fonts. The format is version two of [[EGA Font format]]

Fonts are used in the game for game texts. They can be stretched or colored and are always transparent, meaning they look strange when not over a single-color background.

=== Pictures (Bitmaps) ===
These follow game fonts. You need to know how many fonts there are before it to know how what chunk number the first picture is (3+NumFonts). In [[Catacomb 3D]] there are 2 fonts.

Bitmaps are stored as standard [[Raw EGA data]], the size of each plane being the decompressed chunk size \ 4. In order to be displayed properly these need information from chunk 0, the picture table. Note that unlike fonts, data does not wrap, so a 4x4 sprite will consist of four planes of four bytes in size, the same as an 8x4 sprite.

Pictures are used for things such as title screens or pictures in game texts. Most of the main menu will consist of pictures. They are also used for the wall textures in EGA FPS games like [[Catacomb 3D]], in which case the wall textures come near the end of the picture list (starting from chunk 137 in [[Catacomb 3D]]).

The number of picture chunks is given in chunk 0, the picture table.

=== Masked pictures (Masked bitmaps) ===

These follow Pictures. You need to know how many fonts there are, to know how what chunk number the first Masked picture is (3+NumFonts+numpics).

These are identical to the picture chunks in every respect, except being masked, they consist of five EGA planes, not four. Some games (E.g [[Catacomb 3D]] make great use of masked pictures, but most games have only one or two. They need information from chunk 1, the masked picture table, to be displayed correctly.

The number of Masked pictures is given in Chunk 1.

=== Sprites ===

The first Sprite chunk is at 3+numfonts+numpics+numMaskedPics.

Sprites are exactly identical to masked pictures in format except the game cannot stretch or warp them. Most games use these for enemy, player or item graphics during gameplay. They need information from chunk 2, the sprite table both to display correctly and to interact correctly, something more complex than masked pictures.

There are usually several hundred sprite chunks, but [[Catacomb 3D]] only has 3 sprite chunks and they are used for the PaddleWar game (Skulls and Bones). The number of sprite chunks is given in chunk 2.

=== 8x8 tiles ===

8x8 tiles are used by most games in status windows or foe the borders of message windows in-game. (And occasionally by TED5 to display levels.) There are thus not many of them. ALL the masked or unmasked tiles are one single chunk stored as four and five plane [[Raw EGA data]] respectively. The number of 8x8 unmasked and masked tiles is hard-coded into the executable, but can be worked out by dividing the decompressed chunk size by 32 and 40 (The size of one tile's data) respectively.

=== 16x16 and 32x32 tiles ===

32x32 tiles are optional and rarely used. 16x16 tiles are only occasionally absent and for most games make up the bulk of graphics. There are usually several thousand entries. Both come in masked and unmasked kinds, just like 8x8 tiles.

These chunks have notable differences. Each tile is an individual chunk and stored as four or five plane [[Raw EGA data]] with a plane size of 128 (16x16) or 512 (32x32). These chunks do NOT have a dword specifying their decompressed size as this is hard-coded into the executable as a space saving measure. (As is the start and finish of unmasked and masked tile chunks.) This can cause problems for editing programs.

Tiles make up the bulk of 2D game levels but are seldom seen elsewhere.

=== Misc graphics ===

Sometimes following tile graphics are miscellaneous graphics, that cannot be handled by the other chunk types. For example, Keen 4-6 has two misc graphics files used for the 'COOMANDER KEEN terminator text' intro. These graphics usually have unique formats that fit their function and few have been investigated.

==== Keen 4-6 Intro Bitmaps ====

This is the text that displays after the 'Ready, press any key' screen at Keen startup. It is composed of a monochrome bitmap that is scrolled across the screen and distorted for special effects. When displayed it is transparent, with a special palette so it is not black-and-white. It consists of a header followed by RLE compressed monochrome data.

HEADER:
0 2 Img height Height of the image in pixels
2 2 Img width Width of the image in pixels
4 2x Line point Pointers to line 1,2,3..etc of data. There will be [Img height] of
these, each 2 bytes long. The first pointer will have the value
(2 * [Img height] + 4)
+2 ? RLE data RLE-WM compressed data

DATA:
0 2 Black run Number of black pixels to write
2 2 Not-blk run Number of not-black pixels to write
4 2 Black run....
...
..
.
? 2 End $FFFF; end of row.

=== Game texts ===

Also known as ANSII chunks, these are used by the game for things like help screens, and have their own format. In the main they are simply text documents, interspersed with various commands to make things happen. Each file is divided up into a number of 'pages' which are moved between by pressing the up/down arrows. Pages can also have action sequences (Repeated each time the page is moved to, you cannot leave a page until the sequence finishes.)

^P First command in every file. Defines a page start
^E Ends the file
^Cx Change font color to $x until next page or ^C command
^Gx,y,z Display (unmasked) picture chunk z at location x,y (In pixels)
^Tx,y,z,t Display picture chunk z at x,y for z clicks of time
^Bx,y,z,t,b ?
^Lx,y ?

=== Misc files ===

The last few chunks are demo files and occasionally miscellaneous data. Demo file formats are currently unknown but can be recorded using the demo cheat. They contain the level number and difficulty followed by a series of key commands.

= Utilities =
* [[KeenWiki:ModKeen|ModKeen]] (DOS) and [[KeenWiki:LModKeen|LModKeen]] (Linux/Windows) can be used to modify graphics and text stored in EGAGRAPH.CKx for the Commander Keen [[KeenWiki:Goodbye Galaxy|Goodbye Galaxy]] trilogy.
* WDC (Windows) can be used to view or modify the graphics and text stored in EGAGRAPH for [[Catacomb 3D]] and [[Catacomb Abyss]].

[[Category:Catacomb 3D]]
[[Category:Catacomb Abyss]]
[[Category:Commander Keen 4-6]]
[[Category:Commander Keen Dreams]]
[[Category:Dangerous Dave 3]]
[[Category:File Formats]]
[[Category:Graphics Files]]

EGAGraph Format

2011-09-02T04:33:33Z

CarlKenner: /* Dictionary (EGADICT.xxx) */ Catacomb Apocalypse

{{Stub}}

'''EGAGraph''', short for 'EGA graphics library' is the format used to store graphics, demos, fonts, game texts and more in many early [[:Category:Id Software|id Software]] games.

The basic format is that of a number of [[Huffman Compression]] compressed sub-files (Chunks) stored in a similar manner to the [[AudioT Format]]. Like this format it has three main files, <tt>EGAGRAPH</tt>, <tt>EGAHEAD</tt>and <tt>EGADICT</tt>.

It is notable that while the <tt>EGAHEAD</tt> and <tt>EGADICT</tt> are stored in the game executable, they need to be extracted for [[TED5]] to edit levels.

This article may contain errors as some formats have not been worked out completely.

= File formats =

== EGA Head (EGAHEAD.xxx) ==
There are two versions of this format. The first version, used in [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], is an array of 4-byte little-endian values. The second version, used in all later EGA games, is an array of 3-byte unsigned little-endian values.

This header file may be present as an external file (eg. [[Bio Menace]]), or it may be included in the EXE file (eg. [[Catacomb 3D]]). The last entry in the header is always the size of the EGAGRAPH.xxx file, and the first entry in the header is always 00 00 00. To find the end, you can search for filesize(EGAGRAPH.xxx) as a 3-byte (little-endian) value in the EXE. Then to find the start, go back 3 bytes at a time until you get to 00 000 00. Or use the fact that entries always get smaller as you go back, unless the entry is $FFFFFF.

In [[Catacomb 3D]] v1.00 <tt>EGAHEAD.C3D</tt> is at $1BFD0 from the start of the decompressed ([[unlzexe]]) EXE file, and is 1437 bytes long, or 479 3-byte entries (for 478 chunks). It is then followed by 3 zero bytes (padding to multiple of 16?) 00 00 00, then the <tt>MAPHEAD.C3D</tt>.

In [[Catacomb Abyss]] v1.13 <tt>EGAHEAD.ABS</tt> is at $1BDB0 from the start of the decompressed ([[unlzexe]]) EXE file, and 1881 bytes long and has 626 chunks.

This header file stores the offsets (relative to the start of the <tt>EGAGRAPH</tt> file) for each sub file. The format is trivial, in that the header file is simply an array of 3 or 4 byte variables. (Each variable is a "slot", as the game refers to files by index.) Most games use the more compact 3-byte variables, since the <tt>EGAGRAPH</tt> file is never larger than $FFFFFF. However games based on older engines, notably [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], use 4-byte vales as these are easier to deal with.

It is important to note that not every 'slot' will be in use. Tiles especially if blank (All black) will be considered 'empty' and not worth adding to the file. (A blank 16x16 tile takes up 12 bytes of space if compressed, and there can be hundreds.) There are two ways of dealing with this; by default the games set all empty slot headers to -1, but many programs compress the graphics anyway and use a normal header value for them.

The last offset in the file will be an offset to the end of the main <tt>EGAGRAPH</tt> file. (So this should be ignored when reading the file to prevent a zero-byte file appearing last.) The total size of the <tt>EGAHEAD</tt> divided by 3 (Or 4) will also give the number of 'slots' the game uses. This varies, but is usually about 10,000.

Unlike [[AudioT Format]] files, <tt>EGAGRAPH</tt> is always compressed. However the header and dictionary may be external (E.g [[Bio Menace]])

== Dictionary (EGADICT.xxx) ==

The same [[Huffman Compression]] scheme used elsewhere in the [[Commander Keen]] series is used. When reading compressed chunks out of the <tt>EGAGRAPH</tt> file, the first four bytes located at the <tt>EGAHEAD</tt> offset are a UINT32LE specifying the file's decompressed size (which is required for the decompression algorithm.) The compressed data then follows, EXCEPT for 16x16 masked and unmasked tiles. Since these all have a similar decompressed size (128 bytes for unmasked, 256 bytes for masked) it is a waste of space to include these (Since tiles make up the bulk of graphics slots.) The decompressed size is hard coded, as are the start and finish of the unmasked and masked tile slots.

Note that the <tt>EGADICT</tt> file is not often obvious, usually being embedded in the main exe file. It is possible to locate it by looking for the string <tt>$FD $01 $00 $00 $00 $00</tt>, which appears at the end of nearly all Huffman dictionaries. They are the last 6 bytes out of 1024 (256*2*2), so add 6 and subtract 1024 to get the start (pointer to the head node).

Executables usually contain two or more dictionaries, but the <tt>EGADICT</tt> is usually '''the second one''', except in the case of early games like Keen Dreams, where it is the first of THREE. (A simple check of whether it decompresses the data sensibly works.)

In [[Catacomb 3D]] <tt>EGADICT.C3D</tt> is at offset $24464 from the start of the decompressed CAT3D.EXE, which is the second Huffman dictionary and comes immediately after the first Huffman Dictionary.

In [[Catacomb Abyss]] v1.13 <tt>EGADICT.ABS</tt> is at offset $2734C from the start of the decompressed CATABYSS.EXE, which is the second Huffman dictionary.

In [[Catacomb Apocalypse]] v1.00b <tt>EGADICT.APC</tt> is at offset $26B24 from the start of the decompressed CATAPOC.EXE, which is the second Huffman dictionary.

== Main file (EGAGRAPH.xxx) ==

This file is simply an array of data files. Each file starts at the offset specified in the <tt>EGAHEAD</tt> file, and as there are no filenames each file is referred to by its index/slot number. Slots containing dummy values in the header are treated as if they don't exist in the <tt>EGAGRAPH</tt> file. Each file within the <tt>EGAGRAPH</tt> file is individually compressed. A file can be read by opening the <tt>EGAGRAPH</tt> file, seeking to the offset specified in the <tt>EGAHEAD</tt> file, reading a UINT32LE for the decompressed file size, and then decompressing the data from that point onwards using standard Huffman decompression techniques.

Note that in early games such as Keen Dreams, the <tt>EGAGRAPH</tt> file may have another name. (Such as <tt>KDREAM.EGA</tt> in that case.) in which case it is always the largest non-executable file.

=== Chunks ===

Each sub file or chunk is a separate graphic, ordered in the <tt>EGAGRAPH</tt> file in a specific way, much again like the [[AudioT Format]]. Chunks are labeled starting at 0. The game executable is hard coded for the start and finish of various chunk types. For example, Keen 4 has chunks 124-520 as sprites and will treat any chunk between these two numbers as a sprite. (Note that the game refers to chunk number, not sprite number.) The usual order of chunks is:

Picture table
Masked picture table
Sprite table
Fonts
Pictures (Unmasked bitmaps)
Masked pictures
8x8 unmasked tiles (Single chunk)
8x8 masked tiles (Single chunk)
16x16 unmasked tiles
16x16 masked tiles
32x32 unmasked tiles (Optional)
32x32 masked tiles (Optional)
Misc graphics (Optional)
Game texts
Demo files (Optional)
Misc data (Optional)

== Chunk formats ==

When compressed each chunk consists of a dword giving the decompressed size followed by the compressed data.

=== Picture table ===

Always present and always the first chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying picture chunks.

Divide the decompressed size by 4 to get numpics.

=== Masked picture table ===

Always present and always the second chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying masked picture chunks. This table is often quite small.

=== Sprite table ===

Always present and always the third chunk, the picture table is 16 * numspr bytes long, consisting of eight words for each sprite image (Width\8, Height, x offset, y offset and clipping rectangle left, top, right and bottom; all in that order.) used when displaying sprite chunks. This is what Modkeen exports as its notoriously hard to use xSPRITES.txt file.

=== Fonts ===

Always present and always starting at the fourth chunk. (Chunk 3!) Each chunk is a single monochrome font, containing all the entries from 0-255 for that font. Each 'letter' is stored separately and consecutively. A game will have at least one, and usually 3 fonts. The format is version two of [[EGA Font format]]

Fonts are used in the game for game texts. They can be stretched or colored and are always transparent, meaning they look strange when not over a single-color background.

=== Pictures (Bitmaps) ===
These follow game fonts. You need to know how many fonts there are before it to know how what chunk number the first picture is (3+NumFonts). In [[Catacomb 3D]] there are 2 fonts.

Bitmaps are stored as standard [[Raw EGA data]], the size of each plane being the decompressed chunk size \ 4. In order to be displayed properly these need information from chunk 0, the picture table. Note that unlike fonts, data does not wrap, so a 4x4 sprite will consist of four planes of four bytes in size, the same as an 8x4 sprite.

Pictures are used for things such as title screens or pictures in game texts. Most of the main menu will consist of pictures. They are also used for the wall textures in EGA FPS games like [[Catacomb 3D]], in which case the wall textures come near the end of the picture list (starting from chunk 137 in [[Catacomb 3D]]).

The number of picture chunks is given in chunk 0, the picture table.

=== Masked pictures (Masked bitmaps) ===

These follow Pictures. You need to know how many fonts there are, to know how what chunk number the first Masked picture is (3+NumFonts+numpics).

These are identical to the picture chunks in every respect, except being masked, they consist of five EGA planes, not four. Some games (E.g [[Catacomb 3D]] make great use of masked pictures, but most games have only one or two. They need information from chunk 1, the masked picture table, to be displayed correctly.

The number of Masked pictures is given in Chunk 1.

=== Sprites ===

The first Sprite chunk is at 3+numfonts+numpics+numMaskedPics.

Sprites are exactly identical to masked pictures in format except the game cannot stretch or warp them. Most games use these for enemy, player or item graphics during gameplay. They need information from chunk 2, the sprite table both to display correctly and to interact correctly, something more complex than masked pictures.

There are usually several hundred sprite chunks, but [[Catacomb 3D]] only has 3 sprite chunks and they are used for the PaddleWar game (Skulls and Bones). The number of sprite chunks is given in chunk 2.

=== 8x8 tiles ===

8x8 tiles are used by most games in status windows or foe the borders of message windows in-game. (And occasionally by TED5 to display levels.) There are thus not many of them. ALL the masked or unmasked tiles are one single chunk stored as four and five plane [[Raw EGA data]] respectively. The number of 8x8 unmasked and masked tiles is hard-coded into the executable, but can be worked out by dividing the decompressed chunk size by 32 and 40 (The size of one tile's data) respectively.

=== 16x16 and 32x32 tiles ===

32x32 tiles are optional and rarely used. 16x16 tiles are only occasionally absent and for most games make up the bulk of graphics. There are usually several thousand entries. Both come in masked and unmasked kinds, just like 8x8 tiles.

These chunks have notable differences. Each tile is an individual chunk and stored as four or five plane [[Raw EGA data]] with a plane size of 128 (16x16) or 512 (32x32). These chunks do NOT have a dword specifying their decompressed size as this is hard-coded into the executable as a space saving measure. (As is the start and finish of unmasked and masked tile chunks.) This can cause problems for editing programs.

Tiles make up the bulk of 2D game levels but are seldom seen elsewhere.

=== Misc graphics ===

Sometimes following tile graphics are miscellaneous graphics, that cannot be handled by the other chunk types. For example, Keen 4-6 has two misc graphics files used for the 'COOMANDER KEEN terminator text' intro. These graphics usually have unique formats that fit their function and few have been investigated.

==== Keen 4-6 Intro Bitmaps ====

This is the text that displays after the 'Ready, press any key' screen at Keen startup. It is composed of a monochrome bitmap that is scrolled across the screen and distorted for special effects. When displayed it is transparent, with a special palette so it is not black-and-white. It consists of a header followed by RLE compressed monochrome data.

HEADER:
0 2 Img height Height of the image in pixels
2 2 Img width Width of the image in pixels
4 2x Line point Pointers to line 1,2,3..etc of data. There will be [Img height] of
these, each 2 bytes long. The first pointer will have the value
(2 * [Img height] + 4)
+2 ? RLE data RLE-WM compressed data

DATA:
0 2 Black run Number of black pixels to write
2 2 Not-blk run Number of not-black pixels to write
4 2 Black run....
...
..
.
? 2 End $FFFF; end of row.

=== Game texts ===

Also known as ANSII chunks, these are used by the game for things like help screens, and have their own format. In the main they are simply text documents, interspersed with various commands to make things happen. Each file is divided up into a number of 'pages' which are moved between by pressing the up/down arrows. Pages can also have action sequences (Repeated each time the page is moved to, you cannot leave a page until the sequence finishes.)

^P First command in every file. Defines a page start
^E Ends the file
^Cx Change font color to $x until next page or ^C command
^Gx,y,z Display (unmasked) picture chunk z at location x,y (In pixels)
^Tx,y,z,t Display picture chunk z at x,y for z clicks of time
^Bx,y,z,t,b ?
^Lx,y ?

=== Misc files ===

The last few chunks are demo files and occasionally miscellaneous data. Demo file formats are currently unknown but can be recorded using the demo cheat. They contain the level number and difficulty followed by a series of key commands.

= Utilities =
* [[KeenWiki:ModKeen|ModKeen]] (DOS) and [[KeenWiki:LModKeen|LModKeen]] (Linux/Windows) can be used to modify graphics and text stored in EGAGRAPH.CKx for the Commander Keen [[KeenWiki:Goodbye Galaxy|Goodbye Galaxy]] trilogy.
* WDC (Windows) can be used to view or modify the graphics and text stored in EGAGRAPH for [[Catacomb 3D]] and [[Catacomb Abyss]].

[[Category:Catacomb 3D]]
[[Category:Catacomb Abyss]]
[[Category:Commander Keen 4-6]]
[[Category:Commander Keen Dreams]]
[[Category:Dangerous Dave 3]]
[[Category:File Formats]]
[[Category:Graphics Files]]

EGAGraph Format

2011-09-02T02:20:24Z

CarlKenner: /* Utilities */

{{Stub}}

'''EGAGraph''', short for 'EGA graphics library' is the format used to store graphics, demos, fonts, game texts and more in many early [[:Category:Id Software|id Software]] games.

The basic format is that of a number of [[Huffman Compression]] compressed sub-files (Chunks) stored in a similar manner to the [[AudioT Format]]. Like this format it has three main files, <tt>EGAGRAPH</tt>, <tt>EGAHEAD</tt>and <tt>EGADICT</tt>.

It is notable that while the <tt>EGAHEAD</tt> and <tt>EGADICT</tt> are stored in the game executable, they need to be extracted for [[TED5]] to edit levels.

This article may contain errors as some formats have not been worked out completely.

= File formats =

== EGA Head (EGAHEAD.xxx) ==
There are two versions of this format. The first version, used in [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], is an array of 4-byte little-endian values. The second version, used in all later EGA games, is an array of 3-byte unsigned little-endian values.

This header file may be present as an external file (eg. [[Bio Menace]]), or it may be included in the EXE file (eg. [[Catacomb 3D]]). The last entry in the header is always the size of the EGAGRAPH.xxx file, and the first entry in the header is always 00 00 00. To find the end, you can search for filesize(EGAGRAPH.xxx) as a 3-byte (little-endian) value in the EXE. Then to find the start, go back 3 bytes at a time until you get to 00 000 00. Or use the fact that entries always get smaller as you go back, unless the entry is $FFFFFF.

In [[Catacomb 3D]] v1.00 <tt>EGAHEAD.C3D</tt> is at $1BFD0 from the start of the decompressed ([[unlzexe]]) EXE file, and is 1437 bytes long, or 479 3-byte entries (for 478 chunks). It is then followed by 3 zero bytes (padding to multiple of 16?) 00 00 00, then the <tt>MAPHEAD.C3D</tt>.

In [[Catacomb Abyss]] v1.13 <tt>EGAHEAD.ABS</tt> is at $1BDB0 from the start of the decompressed ([[unlzexe]]) EXE file, and 1881 bytes long and has 626 chunks.

This header file stores the offsets (relative to the start of the <tt>EGAGRAPH</tt> file) for each sub file. The format is trivial, in that the header file is simply an array of 3 or 4 byte variables. (Each variable is a "slot", as the game refers to files by index.) Most games use the more compact 3-byte variables, since the <tt>EGAGRAPH</tt> file is never larger than $FFFFFF. However games based on older engines, notably [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], use 4-byte vales as these are easier to deal with.

It is important to note that not every 'slot' will be in use. Tiles especially if blank (All black) will be considered 'empty' and not worth adding to the file. (A blank 16x16 tile takes up 12 bytes of space if compressed, and there can be hundreds.) There are two ways of dealing with this; by default the games set all empty slot headers to -1, but many programs compress the graphics anyway and use a normal header value for them.

The last offset in the file will be an offset to the end of the main <tt>EGAGRAPH</tt> file. (So this should be ignored when reading the file to prevent a zero-byte file appearing last.) The total size of the <tt>EGAHEAD</tt> divided by 3 (Or 4) will also give the number of 'slots' the game uses. This varies, but is usually about 10,000.

Unlike [[AudioT Format]] files, <tt>EGAGRAPH</tt> is always compressed. However the header and dictionary may be external (E.g [[Bio Menace]])

== Dictionary (EGADICT.xxx) ==

The same [[Huffman Compression]] scheme used elsewhere in the [[Commander Keen]] series is used. When reading compressed chunks out of the <tt>EGAGRAPH</tt> file, the first four bytes located at the <tt>EGAHEAD</tt> offset are a UINT32LE specifying the file's decompressed size (which is required for the decompression algorithm.) The compressed data then follows, EXCEPT for 16x16 masked and unmasked tiles. Since these all have a similar decompressed size (128 bytes for unmasked, 256 bytes for masked) it is a waste of space to include these (Since tiles make up the bulk of graphics slots.) The decompressed size is hard coded, as are the start and finish of the unmasked and masked tile slots.

Note that the <tt>EGADICT</tt> file is not often obvious, usually being embedded in the main exe file. It is possible to locate it by looking for the string <tt>$FD $01 $00 $00 $00 $00</tt>, which appears at the end of nearly all Huffman dictionaries. They are the last 6 bytes out of 1024 (256*2*2), so add 6 and subtract 1024 to get the start (pointer to the head node).

Executables usually contain two or more dictionaries, but the <tt>EGADICT</tt> is usually '''the second one''', except in the case of early games like Keen Dreams, where it is the first of THREE. (A simple check of whether it decompresses the data sensibly works.)

In [[Catacomb 3D]] <tt>EGADICT.C3D</tt> is at offset $24464 from the start of the decompressed CAT3D.EXE, which is the second Huffman dictionary and comes immediately after the first Huffman Dictionary.

In [[Catacomb Abyss]] v1.13 <tt>EGADICT.ABS</tt> is at offset $2734C from the start of the decompressed CATABYSS.EXE, which is the second Huffman dictionary.

== Main file (EGAGRAPH.xxx) ==

This file is simply an array of data files. Each file starts at the offset specified in the <tt>EGAHEAD</tt> file, and as there are no filenames each file is referred to by its index/slot number. Slots containing dummy values in the header are treated as if they don't exist in the <tt>EGAGRAPH</tt> file. Each file within the <tt>EGAGRAPH</tt> file is individually compressed. A file can be read by opening the <tt>EGAGRAPH</tt> file, seeking to the offset specified in the <tt>EGAHEAD</tt> file, reading a UINT32LE for the decompressed file size, and then decompressing the data from that point onwards using standard Huffman decompression techniques.

Note that in early games such as Keen Dreams, the <tt>EGAGRAPH</tt> file may have another name. (Such as <tt>KDREAM.EGA</tt> in that case.) in which case it is always the largest non-executable file.

=== Chunks ===

Each sub file or chunk is a separate graphic, ordered in the <tt>EGAGRAPH</tt> file in a specific way, much again like the [[AudioT Format]]. Chunks are labeled starting at 0. The game executable is hard coded for the start and finish of various chunk types. For example, Keen 4 has chunks 124-520 as sprites and will treat any chunk between these two numbers as a sprite. (Note that the game refers to chunk number, not sprite number.) The usual order of chunks is:

Picture table
Masked picture table
Sprite table
Fonts
Pictures (Unmasked bitmaps)
Masked pictures
8x8 unmasked tiles (Single chunk)
8x8 masked tiles (Single chunk)
16x16 unmasked tiles
16x16 masked tiles
32x32 unmasked tiles (Optional)
32x32 masked tiles (Optional)
Misc graphics (Optional)
Game texts
Demo files (Optional)
Misc data (Optional)

== Chunk formats ==

When compressed each chunk consists of a dword giving the decompressed size followed by the compressed data.

=== Picture table ===

Always present and always the first chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying picture chunks.

Divide the decompressed size by 4 to get numpics.

=== Masked picture table ===

Always present and always the second chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying masked picture chunks. This table is often quite small.

=== Sprite table ===

Always present and always the third chunk, the picture table is 16 * numspr bytes long, consisting of eight words for each sprite image (Width\8, Height, x offset, y offset and clipping rectangle left, top, right and bottom; all in that order.) used when displaying sprite chunks. This is what Modkeen exports as its notoriously hard to use xSPRITES.txt file.

=== Fonts ===

Always present and always starting at the fourth chunk. (Chunk 3!) Each chunk is a single monochrome font, containing all the entries from 0-255 for that font. Each 'letter' is stored separately and consecutively. A game will have at least one, and usually 3 fonts. The format is version two of [[EGA Font format]]

Fonts are used in the game for game texts. They can be stretched or colored and are always transparent, meaning they look strange when not over a single-color background.

=== Pictures (Bitmaps) ===
These follow game fonts. You need to know how many fonts there are before it to know how what chunk number the first picture is (3+NumFonts). In [[Catacomb 3D]] there are 2 fonts.

Bitmaps are stored as standard [[Raw EGA data]], the size of each plane being the decompressed chunk size \ 4. In order to be displayed properly these need information from chunk 0, the picture table. Note that unlike fonts, data does not wrap, so a 4x4 sprite will consist of four planes of four bytes in size, the same as an 8x4 sprite.

Pictures are used for things such as title screens or pictures in game texts. Most of the main menu will consist of pictures. They are also used for the wall textures in EGA FPS games like [[Catacomb 3D]], in which case the wall textures come near the end of the picture list (starting from chunk 137 in [[Catacomb 3D]]).

The number of picture chunks is given in chunk 0, the picture table.

=== Masked pictures (Masked bitmaps) ===

These follow Pictures. You need to know how many fonts there are, to know how what chunk number the first Masked picture is (3+NumFonts+numpics).

These are identical to the picture chunks in every respect, except being masked, they consist of five EGA planes, not four. Some games (E.g [[Catacomb 3D]] make great use of masked pictures, but most games have only one or two. They need information from chunk 1, the masked picture table, to be displayed correctly.

The number of Masked pictures is given in Chunk 1.

=== Sprites ===

The first Sprite chunk is at 3+numfonts+numpics+numMaskedPics.

Sprites are exactly identical to masked pictures in format except the game cannot stretch or warp them. Most games use these for enemy, player or item graphics during gameplay. They need information from chunk 2, the sprite table both to display correctly and to interact correctly, something more complex than masked pictures.

There are usually several hundred sprite chunks, but [[Catacomb 3D]] only has 3 sprite chunks and they are used for the PaddleWar game (Skulls and Bones). The number of sprite chunks is given in chunk 2.

=== 8x8 tiles ===

8x8 tiles are used by most games in status windows or foe the borders of message windows in-game. (And occasionally by TED5 to display levels.) There are thus not many of them. ALL the masked or unmasked tiles are one single chunk stored as four and five plane [[Raw EGA data]] respectively. The number of 8x8 unmasked and masked tiles is hard-coded into the executable, but can be worked out by dividing the decompressed chunk size by 32 and 40 (The size of one tile's data) respectively.

=== 16x16 and 32x32 tiles ===

32x32 tiles are optional and rarely used. 16x16 tiles are only occasionally absent and for most games make up the bulk of graphics. There are usually several thousand entries. Both come in masked and unmasked kinds, just like 8x8 tiles.

These chunks have notable differences. Each tile is an individual chunk and stored as four or five plane [[Raw EGA data]] with a plane size of 128 (16x16) or 512 (32x32). These chunks do NOT have a dword specifying their decompressed size as this is hard-coded into the executable as a space saving measure. (As is the start and finish of unmasked and masked tile chunks.) This can cause problems for editing programs.

Tiles make up the bulk of 2D game levels but are seldom seen elsewhere.

=== Misc graphics ===

Sometimes following tile graphics are miscellaneous graphics, that cannot be handled by the other chunk types. For example, Keen 4-6 has two misc graphics files used for the 'COOMANDER KEEN terminator text' intro. These graphics usually have unique formats that fit their function and few have been investigated.

==== Keen 4-6 Intro Bitmaps ====

This is the text that displays after the 'Ready, press any key' screen at Keen startup. It is composed of a monochrome bitmap that is scrolled across the screen and distorted for special effects. When displayed it is transparent, with a special palette so it is not black-and-white. It consists of a header followed by RLE compressed monochrome data.

HEADER:
0 2 Img height Height of the image in pixels
2 2 Img width Width of the image in pixels
4 2x Line point Pointers to line 1,2,3..etc of data. There will be [Img height] of
these, each 2 bytes long. The first pointer will have the value
(2 * [Img height] + 4)
+2 ? RLE data RLE-WM compressed data

DATA:
0 2 Black run Number of black pixels to write
2 2 Not-blk run Number of not-black pixels to write
4 2 Black run....
...
..
.
? 2 End $FFFF; end of row.

=== Game texts ===

Also known as ANSII chunks, these are used by the game for things like help screens, and have their own format. In the main they are simply text documents, interspersed with various commands to make things happen. Each file is divided up into a number of 'pages' which are moved between by pressing the up/down arrows. Pages can also have action sequences (Repeated each time the page is moved to, you cannot leave a page until the sequence finishes.)

^P First command in every file. Defines a page start
^E Ends the file
^Cx Change font color to $x until next page or ^C command
^Gx,y,z Display (unmasked) picture chunk z at location x,y (In pixels)
^Tx,y,z,t Display picture chunk z at x,y for z clicks of time
^Bx,y,z,t,b ?
^Lx,y ?

=== Misc files ===

The last few chunks are demo files and occasionally miscellaneous data. Demo file formats are currently unknown but can be recorded using the demo cheat. They contain the level number and difficulty followed by a series of key commands.

= Utilities =
* [[KeenWiki:ModKeen|ModKeen]] (DOS) and [[KeenWiki:LModKeen|LModKeen]] (Linux/Windows) can be used to modify graphics and text stored in EGAGRAPH.CKx for the Commander Keen [[KeenWiki:Goodbye Galaxy|Goodbye Galaxy]] trilogy.
* WDC (Windows) can be used to view or modify the graphics and text stored in EGAGRAPH for [[Catacomb 3D]] and [[Catacomb Abyss]].

[[Category:Catacomb 3D]]
[[Category:Catacomb Abyss]]
[[Category:Commander Keen 4-6]]
[[Category:Commander Keen Dreams]]
[[Category:Dangerous Dave 3]]
[[Category:File Formats]]
[[Category:Graphics Files]]

EGAGraph Format

2011-09-02T02:05:36Z

CarlKenner: /* Dictionary (EGADICT.xxx) */ Catacomb Abyss

{{Stub}}

'''EGAGraph''', short for 'EGA graphics library' is the format used to store graphics, demos, fonts, game texts and more in many early [[:Category:Id Software|id Software]] games.

The basic format is that of a number of [[Huffman Compression]] compressed sub-files (Chunks) stored in a similar manner to the [[AudioT Format]]. Like this format it has three main files, <tt>EGAGRAPH</tt>, <tt>EGAHEAD</tt>and <tt>EGADICT</tt>.

It is notable that while the <tt>EGAHEAD</tt> and <tt>EGADICT</tt> are stored in the game executable, they need to be extracted for [[TED5]] to edit levels.

This article may contain errors as some formats have not been worked out completely.

= File formats =

== EGA Head (EGAHEAD.xxx) ==
There are two versions of this format. The first version, used in [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], is an array of 4-byte little-endian values. The second version, used in all later EGA games, is an array of 3-byte unsigned little-endian values.

This header file may be present as an external file (eg. [[Bio Menace]]), or it may be included in the EXE file (eg. [[Catacomb 3D]]). The last entry in the header is always the size of the EGAGRAPH.xxx file, and the first entry in the header is always 00 00 00. To find the end, you can search for filesize(EGAGRAPH.xxx) as a 3-byte (little-endian) value in the EXE. Then to find the start, go back 3 bytes at a time until you get to 00 000 00. Or use the fact that entries always get smaller as you go back, unless the entry is $FFFFFF.

In [[Catacomb 3D]] v1.00 <tt>EGAHEAD.C3D</tt> is at $1BFD0 from the start of the decompressed ([[unlzexe]]) EXE file, and is 1437 bytes long, or 479 3-byte entries (for 478 chunks). It is then followed by 3 zero bytes (padding to multiple of 16?) 00 00 00, then the <tt>MAPHEAD.C3D</tt>.

In [[Catacomb Abyss]] v1.13 <tt>EGAHEAD.ABS</tt> is at $1BDB0 from the start of the decompressed ([[unlzexe]]) EXE file, and 1881 bytes long and has 626 chunks.

This header file stores the offsets (relative to the start of the <tt>EGAGRAPH</tt> file) for each sub file. The format is trivial, in that the header file is simply an array of 3 or 4 byte variables. (Each variable is a "slot", as the game refers to files by index.) Most games use the more compact 3-byte variables, since the <tt>EGAGRAPH</tt> file is never larger than $FFFFFF. However games based on older engines, notably [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], use 4-byte vales as these are easier to deal with.

It is important to note that not every 'slot' will be in use. Tiles especially if blank (All black) will be considered 'empty' and not worth adding to the file. (A blank 16x16 tile takes up 12 bytes of space if compressed, and there can be hundreds.) There are two ways of dealing with this; by default the games set all empty slot headers to -1, but many programs compress the graphics anyway and use a normal header value for them.

The last offset in the file will be an offset to the end of the main <tt>EGAGRAPH</tt> file. (So this should be ignored when reading the file to prevent a zero-byte file appearing last.) The total size of the <tt>EGAHEAD</tt> divided by 3 (Or 4) will also give the number of 'slots' the game uses. This varies, but is usually about 10,000.

Unlike [[AudioT Format]] files, <tt>EGAGRAPH</tt> is always compressed. However the header and dictionary may be external (E.g [[Bio Menace]])

== Dictionary (EGADICT.xxx) ==

The same [[Huffman Compression]] scheme used elsewhere in the [[Commander Keen]] series is used. When reading compressed chunks out of the <tt>EGAGRAPH</tt> file, the first four bytes located at the <tt>EGAHEAD</tt> offset are a UINT32LE specifying the file's decompressed size (which is required for the decompression algorithm.) The compressed data then follows, EXCEPT for 16x16 masked and unmasked tiles. Since these all have a similar decompressed size (128 bytes for unmasked, 256 bytes for masked) it is a waste of space to include these (Since tiles make up the bulk of graphics slots.) The decompressed size is hard coded, as are the start and finish of the unmasked and masked tile slots.

Note that the <tt>EGADICT</tt> file is not often obvious, usually being embedded in the main exe file. It is possible to locate it by looking for the string <tt>$FD $01 $00 $00 $00 $00</tt>, which appears at the end of nearly all Huffman dictionaries. They are the last 6 bytes out of 1024 (256*2*2), so add 6 and subtract 1024 to get the start (pointer to the head node).

Executables usually contain two or more dictionaries, but the <tt>EGADICT</tt> is usually '''the second one''', except in the case of early games like Keen Dreams, where it is the first of THREE. (A simple check of whether it decompresses the data sensibly works.)

In [[Catacomb 3D]] <tt>EGADICT.C3D</tt> is at offset $24464 from the start of the decompressed CAT3D.EXE, which is the second Huffman dictionary and comes immediately after the first Huffman Dictionary.

In [[Catacomb Abyss]] v1.13 <tt>EGADICT.ABS</tt> is at offset $2734C from the start of the decompressed CATABYSS.EXE, which is the second Huffman dictionary.

== Main file (EGAGRAPH.xxx) ==

This file is simply an array of data files. Each file starts at the offset specified in the <tt>EGAHEAD</tt> file, and as there are no filenames each file is referred to by its index/slot number. Slots containing dummy values in the header are treated as if they don't exist in the <tt>EGAGRAPH</tt> file. Each file within the <tt>EGAGRAPH</tt> file is individually compressed. A file can be read by opening the <tt>EGAGRAPH</tt> file, seeking to the offset specified in the <tt>EGAHEAD</tt> file, reading a UINT32LE for the decompressed file size, and then decompressing the data from that point onwards using standard Huffman decompression techniques.

Note that in early games such as Keen Dreams, the <tt>EGAGRAPH</tt> file may have another name. (Such as <tt>KDREAM.EGA</tt> in that case.) in which case it is always the largest non-executable file.

=== Chunks ===

Each sub file or chunk is a separate graphic, ordered in the <tt>EGAGRAPH</tt> file in a specific way, much again like the [[AudioT Format]]. Chunks are labeled starting at 0. The game executable is hard coded for the start and finish of various chunk types. For example, Keen 4 has chunks 124-520 as sprites and will treat any chunk between these two numbers as a sprite. (Note that the game refers to chunk number, not sprite number.) The usual order of chunks is:

Picture table
Masked picture table
Sprite table
Fonts
Pictures (Unmasked bitmaps)
Masked pictures
8x8 unmasked tiles (Single chunk)
8x8 masked tiles (Single chunk)
16x16 unmasked tiles
16x16 masked tiles
32x32 unmasked tiles (Optional)
32x32 masked tiles (Optional)
Misc graphics (Optional)
Game texts
Demo files (Optional)
Misc data (Optional)

== Chunk formats ==

When compressed each chunk consists of a dword giving the decompressed size followed by the compressed data.

=== Picture table ===

Always present and always the first chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying picture chunks.

Divide the decompressed size by 4 to get numpics.

=== Masked picture table ===

Always present and always the second chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying masked picture chunks. This table is often quite small.

=== Sprite table ===

Always present and always the third chunk, the picture table is 16 * numspr bytes long, consisting of eight words for each sprite image (Width\8, Height, x offset, y offset and clipping rectangle left, top, right and bottom; all in that order.) used when displaying sprite chunks. This is what Modkeen exports as its notoriously hard to use xSPRITES.txt file.

=== Fonts ===

Always present and always starting at the fourth chunk. (Chunk 3!) Each chunk is a single monochrome font, containing all the entries from 0-255 for that font. Each 'letter' is stored separately and consecutively. A game will have at least one, and usually 3 fonts. The format is version two of [[EGA Font format]]

Fonts are used in the game for game texts. They can be stretched or colored and are always transparent, meaning they look strange when not over a single-color background.

=== Pictures (Bitmaps) ===
These follow game fonts. You need to know how many fonts there are before it to know how what chunk number the first picture is (3+NumFonts). In [[Catacomb 3D]] there are 2 fonts.

Bitmaps are stored as standard [[Raw EGA data]], the size of each plane being the decompressed chunk size \ 4. In order to be displayed properly these need information from chunk 0, the picture table. Note that unlike fonts, data does not wrap, so a 4x4 sprite will consist of four planes of four bytes in size, the same as an 8x4 sprite.

Pictures are used for things such as title screens or pictures in game texts. Most of the main menu will consist of pictures. They are also used for the wall textures in EGA FPS games like [[Catacomb 3D]], in which case the wall textures come near the end of the picture list (starting from chunk 137 in [[Catacomb 3D]]).

The number of picture chunks is given in chunk 0, the picture table.

=== Masked pictures (Masked bitmaps) ===

These follow Pictures. You need to know how many fonts there are, to know how what chunk number the first Masked picture is (3+NumFonts+numpics).

These are identical to the picture chunks in every respect, except being masked, they consist of five EGA planes, not four. Some games (E.g [[Catacomb 3D]] make great use of masked pictures, but most games have only one or two. They need information from chunk 1, the masked picture table, to be displayed correctly.

The number of Masked pictures is given in Chunk 1.

=== Sprites ===

The first Sprite chunk is at 3+numfonts+numpics+numMaskedPics.

Sprites are exactly identical to masked pictures in format except the game cannot stretch or warp them. Most games use these for enemy, player or item graphics during gameplay. They need information from chunk 2, the sprite table both to display correctly and to interact correctly, something more complex than masked pictures.

There are usually several hundred sprite chunks, but [[Catacomb 3D]] only has 3 sprite chunks and they are used for the PaddleWar game (Skulls and Bones). The number of sprite chunks is given in chunk 2.

=== 8x8 tiles ===

8x8 tiles are used by most games in status windows or foe the borders of message windows in-game. (And occasionally by TED5 to display levels.) There are thus not many of them. ALL the masked or unmasked tiles are one single chunk stored as four and five plane [[Raw EGA data]] respectively. The number of 8x8 unmasked and masked tiles is hard-coded into the executable, but can be worked out by dividing the decompressed chunk size by 32 and 40 (The size of one tile's data) respectively.

=== 16x16 and 32x32 tiles ===

32x32 tiles are optional and rarely used. 16x16 tiles are only occasionally absent and for most games make up the bulk of graphics. There are usually several thousand entries. Both come in masked and unmasked kinds, just like 8x8 tiles.

These chunks have notable differences. Each tile is an individual chunk and stored as four or five plane [[Raw EGA data]] with a plane size of 128 (16x16) or 512 (32x32). These chunks do NOT have a dword specifying their decompressed size as this is hard-coded into the executable as a space saving measure. (As is the start and finish of unmasked and masked tile chunks.) This can cause problems for editing programs.

Tiles make up the bulk of 2D game levels but are seldom seen elsewhere.

=== Misc graphics ===

Sometimes following tile graphics are miscellaneous graphics, that cannot be handled by the other chunk types. For example, Keen 4-6 has two misc graphics files used for the 'COOMANDER KEEN terminator text' intro. These graphics usually have unique formats that fit their function and few have been investigated.

==== Keen 4-6 Intro Bitmaps ====

This is the text that displays after the 'Ready, press any key' screen at Keen startup. It is composed of a monochrome bitmap that is scrolled across the screen and distorted for special effects. When displayed it is transparent, with a special palette so it is not black-and-white. It consists of a header followed by RLE compressed monochrome data.

HEADER:
0 2 Img height Height of the image in pixels
2 2 Img width Width of the image in pixels
4 2x Line point Pointers to line 1,2,3..etc of data. There will be [Img height] of
these, each 2 bytes long. The first pointer will have the value
(2 * [Img height] + 4)
+2 ? RLE data RLE-WM compressed data

DATA:
0 2 Black run Number of black pixels to write
2 2 Not-blk run Number of not-black pixels to write
4 2 Black run....
...
..
.
? 2 End $FFFF; end of row.

=== Game texts ===

Also known as ANSII chunks, these are used by the game for things like help screens, and have their own format. In the main they are simply text documents, interspersed with various commands to make things happen. Each file is divided up into a number of 'pages' which are moved between by pressing the up/down arrows. Pages can also have action sequences (Repeated each time the page is moved to, you cannot leave a page until the sequence finishes.)

^P First command in every file. Defines a page start
^E Ends the file
^Cx Change font color to $x until next page or ^C command
^Gx,y,z Display (unmasked) picture chunk z at location x,y (In pixels)
^Tx,y,z,t Display picture chunk z at x,y for z clicks of time
^Bx,y,z,t,b ?
^Lx,y ?

=== Misc files ===

The last few chunks are demo files and occasionally miscellaneous data. Demo file formats are currently unknown but can be recorded using the demo cheat. They contain the level number and difficulty followed by a series of key commands.

= Utilities =
* [[KeenWiki:ModKeen|ModKeen]] (DOS) and [[KeenWiki:LModKeen|LModKeen]] (Linux/Windows) can be used to modify graphics and text stored in EGAGRAPH.CKx for the Commander Keen [[KeenWiki:Goodbye Galaxy|Goodbye Galaxy]] trilogy.
* WDC (Windows) can be used to view or modify the graphics and text stored in EGAGRAPH.C3D for [[Catacomb 3D]].

[[Category:Catacomb 3D]]
[[Category:Commander Keen 4-6]]
[[Category:Commander Keen Dreams]]
[[Category:Dangerous Dave 3]]
[[Category:File Formats]]
[[Category:Graphics Files]]

EGAGraph Format

2011-09-02T02:00:55Z

CarlKenner: /* EGA Head (EGAHEAD.xxx) */ Catacomb Abyss

{{Stub}}

'''EGAGraph''', short for 'EGA graphics library' is the format used to store graphics, demos, fonts, game texts and more in many early [[:Category:Id Software|id Software]] games.

The basic format is that of a number of [[Huffman Compression]] compressed sub-files (Chunks) stored in a similar manner to the [[AudioT Format]]. Like this format it has three main files, <tt>EGAGRAPH</tt>, <tt>EGAHEAD</tt>and <tt>EGADICT</tt>.

It is notable that while the <tt>EGAHEAD</tt> and <tt>EGADICT</tt> are stored in the game executable, they need to be extracted for [[TED5]] to edit levels.

This article may contain errors as some formats have not been worked out completely.

= File formats =

== EGA Head (EGAHEAD.xxx) ==
There are two versions of this format. The first version, used in [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], is an array of 4-byte little-endian values. The second version, used in all later EGA games, is an array of 3-byte unsigned little-endian values.

This header file may be present as an external file (eg. [[Bio Menace]]), or it may be included in the EXE file (eg. [[Catacomb 3D]]). The last entry in the header is always the size of the EGAGRAPH.xxx file, and the first entry in the header is always 00 00 00. To find the end, you can search for filesize(EGAGRAPH.xxx) as a 3-byte (little-endian) value in the EXE. Then to find the start, go back 3 bytes at a time until you get to 00 000 00. Or use the fact that entries always get smaller as you go back, unless the entry is $FFFFFF.

In [[Catacomb 3D]] v1.00 <tt>EGAHEAD.C3D</tt> is at $1BFD0 from the start of the decompressed ([[unlzexe]]) EXE file, and is 1437 bytes long, or 479 3-byte entries (for 478 chunks). It is then followed by 3 zero bytes (padding to multiple of 16?) 00 00 00, then the <tt>MAPHEAD.C3D</tt>.

In [[Catacomb Abyss]] v1.13 <tt>EGAHEAD.ABS</tt> is at $1BDB0 from the start of the decompressed ([[unlzexe]]) EXE file, and 1881 bytes long and has 626 chunks.

This header file stores the offsets (relative to the start of the <tt>EGAGRAPH</tt> file) for each sub file. The format is trivial, in that the header file is simply an array of 3 or 4 byte variables. (Each variable is a "slot", as the game refers to files by index.) Most games use the more compact 3-byte variables, since the <tt>EGAGRAPH</tt> file is never larger than $FFFFFF. However games based on older engines, notably [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], use 4-byte vales as these are easier to deal with.

It is important to note that not every 'slot' will be in use. Tiles especially if blank (All black) will be considered 'empty' and not worth adding to the file. (A blank 16x16 tile takes up 12 bytes of space if compressed, and there can be hundreds.) There are two ways of dealing with this; by default the games set all empty slot headers to -1, but many programs compress the graphics anyway and use a normal header value for them.

The last offset in the file will be an offset to the end of the main <tt>EGAGRAPH</tt> file. (So this should be ignored when reading the file to prevent a zero-byte file appearing last.) The total size of the <tt>EGAHEAD</tt> divided by 3 (Or 4) will also give the number of 'slots' the game uses. This varies, but is usually about 10,000.

Unlike [[AudioT Format]] files, <tt>EGAGRAPH</tt> is always compressed. However the header and dictionary may be external (E.g [[Bio Menace]])

== Dictionary (EGADICT.xxx) ==

The same [[Huffman Compression]] scheme used elsewhere in the [[Commander Keen]] series is used. When reading compressed chunks out of the <tt>EGAGRAPH</tt> file, the first four bytes located at the <tt>EGAHEAD</tt> offset are a UINT32LE specifying the file's decompressed size (which is required for the decompression algorithm.) The compressed data then follows, EXCEPT for 16x16 masked and unmasked tiles. Since these all have a similar decompressed size (128 bytes for unmasked, 256 bytes for masked) it is a waste of space to include these (Since tiles make up the bulk of graphics slots.) The decompressed size is hard coded, as are the start and finish of the unmasked and masked tile slots.

Note that the <tt>EGADICT</tt> file is not often obvious, usually being embedded in the main exe file. It is possible to locate it by looking for the string <tt>$FD $01 $00 $00 $00 $00</tt>, which appears at the end of nearly all Huffman dictionaries. They are the last 6 bytes out of 1024 (256*2*2), so add 6 and subtract 1024 to get the start (pointer to the head node).

Executables usually contain two or more dictionaries, but the <tt>EGADICT</tt> is usually '''the second one''', except in the case of early games like Keen Dreams, where it is the first of THREE. (A simple check of whether it decompresses the data sensibly works.)

In [[Catacomb 3D]] <tt>EGADICT.C3D</tt> is at offset $24464 from the start of the decompressed CAT3D.EXE, which is the second Huffman dictionary and comes immediately after the first Huffman Dictionary.

== Main file (EGAGRAPH.xxx) ==

This file is simply an array of data files. Each file starts at the offset specified in the <tt>EGAHEAD</tt> file, and as there are no filenames each file is referred to by its index/slot number. Slots containing dummy values in the header are treated as if they don't exist in the <tt>EGAGRAPH</tt> file. Each file within the <tt>EGAGRAPH</tt> file is individually compressed. A file can be read by opening the <tt>EGAGRAPH</tt> file, seeking to the offset specified in the <tt>EGAHEAD</tt> file, reading a UINT32LE for the decompressed file size, and then decompressing the data from that point onwards using standard Huffman decompression techniques.

Note that in early games such as Keen Dreams, the <tt>EGAGRAPH</tt> file may have another name. (Such as <tt>KDREAM.EGA</tt> in that case.) in which case it is always the largest non-executable file.

=== Chunks ===

Each sub file or chunk is a separate graphic, ordered in the <tt>EGAGRAPH</tt> file in a specific way, much again like the [[AudioT Format]]. Chunks are labeled starting at 0. The game executable is hard coded for the start and finish of various chunk types. For example, Keen 4 has chunks 124-520 as sprites and will treat any chunk between these two numbers as a sprite. (Note that the game refers to chunk number, not sprite number.) The usual order of chunks is:

Picture table
Masked picture table
Sprite table
Fonts
Pictures (Unmasked bitmaps)
Masked pictures
8x8 unmasked tiles (Single chunk)
8x8 masked tiles (Single chunk)
16x16 unmasked tiles
16x16 masked tiles
32x32 unmasked tiles (Optional)
32x32 masked tiles (Optional)
Misc graphics (Optional)
Game texts
Demo files (Optional)
Misc data (Optional)

== Chunk formats ==

When compressed each chunk consists of a dword giving the decompressed size followed by the compressed data.

=== Picture table ===

Always present and always the first chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying picture chunks.

Divide the decompressed size by 4 to get numpics.

=== Masked picture table ===

Always present and always the second chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying masked picture chunks. This table is often quite small.

=== Sprite table ===

Always present and always the third chunk, the picture table is 16 * numspr bytes long, consisting of eight words for each sprite image (Width\8, Height, x offset, y offset and clipping rectangle left, top, right and bottom; all in that order.) used when displaying sprite chunks. This is what Modkeen exports as its notoriously hard to use xSPRITES.txt file.

=== Fonts ===

Always present and always starting at the fourth chunk. (Chunk 3!) Each chunk is a single monochrome font, containing all the entries from 0-255 for that font. Each 'letter' is stored separately and consecutively. A game will have at least one, and usually 3 fonts. The format is version two of [[EGA Font format]]

Fonts are used in the game for game texts. They can be stretched or colored and are always transparent, meaning they look strange when not over a single-color background.

=== Pictures (Bitmaps) ===
These follow game fonts. You need to know how many fonts there are before it to know how what chunk number the first picture is (3+NumFonts). In [[Catacomb 3D]] there are 2 fonts.

Bitmaps are stored as standard [[Raw EGA data]], the size of each plane being the decompressed chunk size \ 4. In order to be displayed properly these need information from chunk 0, the picture table. Note that unlike fonts, data does not wrap, so a 4x4 sprite will consist of four planes of four bytes in size, the same as an 8x4 sprite.

Pictures are used for things such as title screens or pictures in game texts. Most of the main menu will consist of pictures. They are also used for the wall textures in EGA FPS games like [[Catacomb 3D]], in which case the wall textures come near the end of the picture list (starting from chunk 137 in [[Catacomb 3D]]).

The number of picture chunks is given in chunk 0, the picture table.

=== Masked pictures (Masked bitmaps) ===

These follow Pictures. You need to know how many fonts there are, to know how what chunk number the first Masked picture is (3+NumFonts+numpics).

These are identical to the picture chunks in every respect, except being masked, they consist of five EGA planes, not four. Some games (E.g [[Catacomb 3D]] make great use of masked pictures, but most games have only one or two. They need information from chunk 1, the masked picture table, to be displayed correctly.

The number of Masked pictures is given in Chunk 1.

=== Sprites ===

The first Sprite chunk is at 3+numfonts+numpics+numMaskedPics.

Sprites are exactly identical to masked pictures in format except the game cannot stretch or warp them. Most games use these for enemy, player or item graphics during gameplay. They need information from chunk 2, the sprite table both to display correctly and to interact correctly, something more complex than masked pictures.

There are usually several hundred sprite chunks, but [[Catacomb 3D]] only has 3 sprite chunks and they are used for the PaddleWar game (Skulls and Bones). The number of sprite chunks is given in chunk 2.

=== 8x8 tiles ===

8x8 tiles are used by most games in status windows or foe the borders of message windows in-game. (And occasionally by TED5 to display levels.) There are thus not many of them. ALL the masked or unmasked tiles are one single chunk stored as four and five plane [[Raw EGA data]] respectively. The number of 8x8 unmasked and masked tiles is hard-coded into the executable, but can be worked out by dividing the decompressed chunk size by 32 and 40 (The size of one tile's data) respectively.

=== 16x16 and 32x32 tiles ===

32x32 tiles are optional and rarely used. 16x16 tiles are only occasionally absent and for most games make up the bulk of graphics. There are usually several thousand entries. Both come in masked and unmasked kinds, just like 8x8 tiles.

These chunks have notable differences. Each tile is an individual chunk and stored as four or five plane [[Raw EGA data]] with a plane size of 128 (16x16) or 512 (32x32). These chunks do NOT have a dword specifying their decompressed size as this is hard-coded into the executable as a space saving measure. (As is the start and finish of unmasked and masked tile chunks.) This can cause problems for editing programs.

Tiles make up the bulk of 2D game levels but are seldom seen elsewhere.

=== Misc graphics ===

Sometimes following tile graphics are miscellaneous graphics, that cannot be handled by the other chunk types. For example, Keen 4-6 has two misc graphics files used for the 'COOMANDER KEEN terminator text' intro. These graphics usually have unique formats that fit their function and few have been investigated.

==== Keen 4-6 Intro Bitmaps ====

This is the text that displays after the 'Ready, press any key' screen at Keen startup. It is composed of a monochrome bitmap that is scrolled across the screen and distorted for special effects. When displayed it is transparent, with a special palette so it is not black-and-white. It consists of a header followed by RLE compressed monochrome data.

HEADER:
0 2 Img height Height of the image in pixels
2 2 Img width Width of the image in pixels
4 2x Line point Pointers to line 1,2,3..etc of data. There will be [Img height] of
these, each 2 bytes long. The first pointer will have the value
(2 * [Img height] + 4)
+2 ? RLE data RLE-WM compressed data

DATA:
0 2 Black run Number of black pixels to write
2 2 Not-blk run Number of not-black pixels to write
4 2 Black run....
...
..
.
? 2 End $FFFF; end of row.

=== Game texts ===

Also known as ANSII chunks, these are used by the game for things like help screens, and have their own format. In the main they are simply text documents, interspersed with various commands to make things happen. Each file is divided up into a number of 'pages' which are moved between by pressing the up/down arrows. Pages can also have action sequences (Repeated each time the page is moved to, you cannot leave a page until the sequence finishes.)

^P First command in every file. Defines a page start
^E Ends the file
^Cx Change font color to $x until next page or ^C command
^Gx,y,z Display (unmasked) picture chunk z at location x,y (In pixels)
^Tx,y,z,t Display picture chunk z at x,y for z clicks of time
^Bx,y,z,t,b ?
^Lx,y ?

=== Misc files ===

The last few chunks are demo files and occasionally miscellaneous data. Demo file formats are currently unknown but can be recorded using the demo cheat. They contain the level number and difficulty followed by a series of key commands.

= Utilities =
* [[KeenWiki:ModKeen|ModKeen]] (DOS) and [[KeenWiki:LModKeen|LModKeen]] (Linux/Windows) can be used to modify graphics and text stored in EGAGRAPH.CKx for the Commander Keen [[KeenWiki:Goodbye Galaxy|Goodbye Galaxy]] trilogy.
* WDC (Windows) can be used to view or modify the graphics and text stored in EGAGRAPH.C3D for [[Catacomb 3D]].

[[Category:Catacomb 3D]]
[[Category:Commander Keen 4-6]]
[[Category:Commander Keen Dreams]]
[[Category:Dangerous Dave 3]]
[[Category:File Formats]]
[[Category:Graphics Files]]

GameMaps Format

2011-09-02T01:33:11Z

CarlKenner: /* Notes */

The GameMaps format is how the levels in a number of [[Id Software]] games is stored. The filenames and compression varies somewhat across different games but all files stored in this format were produced by the program [[TED5]] or a close analogue.

There are three main varieties of the GameMaps format, uncompressed, [[Carmack compression||carmackized]], and [[Huffman Compression|huffman compressed]]. Each variation has its own file naming scheme and pattern of external\internal files.

There are two main components to the format, the game maps proper, which contain the game level data and the map headers, which contain both the locations of the start of level data in the game maps file and the tile info for the game.

== Structure of uncompressed data ==

The uncompressed format is used by several games including [[Bio Menace]] and [[Wolfenstein 3D]]. In this format map data is stored in <tt>MAPTEMP.xxx<tt> and map headers in <tt>MAPTHEAD.xxx</tt> this is the format saved by [[TED5]] when maps are being edited and can be directly accessed and edited by this utility, allowing changes to be made to the game.

Note that filenames differ and that Wolfenstein 3D uses <tt>GAMEMAPS.WLx</tt> and <tt>MAPHEAD.xxx</tt> files to store data. The main indicator of this format being used is that in the unaltered game the map header file is external.

Only the latest Id Software games lacked compression, when file size was no longer an issue.

=== Map headers ===

0 2 Magic word Magic word for [[RLEW compression]]
2 400 Level ptr 100 4-byte pointers to start of level 0-99 data in game maps file
402 ??? Tileinfo Tileinfo data

The map header file is of varying length and contains three main types of data. The first is the magic word or flag used for RLEW compression, it is always $ABCD. The second is 100 level pointers which give the location of the start of level data in the game maps file, relative to the start of that file. A value of 0 indicates no level. The third is the tileinfo data, tile properties for each tile used in level creation. (These are masked and unmasked and either 8x8, 16x16 or 32x32)

Many programs treat the tileinfo as a separate file from the maphead and it is possible to modify a game in this manner. Indeed, some games, such as Wolfenstein 3D do not have any tileinfo data at all in the map header file (Giving a total file length of 402 bytes.) However TED5 works with any tileinfo data in the maphead.

=== Game maps ===

The game maps file consists of the string 'TED5v1.0.' and a number of RLEW compressed chunks of varying length. Each level in GameMaps will have from two to four chunks (usually four), depending on game, with all levels in a given game having the same number of chunks. These are the level header and 1-3 planes (Foreground, background and sprite or info.) The chunks are in no particular order and it is possible to read through the entire file decompressing chunks as they're found.

Chunks are ordered by the map header file, which will point to the level header chunks which in turn contain pointers to the other chunks used by that level. It is possible the string at the start of the file is to avoid the possibility of a level header at the start of the file which would be read as not existing.

All data in game maps file is in the form of words (Or in the case of pointers, dwords.)

=== Level headers ===

The header for each level that is pointed to by MAPHEAD is 42 bytes long and RLEW compressed, though this is difficult to see since the data is so short and nonrepetitive. For the offsets to level planes a value of 0 indicates the plane does not exist.

Plane 1 is background using unmasked tiles, plane 2 is foreground and uses masked tiles. Plane 3 is sprite or info. Levels must contain c background plane and usually an infoplane.

0 4 Offset in GAMEMAPS to beginning of compressed plane 0 data
4 4 Offset in GAMEMAPS to beginning of compressed plane 1 data
8 4 Offset in GAMEMAPS to beginning of compressed plane 2 data
12 2 Length of compressed plane 0 data (in bytes)
14 2 Length of compressed plane 1 data (in bytes)
16 2 Length of compressed plane 2 data (in bytes)
18 2 Width of level (in tiles)
20 2 Height of level (in tiles)
22 16 Internal name for level (used only by editor, not displayed in-game. null-terminated)

== Carmack compression ==

[[Carmack compression]] is the method used to compress later Id Software games, when file size was an issue. It is the most efficient and complex compression method and was created specifically to work with the word structure of the game maps file. The compression is detailed on its [[Carmack compression|own page.]]

Carmackized game maps files are external <tt>GAMEMAPS.xxx</tt> files and the map header is stored internally in the executable. The map header must be extracted and the game maps decompressed before TED5 can access them. TED5 itself can produce carmackized files and external <tt>MAPHEAD.xxx</tt> files. Carmackization does not replace the RLEW compression used in uncompressed data, but compresses this data, that is, the data is doubly compressed.

== Huffman compression ==

Huffman compression was used by earlier versions of TED5 (possibly not TED5 at all.) before carmackization was created. It uses the same method to compress its data as is used by Id Software games to compress their graphics and sounds. Again this compression method works with RLEW compressed data and has its [[Huffman Compression|own page.]]

Huffman compression is easily detected since it works on the bit level and thus disrupts the word structure of the game data. This is easily seen in a hex editor. Compressed data will not contain the string $00 $00 or indeed even $00 very often (In contrast even carmackized data contains both strings hundreds of times.)

There will be two internal files for this format, the map header and the huffman dictionary (which is always the first dictionary in the executable.) The map header format is also slightly different, being 502 bytes long, the extra 100 bytes being the length of the compressed level headers in the game maps data (These can be ignored when decompressing since huffman data can be read until the decompressed level header's fixed size is reached.)

== Location of internal files ==

Game maps are always external, but in the case of compression the map header file is stored internally in the executable. Executables are themselves compressed wither with LZEXE or PKLite. Once decompressed it is a trivial task to find the map header file as it will start with the word $ABCD For level editing purposes only the first 402 (or 502) bytes of the file needs to be extracted, though it is possible to read the maphead file to calculate its length.

The following addresses are in the locations in the decompressed executables for the start of the Map header files. Values are different for different versions.

TODO: Add all known versions of all games.

External files
Biomenace
Blake Stone 2: Planet Strike
Corridor 7 Demo
Noahs Ark 3D
Operation Body Count
Spear of Destiny
Wolfenstein 3D

Internal files:
Catacomb 3-D: $1C570 from start of file CAT3D.EXE (after file is UNLZEXE)
Catacomb 4 Abyss: $1C510 from start of file CATABYSS.EXE (after file is UNLZEXE)
Catacomb 5 Apocalypse
Catacomb 6 Armageddon
Corridor 7 CD (): $30D50 from start of file CORR7CD.EXE (file is not compressed)
Corridor 7 Floppy (): $24BF0 from start of file C7.EXE (file is not compressed)
Keen 4 (Special Demo): (file is PKLite compressed)
Keen 4 (1.4): $24830
Keen 5 (1.4): $25990
Keen 6 (1.4): $25080
(Keen D (1.13): $1FA50)

== Utilities ==

[[TED5]] This program can edit the <tt>GAMEMAPS</tt> format of any games that use it. It is the original editor used to crate these files.

== Notes ==

This format was reverse engineered by Andrew Durdin (adurdin).

The format is used by other games as well, including [[Bio Menace]], [[Blake Stone]], [[Catacomb 3D]], [[Catacomb Abyss]], [[Corridor 7]], [[Dangerous Dave 3]], [[Dave Goes Nutz]], [[Noah's Ark 3D]], [[Operation Body Count]], [[Spear of Destiny]], and [[Wolfenstein 3D]].

[[Category:File Formats]]
[[Category:Map Files]]

GameMaps Format

2011-09-02T01:32:06Z

CarlKenner: /* Location of internal files */ Catacomb 3D and Catacomb Abyss

The GameMaps format is how the levels in a number of [[Id Software]] games is stored. The filenames and compression varies somewhat across different games but all files stored in this format were produced by the program [[TED5]] or a close analogue.

There are three main varieties of the GameMaps format, uncompressed, [[Carmack compression||carmackized]], and [[Huffman Compression|huffman compressed]]. Each variation has its own file naming scheme and pattern of external\internal files.

There are two main components to the format, the game maps proper, which contain the game level data and the map headers, which contain both the locations of the start of level data in the game maps file and the tile info for the game.

== Structure of uncompressed data ==

The uncompressed format is used by several games including [[Bio Menace]] and [[Wolfenstein 3D]]. In this format map data is stored in <tt>MAPTEMP.xxx<tt> and map headers in <tt>MAPTHEAD.xxx</tt> this is the format saved by [[TED5]] when maps are being edited and can be directly accessed and edited by this utility, allowing changes to be made to the game.

Note that filenames differ and that Wolfenstein 3D uses <tt>GAMEMAPS.WLx</tt> and <tt>MAPHEAD.xxx</tt> files to store data. The main indicator of this format being used is that in the unaltered game the map header file is external.

Only the latest Id Software games lacked compression, when file size was no longer an issue.

=== Map headers ===

0 2 Magic word Magic word for [[RLEW compression]]
2 400 Level ptr 100 4-byte pointers to start of level 0-99 data in game maps file
402 ??? Tileinfo Tileinfo data

The map header file is of varying length and contains three main types of data. The first is the magic word or flag used for RLEW compression, it is always $ABCD. The second is 100 level pointers which give the location of the start of level data in the game maps file, relative to the start of that file. A value of 0 indicates no level. The third is the tileinfo data, tile properties for each tile used in level creation. (These are masked and unmasked and either 8x8, 16x16 or 32x32)

Many programs treat the tileinfo as a separate file from the maphead and it is possible to modify a game in this manner. Indeed, some games, such as Wolfenstein 3D do not have any tileinfo data at all in the map header file (Giving a total file length of 402 bytes.) However TED5 works with any tileinfo data in the maphead.

=== Game maps ===

The game maps file consists of the string 'TED5v1.0.' and a number of RLEW compressed chunks of varying length. Each level in GameMaps will have from two to four chunks (usually four), depending on game, with all levels in a given game having the same number of chunks. These are the level header and 1-3 planes (Foreground, background and sprite or info.) The chunks are in no particular order and it is possible to read through the entire file decompressing chunks as they're found.

Chunks are ordered by the map header file, which will point to the level header chunks which in turn contain pointers to the other chunks used by that level. It is possible the string at the start of the file is to avoid the possibility of a level header at the start of the file which would be read as not existing.

All data in game maps file is in the form of words (Or in the case of pointers, dwords.)

=== Level headers ===

The header for each level that is pointed to by MAPHEAD is 42 bytes long and RLEW compressed, though this is difficult to see since the data is so short and nonrepetitive. For the offsets to level planes a value of 0 indicates the plane does not exist.

Plane 1 is background using unmasked tiles, plane 2 is foreground and uses masked tiles. Plane 3 is sprite or info. Levels must contain c background plane and usually an infoplane.

0 4 Offset in GAMEMAPS to beginning of compressed plane 0 data
4 4 Offset in GAMEMAPS to beginning of compressed plane 1 data
8 4 Offset in GAMEMAPS to beginning of compressed plane 2 data
12 2 Length of compressed plane 0 data (in bytes)
14 2 Length of compressed plane 1 data (in bytes)
16 2 Length of compressed plane 2 data (in bytes)
18 2 Width of level (in tiles)
20 2 Height of level (in tiles)
22 16 Internal name for level (used only by editor, not displayed in-game. null-terminated)

== Carmack compression ==

[[Carmack compression]] is the method used to compress later Id Software games, when file size was an issue. It is the most efficient and complex compression method and was created specifically to work with the word structure of the game maps file. The compression is detailed on its [[Carmack compression|own page.]]

Carmackized game maps files are external <tt>GAMEMAPS.xxx</tt> files and the map header is stored internally in the executable. The map header must be extracted and the game maps decompressed before TED5 can access them. TED5 itself can produce carmackized files and external <tt>MAPHEAD.xxx</tt> files. Carmackization does not replace the RLEW compression used in uncompressed data, but compresses this data, that is, the data is doubly compressed.

== Huffman compression ==

Huffman compression was used by earlier versions of TED5 (possibly not TED5 at all.) before carmackization was created. It uses the same method to compress its data as is used by Id Software games to compress their graphics and sounds. Again this compression method works with RLEW compressed data and has its [[Huffman Compression|own page.]]

Huffman compression is easily detected since it works on the bit level and thus disrupts the word structure of the game data. This is easily seen in a hex editor. Compressed data will not contain the string $00 $00 or indeed even $00 very often (In contrast even carmackized data contains both strings hundreds of times.)

There will be two internal files for this format, the map header and the huffman dictionary (which is always the first dictionary in the executable.) The map header format is also slightly different, being 502 bytes long, the extra 100 bytes being the length of the compressed level headers in the game maps data (These can be ignored when decompressing since huffman data can be read until the decompressed level header's fixed size is reached.)

== Location of internal files ==

Game maps are always external, but in the case of compression the map header file is stored internally in the executable. Executables are themselves compressed wither with LZEXE or PKLite. Once decompressed it is a trivial task to find the map header file as it will start with the word $ABCD For level editing purposes only the first 402 (or 502) bytes of the file needs to be extracted, though it is possible to read the maphead file to calculate its length.

The following addresses are in the locations in the decompressed executables for the start of the Map header files. Values are different for different versions.

TODO: Add all known versions of all games.

External files
Biomenace
Blake Stone 2: Planet Strike
Corridor 7 Demo
Noahs Ark 3D
Operation Body Count
Spear of Destiny
Wolfenstein 3D

Internal files:
Catacomb 3-D: $1C570 from start of file CAT3D.EXE (after file is UNLZEXE)
Catacomb 4 Abyss: $1C510 from start of file CATABYSS.EXE (after file is UNLZEXE)
Catacomb 5 Apocalypse
Catacomb 6 Armageddon
Corridor 7 CD (): $30D50 from start of file CORR7CD.EXE (file is not compressed)
Corridor 7 Floppy (): $24BF0 from start of file C7.EXE (file is not compressed)
Keen 4 (Special Demo): (file is PKLite compressed)
Keen 4 (1.4): $24830
Keen 5 (1.4): $25990
Keen 6 (1.4): $25080
(Keen D (1.13): $1FA50)

== Utilities ==

[[TED5]] This program can edit the <tt>GAMEMAPS</tt> format of any games that use it. It is the original editor used to crate these files.

== Notes ==

This format was reverse engineered by Andrew Durdin (adurdin).

The format is used by other games as well, including [[Bio Menace]], [[Blake Stone]], [[Catacomb 3D]], [[Corridor 7]], [[Dangerous Dave 3]], [[Dave Goes Nutz]], [[Noah's Ark 3D]], [[Operation Body Count]], [[Spear of Destiny]], and [[Wolfenstein 3D]].

[[Category:File Formats]]
[[Category:Map Files]]

EGAGraph Format

2011-09-01T07:47:01Z

CarlKenner: /* Chunks */ 16x16 masked tiles was listed as unmasked.

{{Stub}}

'''EGAGraph''', short for 'EGA graphics library' is the format used to store graphics, demos, fonts, game texts and more in many early [[:Category:Id Software|id Software]] games.

The basic format is that of a number of [[Huffman Compression]] compressed sub-files (Chunks) stored in a similar manner to the [[AudioT Format]]. Like this format it has three main files, <tt>EGAGRAPH</tt>, <tt>EGAHEAD</tt>and <tt>EGADICT</tt>.

It is notable that while the <tt>EGAHEAD</tt> and <tt>EGADICT</tt> are stored in the game executable, they need to be extracted for [[TED5]] to edit levels.

This article may contain errors as some formats have not been worked out completely.

= File formats =

== EGA Head (EGAHEAD.xxx) ==
There are two versions of this format. The first version, used in [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], is an array of 4-byte little-endian values. The second version, used in all later EGA games, is an array of 3-byte unsigned little-endian values.

This header file may be present as an external file (eg. [[Bio Menace]]), or it may be included in the EXE file (eg. [[Catacomb 3D]]). The last entry in the header is always the size of the EGAGRAPH.xxx file, and the first entry in the header is always 00 00 00. To find the end, you can search for filesize(EGAGRAPH.xxx) as a 3-byte (little-endian) value in the EXE. Then to find the start, go back 3 bytes at a time until you get to 00 000 00. Or use the fact that entries always get smaller as you go back, unless the entry is $FFFFFF.

In [[Catacomb 3D]] v1.00 <tt>EGAHEAD.C3D</tt> is at $1BFD0 from the start of the decompressed ([[unlzexe]]) EXE file, and is 1437 bytes long, or 479 3-byte entries (for 478 chunks). It is then followed by 3 zero bytes (padding to multiple of 16?) 00 00 00, then the <tt>MAPHEAD.C3D</tt>.

This header file stores the offsets (relative to the start of the <tt>EGAGRAPH</tt> file) for each sub file. The format is trivial, in that the header file is simply an array of 3 or 4 byte variables. (Each variable is a "slot", as the game refers to files by index.) Most games use the more compact 3-byte variables, since the <tt>EGAGRAPH</tt> file is never larger than $FFFFFF. However games based on older engines, notably [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], use 4-byte vales as these are easier to deal with.

It is important to note that not every 'slot' will be in use. Tiles especially if blank (All black) will be considered 'empty' and not worth adding to the file. (A blank 16x16 tile takes up 12 bytes of space if compressed, and there can be hundreds.) There are two ways of dealing with this; by default the games set all empty slot headers to -1, but many programs compress the graphics anyway and use a normal header value for them.

The last offset in the file will be an offset to the end of the main <tt>EGAGRAPH</tt> file. (So this should be ignored when reading the file to prevent a zero-byte file appearing last.) The total size of the <tt>EGAHEAD</tt> divided by 3 (Or 4) will also give the number of 'slots' the game uses. This varies, but is usually about 10,000.

Unlike [[AudioT Format]] files, <tt>EGAGRAPH</tt> is always compressed. However the header and dictionary may be external (E.g [[Bio Menace]])

== Dictionary (EGADICT.xxx) ==

The same [[Huffman Compression]] scheme used elsewhere in the [[Commander Keen]] series is used. When reading compressed chunks out of the <tt>EGAGRAPH</tt> file, the first four bytes located at the <tt>EGAHEAD</tt> offset are a UINT32LE specifying the file's decompressed size (which is required for the decompression algorithm.) The compressed data then follows, EXCEPT for 16x16 masked and unmasked tiles. Since these all have a similar decompressed size (128 bytes for unmasked, 256 bytes for masked) it is a waste of space to include these (Since tiles make up the bulk of graphics slots.) The decompressed size is hard coded, as are the start and finish of the unmasked and masked tile slots.

Note that the <tt>EGADICT</tt> file is not often obvious, usually being embedded in the main exe file. It is possible to locate it by looking for the string <tt>$FD $01 $00 $00 $00 $00</tt>, which appears at the end of nearly all Huffman dictionaries. They are the last 6 bytes out of 1024 (256*2*2), so add 6 and subtract 1024 to get the start (pointer to the head node).

Executables usually contain two or more dictionaries, but the <tt>EGADICT</tt> is usually '''the second one''', except in the case of early games like Keen Dreams, where it is the first of THREE. (A simple check of whether it decompresses the data sensibly works.)

In [[Catacomb 3D]] <tt>EGADICT.C3D</tt> is at offset $24464 from the start of the decompressed CAT3D.EXE, which is the second Huffman dictionary and comes immediately after the first Huffman Dictionary.

== Main file (EGAGRAPH.xxx) ==

This file is simply an array of data files. Each file starts at the offset specified in the <tt>EGAHEAD</tt> file, and as there are no filenames each file is referred to by its index/slot number. Slots containing dummy values in the header are treated as if they don't exist in the <tt>EGAGRAPH</tt> file. Each file within the <tt>EGAGRAPH</tt> file is individually compressed. A file can be read by opening the <tt>EGAGRAPH</tt> file, seeking to the offset specified in the <tt>EGAHEAD</tt> file, reading a UINT32LE for the decompressed file size, and then decompressing the data from that point onwards using standard Huffman decompression techniques.

Note that in early games such as Keen Dreams, the <tt>EGAGRAPH</tt> file may have another name. (Such as <tt>KDREAM.EGA</tt> in that case.) in which case it is always the largest non-executable file.

=== Chunks ===

Each sub file or chunk is a separate graphic, ordered in the <tt>EGAGRAPH</tt> file in a specific way, much again like the [[AudioT Format]]. Chunks are labeled starting at 0. The game executable is hard coded for the start and finish of various chunk types. For example, Keen 4 has chunks 124-520 as sprites and will treat any chunk between these two numbers as a sprite. (Note that the game refers to chunk number, not sprite number.) The usual order of chunks is:

Picture table
Masked picture table
Sprite table
Fonts
Pictures (Unmasked bitmaps)
Masked pictures
8x8 unmasked tiles (Single chunk)
8x8 masked tiles (Single chunk)
16x16 unmasked tiles
16x16 masked tiles
32x32 unmasked tiles (Optional)
32x32 masked tiles (Optional)
Misc graphics (Optional)
Game texts
Demo files (Optional)
Misc data (Optional)

== Chunk formats ==

When compressed each chunk consists of a dword giving the decompressed size followed by the compressed data.

=== Picture table ===

Always present and always the first chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying picture chunks.

Divide the decompressed size by 4 to get numpics.

=== Masked picture table ===

Always present and always the second chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying masked picture chunks. This table is often quite small.

=== Sprite table ===

Always present and always the third chunk, the picture table is 16 * numspr bytes long, consisting of eight words for each sprite image (Width\8, Height, x offset, y offset and clipping rectangle left, top, right and bottom; all in that order.) used when displaying sprite chunks. This is what Modkeen exports as its notoriously hard to use xSPRITES.txt file.

=== Fonts ===

Always present and always starting at the fourth chunk. (Chunk 3!) Each chunk is a single monochrome font, containing all the entries from 0-255 for that font. Each 'letter' is stored separately and consecutively. A game will have at least one, and usually 3 fonts. The format is version two of [[EGA Font format]]

Fonts are used in the game for game texts. They can be stretched or colored and are always transparent, meaning they look strange when not over a single-color background.

=== Pictures (Bitmaps) ===
These follow game fonts. You need to know how many fonts there are before it to know how what chunk number the first picture is (3+NumFonts). In [[Catacomb 3D]] there are 2 fonts.

Bitmaps are stored as standard [[Raw EGA data]], the size of each plane being the decompressed chunk size \ 4. In order to be displayed properly these need information from chunk 0, the picture table. Note that unlike fonts, data does not wrap, so a 4x4 sprite will consist of four planes of four bytes in size, the same as an 8x4 sprite.

Pictures are used for things such as title screens or pictures in game texts. Most of the main menu will consist of pictures. They are also used for the wall textures in EGA FPS games like [[Catacomb 3D]], in which case the wall textures come near the end of the picture list (starting from chunk 137 in [[Catacomb 3D]]).

The number of picture chunks is given in chunk 0, the picture table.

=== Masked pictures (Masked bitmaps) ===

These follow Pictures. You need to know how many fonts there are, to know how what chunk number the first Masked picture is (3+NumFonts+numpics).

These are identical to the picture chunks in every respect, except being masked, they consist of five EGA planes, not four. Some games (E.g [[Catacomb 3D]] make great use of masked pictures, but most games have only one or two. They need information from chunk 1, the masked picture table, to be displayed correctly.

The number of Masked pictures is given in Chunk 1.

=== Sprites ===

The first Sprite chunk is at 3+numfonts+numpics+numMaskedPics.

Sprites are exactly identical to masked pictures in format except the game cannot stretch or warp them. Most games use these for enemy, player or item graphics during gameplay. They need information from chunk 2, the sprite table both to display correctly and to interact correctly, something more complex than masked pictures.

There are usually several hundred sprite chunks, but [[Catacomb 3D]] only has 3 sprite chunks and they are used for the PaddleWar game (Skulls and Bones). The number of sprite chunks is given in chunk 2.

=== 8x8 tiles ===

8x8 tiles are used by most games in status windows or foe the borders of message windows in-game. (And occasionally by TED5 to display levels.) There are thus not many of them. ALL the masked or unmasked tiles are one single chunk stored as four and five plane [[Raw EGA data]] respectively. The number of 8x8 unmasked and masked tiles is hard-coded into the executable, but can be worked out by dividing the decompressed chunk size by 32 and 40 (The size of one tile's data) respectively.

=== 16x16 and 32x32 tiles ===

32x32 tiles are optional and rarely used. 16x16 tiles are only occasionally absent and for most games make up the bulk of graphics. There are usually several thousand entries. Both come in masked and unmasked kinds, just like 8x8 tiles.

These chunks have notable differences. Each tile is an individual chunk and stored as four or five plane [[Raw EGA data]] with a plane size of 128 (16x16) or 512 (32x32). These chunks do NOT have a dword specifying their decompressed size as this is hard-coded into the executable as a space saving measure. (As is the start and finish of unmasked and masked tile chunks.) This can cause problems for editing programs.

Tiles make up the bulk of 2D game levels but are seldom seen elsewhere.

=== Misc graphics ===

Sometimes following tile graphics are miscellaneous graphics, that cannot be handled by the other chunk types. For example, Keen 4-6 has two misc graphics files used for the 'COOMANDER KEEN terminator text' intro. These graphics usually have unique formats that fit their function and few have been investigated.

==== Keen 4-6 Intro Bitmaps ====

This is the text that displays after the 'Ready, press any key' screen at Keen startup. It is composed of a monochrome bitmap that is scrolled across the screen and distorted for special effects. When displayed it is transparent, with a special palette so it is not black-and-white. It consists of a header followed by RLE compressed monochrome data.

HEADER:
0 2 Img height Height of the image in pixels
2 2 Img width Width of the image in pixels
4 2x Line point Pointers to line 1,2,3..etc of data. There will be [Img height] of
these, each 2 bytes long. The first pointer will have the value
(2 * [Img height] + 4)
+2 ? RLE data RLE-WM compressed data

DATA:
0 2 Black run Number of black pixels to write
2 2 Not-blk run Number of not-black pixels to write
4 2 Black run....
...
..
.
? 2 End $FFFF; end of row.

=== Game texts ===

Also known as ANSII chunks, these are used by the game for things like help screens, and have their own format. In the main they are simply text documents, interspersed with various commands to make things happen. Each file is divided up into a number of 'pages' which are moved between by pressing the up/down arrows. Pages can also have action sequences (Repeated each time the page is moved to, you cannot leave a page until the sequence finishes.)

^P First command in every file. Defines a page start
^E Ends the file
^Cx Change font color to $x until next page or ^C command
^Gx,y,z Display (unmasked) picture chunk z at location x,y (In pixels)
^Tx,y,z,t Display picture chunk z at x,y for z clicks of time
^Bx,y,z,t,b ?
^Lx,y ?

=== Misc files ===

The last few chunks are demo files and occasionally miscellaneous data. Demo file formats are currently unknown but can be recorded using the demo cheat. They contain the level number and difficulty followed by a series of key commands.

= Utilities =
* [[KeenWiki:ModKeen|ModKeen]] (DOS) and [[KeenWiki:LModKeen|LModKeen]] (Linux/Windows) can be used to modify graphics and text stored in EGAGRAPH.CKx for the Commander Keen [[KeenWiki:Goodbye Galaxy|Goodbye Galaxy]] trilogy.
* WDC (Windows) can be used to view or modify the graphics and text stored in EGAGRAPH.C3D for [[Catacomb 3D]].

[[Category:Catacomb 3D]]
[[Category:Commander Keen 4-6]]
[[Category:Commander Keen Dreams]]
[[Category:Dangerous Dave 3]]
[[Category:File Formats]]
[[Category:Graphics Files]]

EGAGraph Format

2011-09-01T07:43:56Z

CarlKenner: /* Dictionary (EGADICT.xxx) */

{{Stub}}

'''EGAGraph''', short for 'EGA graphics library' is the format used to store graphics, demos, fonts, game texts and more in many early [[:Category:Id Software|id Software]] games.

The basic format is that of a number of [[Huffman Compression]] compressed sub-files (Chunks) stored in a similar manner to the [[AudioT Format]]. Like this format it has three main files, <tt>EGAGRAPH</tt>, <tt>EGAHEAD</tt>and <tt>EGADICT</tt>.

It is notable that while the <tt>EGAHEAD</tt> and <tt>EGADICT</tt> are stored in the game executable, they need to be extracted for [[TED5]] to edit levels.

This article may contain errors as some formats have not been worked out completely.

= File formats =

== EGA Head (EGAHEAD.xxx) ==
There are two versions of this format. The first version, used in [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], is an array of 4-byte little-endian values. The second version, used in all later EGA games, is an array of 3-byte unsigned little-endian values.

This header file may be present as an external file (eg. [[Bio Menace]]), or it may be included in the EXE file (eg. [[Catacomb 3D]]). The last entry in the header is always the size of the EGAGRAPH.xxx file, and the first entry in the header is always 00 00 00. To find the end, you can search for filesize(EGAGRAPH.xxx) as a 3-byte (little-endian) value in the EXE. Then to find the start, go back 3 bytes at a time until you get to 00 000 00. Or use the fact that entries always get smaller as you go back, unless the entry is $FFFFFF.

In [[Catacomb 3D]] v1.00 <tt>EGAHEAD.C3D</tt> is at $1BFD0 from the start of the decompressed ([[unlzexe]]) EXE file, and is 1437 bytes long, or 479 3-byte entries (for 478 chunks). It is then followed by 3 zero bytes (padding to multiple of 16?) 00 00 00, then the <tt>MAPHEAD.C3D</tt>.

This header file stores the offsets (relative to the start of the <tt>EGAGRAPH</tt> file) for each sub file. The format is trivial, in that the header file is simply an array of 3 or 4 byte variables. (Each variable is a "slot", as the game refers to files by index.) Most games use the more compact 3-byte variables, since the <tt>EGAGRAPH</tt> file is never larger than $FFFFFF. However games based on older engines, notably [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], use 4-byte vales as these are easier to deal with.

It is important to note that not every 'slot' will be in use. Tiles especially if blank (All black) will be considered 'empty' and not worth adding to the file. (A blank 16x16 tile takes up 12 bytes of space if compressed, and there can be hundreds.) There are two ways of dealing with this; by default the games set all empty slot headers to -1, but many programs compress the graphics anyway and use a normal header value for them.

The last offset in the file will be an offset to the end of the main <tt>EGAGRAPH</tt> file. (So this should be ignored when reading the file to prevent a zero-byte file appearing last.) The total size of the <tt>EGAHEAD</tt> divided by 3 (Or 4) will also give the number of 'slots' the game uses. This varies, but is usually about 10,000.

Unlike [[AudioT Format]] files, <tt>EGAGRAPH</tt> is always compressed. However the header and dictionary may be external (E.g [[Bio Menace]])

== Dictionary (EGADICT.xxx) ==

The same [[Huffman Compression]] scheme used elsewhere in the [[Commander Keen]] series is used. When reading compressed chunks out of the <tt>EGAGRAPH</tt> file, the first four bytes located at the <tt>EGAHEAD</tt> offset are a UINT32LE specifying the file's decompressed size (which is required for the decompression algorithm.) The compressed data then follows, EXCEPT for 16x16 masked and unmasked tiles. Since these all have a similar decompressed size (128 bytes for unmasked, 256 bytes for masked) it is a waste of space to include these (Since tiles make up the bulk of graphics slots.) The decompressed size is hard coded, as are the start and finish of the unmasked and masked tile slots.

Note that the <tt>EGADICT</tt> file is not often obvious, usually being embedded in the main exe file. It is possible to locate it by looking for the string <tt>$FD $01 $00 $00 $00 $00</tt>, which appears at the end of nearly all Huffman dictionaries. They are the last 6 bytes out of 1024 (256*2*2), so add 6 and subtract 1024 to get the start (pointer to the head node).

Executables usually contain two or more dictionaries, but the <tt>EGADICT</tt> is usually '''the second one''', except in the case of early games like Keen Dreams, where it is the first of THREE. (A simple check of whether it decompresses the data sensibly works.)

In [[Catacomb 3D]] <tt>EGADICT.C3D</tt> is at offset $24464 from the start of the decompressed CAT3D.EXE, which is the second Huffman dictionary and comes immediately after the first Huffman Dictionary.

== Main file (EGAGRAPH.xxx) ==

This file is simply an array of data files. Each file starts at the offset specified in the <tt>EGAHEAD</tt> file, and as there are no filenames each file is referred to by its index/slot number. Slots containing dummy values in the header are treated as if they don't exist in the <tt>EGAGRAPH</tt> file. Each file within the <tt>EGAGRAPH</tt> file is individually compressed. A file can be read by opening the <tt>EGAGRAPH</tt> file, seeking to the offset specified in the <tt>EGAHEAD</tt> file, reading a UINT32LE for the decompressed file size, and then decompressing the data from that point onwards using standard Huffman decompression techniques.

Note that in early games such as Keen Dreams, the <tt>EGAGRAPH</tt> file may have another name. (Such as <tt>KDREAM.EGA</tt> in that case.) in which case it is always the largest non-executable file.

=== Chunks ===

Each sub file or chunk is a separate graphic, ordered in the <tt>EGAGRAPH</tt> file in a specific way, much again like the [[AudioT Format]]. Chunks are labeled starting at 0. The game executable is hard coded for the start and finish of various chunk types. For example, Keen 4 has chunks 124-520 as sprites and will treat any chunk between these two numbers as a sprite. (Note that the game refers to chunk number, not sprite number.) The usual order of chunks is:

Picture table
Masked picture table
Sprite table
Fonts
Pictures (Unmasked bitmaps)
Masked pictures
8x8 unmasked tiles (Single chunk)
8x8 masked tiles (Single chunk)
16x16 unmasked tiles
16x16 unmasked tiles
32x32 unmasked tiles (Optional)
32x32 masked tiles (Optional)
Misc graphics (Optional)
Game texts
Demo files (Optional)
Misc data (Optional)

== Chunk formats ==

When compressed each chunk consists of a dword giving the decompressed size followed by the compressed data.

=== Picture table ===

Always present and always the first chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying picture chunks.

Divide the decompressed size by 4 to get numpics.

=== Masked picture table ===

Always present and always the second chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying masked picture chunks. This table is often quite small.

=== Sprite table ===

Always present and always the third chunk, the picture table is 16 * numspr bytes long, consisting of eight words for each sprite image (Width\8, Height, x offset, y offset and clipping rectangle left, top, right and bottom; all in that order.) used when displaying sprite chunks. This is what Modkeen exports as its notoriously hard to use xSPRITES.txt file.

=== Fonts ===

Always present and always starting at the fourth chunk. (Chunk 3!) Each chunk is a single monochrome font, containing all the entries from 0-255 for that font. Each 'letter' is stored separately and consecutively. A game will have at least one, and usually 3 fonts. The format is version two of [[EGA Font format]]

Fonts are used in the game for game texts. They can be stretched or colored and are always transparent, meaning they look strange when not over a single-color background.

=== Pictures (Bitmaps) ===
These follow game fonts. You need to know how many fonts there are before it to know how what chunk number the first picture is (3+NumFonts). In [[Catacomb 3D]] there are 2 fonts.

Bitmaps are stored as standard [[Raw EGA data]], the size of each plane being the decompressed chunk size \ 4. In order to be displayed properly these need information from chunk 0, the picture table. Note that unlike fonts, data does not wrap, so a 4x4 sprite will consist of four planes of four bytes in size, the same as an 8x4 sprite.

Pictures are used for things such as title screens or pictures in game texts. Most of the main menu will consist of pictures. They are also used for the wall textures in EGA FPS games like [[Catacomb 3D]], in which case the wall textures come near the end of the picture list (starting from chunk 137 in [[Catacomb 3D]]).

The number of picture chunks is given in chunk 0, the picture table.

=== Masked pictures (Masked bitmaps) ===

These follow Pictures. You need to know how many fonts there are, to know how what chunk number the first Masked picture is (3+NumFonts+numpics).

These are identical to the picture chunks in every respect, except being masked, they consist of five EGA planes, not four. Some games (E.g [[Catacomb 3D]] make great use of masked pictures, but most games have only one or two. They need information from chunk 1, the masked picture table, to be displayed correctly.

The number of Masked pictures is given in Chunk 1.

=== Sprites ===

The first Sprite chunk is at 3+numfonts+numpics+numMaskedPics.

Sprites are exactly identical to masked pictures in format except the game cannot stretch or warp them. Most games use these for enemy, player or item graphics during gameplay. They need information from chunk 2, the sprite table both to display correctly and to interact correctly, something more complex than masked pictures.

There are usually several hundred sprite chunks, but [[Catacomb 3D]] only has 3 sprite chunks and they are used for the PaddleWar game (Skulls and Bones). The number of sprite chunks is given in chunk 2.

=== 8x8 tiles ===

8x8 tiles are used by most games in status windows or foe the borders of message windows in-game. (And occasionally by TED5 to display levels.) There are thus not many of them. ALL the masked or unmasked tiles are one single chunk stored as four and five plane [[Raw EGA data]] respectively. The number of 8x8 unmasked and masked tiles is hard-coded into the executable, but can be worked out by dividing the decompressed chunk size by 32 and 40 (The size of one tile's data) respectively.

=== 16x16 and 32x32 tiles ===

32x32 tiles are optional and rarely used. 16x16 tiles are only occasionally absent and for most games make up the bulk of graphics. There are usually several thousand entries. Both come in masked and unmasked kinds, just like 8x8 tiles.

These chunks have notable differences. Each tile is an individual chunk and stored as four or five plane [[Raw EGA data]] with a plane size of 128 (16x16) or 512 (32x32). These chunks do NOT have a dword specifying their decompressed size as this is hard-coded into the executable as a space saving measure. (As is the start and finish of unmasked and masked tile chunks.) This can cause problems for editing programs.

Tiles make up the bulk of 2D game levels but are seldom seen elsewhere.

=== Misc graphics ===

Sometimes following tile graphics are miscellaneous graphics, that cannot be handled by the other chunk types. For example, Keen 4-6 has two misc graphics files used for the 'COOMANDER KEEN terminator text' intro. These graphics usually have unique formats that fit their function and few have been investigated.

==== Keen 4-6 Intro Bitmaps ====

This is the text that displays after the 'Ready, press any key' screen at Keen startup. It is composed of a monochrome bitmap that is scrolled across the screen and distorted for special effects. When displayed it is transparent, with a special palette so it is not black-and-white. It consists of a header followed by RLE compressed monochrome data.

HEADER:
0 2 Img height Height of the image in pixels
2 2 Img width Width of the image in pixels
4 2x Line point Pointers to line 1,2,3..etc of data. There will be [Img height] of
these, each 2 bytes long. The first pointer will have the value
(2 * [Img height] + 4)
+2 ? RLE data RLE-WM compressed data

DATA:
0 2 Black run Number of black pixels to write
2 2 Not-blk run Number of not-black pixels to write
4 2 Black run....
...
..
.
? 2 End $FFFF; end of row.

=== Game texts ===

Also known as ANSII chunks, these are used by the game for things like help screens, and have their own format. In the main they are simply text documents, interspersed with various commands to make things happen. Each file is divided up into a number of 'pages' which are moved between by pressing the up/down arrows. Pages can also have action sequences (Repeated each time the page is moved to, you cannot leave a page until the sequence finishes.)

^P First command in every file. Defines a page start
^E Ends the file
^Cx Change font color to $x until next page or ^C command
^Gx,y,z Display (unmasked) picture chunk z at location x,y (In pixels)
^Tx,y,z,t Display picture chunk z at x,y for z clicks of time
^Bx,y,z,t,b ?
^Lx,y ?

=== Misc files ===

The last few chunks are demo files and occasionally miscellaneous data. Demo file formats are currently unknown but can be recorded using the demo cheat. They contain the level number and difficulty followed by a series of key commands.

= Utilities =
* [[KeenWiki:ModKeen|ModKeen]] (DOS) and [[KeenWiki:LModKeen|LModKeen]] (Linux/Windows) can be used to modify graphics and text stored in EGAGRAPH.CKx for the Commander Keen [[KeenWiki:Goodbye Galaxy|Goodbye Galaxy]] trilogy.
* WDC (Windows) can be used to view or modify the graphics and text stored in EGAGRAPH.C3D for [[Catacomb 3D]].

[[Category:Catacomb 3D]]
[[Category:Commander Keen 4-6]]
[[Category:Commander Keen Dreams]]
[[Category:Dangerous Dave 3]]
[[Category:File Formats]]
[[Category:Graphics Files]]

EGAGraph Format

2011-09-01T06:35:41Z

CarlKenner: /* EGA Head (EGAHEAD.xxx) */

{{Stub}}

'''EGAGraph''', short for 'EGA graphics library' is the format used to store graphics, demos, fonts, game texts and more in many early [[:Category:Id Software|id Software]] games.

The basic format is that of a number of [[Huffman Compression]] compressed sub-files (Chunks) stored in a similar manner to the [[AudioT Format]]. Like this format it has three main files, <tt>EGAGRAPH</tt>, <tt>EGAHEAD</tt>and <tt>EGADICT</tt>.

It is notable that while the <tt>EGAHEAD</tt> and <tt>EGADICT</tt> are stored in the game executable, they need to be extracted for [[TED5]] to edit levels.

This article may contain errors as some formats have not been worked out completely.

= File formats =

== EGA Head (EGAHEAD.xxx) ==
There are two versions of this format. The first version, used in [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], is an array of 4-byte little-endian values. The second version, used in all later EGA games, is an array of 3-byte unsigned little-endian values.

This header file may be present as an external file (eg. [[Bio Menace]]), or it may be included in the EXE file (eg. [[Catacomb 3D]]). The last entry in the header is always the size of the EGAGRAPH.xxx file, and the first entry in the header is always 00 00 00. To find the end, you can search for filesize(EGAGRAPH.xxx) as a 3-byte (little-endian) value in the EXE. Then to find the start, go back 3 bytes at a time until you get to 00 000 00. Or use the fact that entries always get smaller as you go back, unless the entry is $FFFFFF.

In [[Catacomb 3D]] v1.00 <tt>EGAHEAD.C3D</tt> is at $1BFD0 from the start of the decompressed ([[unlzexe]]) EXE file, and is 1437 bytes long, or 479 3-byte entries (for 478 chunks). It is then followed by 3 zero bytes (padding to multiple of 16?) 00 00 00, then the <tt>MAPHEAD.C3D</tt>.

This header file stores the offsets (relative to the start of the <tt>EGAGRAPH</tt> file) for each sub file. The format is trivial, in that the header file is simply an array of 3 or 4 byte variables. (Each variable is a "slot", as the game refers to files by index.) Most games use the more compact 3-byte variables, since the <tt>EGAGRAPH</tt> file is never larger than $FFFFFF. However games based on older engines, notably [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], use 4-byte vales as these are easier to deal with.

It is important to note that not every 'slot' will be in use. Tiles especially if blank (All black) will be considered 'empty' and not worth adding to the file. (A blank 16x16 tile takes up 12 bytes of space if compressed, and there can be hundreds.) There are two ways of dealing with this; by default the games set all empty slot headers to -1, but many programs compress the graphics anyway and use a normal header value for them.

The last offset in the file will be an offset to the end of the main <tt>EGAGRAPH</tt> file. (So this should be ignored when reading the file to prevent a zero-byte file appearing last.) The total size of the <tt>EGAHEAD</tt> divided by 3 (Or 4) will also give the number of 'slots' the game uses. This varies, but is usually about 10,000.

Unlike [[AudioT Format]] files, <tt>EGAGRAPH</tt> is always compressed. However the header and dictionary may be external (E.g [[Bio Menace]])

== Dictionary (EGADICT.xxx) ==

The same [[Huffman Compression]] scheme used elsewhere in the [[Commander Keen]] series is used. When reading compressed files out of the <tt>EGAGRAPH</tt> file, the first four bytes located at the <tt>EGAHEAD</tt> offset are a UINT32LE specifying the file's decompressed size (which is required for the decompression algorithm.) The compressed data then follows, EXCEPT for 16x16 masked and unmasked tiles. Since these all have a similar decompressed size (128 bytes for unmasked, 256 bytes for masked) it is a waste of space to include these (Since tiles make up the bulk of graphics slots.) The decompressed size is hard coded, as are the start and finish of the unmasked and masked tile slots.

Note that the <tt>EGADICT</tt> file is not often obvious, usually being embedded in the main exe file. It is possible to locate it by looking for the string <tt>$FD $01 $00 $00 $00 $00</tt>, which appears at the end of nearly all Huffman dictionaries. Executables usually contain two or more dictionaries, but the <tt>EGADICT</tt> is usually the second one, except in the case of early games like Keen Dreams, where it is the first of THREE. (A simple check of whether it decompresses the data sensibly works.)

== Main file (EGAGRAPH.xxx) ==

This file is simply an array of data files. Each file starts at the offset specified in the <tt>EGAHEAD</tt> file, and as there are no filenames each file is referred to by its index/slot number. Slots containing dummy values in the header are treated as if they don't exist in the <tt>EGAGRAPH</tt> file. Each file within the <tt>EGAGRAPH</tt> file is individually compressed. A file can be read by opening the <tt>EGAGRAPH</tt> file, seeking to the offset specified in the <tt>EGAHEAD</tt> file, reading a UINT32LE for the decompressed file size, and then decompressing the data from that point onwards using standard Huffman decompression techniques.

Note that in early games such as Keen Dreams, the <tt>EGAGRAPH</tt> file may have another name. (Such as <tt>KDREAM.EGA</tt> in that case.) in which case it is always the largest non-executable file.

=== Chunks ===

Each sub file or chunk is a separate graphic, ordered in the <tt>EGAGRAPH</tt> file in a specific way, much again like the [[AudioT Format]]. Chunks are labeled starting at 0. The game executable is hard coded for the start and finish of various chunk types. For example, Keen 4 has chunks 124-520 as sprites and will treat any chunk between these two numbers as a sprite. (Note that the game refers to chunk number, not sprite number.) The usual order of chunks is:

Picture table
Masked picture table
Sprite table
Fonts
Pictures (Unmasked bitmaps)
Masked pictures
8x8 unmasked tiles (Single chunk)
8x8 masked tiles (Single chunk)
16x16 unmasked tiles
16x16 unmasked tiles
32x32 unmasked tiles (Optional)
32x32 masked tiles (Optional)
Misc graphics (Optional)
Game texts
Demo files (Optional)
Misc data (Optional)

== Chunk formats ==

When compressed each chunk consists of a dword giving the decompressed size followed by the compressed data.

=== Picture table ===

Always present and always the first chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying picture chunks.

Divide the decompressed size by 4 to get numpics.

=== Masked picture table ===

Always present and always the second chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying masked picture chunks. This table is often quite small.

=== Sprite table ===

Always present and always the third chunk, the picture table is 16 * numspr bytes long, consisting of eight words for each sprite image (Width\8, Height, x offset, y offset and clipping rectangle left, top, right and bottom; all in that order.) used when displaying sprite chunks. This is what Modkeen exports as its notoriously hard to use xSPRITES.txt file.

=== Fonts ===

Always present and always starting at the fourth chunk. (Chunk 3!) Each chunk is a single monochrome font, containing all the entries from 0-255 for that font. Each 'letter' is stored separately and consecutively. A game will have at least one, and usually 3 fonts. The format is version two of [[EGA Font format]]

Fonts are used in the game for game texts. They can be stretched or colored and are always transparent, meaning they look strange when not over a single-color background.

=== Pictures (Bitmaps) ===
These follow game fonts. You need to know how many fonts there are before it to know how what chunk number the first picture is (3+NumFonts). In [[Catacomb 3D]] there are 2 fonts.

Bitmaps are stored as standard [[Raw EGA data]], the size of each plane being the decompressed chunk size \ 4. In order to be displayed properly these need information from chunk 0, the picture table. Note that unlike fonts, data does not wrap, so a 4x4 sprite will consist of four planes of four bytes in size, the same as an 8x4 sprite.

Pictures are used for things such as title screens or pictures in game texts. Most of the main menu will consist of pictures. They are also used for the wall textures in EGA FPS games like [[Catacomb 3D]], in which case the wall textures come near the end of the picture list (starting from chunk 137 in [[Catacomb 3D]]).

The number of picture chunks is given in chunk 0, the picture table.

=== Masked pictures (Masked bitmaps) ===

These follow Pictures. You need to know how many fonts there are, to know how what chunk number the first Masked picture is (3+NumFonts+numpics).

These are identical to the picture chunks in every respect, except being masked, they consist of five EGA planes, not four. Some games (E.g [[Catacomb 3D]] make great use of masked pictures, but most games have only one or two. They need information from chunk 1, the masked picture table, to be displayed correctly.

The number of Masked pictures is given in Chunk 1.

=== Sprites ===

The first Sprite chunk is at 3+numfonts+numpics+numMaskedPics.

Sprites are exactly identical to masked pictures in format except the game cannot stretch or warp them. Most games use these for enemy, player or item graphics during gameplay. They need information from chunk 2, the sprite table both to display correctly and to interact correctly, something more complex than masked pictures.

There are usually several hundred sprite chunks, but [[Catacomb 3D]] only has 3 sprite chunks and they are used for the PaddleWar game (Skulls and Bones). The number of sprite chunks is given in chunk 2.

=== 8x8 tiles ===

8x8 tiles are used by most games in status windows or foe the borders of message windows in-game. (And occasionally by TED5 to display levels.) There are thus not many of them. ALL the masked or unmasked tiles are one single chunk stored as four and five plane [[Raw EGA data]] respectively. The number of 8x8 unmasked and masked tiles is hard-coded into the executable, but can be worked out by dividing the decompressed chunk size by 32 and 40 (The size of one tile's data) respectively.

=== 16x16 and 32x32 tiles ===

32x32 tiles are optional and rarely used. 16x16 tiles are only occasionally absent and for most games make up the bulk of graphics. There are usually several thousand entries. Both come in masked and unmasked kinds, just like 8x8 tiles.

These chunks have notable differences. Each tile is an individual chunk and stored as four or five plane [[Raw EGA data]] with a plane size of 128 (16x16) or 512 (32x32). These chunks do NOT have a dword specifying their decompressed size as this is hard-coded into the executable as a space saving measure. (As is the start and finish of unmasked and masked tile chunks.) This can cause problems for editing programs.

Tiles make up the bulk of 2D game levels but are seldom seen elsewhere.

=== Misc graphics ===

Sometimes following tile graphics are miscellaneous graphics, that cannot be handled by the other chunk types. For example, Keen 4-6 has two misc graphics files used for the 'COOMANDER KEEN terminator text' intro. These graphics usually have unique formats that fit their function and few have been investigated.

==== Keen 4-6 Intro Bitmaps ====

This is the text that displays after the 'Ready, press any key' screen at Keen startup. It is composed of a monochrome bitmap that is scrolled across the screen and distorted for special effects. When displayed it is transparent, with a special palette so it is not black-and-white. It consists of a header followed by RLE compressed monochrome data.

HEADER:
0 2 Img height Height of the image in pixels
2 2 Img width Width of the image in pixels
4 2x Line point Pointers to line 1,2,3..etc of data. There will be [Img height] of
these, each 2 bytes long. The first pointer will have the value
(2 * [Img height] + 4)
+2 ? RLE data RLE-WM compressed data

DATA:
0 2 Black run Number of black pixels to write
2 2 Not-blk run Number of not-black pixels to write
4 2 Black run....
...
..
.
? 2 End $FFFF; end of row.

=== Game texts ===

Also known as ANSII chunks, these are used by the game for things like help screens, and have their own format. In the main they are simply text documents, interspersed with various commands to make things happen. Each file is divided up into a number of 'pages' which are moved between by pressing the up/down arrows. Pages can also have action sequences (Repeated each time the page is moved to, you cannot leave a page until the sequence finishes.)

^P First command in every file. Defines a page start
^E Ends the file
^Cx Change font color to $x until next page or ^C command
^Gx,y,z Display (unmasked) picture chunk z at location x,y (In pixels)
^Tx,y,z,t Display picture chunk z at x,y for z clicks of time
^Bx,y,z,t,b ?
^Lx,y ?

=== Misc files ===

The last few chunks are demo files and occasionally miscellaneous data. Demo file formats are currently unknown but can be recorded using the demo cheat. They contain the level number and difficulty followed by a series of key commands.

= Utilities =
* [[KeenWiki:ModKeen|ModKeen]] (DOS) and [[KeenWiki:LModKeen|LModKeen]] (Linux/Windows) can be used to modify graphics and text stored in EGAGRAPH.CKx for the Commander Keen [[KeenWiki:Goodbye Galaxy|Goodbye Galaxy]] trilogy.
* WDC (Windows) can be used to view or modify the graphics and text stored in EGAGRAPH.C3D for [[Catacomb 3D]].

[[Category:Catacomb 3D]]
[[Category:Commander Keen 4-6]]
[[Category:Commander Keen Dreams]]
[[Category:Dangerous Dave 3]]
[[Category:File Formats]]
[[Category:Graphics Files]]

EGAGraph Format

2011-09-01T06:22:30Z

CarlKenner: /* Utilities */

{{Stub}}

'''EGAGraph''', short for 'EGA graphics library' is the format used to store graphics, demos, fonts, game texts and more in many early [[:Category:Id Software|id Software]] games.

The basic format is that of a number of [[Huffman Compression]] compressed sub-files (Chunks) stored in a similar manner to the [[AudioT Format]]. Like this format it has three main files, <tt>EGAGRAPH</tt>, <tt>EGAHEAD</tt>and <tt>EGADICT</tt>.

It is notable that while the <tt>EGAHEAD</tt> and <tt>EGADICT</tt> are stored in the game executable, they need to be extracted for [[TED5]] to edit levels.

This article may contain errors as some formats have not been worked out completely.

= File formats =

== EGA Head (EGAHEAD.xxx) ==
There are two versions of this format. The first version, used in [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], is an array of 4-byte little-endian values. The second version, used in all later EGA games, is an array of 3-byte unsigned little-endian values.

This header file may be present as an external file (eg. [[Bio Menace]]), or it may be included in the EXE file (eg. [[Catacomb 3D]]). The last entry in the header is always the size of the EGAGRAPH.xxx file, and the first entry in the header is always 00 00 00. To find the end, you can search for filesize(EGAGRAPH.xxx) as a 3-byte (little-endian) value in the EXE. Then to find the start, go back 3 bytes at a time until you get to 00 000 00. Or use the fact that entries always get smaller as you go back, unless the entry is $FFFFFF.

In [[Catacomb 3D]] v1.00 EGAHEAD.C3D is at $1BFD0 from the start of the decompressed ([[unlzexe]]) EXE file, and is 1436 bytes long, or 478 3-byte entries (for 477 chunks). It is then followed by 4 bytes (padding to multiple of 16?) 03 00 00 00, then the MAPHEAD.C3D.

This header file stores the offsets (relative to the start of the <tt>EGAGRAPH</tt> file) for each sub file. The format is trivial, in that the header file is simply an array of 3 or 4 byte variables. (Each variable is a "slot", as the game refers to files by index.) Most games use the more compact 3-byte variables, since the <tt>EGAGRAPH</tt> file is never larger than $FFFFFF. However games based on older engines, notably [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], use 4-byte vales as these are easier to deal with.

It is important to note that not every 'slot' will be in use. Tiles especially if blank (All black) will be considered 'empty' and not worth adding to the file. (A blank 16x16 tile takes up 12 bytes of space if compressed, and there can be hundreds.) There are two ways of dealing with this; by default the games set all empty slot headers to -1, but many programs compress the graphics anyway and use a normal header value for them.

The last offset in the file will be an offset to the end of the main <tt>EGAGRAPH</tt> file. (So this should be ignored when reading the file to prevent a zero-byte file appearing last.) The total size of the <tt>EGAHEAD</tt> divided by 3 (Or 4) will also give the number of 'slots' the game uses. This varies, but is usually about 10,000.

Unlike [[AudioT Format]] files, <tt>EGAGRAPH</tt> is always compressed. However the header and dictionary may be external (E.g [[Bio Menace]])

== Dictionary (EGADICT.xxx) ==

The same [[Huffman Compression]] scheme used elsewhere in the [[Commander Keen]] series is used. When reading compressed files out of the <tt>EGAGRAPH</tt> file, the first four bytes located at the <tt>EGAHEAD</tt> offset are a UINT32LE specifying the file's decompressed size (which is required for the decompression algorithm.) The compressed data then follows, EXCEPT for 16x16 masked and unmasked tiles. Since these all have a similar decompressed size (128 bytes for unmasked, 256 bytes for masked) it is a waste of space to include these (Since tiles make up the bulk of graphics slots.) The decompressed size is hard coded, as are the start and finish of the unmasked and masked tile slots.

Note that the <tt>EGADICT</tt> file is not often obvious, usually being embedded in the main exe file. It is possible to locate it by looking for the string <tt>$FD $01 $00 $00 $00 $00</tt>, which appears at the end of nearly all Huffman dictionaries. Executables usually contain two or more dictionaries, but the <tt>EGADICT</tt> is usually the second one, except in the case of early games like Keen Dreams, where it is the first of THREE. (A simple check of whether it decompresses the data sensibly works.)

== Main file (EGAGRAPH.xxx) ==

This file is simply an array of data files. Each file starts at the offset specified in the <tt>EGAHEAD</tt> file, and as there are no filenames each file is referred to by its index/slot number. Slots containing dummy values in the header are treated as if they don't exist in the <tt>EGAGRAPH</tt> file. Each file within the <tt>EGAGRAPH</tt> file is individually compressed. A file can be read by opening the <tt>EGAGRAPH</tt> file, seeking to the offset specified in the <tt>EGAHEAD</tt> file, reading a UINT32LE for the decompressed file size, and then decompressing the data from that point onwards using standard Huffman decompression techniques.

Note that in early games such as Keen Dreams, the <tt>EGAGRAPH</tt> file may have another name. (Such as <tt>KDREAM.EGA</tt> in that case.) in which case it is always the largest non-executable file.

=== Chunks ===

Each sub file or chunk is a separate graphic, ordered in the <tt>EGAGRAPH</tt> file in a specific way, much again like the [[AudioT Format]]. Chunks are labeled starting at 0. The game executable is hard coded for the start and finish of various chunk types. For example, Keen 4 has chunks 124-520 as sprites and will treat any chunk between these two numbers as a sprite. (Note that the game refers to chunk number, not sprite number.) The usual order of chunks is:

Picture table
Masked picture table
Sprite table
Fonts
Pictures (Unmasked bitmaps)
Masked pictures
8x8 unmasked tiles (Single chunk)
8x8 masked tiles (Single chunk)
16x16 unmasked tiles
16x16 unmasked tiles
32x32 unmasked tiles (Optional)
32x32 masked tiles (Optional)
Misc graphics (Optional)
Game texts
Demo files (Optional)
Misc data (Optional)

== Chunk formats ==

When compressed each chunk consists of a dword giving the decompressed size followed by the compressed data.

=== Picture table ===

Always present and always the first chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying picture chunks.

Divide the decompressed size by 4 to get numpics.

=== Masked picture table ===

Always present and always the second chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying masked picture chunks. This table is often quite small.

=== Sprite table ===

Always present and always the third chunk, the picture table is 16 * numspr bytes long, consisting of eight words for each sprite image (Width\8, Height, x offset, y offset and clipping rectangle left, top, right and bottom; all in that order.) used when displaying sprite chunks. This is what Modkeen exports as its notoriously hard to use xSPRITES.txt file.

=== Fonts ===

Always present and always starting at the fourth chunk. (Chunk 3!) Each chunk is a single monochrome font, containing all the entries from 0-255 for that font. Each 'letter' is stored separately and consecutively. A game will have at least one, and usually 3 fonts. The format is version two of [[EGA Font format]]

Fonts are used in the game for game texts. They can be stretched or colored and are always transparent, meaning they look strange when not over a single-color background.

=== Pictures (Bitmaps) ===
These follow game fonts. You need to know how many fonts there are before it to know how what chunk number the first picture is (3+NumFonts). In [[Catacomb 3D]] there are 2 fonts.

Bitmaps are stored as standard [[Raw EGA data]], the size of each plane being the decompressed chunk size \ 4. In order to be displayed properly these need information from chunk 0, the picture table. Note that unlike fonts, data does not wrap, so a 4x4 sprite will consist of four planes of four bytes in size, the same as an 8x4 sprite.

Pictures are used for things such as title screens or pictures in game texts. Most of the main menu will consist of pictures. They are also used for the wall textures in EGA FPS games like [[Catacomb 3D]], in which case the wall textures come near the end of the picture list (starting from chunk 137 in [[Catacomb 3D]]).

The number of picture chunks is given in chunk 0, the picture table.

=== Masked pictures (Masked bitmaps) ===

These follow Pictures. You need to know how many fonts there are, to know how what chunk number the first Masked picture is (3+NumFonts+numpics).

These are identical to the picture chunks in every respect, except being masked, they consist of five EGA planes, not four. Some games (E.g [[Catacomb 3D]] make great use of masked pictures, but most games have only one or two. They need information from chunk 1, the masked picture table, to be displayed correctly.

The number of Masked pictures is given in Chunk 1.

=== Sprites ===

The first Sprite chunk is at 3+numfonts+numpics+numMaskedPics.

Sprites are exactly identical to masked pictures in format except the game cannot stretch or warp them. Most games use these for enemy, player or item graphics during gameplay. They need information from chunk 2, the sprite table both to display correctly and to interact correctly, something more complex than masked pictures.

There are usually several hundred sprite chunks, but [[Catacomb 3D]] only has 3 sprite chunks and they are used for the PaddleWar game (Skulls and Bones). The number of sprite chunks is given in chunk 2.

=== 8x8 tiles ===

8x8 tiles are used by most games in status windows or foe the borders of message windows in-game. (And occasionally by TED5 to display levels.) There are thus not many of them. ALL the masked or unmasked tiles are one single chunk stored as four and five plane [[Raw EGA data]] respectively. The number of 8x8 unmasked and masked tiles is hard-coded into the executable, but can be worked out by dividing the decompressed chunk size by 32 and 40 (The size of one tile's data) respectively.

=== 16x16 and 32x32 tiles ===

32x32 tiles are optional and rarely used. 16x16 tiles are only occasionally absent and for most games make up the bulk of graphics. There are usually several thousand entries. Both come in masked and unmasked kinds, just like 8x8 tiles.

These chunks have notable differences. Each tile is an individual chunk and stored as four or five plane [[Raw EGA data]] with a plane size of 128 (16x16) or 512 (32x32). These chunks do NOT have a dword specifying their decompressed size as this is hard-coded into the executable as a space saving measure. (As is the start and finish of unmasked and masked tile chunks.) This can cause problems for editing programs.

Tiles make up the bulk of 2D game levels but are seldom seen elsewhere.

=== Misc graphics ===

Sometimes following tile graphics are miscellaneous graphics, that cannot be handled by the other chunk types. For example, Keen 4-6 has two misc graphics files used for the 'COOMANDER KEEN terminator text' intro. These graphics usually have unique formats that fit their function and few have been investigated.

==== Keen 4-6 Intro Bitmaps ====

This is the text that displays after the 'Ready, press any key' screen at Keen startup. It is composed of a monochrome bitmap that is scrolled across the screen and distorted for special effects. When displayed it is transparent, with a special palette so it is not black-and-white. It consists of a header followed by RLE compressed monochrome data.

HEADER:
0 2 Img height Height of the image in pixels
2 2 Img width Width of the image in pixels
4 2x Line point Pointers to line 1,2,3..etc of data. There will be [Img height] of
these, each 2 bytes long. The first pointer will have the value
(2 * [Img height] + 4)
+2 ? RLE data RLE-WM compressed data

DATA:
0 2 Black run Number of black pixels to write
2 2 Not-blk run Number of not-black pixels to write
4 2 Black run....
...
..
.
? 2 End $FFFF; end of row.

=== Game texts ===

Also known as ANSII chunks, these are used by the game for things like help screens, and have their own format. In the main they are simply text documents, interspersed with various commands to make things happen. Each file is divided up into a number of 'pages' which are moved between by pressing the up/down arrows. Pages can also have action sequences (Repeated each time the page is moved to, you cannot leave a page until the sequence finishes.)

^P First command in every file. Defines a page start
^E Ends the file
^Cx Change font color to $x until next page or ^C command
^Gx,y,z Display (unmasked) picture chunk z at location x,y (In pixels)
^Tx,y,z,t Display picture chunk z at x,y for z clicks of time
^Bx,y,z,t,b ?
^Lx,y ?

=== Misc files ===

The last few chunks are demo files and occasionally miscellaneous data. Demo file formats are currently unknown but can be recorded using the demo cheat. They contain the level number and difficulty followed by a series of key commands.

= Utilities =
* [[KeenWiki:ModKeen|ModKeen]] (DOS) and [[KeenWiki:LModKeen|LModKeen]] (Linux/Windows) can be used to modify graphics and text stored in EGAGRAPH.CKx for the Commander Keen [[KeenWiki:Goodbye Galaxy|Goodbye Galaxy]] trilogy.
* WDC (Windows) can be used to view or modify the graphics and text stored in EGAGRAPH.C3D for [[Catacomb 3D]].

[[Category:Catacomb 3D]]
[[Category:Commander Keen 4-6]]
[[Category:Commander Keen Dreams]]
[[Category:Dangerous Dave 3]]
[[Category:File Formats]]
[[Category:Graphics Files]]

EGAGraph Format

2011-09-01T06:19:43Z

CarlKenner: /* Sprites */

{{Stub}}

'''EGAGraph''', short for 'EGA graphics library' is the format used to store graphics, demos, fonts, game texts and more in many early [[:Category:Id Software|id Software]] games.

The basic format is that of a number of [[Huffman Compression]] compressed sub-files (Chunks) stored in a similar manner to the [[AudioT Format]]. Like this format it has three main files, <tt>EGAGRAPH</tt>, <tt>EGAHEAD</tt>and <tt>EGADICT</tt>.

It is notable that while the <tt>EGAHEAD</tt> and <tt>EGADICT</tt> are stored in the game executable, they need to be extracted for [[TED5]] to edit levels.

This article may contain errors as some formats have not been worked out completely.

= File formats =

== EGA Head (EGAHEAD.xxx) ==
There are two versions of this format. The first version, used in [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], is an array of 4-byte little-endian values. The second version, used in all later EGA games, is an array of 3-byte unsigned little-endian values.

This header file may be present as an external file (eg. [[Bio Menace]]), or it may be included in the EXE file (eg. [[Catacomb 3D]]). The last entry in the header is always the size of the EGAGRAPH.xxx file, and the first entry in the header is always 00 00 00. To find the end, you can search for filesize(EGAGRAPH.xxx) as a 3-byte (little-endian) value in the EXE. Then to find the start, go back 3 bytes at a time until you get to 00 000 00. Or use the fact that entries always get smaller as you go back, unless the entry is $FFFFFF.

In [[Catacomb 3D]] v1.00 EGAHEAD.C3D is at $1BFD0 from the start of the decompressed ([[unlzexe]]) EXE file, and is 1436 bytes long, or 478 3-byte entries (for 477 chunks). It is then followed by 4 bytes (padding to multiple of 16?) 03 00 00 00, then the MAPHEAD.C3D.

This header file stores the offsets (relative to the start of the <tt>EGAGRAPH</tt> file) for each sub file. The format is trivial, in that the header file is simply an array of 3 or 4 byte variables. (Each variable is a "slot", as the game refers to files by index.) Most games use the more compact 3-byte variables, since the <tt>EGAGRAPH</tt> file is never larger than $FFFFFF. However games based on older engines, notably [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], use 4-byte vales as these are easier to deal with.

It is important to note that not every 'slot' will be in use. Tiles especially if blank (All black) will be considered 'empty' and not worth adding to the file. (A blank 16x16 tile takes up 12 bytes of space if compressed, and there can be hundreds.) There are two ways of dealing with this; by default the games set all empty slot headers to -1, but many programs compress the graphics anyway and use a normal header value for them.

The last offset in the file will be an offset to the end of the main <tt>EGAGRAPH</tt> file. (So this should be ignored when reading the file to prevent a zero-byte file appearing last.) The total size of the <tt>EGAHEAD</tt> divided by 3 (Or 4) will also give the number of 'slots' the game uses. This varies, but is usually about 10,000.

Unlike [[AudioT Format]] files, <tt>EGAGRAPH</tt> is always compressed. However the header and dictionary may be external (E.g [[Bio Menace]])

== Dictionary (EGADICT.xxx) ==

The same [[Huffman Compression]] scheme used elsewhere in the [[Commander Keen]] series is used. When reading compressed files out of the <tt>EGAGRAPH</tt> file, the first four bytes located at the <tt>EGAHEAD</tt> offset are a UINT32LE specifying the file's decompressed size (which is required for the decompression algorithm.) The compressed data then follows, EXCEPT for 16x16 masked and unmasked tiles. Since these all have a similar decompressed size (128 bytes for unmasked, 256 bytes for masked) it is a waste of space to include these (Since tiles make up the bulk of graphics slots.) The decompressed size is hard coded, as are the start and finish of the unmasked and masked tile slots.

Note that the <tt>EGADICT</tt> file is not often obvious, usually being embedded in the main exe file. It is possible to locate it by looking for the string <tt>$FD $01 $00 $00 $00 $00</tt>, which appears at the end of nearly all Huffman dictionaries. Executables usually contain two or more dictionaries, but the <tt>EGADICT</tt> is usually the second one, except in the case of early games like Keen Dreams, where it is the first of THREE. (A simple check of whether it decompresses the data sensibly works.)

== Main file (EGAGRAPH.xxx) ==

This file is simply an array of data files. Each file starts at the offset specified in the <tt>EGAHEAD</tt> file, and as there are no filenames each file is referred to by its index/slot number. Slots containing dummy values in the header are treated as if they don't exist in the <tt>EGAGRAPH</tt> file. Each file within the <tt>EGAGRAPH</tt> file is individually compressed. A file can be read by opening the <tt>EGAGRAPH</tt> file, seeking to the offset specified in the <tt>EGAHEAD</tt> file, reading a UINT32LE for the decompressed file size, and then decompressing the data from that point onwards using standard Huffman decompression techniques.

Note that in early games such as Keen Dreams, the <tt>EGAGRAPH</tt> file may have another name. (Such as <tt>KDREAM.EGA</tt> in that case.) in which case it is always the largest non-executable file.

=== Chunks ===

Each sub file or chunk is a separate graphic, ordered in the <tt>EGAGRAPH</tt> file in a specific way, much again like the [[AudioT Format]]. Chunks are labeled starting at 0. The game executable is hard coded for the start and finish of various chunk types. For example, Keen 4 has chunks 124-520 as sprites and will treat any chunk between these two numbers as a sprite. (Note that the game refers to chunk number, not sprite number.) The usual order of chunks is:

Picture table
Masked picture table
Sprite table
Fonts
Pictures (Unmasked bitmaps)
Masked pictures
8x8 unmasked tiles (Single chunk)
8x8 masked tiles (Single chunk)
16x16 unmasked tiles
16x16 unmasked tiles
32x32 unmasked tiles (Optional)
32x32 masked tiles (Optional)
Misc graphics (Optional)
Game texts
Demo files (Optional)
Misc data (Optional)

== Chunk formats ==

When compressed each chunk consists of a dword giving the decompressed size followed by the compressed data.

=== Picture table ===

Always present and always the first chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying picture chunks.

Divide the decompressed size by 4 to get numpics.

=== Masked picture table ===

Always present and always the second chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying masked picture chunks. This table is often quite small.

=== Sprite table ===

Always present and always the third chunk, the picture table is 16 * numspr bytes long, consisting of eight words for each sprite image (Width\8, Height, x offset, y offset and clipping rectangle left, top, right and bottom; all in that order.) used when displaying sprite chunks. This is what Modkeen exports as its notoriously hard to use xSPRITES.txt file.

=== Fonts ===

Always present and always starting at the fourth chunk. (Chunk 3!) Each chunk is a single monochrome font, containing all the entries from 0-255 for that font. Each 'letter' is stored separately and consecutively. A game will have at least one, and usually 3 fonts. The format is version two of [[EGA Font format]]

Fonts are used in the game for game texts. They can be stretched or colored and are always transparent, meaning they look strange when not over a single-color background.

=== Pictures (Bitmaps) ===
These follow game fonts. You need to know how many fonts there are before it to know how what chunk number the first picture is (3+NumFonts). In [[Catacomb 3D]] there are 2 fonts.

Bitmaps are stored as standard [[Raw EGA data]], the size of each plane being the decompressed chunk size \ 4. In order to be displayed properly these need information from chunk 0, the picture table. Note that unlike fonts, data does not wrap, so a 4x4 sprite will consist of four planes of four bytes in size, the same as an 8x4 sprite.

Pictures are used for things such as title screens or pictures in game texts. Most of the main menu will consist of pictures. They are also used for the wall textures in EGA FPS games like [[Catacomb 3D]], in which case the wall textures come near the end of the picture list (starting from chunk 137 in [[Catacomb 3D]]).

The number of picture chunks is given in chunk 0, the picture table.

=== Masked pictures (Masked bitmaps) ===

These follow Pictures. You need to know how many fonts there are, to know how what chunk number the first Masked picture is (3+NumFonts+numpics).

These are identical to the picture chunks in every respect, except being masked, they consist of five EGA planes, not four. Some games (E.g [[Catacomb 3D]] make great use of masked pictures, but most games have only one or two. They need information from chunk 1, the masked picture table, to be displayed correctly.

The number of Masked pictures is given in Chunk 1.

=== Sprites ===

The first Sprite chunk is at 3+numfonts+numpics+numMaskedPics.

Sprites are exactly identical to masked pictures in format except the game cannot stretch or warp them. Most games use these for enemy, player or item graphics during gameplay. They need information from chunk 2, the sprite table both to display correctly and to interact correctly, something more complex than masked pictures.

There are usually several hundred sprite chunks, but [[Catacomb 3D]] only has 3 sprite chunks and they are used for the PaddleWar game (Skulls and Bones). The number of sprite chunks is given in chunk 2.

=== 8x8 tiles ===

8x8 tiles are used by most games in status windows or foe the borders of message windows in-game. (And occasionally by TED5 to display levels.) There are thus not many of them. ALL the masked or unmasked tiles are one single chunk stored as four and five plane [[Raw EGA data]] respectively. The number of 8x8 unmasked and masked tiles is hard-coded into the executable, but can be worked out by dividing the decompressed chunk size by 32 and 40 (The size of one tile's data) respectively.

=== 16x16 and 32x32 tiles ===

32x32 tiles are optional and rarely used. 16x16 tiles are only occasionally absent and for most games make up the bulk of graphics. There are usually several thousand entries. Both come in masked and unmasked kinds, just like 8x8 tiles.

These chunks have notable differences. Each tile is an individual chunk and stored as four or five plane [[Raw EGA data]] with a plane size of 128 (16x16) or 512 (32x32). These chunks do NOT have a dword specifying their decompressed size as this is hard-coded into the executable as a space saving measure. (As is the start and finish of unmasked and masked tile chunks.) This can cause problems for editing programs.

Tiles make up the bulk of 2D game levels but are seldom seen elsewhere.

=== Misc graphics ===

Sometimes following tile graphics are miscellaneous graphics, that cannot be handled by the other chunk types. For example, Keen 4-6 has two misc graphics files used for the 'COOMANDER KEEN terminator text' intro. These graphics usually have unique formats that fit their function and few have been investigated.

==== Keen 4-6 Intro Bitmaps ====

This is the text that displays after the 'Ready, press any key' screen at Keen startup. It is composed of a monochrome bitmap that is scrolled across the screen and distorted for special effects. When displayed it is transparent, with a special palette so it is not black-and-white. It consists of a header followed by RLE compressed monochrome data.

HEADER:
0 2 Img height Height of the image in pixels
2 2 Img width Width of the image in pixels
4 2x Line point Pointers to line 1,2,3..etc of data. There will be [Img height] of
these, each 2 bytes long. The first pointer will have the value
(2 * [Img height] + 4)
+2 ? RLE data RLE-WM compressed data

DATA:
0 2 Black run Number of black pixels to write
2 2 Not-blk run Number of not-black pixels to write
4 2 Black run....
...
..
.
? 2 End $FFFF; end of row.

=== Game texts ===

Also known as ANSII chunks, these are used by the game for things like help screens, and have their own format. In the main they are simply text documents, interspersed with various commands to make things happen. Each file is divided up into a number of 'pages' which are moved between by pressing the up/down arrows. Pages can also have action sequences (Repeated each time the page is moved to, you cannot leave a page until the sequence finishes.)

^P First command in every file. Defines a page start
^E Ends the file
^Cx Change font color to $x until next page or ^C command
^Gx,y,z Display (unmasked) picture chunk z at location x,y (In pixels)
^Tx,y,z,t Display picture chunk z at x,y for z clicks of time
^Bx,y,z,t,b ?
^Lx,y ?

=== Misc files ===

The last few chunks are demo files and occasionally miscellaneous data. Demo file formats are currently unknown but can be recorded using the demo cheat. They contain the level number and difficulty followed by a series of key commands.

= Utilities =
* [[KeenWiki:ModKeen|ModKeen]] (DOS) and [[KeenWiki:LModKeen|LModKeen]] (Linux/Windows) can be used to modify graphics and text stored in EGAGRAPH.CKx for the Commander Keen [[KeenWiki:Goodbye Galaxy|Goodbye Galaxy]] trilogy.

[[Category:Catacomb 3D]]
[[Category:Commander Keen 4-6]]
[[Category:Commander Keen Dreams]]
[[Category:Dangerous Dave 3]]
[[Category:File Formats]]
[[Category:Graphics Files]]

EGAGraph Format

2011-09-01T06:15:04Z

CarlKenner: /* Masked pictures (Masked bitmaps) */

{{Stub}}

'''EGAGraph''', short for 'EGA graphics library' is the format used to store graphics, demos, fonts, game texts and more in many early [[:Category:Id Software|id Software]] games.

The basic format is that of a number of [[Huffman Compression]] compressed sub-files (Chunks) stored in a similar manner to the [[AudioT Format]]. Like this format it has three main files, <tt>EGAGRAPH</tt>, <tt>EGAHEAD</tt>and <tt>EGADICT</tt>.

It is notable that while the <tt>EGAHEAD</tt> and <tt>EGADICT</tt> are stored in the game executable, they need to be extracted for [[TED5]] to edit levels.

This article may contain errors as some formats have not been worked out completely.

= File formats =

== EGA Head (EGAHEAD.xxx) ==
There are two versions of this format. The first version, used in [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], is an array of 4-byte little-endian values. The second version, used in all later EGA games, is an array of 3-byte unsigned little-endian values.

This header file may be present as an external file (eg. [[Bio Menace]]), or it may be included in the EXE file (eg. [[Catacomb 3D]]). The last entry in the header is always the size of the EGAGRAPH.xxx file, and the first entry in the header is always 00 00 00. To find the end, you can search for filesize(EGAGRAPH.xxx) as a 3-byte (little-endian) value in the EXE. Then to find the start, go back 3 bytes at a time until you get to 00 000 00. Or use the fact that entries always get smaller as you go back, unless the entry is $FFFFFF.

In [[Catacomb 3D]] v1.00 EGAHEAD.C3D is at $1BFD0 from the start of the decompressed ([[unlzexe]]) EXE file, and is 1436 bytes long, or 478 3-byte entries (for 477 chunks). It is then followed by 4 bytes (padding to multiple of 16?) 03 00 00 00, then the MAPHEAD.C3D.

This header file stores the offsets (relative to the start of the <tt>EGAGRAPH</tt> file) for each sub file. The format is trivial, in that the header file is simply an array of 3 or 4 byte variables. (Each variable is a "slot", as the game refers to files by index.) Most games use the more compact 3-byte variables, since the <tt>EGAGRAPH</tt> file is never larger than $FFFFFF. However games based on older engines, notably [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], use 4-byte vales as these are easier to deal with.

It is important to note that not every 'slot' will be in use. Tiles especially if blank (All black) will be considered 'empty' and not worth adding to the file. (A blank 16x16 tile takes up 12 bytes of space if compressed, and there can be hundreds.) There are two ways of dealing with this; by default the games set all empty slot headers to -1, but many programs compress the graphics anyway and use a normal header value for them.

The last offset in the file will be an offset to the end of the main <tt>EGAGRAPH</tt> file. (So this should be ignored when reading the file to prevent a zero-byte file appearing last.) The total size of the <tt>EGAHEAD</tt> divided by 3 (Or 4) will also give the number of 'slots' the game uses. This varies, but is usually about 10,000.

Unlike [[AudioT Format]] files, <tt>EGAGRAPH</tt> is always compressed. However the header and dictionary may be external (E.g [[Bio Menace]])

== Dictionary (EGADICT.xxx) ==

The same [[Huffman Compression]] scheme used elsewhere in the [[Commander Keen]] series is used. When reading compressed files out of the <tt>EGAGRAPH</tt> file, the first four bytes located at the <tt>EGAHEAD</tt> offset are a UINT32LE specifying the file's decompressed size (which is required for the decompression algorithm.) The compressed data then follows, EXCEPT for 16x16 masked and unmasked tiles. Since these all have a similar decompressed size (128 bytes for unmasked, 256 bytes for masked) it is a waste of space to include these (Since tiles make up the bulk of graphics slots.) The decompressed size is hard coded, as are the start and finish of the unmasked and masked tile slots.

Note that the <tt>EGADICT</tt> file is not often obvious, usually being embedded in the main exe file. It is possible to locate it by looking for the string <tt>$FD $01 $00 $00 $00 $00</tt>, which appears at the end of nearly all Huffman dictionaries. Executables usually contain two or more dictionaries, but the <tt>EGADICT</tt> is usually the second one, except in the case of early games like Keen Dreams, where it is the first of THREE. (A simple check of whether it decompresses the data sensibly works.)

== Main file (EGAGRAPH.xxx) ==

This file is simply an array of data files. Each file starts at the offset specified in the <tt>EGAHEAD</tt> file, and as there are no filenames each file is referred to by its index/slot number. Slots containing dummy values in the header are treated as if they don't exist in the <tt>EGAGRAPH</tt> file. Each file within the <tt>EGAGRAPH</tt> file is individually compressed. A file can be read by opening the <tt>EGAGRAPH</tt> file, seeking to the offset specified in the <tt>EGAHEAD</tt> file, reading a UINT32LE for the decompressed file size, and then decompressing the data from that point onwards using standard Huffman decompression techniques.

Note that in early games such as Keen Dreams, the <tt>EGAGRAPH</tt> file may have another name. (Such as <tt>KDREAM.EGA</tt> in that case.) in which case it is always the largest non-executable file.

=== Chunks ===

Each sub file or chunk is a separate graphic, ordered in the <tt>EGAGRAPH</tt> file in a specific way, much again like the [[AudioT Format]]. Chunks are labeled starting at 0. The game executable is hard coded for the start and finish of various chunk types. For example, Keen 4 has chunks 124-520 as sprites and will treat any chunk between these two numbers as a sprite. (Note that the game refers to chunk number, not sprite number.) The usual order of chunks is:

Picture table
Masked picture table
Sprite table
Fonts
Pictures (Unmasked bitmaps)
Masked pictures
8x8 unmasked tiles (Single chunk)
8x8 masked tiles (Single chunk)
16x16 unmasked tiles
16x16 unmasked tiles
32x32 unmasked tiles (Optional)
32x32 masked tiles (Optional)
Misc graphics (Optional)
Game texts
Demo files (Optional)
Misc data (Optional)

== Chunk formats ==

When compressed each chunk consists of a dword giving the decompressed size followed by the compressed data.

=== Picture table ===

Always present and always the first chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying picture chunks.

Divide the decompressed size by 4 to get numpics.

=== Masked picture table ===

Always present and always the second chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying masked picture chunks. This table is often quite small.

=== Sprite table ===

Always present and always the third chunk, the picture table is 16 * numspr bytes long, consisting of eight words for each sprite image (Width\8, Height, x offset, y offset and clipping rectangle left, top, right and bottom; all in that order.) used when displaying sprite chunks. This is what Modkeen exports as its notoriously hard to use xSPRITES.txt file.

=== Fonts ===

Always present and always starting at the fourth chunk. (Chunk 3!) Each chunk is a single monochrome font, containing all the entries from 0-255 for that font. Each 'letter' is stored separately and consecutively. A game will have at least one, and usually 3 fonts. The format is version two of [[EGA Font format]]

Fonts are used in the game for game texts. They can be stretched or colored and are always transparent, meaning they look strange when not over a single-color background.

=== Pictures (Bitmaps) ===
These follow game fonts. You need to know how many fonts there are before it to know how what chunk number the first picture is (3+NumFonts). In [[Catacomb 3D]] there are 2 fonts.

Bitmaps are stored as standard [[Raw EGA data]], the size of each plane being the decompressed chunk size \ 4. In order to be displayed properly these need information from chunk 0, the picture table. Note that unlike fonts, data does not wrap, so a 4x4 sprite will consist of four planes of four bytes in size, the same as an 8x4 sprite.

Pictures are used for things such as title screens or pictures in game texts. Most of the main menu will consist of pictures. They are also used for the wall textures in EGA FPS games like [[Catacomb 3D]], in which case the wall textures come near the end of the picture list (starting from chunk 137 in [[Catacomb 3D]]).

The number of picture chunks is given in chunk 0, the picture table.

=== Masked pictures (Masked bitmaps) ===

These follow Pictures. You need to know how many fonts there are, to know how what chunk number the first Masked picture is (3+NumFonts+numpics).

These are identical to the picture chunks in every respect, except being masked, they consist of five EGA planes, not four. Some games (E.g [[Catacomb 3D]] make great use of masked pictures, but most games have only one or two. They need information from chunk 1, the masked picture table, to be displayed correctly.

The number of Masked pictures is given in Chunk 1.

=== Sprites ===

Sprites are exactly identical to masked pictures in format except the game cannot stretch or warp them. Most games use these for enemy, player or item graphics during gameplay. They need information from chunk 2, the sprite table both to display correctly and to interact correctly, something more complex than masked pictures. There are usually several hundred sprite chunks.

=== 8x8 tiles ===

8x8 tiles are used by most games in status windows or foe the borders of message windows in-game. (And occasionally by TED5 to display levels.) There are thus not many of them. ALL the masked or unmasked tiles are one single chunk stored as four and five plane [[Raw EGA data]] respectively. The number of 8x8 unmasked and masked tiles is hard-coded into the executable, but can be worked out by dividing the decompressed chunk size by 32 and 40 (The size of one tile's data) respectively.

=== 16x16 and 32x32 tiles ===

32x32 tiles are optional and rarely used. 16x16 tiles are only occasionally absent and for most games make up the bulk of graphics. There are usually several thousand entries. Both come in masked and unmasked kinds, just like 8x8 tiles.

These chunks have notable differences. Each tile is an individual chunk and stored as four or five plane [[Raw EGA data]] with a plane size of 128 (16x16) or 512 (32x32). These chunks do NOT have a dword specifying their decompressed size as this is hard-coded into the executable as a space saving measure. (As is the start and finish of unmasked and masked tile chunks.) This can cause problems for editing programs.

Tiles make up the bulk of 2D game levels but are seldom seen elsewhere.

=== Misc graphics ===

Sometimes following tile graphics are miscellaneous graphics, that cannot be handled by the other chunk types. For example, Keen 4-6 has two misc graphics files used for the 'COOMANDER KEEN terminator text' intro. These graphics usually have unique formats that fit their function and few have been investigated.

==== Keen 4-6 Intro Bitmaps ====

This is the text that displays after the 'Ready, press any key' screen at Keen startup. It is composed of a monochrome bitmap that is scrolled across the screen and distorted for special effects. When displayed it is transparent, with a special palette so it is not black-and-white. It consists of a header followed by RLE compressed monochrome data.

HEADER:
0 2 Img height Height of the image in pixels
2 2 Img width Width of the image in pixels
4 2x Line point Pointers to line 1,2,3..etc of data. There will be [Img height] of
these, each 2 bytes long. The first pointer will have the value
(2 * [Img height] + 4)
+2 ? RLE data RLE-WM compressed data

DATA:
0 2 Black run Number of black pixels to write
2 2 Not-blk run Number of not-black pixels to write
4 2 Black run....
...
..
.
? 2 End $FFFF; end of row.

=== Game texts ===

Also known as ANSII chunks, these are used by the game for things like help screens, and have their own format. In the main they are simply text documents, interspersed with various commands to make things happen. Each file is divided up into a number of 'pages' which are moved between by pressing the up/down arrows. Pages can also have action sequences (Repeated each time the page is moved to, you cannot leave a page until the sequence finishes.)

^P First command in every file. Defines a page start
^E Ends the file
^Cx Change font color to $x until next page or ^C command
^Gx,y,z Display (unmasked) picture chunk z at location x,y (In pixels)
^Tx,y,z,t Display picture chunk z at x,y for z clicks of time
^Bx,y,z,t,b ?
^Lx,y ?

=== Misc files ===

The last few chunks are demo files and occasionally miscellaneous data. Demo file formats are currently unknown but can be recorded using the demo cheat. They contain the level number and difficulty followed by a series of key commands.

= Utilities =
* [[KeenWiki:ModKeen|ModKeen]] (DOS) and [[KeenWiki:LModKeen|LModKeen]] (Linux/Windows) can be used to modify graphics and text stored in EGAGRAPH.CKx for the Commander Keen [[KeenWiki:Goodbye Galaxy|Goodbye Galaxy]] trilogy.

[[Category:Catacomb 3D]]
[[Category:Commander Keen 4-6]]
[[Category:Commander Keen Dreams]]
[[Category:Dangerous Dave 3]]
[[Category:File Formats]]
[[Category:Graphics Files]]

EGAGraph Format

2011-09-01T06:12:38Z

CarlKenner: /* Pictures (Bitmaps) */

{{Stub}}

'''EGAGraph''', short for 'EGA graphics library' is the format used to store graphics, demos, fonts, game texts and more in many early [[:Category:Id Software|id Software]] games.

The basic format is that of a number of [[Huffman Compression]] compressed sub-files (Chunks) stored in a similar manner to the [[AudioT Format]]. Like this format it has three main files, <tt>EGAGRAPH</tt>, <tt>EGAHEAD</tt>and <tt>EGADICT</tt>.

It is notable that while the <tt>EGAHEAD</tt> and <tt>EGADICT</tt> are stored in the game executable, they need to be extracted for [[TED5]] to edit levels.

This article may contain errors as some formats have not been worked out completely.

= File formats =

== EGA Head (EGAHEAD.xxx) ==
There are two versions of this format. The first version, used in [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], is an array of 4-byte little-endian values. The second version, used in all later EGA games, is an array of 3-byte unsigned little-endian values.

This header file may be present as an external file (eg. [[Bio Menace]]), or it may be included in the EXE file (eg. [[Catacomb 3D]]). The last entry in the header is always the size of the EGAGRAPH.xxx file, and the first entry in the header is always 00 00 00. To find the end, you can search for filesize(EGAGRAPH.xxx) as a 3-byte (little-endian) value in the EXE. Then to find the start, go back 3 bytes at a time until you get to 00 000 00. Or use the fact that entries always get smaller as you go back, unless the entry is $FFFFFF.

In [[Catacomb 3D]] v1.00 EGAHEAD.C3D is at $1BFD0 from the start of the decompressed ([[unlzexe]]) EXE file, and is 1436 bytes long, or 478 3-byte entries (for 477 chunks). It is then followed by 4 bytes (padding to multiple of 16?) 03 00 00 00, then the MAPHEAD.C3D.

This header file stores the offsets (relative to the start of the <tt>EGAGRAPH</tt> file) for each sub file. The format is trivial, in that the header file is simply an array of 3 or 4 byte variables. (Each variable is a "slot", as the game refers to files by index.) Most games use the more compact 3-byte variables, since the <tt>EGAGRAPH</tt> file is never larger than $FFFFFF. However games based on older engines, notably [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], use 4-byte vales as these are easier to deal with.

It is important to note that not every 'slot' will be in use. Tiles especially if blank (All black) will be considered 'empty' and not worth adding to the file. (A blank 16x16 tile takes up 12 bytes of space if compressed, and there can be hundreds.) There are two ways of dealing with this; by default the games set all empty slot headers to -1, but many programs compress the graphics anyway and use a normal header value for them.

The last offset in the file will be an offset to the end of the main <tt>EGAGRAPH</tt> file. (So this should be ignored when reading the file to prevent a zero-byte file appearing last.) The total size of the <tt>EGAHEAD</tt> divided by 3 (Or 4) will also give the number of 'slots' the game uses. This varies, but is usually about 10,000.

Unlike [[AudioT Format]] files, <tt>EGAGRAPH</tt> is always compressed. However the header and dictionary may be external (E.g [[Bio Menace]])

== Dictionary (EGADICT.xxx) ==

The same [[Huffman Compression]] scheme used elsewhere in the [[Commander Keen]] series is used. When reading compressed files out of the <tt>EGAGRAPH</tt> file, the first four bytes located at the <tt>EGAHEAD</tt> offset are a UINT32LE specifying the file's decompressed size (which is required for the decompression algorithm.) The compressed data then follows, EXCEPT for 16x16 masked and unmasked tiles. Since these all have a similar decompressed size (128 bytes for unmasked, 256 bytes for masked) it is a waste of space to include these (Since tiles make up the bulk of graphics slots.) The decompressed size is hard coded, as are the start and finish of the unmasked and masked tile slots.

Note that the <tt>EGADICT</tt> file is not often obvious, usually being embedded in the main exe file. It is possible to locate it by looking for the string <tt>$FD $01 $00 $00 $00 $00</tt>, which appears at the end of nearly all Huffman dictionaries. Executables usually contain two or more dictionaries, but the <tt>EGADICT</tt> is usually the second one, except in the case of early games like Keen Dreams, where it is the first of THREE. (A simple check of whether it decompresses the data sensibly works.)

== Main file (EGAGRAPH.xxx) ==

This file is simply an array of data files. Each file starts at the offset specified in the <tt>EGAHEAD</tt> file, and as there are no filenames each file is referred to by its index/slot number. Slots containing dummy values in the header are treated as if they don't exist in the <tt>EGAGRAPH</tt> file. Each file within the <tt>EGAGRAPH</tt> file is individually compressed. A file can be read by opening the <tt>EGAGRAPH</tt> file, seeking to the offset specified in the <tt>EGAHEAD</tt> file, reading a UINT32LE for the decompressed file size, and then decompressing the data from that point onwards using standard Huffman decompression techniques.

Note that in early games such as Keen Dreams, the <tt>EGAGRAPH</tt> file may have another name. (Such as <tt>KDREAM.EGA</tt> in that case.) in which case it is always the largest non-executable file.

=== Chunks ===

Each sub file or chunk is a separate graphic, ordered in the <tt>EGAGRAPH</tt> file in a specific way, much again like the [[AudioT Format]]. Chunks are labeled starting at 0. The game executable is hard coded for the start and finish of various chunk types. For example, Keen 4 has chunks 124-520 as sprites and will treat any chunk between these two numbers as a sprite. (Note that the game refers to chunk number, not sprite number.) The usual order of chunks is:

Picture table
Masked picture table
Sprite table
Fonts
Pictures (Unmasked bitmaps)
Masked pictures
8x8 unmasked tiles (Single chunk)
8x8 masked tiles (Single chunk)
16x16 unmasked tiles
16x16 unmasked tiles
32x32 unmasked tiles (Optional)
32x32 masked tiles (Optional)
Misc graphics (Optional)
Game texts
Demo files (Optional)
Misc data (Optional)

== Chunk formats ==

When compressed each chunk consists of a dword giving the decompressed size followed by the compressed data.

=== Picture table ===

Always present and always the first chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying picture chunks.

Divide the decompressed size by 4 to get numpics.

=== Masked picture table ===

Always present and always the second chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying masked picture chunks. This table is often quite small.

=== Sprite table ===

Always present and always the third chunk, the picture table is 16 * numspr bytes long, consisting of eight words for each sprite image (Width\8, Height, x offset, y offset and clipping rectangle left, top, right and bottom; all in that order.) used when displaying sprite chunks. This is what Modkeen exports as its notoriously hard to use xSPRITES.txt file.

=== Fonts ===

Always present and always starting at the fourth chunk. (Chunk 3!) Each chunk is a single monochrome font, containing all the entries from 0-255 for that font. Each 'letter' is stored separately and consecutively. A game will have at least one, and usually 3 fonts. The format is version two of [[EGA Font format]]

Fonts are used in the game for game texts. They can be stretched or colored and are always transparent, meaning they look strange when not over a single-color background.

=== Pictures (Bitmaps) ===
These follow game fonts. You need to know how many fonts there are before it to know how what chunk number the first picture is (3+NumFonts). In [[Catacomb 3D]] there are 2 fonts.

Bitmaps are stored as standard [[Raw EGA data]], the size of each plane being the decompressed chunk size \ 4. In order to be displayed properly these need information from chunk 0, the picture table. Note that unlike fonts, data does not wrap, so a 4x4 sprite will consist of four planes of four bytes in size, the same as an 8x4 sprite.

Pictures are used for things such as title screens or pictures in game texts. Most of the main menu will consist of pictures. They are also used for the wall textures in EGA FPS games like [[Catacomb 3D]], in which case the wall textures come near the end of the picture list (starting from chunk 137 in [[Catacomb 3D]]).

The number of picture chunks is given in chunk 0, the picture table.

=== Masked pictures (Masked bitmaps) ===

These are identical to the picture chunks in every respect, except being masked, they consist of five EGA planes, not four. Some games (E.g [[Catacomb 3D]] make great use of masked pictures, but most games have only one or two. They need information from chunk 1, the masked picture table, to be displayed correctly.

=== Sprites ===

Sprites are exactly identical to masked pictures in format except the game cannot stretch or warp them. Most games use these for enemy, player or item graphics during gameplay. They need information from chunk 2, the sprite table both to display correctly and to interact correctly, something more complex than masked pictures. There are usually several hundred sprite chunks.

=== 8x8 tiles ===

8x8 tiles are used by most games in status windows or foe the borders of message windows in-game. (And occasionally by TED5 to display levels.) There are thus not many of them. ALL the masked or unmasked tiles are one single chunk stored as four and five plane [[Raw EGA data]] respectively. The number of 8x8 unmasked and masked tiles is hard-coded into the executable, but can be worked out by dividing the decompressed chunk size by 32 and 40 (The size of one tile's data) respectively.

=== 16x16 and 32x32 tiles ===

32x32 tiles are optional and rarely used. 16x16 tiles are only occasionally absent and for most games make up the bulk of graphics. There are usually several thousand entries. Both come in masked and unmasked kinds, just like 8x8 tiles.

These chunks have notable differences. Each tile is an individual chunk and stored as four or five plane [[Raw EGA data]] with a plane size of 128 (16x16) or 512 (32x32). These chunks do NOT have a dword specifying their decompressed size as this is hard-coded into the executable as a space saving measure. (As is the start and finish of unmasked and masked tile chunks.) This can cause problems for editing programs.

Tiles make up the bulk of 2D game levels but are seldom seen elsewhere.

=== Misc graphics ===

Sometimes following tile graphics are miscellaneous graphics, that cannot be handled by the other chunk types. For example, Keen 4-6 has two misc graphics files used for the 'COOMANDER KEEN terminator text' intro. These graphics usually have unique formats that fit their function and few have been investigated.

==== Keen 4-6 Intro Bitmaps ====

This is the text that displays after the 'Ready, press any key' screen at Keen startup. It is composed of a monochrome bitmap that is scrolled across the screen and distorted for special effects. When displayed it is transparent, with a special palette so it is not black-and-white. It consists of a header followed by RLE compressed monochrome data.

HEADER:
0 2 Img height Height of the image in pixels
2 2 Img width Width of the image in pixels
4 2x Line point Pointers to line 1,2,3..etc of data. There will be [Img height] of
these, each 2 bytes long. The first pointer will have the value
(2 * [Img height] + 4)
+2 ? RLE data RLE-WM compressed data

DATA:
0 2 Black run Number of black pixels to write
2 2 Not-blk run Number of not-black pixels to write
4 2 Black run....
...
..
.
? 2 End $FFFF; end of row.

=== Game texts ===

Also known as ANSII chunks, these are used by the game for things like help screens, and have their own format. In the main they are simply text documents, interspersed with various commands to make things happen. Each file is divided up into a number of 'pages' which are moved between by pressing the up/down arrows. Pages can also have action sequences (Repeated each time the page is moved to, you cannot leave a page until the sequence finishes.)

^P First command in every file. Defines a page start
^E Ends the file
^Cx Change font color to $x until next page or ^C command
^Gx,y,z Display (unmasked) picture chunk z at location x,y (In pixels)
^Tx,y,z,t Display picture chunk z at x,y for z clicks of time
^Bx,y,z,t,b ?
^Lx,y ?

=== Misc files ===

The last few chunks are demo files and occasionally miscellaneous data. Demo file formats are currently unknown but can be recorded using the demo cheat. They contain the level number and difficulty followed by a series of key commands.

= Utilities =
* [[KeenWiki:ModKeen|ModKeen]] (DOS) and [[KeenWiki:LModKeen|LModKeen]] (Linux/Windows) can be used to modify graphics and text stored in EGAGRAPH.CKx for the Commander Keen [[KeenWiki:Goodbye Galaxy|Goodbye Galaxy]] trilogy.

[[Category:Catacomb 3D]]
[[Category:Commander Keen 4-6]]
[[Category:Commander Keen Dreams]]
[[Category:Dangerous Dave 3]]
[[Category:File Formats]]
[[Category:Graphics Files]]

EGAGraph Format

2011-09-01T06:11:00Z

CarlKenner: /* Picture table */

{{Stub}}

'''EGAGraph''', short for 'EGA graphics library' is the format used to store graphics, demos, fonts, game texts and more in many early [[:Category:Id Software|id Software]] games.

The basic format is that of a number of [[Huffman Compression]] compressed sub-files (Chunks) stored in a similar manner to the [[AudioT Format]]. Like this format it has three main files, <tt>EGAGRAPH</tt>, <tt>EGAHEAD</tt>and <tt>EGADICT</tt>.

It is notable that while the <tt>EGAHEAD</tt> and <tt>EGADICT</tt> are stored in the game executable, they need to be extracted for [[TED5]] to edit levels.

This article may contain errors as some formats have not been worked out completely.

= File formats =

== EGA Head (EGAHEAD.xxx) ==
There are two versions of this format. The first version, used in [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], is an array of 4-byte little-endian values. The second version, used in all later EGA games, is an array of 3-byte unsigned little-endian values.

This header file may be present as an external file (eg. [[Bio Menace]]), or it may be included in the EXE file (eg. [[Catacomb 3D]]). The last entry in the header is always the size of the EGAGRAPH.xxx file, and the first entry in the header is always 00 00 00. To find the end, you can search for filesize(EGAGRAPH.xxx) as a 3-byte (little-endian) value in the EXE. Then to find the start, go back 3 bytes at a time until you get to 00 000 00. Or use the fact that entries always get smaller as you go back, unless the entry is $FFFFFF.

In [[Catacomb 3D]] v1.00 EGAHEAD.C3D is at $1BFD0 from the start of the decompressed ([[unlzexe]]) EXE file, and is 1436 bytes long, or 478 3-byte entries (for 477 chunks). It is then followed by 4 bytes (padding to multiple of 16?) 03 00 00 00, then the MAPHEAD.C3D.

This header file stores the offsets (relative to the start of the <tt>EGAGRAPH</tt> file) for each sub file. The format is trivial, in that the header file is simply an array of 3 or 4 byte variables. (Each variable is a "slot", as the game refers to files by index.) Most games use the more compact 3-byte variables, since the <tt>EGAGRAPH</tt> file is never larger than $FFFFFF. However games based on older engines, notably [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], use 4-byte vales as these are easier to deal with.

It is important to note that not every 'slot' will be in use. Tiles especially if blank (All black) will be considered 'empty' and not worth adding to the file. (A blank 16x16 tile takes up 12 bytes of space if compressed, and there can be hundreds.) There are two ways of dealing with this; by default the games set all empty slot headers to -1, but many programs compress the graphics anyway and use a normal header value for them.

The last offset in the file will be an offset to the end of the main <tt>EGAGRAPH</tt> file. (So this should be ignored when reading the file to prevent a zero-byte file appearing last.) The total size of the <tt>EGAHEAD</tt> divided by 3 (Or 4) will also give the number of 'slots' the game uses. This varies, but is usually about 10,000.

Unlike [[AudioT Format]] files, <tt>EGAGRAPH</tt> is always compressed. However the header and dictionary may be external (E.g [[Bio Menace]])

== Dictionary (EGADICT.xxx) ==

The same [[Huffman Compression]] scheme used elsewhere in the [[Commander Keen]] series is used. When reading compressed files out of the <tt>EGAGRAPH</tt> file, the first four bytes located at the <tt>EGAHEAD</tt> offset are a UINT32LE specifying the file's decompressed size (which is required for the decompression algorithm.) The compressed data then follows, EXCEPT for 16x16 masked and unmasked tiles. Since these all have a similar decompressed size (128 bytes for unmasked, 256 bytes for masked) it is a waste of space to include these (Since tiles make up the bulk of graphics slots.) The decompressed size is hard coded, as are the start and finish of the unmasked and masked tile slots.

Note that the <tt>EGADICT</tt> file is not often obvious, usually being embedded in the main exe file. It is possible to locate it by looking for the string <tt>$FD $01 $00 $00 $00 $00</tt>, which appears at the end of nearly all Huffman dictionaries. Executables usually contain two or more dictionaries, but the <tt>EGADICT</tt> is usually the second one, except in the case of early games like Keen Dreams, where it is the first of THREE. (A simple check of whether it decompresses the data sensibly works.)

== Main file (EGAGRAPH.xxx) ==

This file is simply an array of data files. Each file starts at the offset specified in the <tt>EGAHEAD</tt> file, and as there are no filenames each file is referred to by its index/slot number. Slots containing dummy values in the header are treated as if they don't exist in the <tt>EGAGRAPH</tt> file. Each file within the <tt>EGAGRAPH</tt> file is individually compressed. A file can be read by opening the <tt>EGAGRAPH</tt> file, seeking to the offset specified in the <tt>EGAHEAD</tt> file, reading a UINT32LE for the decompressed file size, and then decompressing the data from that point onwards using standard Huffman decompression techniques.

Note that in early games such as Keen Dreams, the <tt>EGAGRAPH</tt> file may have another name. (Such as <tt>KDREAM.EGA</tt> in that case.) in which case it is always the largest non-executable file.

=== Chunks ===

Each sub file or chunk is a separate graphic, ordered in the <tt>EGAGRAPH</tt> file in a specific way, much again like the [[AudioT Format]]. Chunks are labeled starting at 0. The game executable is hard coded for the start and finish of various chunk types. For example, Keen 4 has chunks 124-520 as sprites and will treat any chunk between these two numbers as a sprite. (Note that the game refers to chunk number, not sprite number.) The usual order of chunks is:

Picture table
Masked picture table
Sprite table
Fonts
Pictures (Unmasked bitmaps)
Masked pictures
8x8 unmasked tiles (Single chunk)
8x8 masked tiles (Single chunk)
16x16 unmasked tiles
16x16 unmasked tiles
32x32 unmasked tiles (Optional)
32x32 masked tiles (Optional)
Misc graphics (Optional)
Game texts
Demo files (Optional)
Misc data (Optional)

== Chunk formats ==

When compressed each chunk consists of a dword giving the decompressed size followed by the compressed data.

=== Picture table ===

Always present and always the first chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying picture chunks.

Divide the decompressed size by 4 to get numpics.

=== Masked picture table ===

Always present and always the second chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying masked picture chunks. This table is often quite small.

=== Sprite table ===

Always present and always the third chunk, the picture table is 16 * numspr bytes long, consisting of eight words for each sprite image (Width\8, Height, x offset, y offset and clipping rectangle left, top, right and bottom; all in that order.) used when displaying sprite chunks. This is what Modkeen exports as its notoriously hard to use xSPRITES.txt file.

=== Fonts ===

Always present and always starting at the fourth chunk. (Chunk 3!) Each chunk is a single monochrome font, containing all the entries from 0-255 for that font. Each 'letter' is stored separately and consecutively. A game will have at least one, and usually 3 fonts. The format is version two of [[EGA Font format]]

Fonts are used in the game for game texts. They can be stretched or colored and are always transparent, meaning they look strange when not over a single-color background.

=== Pictures (Bitmaps) ===
These follow game fonts. You need to know how many fonts there are before it to know how what chunk number the first picture is (4+NumFonts). In [[Catacomb 3D]] there are 2 fonts.

Bitmaps are stored as standard [[Raw EGA data]], the size of each plane being the decompressed chunk size \ 4. In order to be displayed properly these need information from chunk 0, the picture table. Note that unlike fonts, data does not wrap, so a 4x4 sprite will consist of four planes of four bytes in size, the same as an 8x4 sprite.

Pictures are used for things such as title screens or pictures in game texts. Most of the main menu will consist of pictures. They are also used for the wall textures in EGA FPS games like [[Catacomb 3D]], in which case the wall textures come near the end of the picture list (starting from chunk 137 in [[Catacomb 3D]]).

The number of picture chunks is given in chunk 0, the picture table.

=== Masked pictures (Masked bitmaps) ===

These are identical to the picture chunks in every respect, except being masked, they consist of five EGA planes, not four. Some games (E.g [[Catacomb 3D]] make great use of masked pictures, but most games have only one or two. They need information from chunk 1, the masked picture table, to be displayed correctly.

=== Sprites ===

Sprites are exactly identical to masked pictures in format except the game cannot stretch or warp them. Most games use these for enemy, player or item graphics during gameplay. They need information from chunk 2, the sprite table both to display correctly and to interact correctly, something more complex than masked pictures. There are usually several hundred sprite chunks.

=== 8x8 tiles ===

8x8 tiles are used by most games in status windows or foe the borders of message windows in-game. (And occasionally by TED5 to display levels.) There are thus not many of them. ALL the masked or unmasked tiles are one single chunk stored as four and five plane [[Raw EGA data]] respectively. The number of 8x8 unmasked and masked tiles is hard-coded into the executable, but can be worked out by dividing the decompressed chunk size by 32 and 40 (The size of one tile's data) respectively.

=== 16x16 and 32x32 tiles ===

32x32 tiles are optional and rarely used. 16x16 tiles are only occasionally absent and for most games make up the bulk of graphics. There are usually several thousand entries. Both come in masked and unmasked kinds, just like 8x8 tiles.

These chunks have notable differences. Each tile is an individual chunk and stored as four or five plane [[Raw EGA data]] with a plane size of 128 (16x16) or 512 (32x32). These chunks do NOT have a dword specifying their decompressed size as this is hard-coded into the executable as a space saving measure. (As is the start and finish of unmasked and masked tile chunks.) This can cause problems for editing programs.

Tiles make up the bulk of 2D game levels but are seldom seen elsewhere.

=== Misc graphics ===

Sometimes following tile graphics are miscellaneous graphics, that cannot be handled by the other chunk types. For example, Keen 4-6 has two misc graphics files used for the 'COOMANDER KEEN terminator text' intro. These graphics usually have unique formats that fit their function and few have been investigated.

==== Keen 4-6 Intro Bitmaps ====

This is the text that displays after the 'Ready, press any key' screen at Keen startup. It is composed of a monochrome bitmap that is scrolled across the screen and distorted for special effects. When displayed it is transparent, with a special palette so it is not black-and-white. It consists of a header followed by RLE compressed monochrome data.

HEADER:
0 2 Img height Height of the image in pixels
2 2 Img width Width of the image in pixels
4 2x Line point Pointers to line 1,2,3..etc of data. There will be [Img height] of
these, each 2 bytes long. The first pointer will have the value
(2 * [Img height] + 4)
+2 ? RLE data RLE-WM compressed data

DATA:
0 2 Black run Number of black pixels to write
2 2 Not-blk run Number of not-black pixels to write
4 2 Black run....
...
..
.
? 2 End $FFFF; end of row.

=== Game texts ===

Also known as ANSII chunks, these are used by the game for things like help screens, and have their own format. In the main they are simply text documents, interspersed with various commands to make things happen. Each file is divided up into a number of 'pages' which are moved between by pressing the up/down arrows. Pages can also have action sequences (Repeated each time the page is moved to, you cannot leave a page until the sequence finishes.)

^P First command in every file. Defines a page start
^E Ends the file
^Cx Change font color to $x until next page or ^C command
^Gx,y,z Display (unmasked) picture chunk z at location x,y (In pixels)
^Tx,y,z,t Display picture chunk z at x,y for z clicks of time
^Bx,y,z,t,b ?
^Lx,y ?

=== Misc files ===

The last few chunks are demo files and occasionally miscellaneous data. Demo file formats are currently unknown but can be recorded using the demo cheat. They contain the level number and difficulty followed by a series of key commands.

= Utilities =
* [[KeenWiki:ModKeen|ModKeen]] (DOS) and [[KeenWiki:LModKeen|LModKeen]] (Linux/Windows) can be used to modify graphics and text stored in EGAGRAPH.CKx for the Commander Keen [[KeenWiki:Goodbye Galaxy|Goodbye Galaxy]] trilogy.

[[Category:Catacomb 3D]]
[[Category:Commander Keen 4-6]]
[[Category:Commander Keen Dreams]]
[[Category:Dangerous Dave 3]]
[[Category:File Formats]]
[[Category:Graphics Files]]

EGAGraph Format

2011-09-01T06:07:49Z

CarlKenner: /* Pictures (Bitmaps) */

{{Stub}}

'''EGAGraph''', short for 'EGA graphics library' is the format used to store graphics, demos, fonts, game texts and more in many early [[:Category:Id Software|id Software]] games.

The basic format is that of a number of [[Huffman Compression]] compressed sub-files (Chunks) stored in a similar manner to the [[AudioT Format]]. Like this format it has three main files, <tt>EGAGRAPH</tt>, <tt>EGAHEAD</tt>and <tt>EGADICT</tt>.

It is notable that while the <tt>EGAHEAD</tt> and <tt>EGADICT</tt> are stored in the game executable, they need to be extracted for [[TED5]] to edit levels.

This article may contain errors as some formats have not been worked out completely.

= File formats =

== EGA Head (EGAHEAD.xxx) ==
There are two versions of this format. The first version, used in [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], is an array of 4-byte little-endian values. The second version, used in all later EGA games, is an array of 3-byte unsigned little-endian values.

This header file may be present as an external file (eg. [[Bio Menace]]), or it may be included in the EXE file (eg. [[Catacomb 3D]]). The last entry in the header is always the size of the EGAGRAPH.xxx file, and the first entry in the header is always 00 00 00. To find the end, you can search for filesize(EGAGRAPH.xxx) as a 3-byte (little-endian) value in the EXE. Then to find the start, go back 3 bytes at a time until you get to 00 000 00. Or use the fact that entries always get smaller as you go back, unless the entry is $FFFFFF.

In [[Catacomb 3D]] v1.00 EGAHEAD.C3D is at $1BFD0 from the start of the decompressed ([[unlzexe]]) EXE file, and is 1436 bytes long, or 478 3-byte entries (for 477 chunks). It is then followed by 4 bytes (padding to multiple of 16?) 03 00 00 00, then the MAPHEAD.C3D.

This header file stores the offsets (relative to the start of the <tt>EGAGRAPH</tt> file) for each sub file. The format is trivial, in that the header file is simply an array of 3 or 4 byte variables. (Each variable is a "slot", as the game refers to files by index.) Most games use the more compact 3-byte variables, since the <tt>EGAGRAPH</tt> file is never larger than $FFFFFF. However games based on older engines, notably [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], use 4-byte vales as these are easier to deal with.

It is important to note that not every 'slot' will be in use. Tiles especially if blank (All black) will be considered 'empty' and not worth adding to the file. (A blank 16x16 tile takes up 12 bytes of space if compressed, and there can be hundreds.) There are two ways of dealing with this; by default the games set all empty slot headers to -1, but many programs compress the graphics anyway and use a normal header value for them.

The last offset in the file will be an offset to the end of the main <tt>EGAGRAPH</tt> file. (So this should be ignored when reading the file to prevent a zero-byte file appearing last.) The total size of the <tt>EGAHEAD</tt> divided by 3 (Or 4) will also give the number of 'slots' the game uses. This varies, but is usually about 10,000.

Unlike [[AudioT Format]] files, <tt>EGAGRAPH</tt> is always compressed. However the header and dictionary may be external (E.g [[Bio Menace]])

== Dictionary (EGADICT.xxx) ==

The same [[Huffman Compression]] scheme used elsewhere in the [[Commander Keen]] series is used. When reading compressed files out of the <tt>EGAGRAPH</tt> file, the first four bytes located at the <tt>EGAHEAD</tt> offset are a UINT32LE specifying the file's decompressed size (which is required for the decompression algorithm.) The compressed data then follows, EXCEPT for 16x16 masked and unmasked tiles. Since these all have a similar decompressed size (128 bytes for unmasked, 256 bytes for masked) it is a waste of space to include these (Since tiles make up the bulk of graphics slots.) The decompressed size is hard coded, as are the start and finish of the unmasked and masked tile slots.

Note that the <tt>EGADICT</tt> file is not often obvious, usually being embedded in the main exe file. It is possible to locate it by looking for the string <tt>$FD $01 $00 $00 $00 $00</tt>, which appears at the end of nearly all Huffman dictionaries. Executables usually contain two or more dictionaries, but the <tt>EGADICT</tt> is usually the second one, except in the case of early games like Keen Dreams, where it is the first of THREE. (A simple check of whether it decompresses the data sensibly works.)

== Main file (EGAGRAPH.xxx) ==

This file is simply an array of data files. Each file starts at the offset specified in the <tt>EGAHEAD</tt> file, and as there are no filenames each file is referred to by its index/slot number. Slots containing dummy values in the header are treated as if they don't exist in the <tt>EGAGRAPH</tt> file. Each file within the <tt>EGAGRAPH</tt> file is individually compressed. A file can be read by opening the <tt>EGAGRAPH</tt> file, seeking to the offset specified in the <tt>EGAHEAD</tt> file, reading a UINT32LE for the decompressed file size, and then decompressing the data from that point onwards using standard Huffman decompression techniques.

Note that in early games such as Keen Dreams, the <tt>EGAGRAPH</tt> file may have another name. (Such as <tt>KDREAM.EGA</tt> in that case.) in which case it is always the largest non-executable file.

=== Chunks ===

Each sub file or chunk is a separate graphic, ordered in the <tt>EGAGRAPH</tt> file in a specific way, much again like the [[AudioT Format]]. Chunks are labeled starting at 0. The game executable is hard coded for the start and finish of various chunk types. For example, Keen 4 has chunks 124-520 as sprites and will treat any chunk between these two numbers as a sprite. (Note that the game refers to chunk number, not sprite number.) The usual order of chunks is:

Picture table
Masked picture table
Sprite table
Fonts
Pictures (Unmasked bitmaps)
Masked pictures
8x8 unmasked tiles (Single chunk)
8x8 masked tiles (Single chunk)
16x16 unmasked tiles
16x16 unmasked tiles
32x32 unmasked tiles (Optional)
32x32 masked tiles (Optional)
Misc graphics (Optional)
Game texts
Demo files (Optional)
Misc data (Optional)

== Chunk formats ==

When compressed each chunk consists of a dword giving the decompressed size followed by the compressed data.

=== Picture table ===

Always present and always the first chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying picture chunks.

=== Masked picture table ===

Always present and always the second chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying masked picture chunks. This table is often quite small.

=== Sprite table ===

Always present and always the third chunk, the picture table is 16 * numspr bytes long, consisting of eight words for each sprite image (Width\8, Height, x offset, y offset and clipping rectangle left, top, right and bottom; all in that order.) used when displaying sprite chunks. This is what Modkeen exports as its notoriously hard to use xSPRITES.txt file.

=== Fonts ===

Always present and always starting at the fourth chunk. (Chunk 3!) Each chunk is a single monochrome font, containing all the entries from 0-255 for that font. Each 'letter' is stored separately and consecutively. A game will have at least one, and usually 3 fonts. The format is version two of [[EGA Font format]]

Fonts are used in the game for game texts. They can be stretched or colored and are always transparent, meaning they look strange when not over a single-color background.

=== Pictures (Bitmaps) ===
These follow game fonts. You need to know how many fonts there are before it to know how what chunk number the first picture is (4+NumFonts). In [[Catacomb 3D]] there are 2 fonts.

Bitmaps are stored as standard [[Raw EGA data]], the size of each plane being the decompressed chunk size \ 4. In order to be displayed properly these need information from chunk 0, the picture table. Note that unlike fonts, data does not wrap, so a 4x4 sprite will consist of four planes of four bytes in size, the same as an 8x4 sprite.

Pictures are used for things such as title screens or pictures in game texts. Most of the main menu will consist of pictures. They are also used for the wall textures in EGA FPS games like [[Catacomb 3D]], in which case the wall textures come near the end of the picture list (starting from chunk 137 in [[Catacomb 3D]]).

The number of picture chunks is given in chunk 0, the picture table.

=== Masked pictures (Masked bitmaps) ===

These are identical to the picture chunks in every respect, except being masked, they consist of five EGA planes, not four. Some games (E.g [[Catacomb 3D]] make great use of masked pictures, but most games have only one or two. They need information from chunk 1, the masked picture table, to be displayed correctly.

=== Sprites ===

Sprites are exactly identical to masked pictures in format except the game cannot stretch or warp them. Most games use these for enemy, player or item graphics during gameplay. They need information from chunk 2, the sprite table both to display correctly and to interact correctly, something more complex than masked pictures. There are usually several hundred sprite chunks.

=== 8x8 tiles ===

8x8 tiles are used by most games in status windows or foe the borders of message windows in-game. (And occasionally by TED5 to display levels.) There are thus not many of them. ALL the masked or unmasked tiles are one single chunk stored as four and five plane [[Raw EGA data]] respectively. The number of 8x8 unmasked and masked tiles is hard-coded into the executable, but can be worked out by dividing the decompressed chunk size by 32 and 40 (The size of one tile's data) respectively.

=== 16x16 and 32x32 tiles ===

32x32 tiles are optional and rarely used. 16x16 tiles are only occasionally absent and for most games make up the bulk of graphics. There are usually several thousand entries. Both come in masked and unmasked kinds, just like 8x8 tiles.

These chunks have notable differences. Each tile is an individual chunk and stored as four or five plane [[Raw EGA data]] with a plane size of 128 (16x16) or 512 (32x32). These chunks do NOT have a dword specifying their decompressed size as this is hard-coded into the executable as a space saving measure. (As is the start and finish of unmasked and masked tile chunks.) This can cause problems for editing programs.

Tiles make up the bulk of 2D game levels but are seldom seen elsewhere.

=== Misc graphics ===

Sometimes following tile graphics are miscellaneous graphics, that cannot be handled by the other chunk types. For example, Keen 4-6 has two misc graphics files used for the 'COOMANDER KEEN terminator text' intro. These graphics usually have unique formats that fit their function and few have been investigated.

==== Keen 4-6 Intro Bitmaps ====

This is the text that displays after the 'Ready, press any key' screen at Keen startup. It is composed of a monochrome bitmap that is scrolled across the screen and distorted for special effects. When displayed it is transparent, with a special palette so it is not black-and-white. It consists of a header followed by RLE compressed monochrome data.

HEADER:
0 2 Img height Height of the image in pixels
2 2 Img width Width of the image in pixels
4 2x Line point Pointers to line 1,2,3..etc of data. There will be [Img height] of
these, each 2 bytes long. The first pointer will have the value
(2 * [Img height] + 4)
+2 ? RLE data RLE-WM compressed data

DATA:
0 2 Black run Number of black pixels to write
2 2 Not-blk run Number of not-black pixels to write
4 2 Black run....
...
..
.
? 2 End $FFFF; end of row.

=== Game texts ===

Also known as ANSII chunks, these are used by the game for things like help screens, and have their own format. In the main they are simply text documents, interspersed with various commands to make things happen. Each file is divided up into a number of 'pages' which are moved between by pressing the up/down arrows. Pages can also have action sequences (Repeated each time the page is moved to, you cannot leave a page until the sequence finishes.)

^P First command in every file. Defines a page start
^E Ends the file
^Cx Change font color to $x until next page or ^C command
^Gx,y,z Display (unmasked) picture chunk z at location x,y (In pixels)
^Tx,y,z,t Display picture chunk z at x,y for z clicks of time
^Bx,y,z,t,b ?
^Lx,y ?

=== Misc files ===

The last few chunks are demo files and occasionally miscellaneous data. Demo file formats are currently unknown but can be recorded using the demo cheat. They contain the level number and difficulty followed by a series of key commands.

= Utilities =
* [[KeenWiki:ModKeen|ModKeen]] (DOS) and [[KeenWiki:LModKeen|LModKeen]] (Linux/Windows) can be used to modify graphics and text stored in EGAGRAPH.CKx for the Commander Keen [[KeenWiki:Goodbye Galaxy|Goodbye Galaxy]] trilogy.

[[Category:Catacomb 3D]]
[[Category:Commander Keen 4-6]]
[[Category:Commander Keen Dreams]]
[[Category:Dangerous Dave 3]]
[[Category:File Formats]]
[[Category:Graphics Files]]

EGAGraph Format

2011-09-01T05:50:04Z

CarlKenner: /* EGA Head (EGAHEAD.xxx) */

{{Stub}}

'''EGAGraph''', short for 'EGA graphics library' is the format used to store graphics, demos, fonts, game texts and more in many early [[:Category:Id Software|id Software]] games.

The basic format is that of a number of [[Huffman Compression]] compressed sub-files (Chunks) stored in a similar manner to the [[AudioT Format]]. Like this format it has three main files, <tt>EGAGRAPH</tt>, <tt>EGAHEAD</tt>and <tt>EGADICT</tt>.

It is notable that while the <tt>EGAHEAD</tt> and <tt>EGADICT</tt> are stored in the game executable, they need to be extracted for [[TED5]] to edit levels.

This article may contain errors as some formats have not been worked out completely.

= File formats =

== EGA Head (EGAHEAD.xxx) ==
There are two versions of this format. The first version, used in [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], is an array of 4-byte little-endian values. The second version, used in all later EGA games, is an array of 3-byte unsigned little-endian values.

This header file may be present as an external file (eg. [[Bio Menace]]), or it may be included in the EXE file (eg. [[Catacomb 3D]]). The last entry in the header is always the size of the EGAGRAPH.xxx file, and the first entry in the header is always 00 00 00. To find the end, you can search for filesize(EGAGRAPH.xxx) as a 3-byte (little-endian) value in the EXE. Then to find the start, go back 3 bytes at a time until you get to 00 000 00. Or use the fact that entries always get smaller as you go back, unless the entry is $FFFFFF.

In [[Catacomb 3D]] v1.00 EGAHEAD.C3D is at $1BFD0 from the start of the decompressed ([[unlzexe]]) EXE file, and is 1436 bytes long, or 478 3-byte entries (for 477 chunks). It is then followed by 4 bytes (padding to multiple of 16?) 03 00 00 00, then the MAPHEAD.C3D.

This header file stores the offsets (relative to the start of the <tt>EGAGRAPH</tt> file) for each sub file. The format is trivial, in that the header file is simply an array of 3 or 4 byte variables. (Each variable is a "slot", as the game refers to files by index.) Most games use the more compact 3-byte variables, since the <tt>EGAGRAPH</tt> file is never larger than $FFFFFF. However games based on older engines, notably [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], use 4-byte vales as these are easier to deal with.

It is important to note that not every 'slot' will be in use. Tiles especially if blank (All black) will be considered 'empty' and not worth adding to the file. (A blank 16x16 tile takes up 12 bytes of space if compressed, and there can be hundreds.) There are two ways of dealing with this; by default the games set all empty slot headers to -1, but many programs compress the graphics anyway and use a normal header value for them.

The last offset in the file will be an offset to the end of the main <tt>EGAGRAPH</tt> file. (So this should be ignored when reading the file to prevent a zero-byte file appearing last.) The total size of the <tt>EGAHEAD</tt> divided by 3 (Or 4) will also give the number of 'slots' the game uses. This varies, but is usually about 10,000.

Unlike [[AudioT Format]] files, <tt>EGAGRAPH</tt> is always compressed. However the header and dictionary may be external (E.g [[Bio Menace]])

== Dictionary (EGADICT.xxx) ==

The same [[Huffman Compression]] scheme used elsewhere in the [[Commander Keen]] series is used. When reading compressed files out of the <tt>EGAGRAPH</tt> file, the first four bytes located at the <tt>EGAHEAD</tt> offset are a UINT32LE specifying the file's decompressed size (which is required for the decompression algorithm.) The compressed data then follows, EXCEPT for 16x16 masked and unmasked tiles. Since these all have a similar decompressed size (128 bytes for unmasked, 256 bytes for masked) it is a waste of space to include these (Since tiles make up the bulk of graphics slots.) The decompressed size is hard coded, as are the start and finish of the unmasked and masked tile slots.

Note that the <tt>EGADICT</tt> file is not often obvious, usually being embedded in the main exe file. It is possible to locate it by looking for the string <tt>$FD $01 $00 $00 $00 $00</tt>, which appears at the end of nearly all Huffman dictionaries. Executables usually contain two or more dictionaries, but the <tt>EGADICT</tt> is usually the second one, except in the case of early games like Keen Dreams, where it is the first of THREE. (A simple check of whether it decompresses the data sensibly works.)

== Main file (EGAGRAPH.xxx) ==

This file is simply an array of data files. Each file starts at the offset specified in the <tt>EGAHEAD</tt> file, and as there are no filenames each file is referred to by its index/slot number. Slots containing dummy values in the header are treated as if they don't exist in the <tt>EGAGRAPH</tt> file. Each file within the <tt>EGAGRAPH</tt> file is individually compressed. A file can be read by opening the <tt>EGAGRAPH</tt> file, seeking to the offset specified in the <tt>EGAHEAD</tt> file, reading a UINT32LE for the decompressed file size, and then decompressing the data from that point onwards using standard Huffman decompression techniques.

Note that in early games such as Keen Dreams, the <tt>EGAGRAPH</tt> file may have another name. (Such as <tt>KDREAM.EGA</tt> in that case.) in which case it is always the largest non-executable file.

=== Chunks ===

Each sub file or chunk is a separate graphic, ordered in the <tt>EGAGRAPH</tt> file in a specific way, much again like the [[AudioT Format]]. Chunks are labeled starting at 0. The game executable is hard coded for the start and finish of various chunk types. For example, Keen 4 has chunks 124-520 as sprites and will treat any chunk between these two numbers as a sprite. (Note that the game refers to chunk number, not sprite number.) The usual order of chunks is:

Picture table
Masked picture table
Sprite table
Fonts
Pictures (Unmasked bitmaps)
Masked pictures
8x8 unmasked tiles (Single chunk)
8x8 masked tiles (Single chunk)
16x16 unmasked tiles
16x16 unmasked tiles
32x32 unmasked tiles (Optional)
32x32 masked tiles (Optional)
Misc graphics (Optional)
Game texts
Demo files (Optional)
Misc data (Optional)

== Chunk formats ==

When compressed each chunk consists of a dword giving the decompressed size followed by the compressed data.

=== Picture table ===

Always present and always the first chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying picture chunks.

=== Masked picture table ===

Always present and always the second chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying masked picture chunks. This table is often quite small.

=== Sprite table ===

Always present and always the third chunk, the picture table is 16 * numspr bytes long, consisting of eight words for each sprite image (Width\8, Height, x offset, y offset and clipping rectangle left, top, right and bottom; all in that order.) used when displaying sprite chunks. This is what Modkeen exports as its notoriously hard to use xSPRITES.txt file.

=== Fonts ===

Always present and always starting at the fourth chunk. (Chunk 3!) Each chunk is a single monochrome font, containing all the entries from 0-255 for that font. Each 'letter' is stored separately and consecutively. A game will have at least one, and usually 3 fonts. The format is version two of [[EGA Font format]]

Fonts are used in the game for game texts. They can be stretched or colored and are always transparent, meaning they look strange when not over a single-color background.

=== Pictures (Bitmaps) ===

These follow game fonts. Bitmaps are stored as standard [[Raw EGA data]], the size of each plane being the decompressed chunk size \ 4. In order to be displayed properly these need information from chunk 0, the picture table. Note that unlike fonts, data does not wrap, so a 4x4 sprite will consist of four planes of four bytes in size, the same as an 8x4 sprite.

Pictures are used for things such as title screens or pictures in game texts. Most of the main menu will consist of pictures.

=== Masked pictures (Masked bitmaps) ===

These are identical to the picture chunks in every respect, except being masked, they consist of five EGA planes, not four. Some games (E.g [[Catacomb 3D]] make great use of masked pictures, but most games have only one or two. They need information from chunk 1, the masked picture table, to be displayed correctly.

=== Sprites ===

Sprites are exactly identical to masked pictures in format except the game cannot stretch or warp them. Most games use these for enemy, player or item graphics during gameplay. They need information from chunk 2, the sprite table both to display correctly and to interact correctly, something more complex than masked pictures. There are usually several hundred sprite chunks.

=== 8x8 tiles ===

8x8 tiles are used by most games in status windows or foe the borders of message windows in-game. (And occasionally by TED5 to display levels.) There are thus not many of them. ALL the masked or unmasked tiles are one single chunk stored as four and five plane [[Raw EGA data]] respectively. The number of 8x8 unmasked and masked tiles is hard-coded into the executable, but can be worked out by dividing the decompressed chunk size by 32 and 40 (The size of one tile's data) respectively.

=== 16x16 and 32x32 tiles ===

32x32 tiles are optional and rarely used. 16x16 tiles are only occasionally absent and for most games make up the bulk of graphics. There are usually several thousand entries. Both come in masked and unmasked kinds, just like 8x8 tiles.

These chunks have notable differences. Each tile is an individual chunk and stored as four or five plane [[Raw EGA data]] with a plane size of 128 (16x16) or 512 (32x32). These chunks do NOT have a dword specifying their decompressed size as this is hard-coded into the executable as a space saving measure. (As is the start and finish of unmasked and masked tile chunks.) This can cause problems for editing programs.

Tiles make up the bulk of 2D game levels but are seldom seen elsewhere.

=== Misc graphics ===

Sometimes following tile graphics are miscellaneous graphics, that cannot be handled by the other chunk types. For example, Keen 4-6 has two misc graphics files used for the 'COOMANDER KEEN terminator text' intro. These graphics usually have unique formats that fit their function and few have been investigated.

==== Keen 4-6 Intro Bitmaps ====

This is the text that displays after the 'Ready, press any key' screen at Keen startup. It is composed of a monochrome bitmap that is scrolled across the screen and distorted for special effects. When displayed it is transparent, with a special palette so it is not black-and-white. It consists of a header followed by RLE compressed monochrome data.

HEADER:
0 2 Img height Height of the image in pixels
2 2 Img width Width of the image in pixels
4 2x Line point Pointers to line 1,2,3..etc of data. There will be [Img height] of
these, each 2 bytes long. The first pointer will have the value
(2 * [Img height] + 4)
+2 ? RLE data RLE-WM compressed data

DATA:
0 2 Black run Number of black pixels to write
2 2 Not-blk run Number of not-black pixels to write
4 2 Black run....
...
..
.
? 2 End $FFFF; end of row.

=== Game texts ===

Also known as ANSII chunks, these are used by the game for things like help screens, and have their own format. In the main they are simply text documents, interspersed with various commands to make things happen. Each file is divided up into a number of 'pages' which are moved between by pressing the up/down arrows. Pages can also have action sequences (Repeated each time the page is moved to, you cannot leave a page until the sequence finishes.)

^P First command in every file. Defines a page start
^E Ends the file
^Cx Change font color to $x until next page or ^C command
^Gx,y,z Display (unmasked) picture chunk z at location x,y (In pixels)
^Tx,y,z,t Display picture chunk z at x,y for z clicks of time
^Bx,y,z,t,b ?
^Lx,y ?

=== Misc files ===

The last few chunks are demo files and occasionally miscellaneous data. Demo file formats are currently unknown but can be recorded using the demo cheat. They contain the level number and difficulty followed by a series of key commands.

= Utilities =
* [[KeenWiki:ModKeen|ModKeen]] (DOS) and [[KeenWiki:LModKeen|LModKeen]] (Linux/Windows) can be used to modify graphics and text stored in EGAGRAPH.CKx for the Commander Keen [[KeenWiki:Goodbye Galaxy|Goodbye Galaxy]] trilogy.

[[Category:Catacomb 3D]]
[[Category:Commander Keen 4-6]]
[[Category:Commander Keen Dreams]]
[[Category:Dangerous Dave 3]]
[[Category:File Formats]]
[[Category:Graphics Files]]

EGAGraph Format

2011-09-01T05:24:15Z

CarlKenner: /* EGA Head (EGAHEAD.xxx) */ Details about internal files and Catacomb 3D,

{{Stub}}

'''EGAGraph''', short for 'EGA graphics library' is the format used to store graphics, demos, fonts, game texts and more in many early [[:Category:Id Software|id Software]] games.

The basic format is that of a number of [[Huffman Compression]] compressed sub-files (Chunks) stored in a similar manner to the [[AudioT Format]]. Like this format it has three main files, <tt>EGAGRAPH</tt>, <tt>EGAHEAD</tt>and <tt>EGADICT</tt>.

It is notable that while the <tt>EGAHEAD</tt> and <tt>EGADICT</tt> are stored in the game executable, they need to be extracted for [[TED5]] to edit levels.

This article may contain errors as some formats have not been worked out completely.

= File formats =

== EGA Head (EGAHEAD.xxx) ==
There are two versions of this format. The first version, used in [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], is an array of 4-byte little-endian values. The second version, used in all later EGA games, is an array of 3-byte unsigned little-endian values.

This header file may be present as an external file (eg. [[Bio Menace]]), or it may be included in the EXE file (eg. [[Catacomb 3D]]). The last entry in the header is always the size of the EGAGRAPH.xxx file, and the first entry in the header is always 00 00 00. To find the end, you can search for filesize(EGAGRAPH.xxx) as a 3-byte (little-endian) value in the EXE. Then to find the start, go back 3 bytes at a time until you get to 00 000 00. Or use the fact that entries always get smaller as you go back, unless the entry is $FFFFFF.

In [[Catacomb 3D]] v1.00 EGAHEAD.C3D is at $1BFD00 from the start of the decompressed ([[unlzexe]]) EXE file, and is 1436 bytes long, or 478 3-byte entries (for 477 chunks). It is then followed by 4 bytes (padding to multiple of 16?) 03 00 00 00, then the MAPHEAD.C3D.

This header file stores the offsets (relative to the start of the <tt>EGAGRAPH</tt> file) for each sub file. The format is trivial, in that the header file is simply an array of 3 or 4 byte variables. (Each variable is a "slot", as the game refers to files by index.) Most games use the more compact 3-byte variables, since the <tt>EGAGRAPH</tt> file is never larger than $FFFFFF. However games based on older engines, notably [[Commander Keen Dreams]], [[Dangerous Dave 3]] and [[Dangerous Dave 4]], use 4-byte vales as these are easier to deal with.

It is important to note that not every 'slot' will be in use. Tiles especially if blank (All black) will be considered 'empty' and not worth adding to the file. (A blank 16x16 tile takes up 12 bytes of space if compressed, and there can be hundreds.) There are two ways of dealing with this; by default the games set all empty slot headers to -1, but many programs compress the graphics anyway and use a normal header value for them.

The last offset in the file will be an offset to the end of the main <tt>EGAGRAPH</tt> file. (So this should be ignored when reading the file to prevent a zero-byte file appearing last.) The total size of the <tt>EGAHEAD</tt> divided by 3 (Or 4) will also give the number of 'slots' the game uses. This varies, but is usually about 10,000.

Unlike [[AudioT Format]] files, <tt>EGAGRAPH</tt> is always compressed. However the header and dictionary may be external (E.g [[Bio Menace]])

== Dictionary (EGADICT.xxx) ==

The same [[Huffman Compression]] scheme used elsewhere in the [[Commander Keen]] series is used. When reading compressed files out of the <tt>EGAGRAPH</tt> file, the first four bytes located at the <tt>EGAHEAD</tt> offset are a UINT32LE specifying the file's decompressed size (which is required for the decompression algorithm.) The compressed data then follows, EXCEPT for 16x16 masked and unmasked tiles. Since these all have a similar decompressed size (128 bytes for unmasked, 256 bytes for masked) it is a waste of space to include these (Since tiles make up the bulk of graphics slots.) The decompressed size is hard coded, as are the start and finish of the unmasked and masked tile slots.

Note that the <tt>EGADICT</tt> file is not often obvious, usually being embedded in the main exe file. It is possible to locate it by looking for the string <tt>$FD $01 $00 $00 $00 $00</tt>, which appears at the end of nearly all Huffman dictionaries. Executables usually contain two or more dictionaries, but the <tt>EGADICT</tt> is usually the second one, except in the case of early games like Keen Dreams, where it is the first of THREE. (A simple check of whether it decompresses the data sensibly works.)

== Main file (EGAGRAPH.xxx) ==

This file is simply an array of data files. Each file starts at the offset specified in the <tt>EGAHEAD</tt> file, and as there are no filenames each file is referred to by its index/slot number. Slots containing dummy values in the header are treated as if they don't exist in the <tt>EGAGRAPH</tt> file. Each file within the <tt>EGAGRAPH</tt> file is individually compressed. A file can be read by opening the <tt>EGAGRAPH</tt> file, seeking to the offset specified in the <tt>EGAHEAD</tt> file, reading a UINT32LE for the decompressed file size, and then decompressing the data from that point onwards using standard Huffman decompression techniques.

Note that in early games such as Keen Dreams, the <tt>EGAGRAPH</tt> file may have another name. (Such as <tt>KDREAM.EGA</tt> in that case.) in which case it is always the largest non-executable file.

=== Chunks ===

Each sub file or chunk is a separate graphic, ordered in the <tt>EGAGRAPH</tt> file in a specific way, much again like the [[AudioT Format]]. Chunks are labeled starting at 0. The game executable is hard coded for the start and finish of various chunk types. For example, Keen 4 has chunks 124-520 as sprites and will treat any chunk between these two numbers as a sprite. (Note that the game refers to chunk number, not sprite number.) The usual order of chunks is:

Picture table
Masked picture table
Sprite table
Fonts
Pictures (Unmasked bitmaps)
Masked pictures
8x8 unmasked tiles (Single chunk)
8x8 masked tiles (Single chunk)
16x16 unmasked tiles
16x16 unmasked tiles
32x32 unmasked tiles (Optional)
32x32 masked tiles (Optional)
Misc graphics (Optional)
Game texts
Demo files (Optional)
Misc data (Optional)

== Chunk formats ==

When compressed each chunk consists of a dword giving the decompressed size followed by the compressed data.

=== Picture table ===

Always present and always the first chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying picture chunks.

=== Masked picture table ===

Always present and always the second chunk, the picture table is 4 * numpics bytes long, consisting of two words for each picture (Width\8, Height) used when displaying masked picture chunks. This table is often quite small.

=== Sprite table ===

Always present and always the third chunk, the picture table is 16 * numspr bytes long, consisting of eight words for each sprite image (Width\8, Height, x offset, y offset and clipping rectangle left, top, right and bottom; all in that order.) used when displaying sprite chunks. This is what Modkeen exports as its notoriously hard to use xSPRITES.txt file.

=== Fonts ===

Always present and always starting at the fourth chunk. (Chunk 3!) Each chunk is a single monochrome font, containing all the entries from 0-255 for that font. Each 'letter' is stored separately and consecutively. A game will have at least one, and usually 3 fonts. The format is version two of [[EGA Font format]]

Fonts are used in the game for game texts. They can be stretched or colored and are always transparent, meaning they look strange when not over a single-color background.

=== Pictures (Bitmaps) ===

These follow game fonts. Bitmaps are stored as standard [[Raw EGA data]], the size of each plane being the decompressed chunk size \ 4. In order to be displayed properly these need information from chunk 0, the picture table. Note that unlike fonts, data does not wrap, so a 4x4 sprite will consist of four planes of four bytes in size, the same as an 8x4 sprite.

Pictures are used for things such as title screens or pictures in game texts. Most of the main menu will consist of pictures.

=== Masked pictures (Masked bitmaps) ===

These are identical to the picture chunks in every respect, except being masked, they consist of five EGA planes, not four. Some games (E.g [[Catacomb 3D]] make great use of masked pictures, but most games have only one or two. They need information from chunk 1, the masked picture table, to be displayed correctly.

=== Sprites ===

Sprites are exactly identical to masked pictures in format except the game cannot stretch or warp them. Most games use these for enemy, player or item graphics during gameplay. They need information from chunk 2, the sprite table both to display correctly and to interact correctly, something more complex than masked pictures. There are usually several hundred sprite chunks.

=== 8x8 tiles ===

8x8 tiles are used by most games in status windows or foe the borders of message windows in-game. (And occasionally by TED5 to display levels.) There are thus not many of them. ALL the masked or unmasked tiles are one single chunk stored as four and five plane [[Raw EGA data]] respectively. The number of 8x8 unmasked and masked tiles is hard-coded into the executable, but can be worked out by dividing the decompressed chunk size by 32 and 40 (The size of one tile's data) respectively.

=== 16x16 and 32x32 tiles ===

32x32 tiles are optional and rarely used. 16x16 tiles are only occasionally absent and for most games make up the bulk of graphics. There are usually several thousand entries. Both come in masked and unmasked kinds, just like 8x8 tiles.

These chunks have notable differences. Each tile is an individual chunk and stored as four or five plane [[Raw EGA data]] with a plane size of 128 (16x16) or 512 (32x32). These chunks do NOT have a dword specifying their decompressed size as this is hard-coded into the executable as a space saving measure. (As is the start and finish of unmasked and masked tile chunks.) This can cause problems for editing programs.

Tiles make up the bulk of 2D game levels but are seldom seen elsewhere.

=== Misc graphics ===

Sometimes following tile graphics are miscellaneous graphics, that cannot be handled by the other chunk types. For example, Keen 4-6 has two misc graphics files used for the 'COOMANDER KEEN terminator text' intro. These graphics usually have unique formats that fit their function and few have been investigated.

==== Keen 4-6 Intro Bitmaps ====

This is the text that displays after the 'Ready, press any key' screen at Keen startup. It is composed of a monochrome bitmap that is scrolled across the screen and distorted for special effects. When displayed it is transparent, with a special palette so it is not black-and-white. It consists of a header followed by RLE compressed monochrome data.

HEADER:
0 2 Img height Height of the image in pixels
2 2 Img width Width of the image in pixels
4 2x Line point Pointers to line 1,2,3..etc of data. There will be [Img height] of
these, each 2 bytes long. The first pointer will have the value
(2 * [Img height] + 4)
+2 ? RLE data RLE-WM compressed data

DATA:
0 2 Black run Number of black pixels to write
2 2 Not-blk run Number of not-black pixels to write
4 2 Black run....
...
..
.
? 2 End $FFFF; end of row.

=== Game texts ===

Also known as ANSII chunks, these are used by the game for things like help screens, and have their own format. In the main they are simply text documents, interspersed with various commands to make things happen. Each file is divided up into a number of 'pages' which are moved between by pressing the up/down arrows. Pages can also have action sequences (Repeated each time the page is moved to, you cannot leave a page until the sequence finishes.)

^P First command in every file. Defines a page start
^E Ends the file
^Cx Change font color to $x until next page or ^C command
^Gx,y,z Display (unmasked) picture chunk z at location x,y (In pixels)
^Tx,y,z,t Display picture chunk z at x,y for z clicks of time
^Bx,y,z,t,b ?
^Lx,y ?

=== Misc files ===

The last few chunks are demo files and occasionally miscellaneous data. Demo file formats are currently unknown but can be recorded using the demo cheat. They contain the level number and difficulty followed by a series of key commands.

= Utilities =
* [[KeenWiki:ModKeen|ModKeen]] (DOS) and [[KeenWiki:LModKeen|LModKeen]] (Linux/Windows) can be used to modify graphics and text stored in EGAGRAPH.CKx for the Commander Keen [[KeenWiki:Goodbye Galaxy|Goodbye Galaxy]] trilogy.

[[Category:Catacomb 3D]]
[[Category:Commander Keen 4-6]]
[[Category:Commander Keen Dreams]]
[[Category:Dangerous Dave 3]]
[[Category:File Formats]]
[[Category:Graphics Files]]

Super 3-D Noah's Ark

2011-09-01T04:07:10Z

CarlKenner: Made like Wolfenstein 3-D page

{{Stub}}

This is the PC version of the game '''Super 3D Noah's Ark''' by Wisdom Tree Games. It uses a modified (licensed) version of the [[Wolfenstein 3-D]] engine, with added support for textured floors and ceilings. Super 3D Noah's Ark's homepage is at [http://www.wisdomtreegames.com/games/super3dnoahsark/ http://www.wisdomtreegames.com/games/super3dnoahsark/], where it is still for sale.

{{GamePage|Noah's Ark 3D}}
== File formats ==

This section lists the major file formats used in the game.

* [[AudioT Format]] - sounds and music are stored in the <tt>AUDIO*.N3D</tt> files
* WLF Format - identical to the [[IMF Format]] only with a faster tempo. (720 vs 560)
* [[GameMaps Format]] - used for storing level maps in the files <tt>GAMEMAPS.N3D</tt> and <tt>MAPHEAD.N3D</tt>
* [[B800 Text]] - there are a couple of text screens in the main .EXE file (including the text displayed on the screen when you quit)

== Tools ==

These tools can be used to edit this game.

* [[TED5]] - the editor used by id Software to create the original levels

== Further information ==

The source code of Wolfenstein 3-D was released by id Software on July 21, 1995. It can be downloaded from id Software's [ftp://ftp.idsoftware.com/idstuff/source/ FTP server].

[[Category:Apogee]]
[[Category:id Software]]
[[Category:3D]]
[[Category:First-Person Shooter]]
[[Category:Noah's Ark 3D]]
[[Category:Wolfenstein 3-D]]

Super 3-D Noah's Ark

2011-09-01T03:59:42Z

CarlKenner: Started stub

This is the PC version of the game '''Super 3D Noah's Ark''' by Wisdom Tree Games. It uses a modified (licensed) version of the [[Wolfenstein 3-D]] engine, with added support for textured floors and ceilings. Super 3D Noah's Ark's homepage is at [http://www.wisdomtreegames.com/games/super3dnoahsark/ http://www.wisdomtreegames.com/games/super3dnoahsark/], where it is still for sale.

{{Stub}}

[[Category:Noah's Ark 3D]]

Category:Super 3-D Noah's Ark

2011-09-01T03:55:10Z

CarlKenner: Created category page

These pages are relevant to the PC version of the game '''Super 3D Noah's Ark''' by Wisdom Tree Games. It uses a modified version of the [[Wolfenstein 3-D]] engine. Super 3D Noah's Ark's homepage is at [http://www.wisdomtreegames.com/games/super3dnoahsark/ http://www.wisdomtreegames.com/games/super3dnoahsark/].

AudioT Format

2011-09-01T03:48:19Z

CarlKenner: Noah's Ark 3D category

The <b>AudioT</b> format is a [[:Category:Group Files|group file]] used by many id/Apogee games to store Adlib music, Adlib sound effects and PC Speaker sound effects in a single file. The basic structure of this format is very similar to the [[EGAGraph_Format]] and the [[GameMaps Format]]. The main audio file can be compressed (<tt>AUDIO.xxx</tt>) or uncompressed (<tt>AUDIOT.xxx</tt>). It requires an associated <tt>AUDIOHED.xxx</tt> which stores the offsets of each file, and an <tt>AUDIODCT.xxx</tt> if the file is compressed.

== File formats ==

=== Audio Head (AUDIOHED.xxx) ===

This header file stores the offsets (relative to the start of the <tt>AUDIO/AUDIOT</tt> file) for each sub file. The format is trivial, in that the header file is simply an array of <tt>UINT32LE</tt> offsets.

The last offset in the file will be an offset to the end of the main <tt>AUDIO/AUDIOT</tt> file. This was probably done so that the size of each chunk could be calculated with the formula below. Thus, the number of offsets (size of the <tt>AUDIOHED</tt> in dwords) minus one will give the number of chunks (slots) included in the main file. This number varies, but is usually about 200. (Games refer to the chunks by index, not by name).

The size of a chunk can be calculated like this:

size[i] = offset[i+1] - offset[i]

Empty chunks (size[i] = 0) should be skipped when extracting chunks.

In most (not all) games, the <tt>AUDIOHED</tt> is included in the executable file. See [[Extracting data files from executables]] for information on how to locate and extract them.

=== Dictionary (AUDIODCT.xxx) ===

If the main audio file is named <tt>AUDIO.xxx</tt>, then the file is compressed. The same [[Huffman Compression]] scheme used elsewhere in the [[Commander Keen]] series is used. When reading compressed files out of the <tt>AUDIO</tt> file, the first four bytes located at the <tt>AUDIOHED</tt> offset are a UINT32LE specifying the file's decompressed size (which is required for the decompression algorithm.) The compressed data then follows.

Note that the <tt>AUDIODCT</tt> file is not often obvious, usually being embedded in the main exe file. Executables usually contain two or more dictionaries, but the <tt>AUDIODCT</tt> is usually the first one. (A simple check of whether it decompresses the data sensibly works.)

=== Main file (AUDIOT.xxx, AUDIO.xxx) ===

This file is simply an array of data files. Each file starts at the offset specified in the <tt>AUDIOHED</tt> file, and as there are no filenames each file is referred to by its index/slot number. Slots containing dummy values in the header are treated as if they don't exist in the <tt>AUDIO/AUDIOT</tt> file.

If compression is in use then the <tt>AUDIO</tt> file itself isn't compressed, rather each file within the <tt>AUDIO</tt> file is individually compressed. A file can be read by opening the <tt>AUDIOT</tt> file, seeking to the offset specified in the <tt>AUDIOHED</tt> file, reading a UINT32LE for the decompressed file size, and then decompressing the data from that point onwards using standard Huffman decompression techniques.

If compression is not in use (<tt>AUDIOT</tt> files) then a file can be read by simply seeking to the offset specified in the AUDIOHED file and reading until the offset of the following file is reached. Since the last offset in the <tt>AUDIOHED</tt> file is the size of the <tt>AUDIOT</tt> file itself, no extra handling should be necessary to correctly read all of the last file.

== Chunks ==

The AUDIOT format is a container format, meaning that any files the game can handle can be stored in it. Traditionally there have been three types of 'sub files' or chunks: Adlib music files (which are [[IMF Format]]), Adlib sound effects and PC Speaker sound effects. By default most games store PC effects first, followed by their Adlib equivalents, with the IMF music filling the last entries.

At least one case is known where the music is in MIDI format ([[Noah's Ark 3D]]) and it is possible that soundblaster 'digital' sound effects have been stored like this as well at some point. There has often been confusion about what part of a file are data and which are the AUDIOT file, for example the IMF file format was widely believed to exist in two types until it was discovered that 'type 1' IMFs were simply 'type 0' that had been incorrectly extracted from the AUDIOT file with their chunk length and tags.

It is notable that a sound file will contain exactly as many PC sounds as adlib sounds (and an arbitrary number of IMF files.) This is because most games have a PC\Adlib option and thus must match every PC sound with an adlib equivalent. The number of sounds appears to be hard-coded, though all games (except for [[Commander Keen Dreams]]) seem to add the string "!ID!" after the last sound of each 'section'. The "!ID!" tags themselves don't seem to be part of any chunk and are probably put in for debugging purposes.

Sound chunks are always in the same order, PC, then adlib then IMF. A simple method to classify the sound chunks is to check the <tt>AUDIOHED</tt> for chunks of size 0. These are 'empty' sounds and seperate sound chunks from music chunks . Every nonzero length chunk after this is an IMF chunk. (IMFs are always the last chunks in the <tt>AUDIOHED</tt> and contain no zero length chunks) For the PC\adlib chunks, the number of chunks before the first zero length chunk can be divided into two, the first half being PC, the second half being adlib.

If the file is uncompressed then the adlib and IMF chunks will be tagged (huffman compression, being a space saving exercise doesn't comoress tags.) PC chunks are never tagged. It appears that the compiler used added a footer to each chunk as they were compiled. For sounds these are null terminated strings of 12 characters or less, including terminator. For IMF files these are 88 byte strings detailed on the [[IMF Format]] page. These names are not read by games (with the possible exception of the aforementioned '!ID!' tags.)

TODO: Need to also document both sound effect formats so the sounds can be edited and not just imported/exported.

=== IMF Music ===

[[IMF Format]] files are packed at the end of the audio file. Occasionally 'empty' music files are found (e.g [[Wolfenstein 3-D]]) that contain no actual data, just footer information placed there by a compiler. They can be extracted as they are and played with various utilities such as Adplug.

=== PC Sounds ===

0 4 Length of sound data (Usually chunk length - 7)
4 2 Priority of sound, only 1 PC sound may play at a time; any sound
will interrupt a sound of equal or lower priority.
6 ? Sound data
? 1 Terminating null

PC sounds use a modified form of the [[Inverse Frequency Sound format]] used in earlier ID games. Briefly, the sound data is stored as bytes, each specifying the inverse of a frequency. (Higher value, higher frequency.) This format is properly detailed elsewhere.

While most PC chunks have no sound name and thus can be identified solely by their first four bytes and their length, it appears they can in fact be named and some games occasionally do name them.

These files can be converted into Wave files easily enough, but currently the only program known to do so is Keenwave: http://levellord.toxicsheep.com/KWbeta.zip

TODO: Document this too (Code exists to do this)

=== Adlib Sounds ===

Adlib sounds are slightly more complex. Their header contains additional information to the PC sounds, used to set up the Adlib chip when playing the sound. Games always reset the chip when playing a new sound and so each sound is given 17 bytes of Adlib data. This information is what, in effect, makes the Adlib sounds sound different from the PC ones. by changing these values, various properties of the sound can be altered. Specifically these refer to the values for each of two operators for a single Adlib channel. (Adlib sounds use two sound waves, a carrier and a modulator, C1 and C2, the modulator moves through the carrier, modifying it as it does so.) Adlib sound effects can be converted into IMF format as IMFs are essentially sequences of Adlib sounds on several instead of just one Adlib channel.

An Adlib sound entry is contained in this structure:

{|class="wikitable"
!Data type!!Name!!Description
|-
| [[UINT32LE]] || length || Length of sound data
|-
| [[UINT16LE]] || priority || Sound priority
|-
| [[BYTE]][] || data || Actual sound data
|-
| [[UINT8]] || terminator || ?
|-
| [[ASCIIZ]][] || name || NULL-terminated sound effect name (optional)
|}

The format of the <tt>data</tt> field is described on the [[Adlib sound effect]] page.

== Utilities ==

[http://www.winamp.com/player Download Winamp] a program capable of playing IMF files with the right library.

KeenWave: This program can extract, and to a limited extent, edit the AUDIOT files into various formats, including wave files. It is currently at the beta stage. http://levellord.toxicsheep.com/KWbeta.zip It also contains:

KeenISF: This program plays the Adlib sounds, if they are extracted as raw data.

[[Category:Adlib]]
[[Category:Audio Files]]
[[Category:File Formats]]
[[Category:Music Files]]
[[Category:Sound Files]]
[[Category:Compressed Files]]
[[Category:Huffman Compression]]


[[Category:Bio Menace]]
[[Category:Catacomb 3D]]
[[Category:Commander Keen 4-6]]
[[Category:Noah's Ark 3D]]
[[Category:Wolfenstein 3-D]]

AudioT Format

2011-09-01T03:46:16Z

CarlKenner: /* Chunks */ Noah's Ark 3D

AudioT Format

2011-09-01T03:43:17Z

CarlKenner: /* IMF Music */

The <b>AudioT</b> format is a [[:Category:Group Files|group file]] used by many id/Apogee games to store Adlib music, Adlib sound effects and PC Speaker sound effects in a single file. The basic structure of this format is very similar to the [[EGAGraph_Format]] and the [[GameMaps Format]]. The main audio file can be compressed (<tt>AUDIO.xxx</tt>) or uncompressed (<tt>AUDIOT.xxx</tt>). It requires an associated <tt>AUDIOHED.xxx</tt> which stores the offsets of each file, and an <tt>AUDIODCT.xxx</tt> if the file is compressed.

== File formats ==

=== Audio Head (AUDIOHED.xxx) ===

This header file stores the offsets (relative to the start of the <tt>AUDIO/AUDIOT</tt> file) for each sub file. The format is trivial, in that the header file is simply an array of <tt>UINT32LE</tt> offsets.

The last offset in the file will be an offset to the end of the main <tt>AUDIO/AUDIOT</tt> file. This was probably done so that the size of each chunk could be calculated with the formula below. Thus, the number of offsets (size of the <tt>AUDIOHED</tt> in dwords) minus one will give the number of chunks (slots) included in the main file. This number varies, but is usually about 200. (Games refer to the chunks by index, not by name).

The size of a chunk can be calculated like this:

size[i] = offset[i+1] - offset[i]

Empty chunks (size[i] = 0) should be skipped when extracting chunks.

In most (not all) games, the <tt>AUDIOHED</tt> is included in the executable file. See [[Extracting data files from executables]] for information on how to locate and extract them.

=== Dictionary (AUDIODCT.xxx) ===

If the main audio file is named <tt>AUDIO.xxx</tt>, then the file is compressed. The same [[Huffman Compression]] scheme used elsewhere in the [[Commander Keen]] series is used. When reading compressed files out of the <tt>AUDIO</tt> file, the first four bytes located at the <tt>AUDIOHED</tt> offset are a UINT32LE specifying the file's decompressed size (which is required for the decompression algorithm.) The compressed data then follows.

Note that the <tt>AUDIODCT</tt> file is not often obvious, usually being embedded in the main exe file. Executables usually contain two or more dictionaries, but the <tt>AUDIODCT</tt> is usually the first one. (A simple check of whether it decompresses the data sensibly works.)

=== Main file (AUDIOT.xxx, AUDIO.xxx) ===

This file is simply an array of data files. Each file starts at the offset specified in the <tt>AUDIOHED</tt> file, and as there are no filenames each file is referred to by its index/slot number. Slots containing dummy values in the header are treated as if they don't exist in the <tt>AUDIO/AUDIOT</tt> file.

If compression is in use then the <tt>AUDIO</tt> file itself isn't compressed, rather each file within the <tt>AUDIO</tt> file is individually compressed. A file can be read by opening the <tt>AUDIOT</tt> file, seeking to the offset specified in the <tt>AUDIOHED</tt> file, reading a UINT32LE for the decompressed file size, and then decompressing the data from that point onwards using standard Huffman decompression techniques.

If compression is not in use (<tt>AUDIOT</tt> files) then a file can be read by simply seeking to the offset specified in the AUDIOHED file and reading until the offset of the following file is reached. Since the last offset in the <tt>AUDIOHED</tt> file is the size of the <tt>AUDIOT</tt> file itself, no extra handling should be necessary to correctly read all of the last file.

== Chunks ==

The AUDIOT format is a container format, meaning that any files the game can handle can be stored in it. Traditionally there have been three types of 'sub files' or chunks: Adlib music files (which are [[IMF Format]]), Adlib sound effects and PC Speaker sound effects. By default most games store PC effects first, followed by their Adlib equivalents, with the IMF music filling the last entries.

At least on case is known where the music is in MIDI format (Noah's Ark 3D) and it is possible that soundblaster 'digital' sound effects have been stored like this as well at some point. There has often been confusion about what part of a file are data and which are the AUDIOT file, for example the IMF file format was widely believed to exist in two types until it was discovered that 'type 1' IMFs were simply 'type 0' that had been incorrectly extracted from the AUDIOT file with their chunk length and tags.

It is notable that a sound file will contain exactly as many PC sounds as adlib sounds (and an arbitrary number of IMF files.) This is because most games have a PC\Adlib option and thus must match every PC sound with an adlib equivalent. The number of sounds appears to be hard-coded, though all games (except for [[Commander Keen Dreams]]) seem to add the string "!ID!" after the last sound of each 'section'. The "!ID!" tags themselves don't seem to be part of any chunk and are probably put in for debugging purposes.

Sound chunks are always in the same order, PC, then adlib then IMF. A simple method to classify the sound chunks is to check the <tt>AUDIOHED</tt> for chunks of size 0. These are 'empty' sounds and seperate sound chunks from music chunks . Every nonzero length chunk after this is an IMF chunk. (IMFs are always the last chunks in the <tt>AUDIOHED</tt> and contain no zero length chunks) For the PC\adlib chunks, the number of chunks before the first zero length chunk can be divided into two, the first half being PC, the second half being adlib.

If the file is uncompressed then the adlib and IMF chunks will be tagged (huffman compression, being a space saving exercise doesn't comoress tags.) PC chunks are never tagged. It appears that the compiler used added a footer to each chunk as they were compiled. For sounds these are null terminated strings of 12 characters or less, including terminator. For IMF files these are 88 byte strings detailed on the [[IMF Format]] page. These names are not read by games (with the possible exception of the aforementioned '!ID!' tags.)

TODO: Need to also document both sound effect formats so the sounds can be edited and not just imported/exported.

=== IMF Music ===

[[IMF Format]] files are packed at the end of the audio file. Occasionally 'empty' music files are found (e.g [[Wolfenstein 3-D]]) that contain no actual data, just footer information placed there by a compiler. They can be extracted as they are and played with various utilities such as Adplug.

=== PC Sounds ===

0 4 Length of sound data (Usually chunk length - 7)
4 2 Priority of sound, only 1 PC sound may play at a time; any sound
will interrupt a sound of equal or lower priority.
6 ? Sound data
? 1 Terminating null

PC sounds use a modified form of the [[Inverse Frequency Sound format]] used in earlier ID games. Briefly, the sound data is stored as bytes, each specifying the inverse of a frequency. (Higher value, higher frequency.) This format is properly detailed elsewhere.

While most PC chunks have no sound name and thus can be identified solely by their first four bytes and their length, it appears they can in fact be named and some games occasionally do name them.

These files can be converted into Wave files easily enough, but currently the only program known to do so is Keenwave: http://levellord.toxicsheep.com/KWbeta.zip

TODO: Document this too (Code exists to do this)

=== Adlib Sounds ===

Adlib sounds are slightly more complex. Their header contains additional information to the PC sounds, used to set up the Adlib chip when playing the sound. Games always reset the chip when playing a new sound and so each sound is given 17 bytes of Adlib data. This information is what, in effect, makes the Adlib sounds sound different from the PC ones. by changing these values, various properties of the sound can be altered. Specifically these refer to the values for each of two operators for a single Adlib channel. (Adlib sounds use two sound waves, a carrier and a modulator, C1 and C2, the modulator moves through the carrier, modifying it as it does so.) Adlib sound effects can be converted into IMF format as IMFs are essentially sequences of Adlib sounds on several instead of just one Adlib channel.

An Adlib sound entry is contained in this structure:

{|class="wikitable"
!Data type!!Name!!Description
|-
| [[UINT32LE]] || length || Length of sound data
|-
| [[UINT16LE]] || priority || Sound priority
|-
| [[BYTE]][] || data || Actual sound data
|-
| [[UINT8]] || terminator || ?
|-
| [[ASCIIZ]][] || name || NULL-terminated sound effect name (optional)
|}

The format of the <tt>data</tt> field is described on the [[Adlib sound effect]] page.

== Utilities ==

[http://www.winamp.com/player Download Winamp] a program capable of playing IMF files with the right library.

KeenWave: This program can extract, and to a limited extent, edit the AUDIOT files into various formats, including wave files. It is currently at the beta stage. http://levellord.toxicsheep.com/KWbeta.zip It also contains:

KeenISF: This program plays the Adlib sounds, if they are extracted as raw data.

[[Category:Adlib]]
[[Category:Audio Files]]
[[Category:File Formats]]
[[Category:Music Files]]
[[Category:Sound Files]]
[[Category:Compressed Files]]
[[Category:Huffman Compression]]


[[Category:Bio Menace]]
[[Category:Catacomb 3D]]
[[Category:Commander Keen 4-6]]
[[Category:Wolfenstein 3-D]]

Carmack compression

2011-09-01T03:37:58Z

CarlKenner: More games that use it, categories

[[Carmack compression]] is used in the [[GameMaps Format|GAMEMAPS file]] in [[Commander Keen 4-6]], [[Catacomb 3D]], [[Wolfenstein 3D]], and [[Noah's Ark 3D]] to further shrink the levels down beyond what [[RLEW compression]] can achieve. Its basic idea is somewhat like LZ (Lempel-Ziv) compression in that it contains pointers back to previous data.

As in RLEW compression, the first word in the Carmack compressed data is the number of bytes (not words) in the decompressed data. This is typically the number of bytes in the compressed RLEW data, as Carmack compression is performed after RLEW compression.

Carmack compression contains two types of references to previous data: near pointers and far pointers.

= Near Pointers =

Near pointers occupy three bytes in the compressed data. The first is the number of words in the referenced sequence, the second is the signal byte of xA7, and the third is the number of words to the start of the reference (counting backwards from the current location). As a concrete example, the three bytes x05 xA7 x0A mean 'repeat the 5 words starting 10 words ago'.

Notice that near pointers only let one refer to the last 255 words. To refer to sequences further back, one must use far pointers.

= Far Pointers =

Far pointers occupy four bytes in the compressed data. The first is, again, the number of words in the referenced sequence, the second is xA8, and the third and fourth are interpreted as a word - a 0-based pointer to the start of the reference (in words). As a concrete example, the four bytes x10 xA8 x01 x20 mean 'repeat the 16 words starting at word number 513'.

= Words with a high byte of $A7 or $A8 =

Words whose high (second) byte is xA7 or xA8 would appear to be issue, as they would be confused with near or far pointers. These are handled by representing them as the three bytes: $00, $Ax $xx, this is recognized as an exception (Repeating zero words would make no sense.)

[[Category:File Formats]]
[[Category:Compressed Files]]
[[Category:Compression Algorithms]]
[[Category:Commander Keen 4-6]]
[[Category:Catacomb 3D]]
[[Category:Wolfenstein 3D]]
[[Category:Noah's Ark 3D]]

GameMaps Format

2011-08-31T16:20:54Z

CarlKenner: /* Location of internal files */

The GameMaps format is how the levels in a number of [[Id Software]] games is stored. The filenames and compression varies somewhat across different games but all files stored in this format were produced by the program [[TED5]] or a close analogue.

There are three main varieties of the GameMaps format, uncompressed, [[Carmack compression||carmackized]], and [[Huffman Compression|huffman compressed]]. Each variation has its own file naming scheme and pattern of external\internal files.

There are two main components to the format, the game maps proper, which contain the game level data and the map headers, which contain both the locations of the start of level data in the game maps file and the tile info for the game.

== Structure of uncompressed data ==

The uncompressed format is used by several games including [[Bio Menace]] and [[Wolfenstein 3D]]. In this format map data is stored in <tt>MAPTEMP.xxx<tt> and map headers in <tt>MAPTHEAD.xxx</tt> this is the format saved by [[TED5]] when maps are being edited and can be directly accessed and edited by this utility, allowing changes to be made to the game.

Note that filenames differ and that Wolfenstein 3D uses <tt>GAMEMAPS.WLx</tt> and <tt>MAPHEAD.xxx</tt> files to store data. The main indicator of this format being used is that in the unaltered game the map header file is external.

Only the latest Id Software games lacked compression, when file size was no longer an issue.

=== Map headers ===

0 2 Magic word Magic word for [[RLEW compression]]
2 400 Level ptr 100 4-byte pointers to start of level 0-99 data in game maps file
402 ??? Tileinfo Tileinfo data

The map header file is of varying length and contains three main types of data. The first is the magic word or flag used for RLEW compression, it is always $ABCD. The second is 100 level pointers which give the location of the start of level data in the game maps file, relative to the start of that file. A value of 0 indicates no level. The third is the tileinfo data, tile properties for each tile used in level creation. (These are masked and unmasked and either 8x8, 16x16 or 32x32)

Many programs treat the tileinfo as a separate file from the maphead and it is possible to modify a game in this manner. Indeed, some games, such as Wolfenstein 3D do not have any tileinfo data at all in the map header file (Giving a total file length of 402 bytes.) However TED5 works with any tileinfo data in the maphead.

=== Game maps ===

The game maps file consists of the string 'TED5v1.0.' and a number of RLEW compressed chunks of varying length. Each level in GameMaps will have from two to four chunks (usually four), depending on game, with all levels in a given game having the same number of chunks. These are the level header and 1-3 planes (Foreground, background and sprite or info.) The chunks are in no particular order and it is possible to read through the entire file decompressing chunks as they're found.

Chunks are ordered by the map header file, which will point to the level header chunks which in turn contain pointers to the other chunks used by that level. It is possible the string at the start of the file is to avoid the possibility of a level header at the start of the file which would be read as not existing.

All data in game maps file is in the form of words (Or in the case of pointers, dwords.)

=== Level headers ===

The header for each level that is pointed to by MAPHEAD is 42 bytes long and RLEW compressed, though this is difficult to see since the data is so short and nonrepetitive. For the offsets to level planes a value of 0 indicates the plane does not exist.

Plane 1 is background using unmasked tiles, plane 2 is foreground and uses masked tiles. Plane 3 is sprite or info. Levels must contain c background plane and usually an infoplane.

0 4 Offset in GAMEMAPS to beginning of compressed plane 0 data
4 4 Offset in GAMEMAPS to beginning of compressed plane 1 data
8 4 Offset in GAMEMAPS to beginning of compressed plane 2 data
12 2 Length of compressed plane 0 data (in bytes)
14 2 Length of compressed plane 1 data (in bytes)
16 2 Length of compressed plane 2 data (in bytes)
18 2 Width of level (in tiles)
20 2 Height of level (in tiles)
22 16 Internal name for level (used only by editor, not displayed in-game. null-terminated)

== Carmack compression ==

[[Carmack compression]] is the method used to compress later Id Software games, when file size was an issue. It is the most efficient and complex compression method and was created specifically to work with the word structure of the game maps file. The compression is detailed on its [[Carmack compression|own page.]]

Carmackized game maps files are external <tt>GAMEMAPS.xxx</tt> files and the map header is stored internally in the executable. The map header must be extracted and the game maps decompressed before TED5 can access them. TED5 itself can produce carmackized files and external <tt>MAPHEAD.xxx</tt> files. Carmackization does not replace the RLEW compression used in uncompressed data, but compresses this data, that is, the data is doubly compressed.

== Huffman compression ==

Huffman compression was used by earlier versions of TED5 (possibly not TED5 at all.) before carmackization was created. It uses the same method to compress its data as is used by Id Software games to compress their graphics and sounds. Again this compression method works with RLEW compressed data and has its [[Huffman Compression|own page.]]

Huffman compression is easily detected since it works on the bit level and thus disrupts the word structure of the game data. This is easily seen in a hex editor. Compressed data will not contain the string $00 $00 or indeed even $00 very often (In contrast even carmackized data contains both strings hundreds of times.)

There will be two internal files for this format, the map header and the huffman dictionary (which is always the first dictionary in the executable.) The map header format is also slightly different, being 502 bytes long, the extra 100 bytes being the length of the compressed level headers in the game maps data (These can be ignored when decompressing since huffman data can be read until the decompressed level header's fixed size is reached.)

== Location of internal files ==

Game maps are always external, but in the case of compression the map header file is stored internally in the executable. Executables are themselves compressed wither with LZEXE or PKLite. Once decompressed it is a trivial task to find the map header file as it will start with the word $ABCD For level editing purposes only the first 402 (or 502) bytes of the file needs to be extracted, though it is possible to read the maphead file to calculate its length.

The following addresses are in the locations in the decompressed executables for the start of the Map header files. Values are different for different versions.

TODO: Add all known versions of all games.

External files
Biomenace
Blake Stone 2: Planet Strike
Corridor 7 Demo
Noahs Ark 3D
Operation Body Count
Spear of Destiny
Wolfenstein 3D

Internal files:
Catacomb 3-D
Catacomb 4 Abyss
Catacomb 5 Apocalypse
Catacomb 6 Armageddon
Corridor 7 CD (): $30D50 from start of file CORR7CD.EXE (file is not compressed)
Corridor 7 Floppy (): $24BF0 from start of file C7.EXE (file is not compressed)
Keen 4 (Special Demo): (file is PKLite compressed)
Keen 4 (1.4): $24830
Keen 5 (1.4): $25990
Keen 6 (1.4): $25080
(Keen D (1.13): $1FA50)

== Utilities ==

[[TED5]] This program can edit the <tt>GAMEMAPS</tt> format of any games that use it. It is the original editor used to crate these files.

== Notes ==

This format was reverse engineered by Andrew Durdin (adurdin).

The format is used by other games as well, including [[Bio Menace]], [[Blake Stone]], [[Catacomb 3D]], [[Corridor 7]], [[Dangerous Dave 3]], [[Dave Goes Nutz]], [[Noah's Ark 3D]], [[Operation Body Count]], [[Spear of Destiny]], and [[Wolfenstein 3D]].

[[Category:File Formats]]
[[Category:Map Files]]

GameMaps Format

2011-08-31T16:02:28Z

CarlKenner: /* Notes */ More games

The GameMaps format is how the levels in a number of [[Id Software]] games is stored. The filenames and compression varies somewhat across different games but all files stored in this format were produced by the program [[TED5]] or a close analogue.

There are three main varieties of the GameMaps format, uncompressed, [[Carmack compression||carmackized]], and [[Huffman Compression|huffman compressed]]. Each variation has its own file naming scheme and pattern of external\internal files.

There are two main components to the format, the game maps proper, which contain the game level data and the map headers, which contain both the locations of the start of level data in the game maps file and the tile info for the game.

== Structure of uncompressed data ==

The uncompressed format is used by several games including [[Bio Menace]] and [[Wolfenstein 3D]]. In this format map data is stored in <tt>MAPTEMP.xxx<tt> and map headers in <tt>MAPTHEAD.xxx</tt> this is the format saved by [[TED5]] when maps are being edited and can be directly accessed and edited by this utility, allowing changes to be made to the game.

Note that filenames differ and that Wolfenstein 3D uses <tt>GAMEMAPS.WLx</tt> and <tt>MAPHEAD.xxx</tt> files to store data. The main indicator of this format being used is that in the unaltered game the map header file is external.

Only the latest Id Software games lacked compression, when file size was no longer an issue.

=== Map headers ===

0 2 Magic word Magic word for [[RLEW compression]]
2 400 Level ptr 100 4-byte pointers to start of level 0-99 data in game maps file
402 ??? Tileinfo Tileinfo data

The map header file is of varying length and contains three main types of data. The first is the magic word or flag used for RLEW compression, it is always $ABCD. The second is 100 level pointers which give the location of the start of level data in the game maps file, relative to the start of that file. A value of 0 indicates no level. The third is the tileinfo data, tile properties for each tile used in level creation. (These are masked and unmasked and either 8x8, 16x16 or 32x32)

Many programs treat the tileinfo as a separate file from the maphead and it is possible to modify a game in this manner. Indeed, some games, such as Wolfenstein 3D do not have any tileinfo data at all in the map header file (Giving a total file length of 402 bytes.) However TED5 works with any tileinfo data in the maphead.

=== Game maps ===

The game maps file consists of the string 'TED5v1.0.' and a number of RLEW compressed chunks of varying length. Each level in GameMaps will have from two to four chunks (usually four), depending on game, with all levels in a given game having the same number of chunks. These are the level header and 1-3 planes (Foreground, background and sprite or info.) The chunks are in no particular order and it is possible to read through the entire file decompressing chunks as they're found.

Chunks are ordered by the map header file, which will point to the level header chunks which in turn contain pointers to the other chunks used by that level. It is possible the string at the start of the file is to avoid the possibility of a level header at the start of the file which would be read as not existing.

All data in game maps file is in the form of words (Or in the case of pointers, dwords.)

=== Level headers ===

The header for each level that is pointed to by MAPHEAD is 42 bytes long and RLEW compressed, though this is difficult to see since the data is so short and nonrepetitive. For the offsets to level planes a value of 0 indicates the plane does not exist.

Plane 1 is background using unmasked tiles, plane 2 is foreground and uses masked tiles. Plane 3 is sprite or info. Levels must contain c background plane and usually an infoplane.

0 4 Offset in GAMEMAPS to beginning of compressed plane 0 data
4 4 Offset in GAMEMAPS to beginning of compressed plane 1 data
8 4 Offset in GAMEMAPS to beginning of compressed plane 2 data
12 2 Length of compressed plane 0 data (in bytes)
14 2 Length of compressed plane 1 data (in bytes)
16 2 Length of compressed plane 2 data (in bytes)
18 2 Width of level (in tiles)
20 2 Height of level (in tiles)
22 16 Internal name for level (used only by editor, not displayed in-game. null-terminated)

== Carmack compression ==

[[Carmack compression]] is the method used to compress later Id Software games, when file size was an issue. It is the most efficient and complex compression method and was created specifically to work with the word structure of the game maps file. The compression is detailed on its [[Carmack compression|own page.]]

Carmackized game maps files are external <tt>GAMEMAPS.xxx</tt> files and the map header is stored internally in the executable. The map header must be extracted and the game maps decompressed before TED5 can access them. TED5 itself can produce carmackized files and external <tt>MAPHEAD.xxx</tt> files. Carmackization does not replace the RLEW compression used in uncompressed data, but compresses this data, that is, the data is doubly compressed.

== Huffman compression ==

Huffman compression was used by earlier versions of TED5 (possibly not TED5 at all.) before carmackization was created. It uses the same method to compress its data as is used by Id Software games to compress their graphics and sounds. Again this compression method works with RLEW compressed data and has its [[Huffman Compression|own page.]]

Huffman compression is easily detected since it works on the bit level and thus disrupts the word structure of the game data. This is easily seen in a hex editor. Compressed data will not contain the string $00 $00 or indeed even $00 very often (In contrast even carmackized data contains both strings hundreds of times.)

There will be two internal files for this format, the map header and the huffman dictionary (which is always the first dictionary in the executable.) The map header format is also slightly different, being 502 bytes long, the extra 100 bytes being the length of the compressed level headers in the game maps data (These can be ignored when decompressing since huffman data can be read until the decompressed level header's fixed size is reached.)

== Location of internal files ==

Game maps are always external, but in the case of compression the map header file is stored internally in the executable. Executables are themselves compressed wither with LZEXE or PKLite. Once decompressed it is a trivial task to find the map header file as it will start with the word $ABCD For level editing purposes only the first 402 (or 502) bytes of the file needs to be extracted, though it is possible to read the maphead file to calculate its length.

The following addresses are in the locations in the decompressed executables for the start of the Map header files. Values are different for different versions.

TODO: Add all known versions of all games.

External files
Biomenace
Blake Stone 2: Planet Strike
Corridor 7 Demo
Noahs Ark 3D
Operation Body Count
Spear of Destiny
Wolfenstein 3D

Internal files:
Keen 4 (Special Demo): (file is PKLite compressed)
Keen 4 (1.4): $24830
Keen 5 (1.4): $25990
Keen 6 (1.4): $25080
(Keen D (1.13): $1FA50)
Corridor 7 CD (): $30D50 from start of file CORR7CD.EXE (file is not compressed)

== Utilities ==

[[TED5]] This program can edit the <tt>GAMEMAPS</tt> format of any games that use it. It is the original editor used to crate these files.

== Notes ==

This format was reverse engineered by Andrew Durdin (adurdin).

The format is used by other games as well, including [[Bio Menace]], [[Blake Stone]], [[Catacomb 3D]], [[Corridor 7]], [[Dangerous Dave 3]], [[Dave Goes Nutz]], [[Noah's Ark 3D]], [[Operation Body Count]], [[Spear of Destiny]], and [[Wolfenstein 3D]].

[[Category:File Formats]]
[[Category:Map Files]]

GameMaps Format

2011-08-31T15:58:11Z

CarlKenner: /* Location of internal files */ Blake Stone 2, Spear of Destiny

The GameMaps format is how the levels in a number of [[Id Software]] games is stored. The filenames and compression varies somewhat across different games but all files stored in this format were produced by the program [[TED5]] or a close analogue.

There are three main varieties of the GameMaps format, uncompressed, [[Carmack compression||carmackized]], and [[Huffman Compression|huffman compressed]]. Each variation has its own file naming scheme and pattern of external\internal files.

There are two main components to the format, the game maps proper, which contain the game level data and the map headers, which contain both the locations of the start of level data in the game maps file and the tile info for the game.

== Structure of uncompressed data ==

The uncompressed format is used by several games including [[Bio Menace]] and [[Wolfenstein 3D]]. In this format map data is stored in <tt>MAPTEMP.xxx<tt> and map headers in <tt>MAPTHEAD.xxx</tt> this is the format saved by [[TED5]] when maps are being edited and can be directly accessed and edited by this utility, allowing changes to be made to the game.

Note that filenames differ and that Wolfenstein 3D uses <tt>GAMEMAPS.WLx</tt> and <tt>MAPHEAD.xxx</tt> files to store data. The main indicator of this format being used is that in the unaltered game the map header file is external.

Only the latest Id Software games lacked compression, when file size was no longer an issue.

=== Map headers ===

0 2 Magic word Magic word for [[RLEW compression]]
2 400 Level ptr 100 4-byte pointers to start of level 0-99 data in game maps file
402 ??? Tileinfo Tileinfo data

The map header file is of varying length and contains three main types of data. The first is the magic word or flag used for RLEW compression, it is always $ABCD. The second is 100 level pointers which give the location of the start of level data in the game maps file, relative to the start of that file. A value of 0 indicates no level. The third is the tileinfo data, tile properties for each tile used in level creation. (These are masked and unmasked and either 8x8, 16x16 or 32x32)

Many programs treat the tileinfo as a separate file from the maphead and it is possible to modify a game in this manner. Indeed, some games, such as Wolfenstein 3D do not have any tileinfo data at all in the map header file (Giving a total file length of 402 bytes.) However TED5 works with any tileinfo data in the maphead.

=== Game maps ===

The game maps file consists of the string 'TED5v1.0.' and a number of RLEW compressed chunks of varying length. Each level in GameMaps will have from two to four chunks (usually four), depending on game, with all levels in a given game having the same number of chunks. These are the level header and 1-3 planes (Foreground, background and sprite or info.) The chunks are in no particular order and it is possible to read through the entire file decompressing chunks as they're found.

Chunks are ordered by the map header file, which will point to the level header chunks which in turn contain pointers to the other chunks used by that level. It is possible the string at the start of the file is to avoid the possibility of a level header at the start of the file which would be read as not existing.

All data in game maps file is in the form of words (Or in the case of pointers, dwords.)

=== Level headers ===

The header for each level that is pointed to by MAPHEAD is 42 bytes long and RLEW compressed, though this is difficult to see since the data is so short and nonrepetitive. For the offsets to level planes a value of 0 indicates the plane does not exist.

Plane 1 is background using unmasked tiles, plane 2 is foreground and uses masked tiles. Plane 3 is sprite or info. Levels must contain c background plane and usually an infoplane.

0 4 Offset in GAMEMAPS to beginning of compressed plane 0 data
4 4 Offset in GAMEMAPS to beginning of compressed plane 1 data
8 4 Offset in GAMEMAPS to beginning of compressed plane 2 data
12 2 Length of compressed plane 0 data (in bytes)
14 2 Length of compressed plane 1 data (in bytes)
16 2 Length of compressed plane 2 data (in bytes)
18 2 Width of level (in tiles)
20 2 Height of level (in tiles)
22 16 Internal name for level (used only by editor, not displayed in-game. null-terminated)

== Carmack compression ==

[[Carmack compression]] is the method used to compress later Id Software games, when file size was an issue. It is the most efficient and complex compression method and was created specifically to work with the word structure of the game maps file. The compression is detailed on its [[Carmack compression|own page.]]

Carmackized game maps files are external <tt>GAMEMAPS.xxx</tt> files and the map header is stored internally in the executable. The map header must be extracted and the game maps decompressed before TED5 can access them. TED5 itself can produce carmackized files and external <tt>MAPHEAD.xxx</tt> files. Carmackization does not replace the RLEW compression used in uncompressed data, but compresses this data, that is, the data is doubly compressed.

== Huffman compression ==

Huffman compression was used by earlier versions of TED5 (possibly not TED5 at all.) before carmackization was created. It uses the same method to compress its data as is used by Id Software games to compress their graphics and sounds. Again this compression method works with RLEW compressed data and has its [[Huffman Compression|own page.]]

Huffman compression is easily detected since it works on the bit level and thus disrupts the word structure of the game data. This is easily seen in a hex editor. Compressed data will not contain the string $00 $00 or indeed even $00 very often (In contrast even carmackized data contains both strings hundreds of times.)

There will be two internal files for this format, the map header and the huffman dictionary (which is always the first dictionary in the executable.) The map header format is also slightly different, being 502 bytes long, the extra 100 bytes being the length of the compressed level headers in the game maps data (These can be ignored when decompressing since huffman data can be read until the decompressed level header's fixed size is reached.)

== Location of internal files ==

Game maps are always external, but in the case of compression the map header file is stored internally in the executable. Executables are themselves compressed wither with LZEXE or PKLite. Once decompressed it is a trivial task to find the map header file as it will start with the word $ABCD For level editing purposes only the first 402 (or 502) bytes of the file needs to be extracted, though it is possible to read the maphead file to calculate its length.

The following addresses are in the locations in the decompressed executables for the start of the Map header files. Values are different for different versions.

TODO: Add all known versions of all games.

External files
Biomenace
Blake Stone 2: Planet Strike
Corridor 7 Demo
Noahs Ark 3D
Operation Body Count
Spear of Destiny
Wolfenstein 3D

Internal files:
Keen 4 (Special Demo): (file is PKLite compressed)
Keen 4 (1.4): $24830
Keen 5 (1.4): $25990
Keen 6 (1.4): $25080
(Keen D (1.13): $1FA50)
Corridor 7 CD (): $30D50 from start of file CORR7CD.EXE (file is not compressed)

== Utilities ==

[[TED5]] This program can edit the <tt>GAMEMAPS</tt> format of any games that use it. It is the original editor used to crate these files.

== Notes ==

This format was reverse engineered by Andrew Durdin (adurdin).

The format is used by other games as well, including [[Bio Menace]], [[Catacomb 3D]], [[Dangerous Dave 3]], [[Dave Goes Nutz]], and [[Wolfenstein 3D]]

[[Category:File Formats]]
[[Category:Map Files]]

GameMaps Format

2011-08-31T14:44:16Z

CarlKenner: /* Location of internal files */ Corridor 7, Operation Body Count, Keen 4 Special Demo

The GameMaps format is how the levels in a number of [[Id Software]] games is stored. The filenames and compression varies somewhat across different games but all files stored in this format were produced by the program [[TED5]] or a close analogue.

There are three main varieties of the GameMaps format, uncompressed, [[Carmack compression||carmackized]], and [[Huffman Compression|huffman compressed]]. Each variation has its own file naming scheme and pattern of external\internal files.

There are two main components to the format, the game maps proper, which contain the game level data and the map headers, which contain both the locations of the start of level data in the game maps file and the tile info for the game.

== Structure of uncompressed data ==

The uncompressed format is used by several games including [[Bio Menace]] and [[Wolfenstein 3D]]. In this format map data is stored in <tt>MAPTEMP.xxx<tt> and map headers in <tt>MAPTHEAD.xxx</tt> this is the format saved by [[TED5]] when maps are being edited and can be directly accessed and edited by this utility, allowing changes to be made to the game.

Note that filenames differ and that Wolfenstein 3D uses <tt>GAMEMAPS.WLx</tt> and <tt>MAPHEAD.xxx</tt> files to store data. The main indicator of this format being used is that in the unaltered game the map header file is external.

Only the latest Id Software games lacked compression, when file size was no longer an issue.

=== Map headers ===

0 2 Magic word Magic word for [[RLEW compression]]
2 400 Level ptr 100 4-byte pointers to start of level 0-99 data in game maps file
402 ??? Tileinfo Tileinfo data

The map header file is of varying length and contains three main types of data. The first is the magic word or flag used for RLEW compression, it is always $ABCD. The second is 100 level pointers which give the location of the start of level data in the game maps file, relative to the start of that file. A value of 0 indicates no level. The third is the tileinfo data, tile properties for each tile used in level creation. (These are masked and unmasked and either 8x8, 16x16 or 32x32)

Many programs treat the tileinfo as a separate file from the maphead and it is possible to modify a game in this manner. Indeed, some games, such as Wolfenstein 3D do not have any tileinfo data at all in the map header file (Giving a total file length of 402 bytes.) However TED5 works with any tileinfo data in the maphead.

=== Game maps ===

The game maps file consists of the string 'TED5v1.0.' and a number of RLEW compressed chunks of varying length. Each level in GameMaps will have from two to four chunks (usually four), depending on game, with all levels in a given game having the same number of chunks. These are the level header and 1-3 planes (Foreground, background and sprite or info.) The chunks are in no particular order and it is possible to read through the entire file decompressing chunks as they're found.

Chunks are ordered by the map header file, which will point to the level header chunks which in turn contain pointers to the other chunks used by that level. It is possible the string at the start of the file is to avoid the possibility of a level header at the start of the file which would be read as not existing.

All data in game maps file is in the form of words (Or in the case of pointers, dwords.)

=== Level headers ===

The header for each level that is pointed to by MAPHEAD is 42 bytes long and RLEW compressed, though this is difficult to see since the data is so short and nonrepetitive. For the offsets to level planes a value of 0 indicates the plane does not exist.

Plane 1 is background using unmasked tiles, plane 2 is foreground and uses masked tiles. Plane 3 is sprite or info. Levels must contain c background plane and usually an infoplane.

0 4 Offset in GAMEMAPS to beginning of compressed plane 0 data
4 4 Offset in GAMEMAPS to beginning of compressed plane 1 data
8 4 Offset in GAMEMAPS to beginning of compressed plane 2 data
12 2 Length of compressed plane 0 data (in bytes)
14 2 Length of compressed plane 1 data (in bytes)
16 2 Length of compressed plane 2 data (in bytes)
18 2 Width of level (in tiles)
20 2 Height of level (in tiles)
22 16 Internal name for level (used only by editor, not displayed in-game. null-terminated)

== Carmack compression ==

[[Carmack compression]] is the method used to compress later Id Software games, when file size was an issue. It is the most efficient and complex compression method and was created specifically to work with the word structure of the game maps file. The compression is detailed on its [[Carmack compression|own page.]]

Carmackized game maps files are external <tt>GAMEMAPS.xxx</tt> files and the map header is stored internally in the executable. The map header must be extracted and the game maps decompressed before TED5 can access them. TED5 itself can produce carmackized files and external <tt>MAPHEAD.xxx</tt> files. Carmackization does not replace the RLEW compression used in uncompressed data, but compresses this data, that is, the data is doubly compressed.

== Huffman compression ==

Huffman compression was used by earlier versions of TED5 (possibly not TED5 at all.) before carmackization was created. It uses the same method to compress its data as is used by Id Software games to compress their graphics and sounds. Again this compression method works with RLEW compressed data and has its [[Huffman Compression|own page.]]

Huffman compression is easily detected since it works on the bit level and thus disrupts the word structure of the game data. This is easily seen in a hex editor. Compressed data will not contain the string $00 $00 or indeed even $00 very often (In contrast even carmackized data contains both strings hundreds of times.)

There will be two internal files for this format, the map header and the huffman dictionary (which is always the first dictionary in the executable.) The map header format is also slightly different, being 502 bytes long, the extra 100 bytes being the length of the compressed level headers in the game maps data (These can be ignored when decompressing since huffman data can be read until the decompressed level header's fixed size is reached.)

== Location of internal files ==

Game maps are always external, but in the case of compression the map header file is stored internally in the executable. Executables are themselves compressed wither with LZEXE or PKLite. Once decompressed it is a trivial task to find the map header file as it will start with the word $ABCD For level editing purposes only the first 402 (or 502) bytes of the file needs to be extracted, though it is possible to read the maphead file to calculate its length.

The following addresses are in the locations in the decompressed executables for the start of the Map header files. Values are different for different versions.

TODO: Add all known versions of all games.

External files
Biomenace
Corridor 7 Demo
Noahs Ark 3D
Operation Body Count
Wolfenstein 3D

Internal files:
Keen 4 (Special Demo): (file is PKLite compressed)
Keen 4 (1.4): $24830
Keen 5 (1.4): $25990
Keen 6 (1.4): $25080
(Keen D (1.13): $1FA50)
Corridor 7 CD (): $30D50 from start of file CORR7CD.EXE (file is not compressed)

== Utilities ==

[[TED5]] This program can edit the <tt>GAMEMAPS</tt> format of any games that use it. It is the original editor used to crate these files.

== Notes ==

This format was reverse engineered by Andrew Durdin (adurdin).

The format is used by other games as well, including [[Bio Menace]], [[Catacomb 3D]], [[Dangerous Dave 3]], [[Dave Goes Nutz]], and [[Wolfenstein 3D]]

[[Category:File Formats]]
[[Category:Map Files]]

Hovertank 3-D

2011-08-26T14:45:55Z

CarlKenner:

'''Hovertank 3D''' is one of the first FPS games and the direct ancestor of most FPS games.

It has EGA graphics with no textures, but with scaled sprites. Like the later [[Wolfenstein 3D]], this game includes different brightnesses of the same tile when viewed from the N/S or E/W.

It was followed by [[Catacomb 3D]], which added textures.

==Level format==

Levels are in LEVELxx.HOV files. They use [[RLEW compression]] with the tag $FEFE.

{{stub}}
[[Category:Game Intro Page]]

Hovertank 3-D

2011-08-26T14:44:05Z

CarlKenner: Category

'''Hovertank 3D''' is one of the first FPS games and the direct ancestor of most FPS games.

It has EGA graphics with no textures, but with scaled sprites. Like the later [[Wolfenstein 3D]], this game includes

It was followed by [[Catacomb 3D]], which added textures.

==Level format==

Levels are in LEVELxx.HOV files. They use [[RLEW compression]] with the tag $FEFE.

{{stub}}
[[Category:Game Intro Page]]

Hovertank 3-D

2011-08-26T14:42:29Z

CarlKenner: Started stub

'''Hovertank 3D''' is one of the first FPS games and the direct ancestor of most FPS games.

It has EGA graphics with no textures, but with scaled sprites. Like the later [[Wolfenstein 3D]], this game includes

It was followed by [[Catacomb 3D]], which added textures.

==Level format==

Levels are in LEVELxx.HOV files. They use [[RLEW compression]] with the tag $FEFE.

{{stub}}

RLE Compression

2011-08-26T14:29:45Z

CarlKenner:

[[RLE Compression]] is a form of lossless compression and one of the simplest ways of compressing and storing data. It is used, in various forms, by nearly every platform game. The essence of this compression method is to compress data by storing it as a collection of length/value pairs. To take a simple case, the string 'aaaaaaaabbb' could be compressed to '8a3b' (8 bytes of a, 3 bytes of b). The trick with RLE is to make sure the data does not end up becoming ''larger'' after processing (for example, 'abracadabra' could become '1a1b1r1a1c1a1d1a1b1r1a')

Most systems also tell the program beforehand the size of the decompressed data, stored as a word or dword at the start of the data block. This is usually done so that the correct amount of memory can be allocated before the data is processed. In modern programs this is an obvious security risk (it is trivial to [[wp:Buffer overflow|write past the end of the allocated buffer]]) however this was not an issue at the time the games were written.

== Length/value vs Flag Systems ==

There are two main systems of RLE divided by how length is differentiated from value.

In the first system, the first data is assumed to be a length, the next data(s) to be values. So using the simple system above the string '3a3b2c' would be read as 'Length 3 of a, length 3 of b...' Any individual piece of data may be a length or a value and can only be told apart by what type came before it. This system is simplest, but has the disadvantage that the data must be read from the start and that any error will ruin all data that follows it.

In the second system a 'flag' is used. This is a special value that tells a program to do something whenever it is encountered. So the string '*3abcd*3e' could be decompressed to 'aaabcdeee' where '*' is the flag value, indicating that a length/value pair is coming up next. Flags may indicate a number of things (Usually a value/length pair) and be words, bytes or bits (E.g all values over 128 are length bytes, all under value bytes.) This is more complex and is less efficient space-wise than the first system, but has the advantage that the data can usually be read from multiple points and an error may not corrupt all following data. (Since ever flag 'resets' the decompression.)

==== Literals ====

A literal is a piece of data that is uncompressed and should be copied directly to output. If RLE is used to compress something like text for example, there will be a large amount of data (Words) with no pattern that cannot readily be compressed and so would appear 'as-is' in the compressed data.

While simpler systems must encode everything as length/value pairs, flag systems especially assume that all data is literal and just copy it to output until they come across a flag. This saves a lot of space since incompressible data won't increase in size. This is why flag systems are the more common way of compressing data (Especially graphics.) and are more common in older games.

== Compression units ==

RLE compression can occur at several levels, bits, bytes or words (RLEB, RLE, [[RLEW compression |RLEW]]) Most common is at the byte level. The type used often depends on what data is being compressed. For example, most tilesets work on the byte level (1 pixel = 1 byte) and so will be compressed at the byte level, while most levels are made of two byte values for tiles and will be compressed at the word level.

[[RLEW compression]] has its own page which describes how it it commonly implemented.

== Types of RLE ==

* Supplemental data in [[Duke Nukem II Map Format#Supplemental foreground data|Duke Nukem II map files]]
* [[Keen 1-3 RLE compression]] also used by [[Dangerous Dave]] and [[Catacomb]]
* [[Jazz Jackrabbit RLE compression]]
* [[DAT Format (Monster Bash)#RLE|Monster Bash RLE]] used in .DAT file
* [[RLEW compression]]

[[Category:File Formats]]
[[Category:Compressed Files]]

Keen 1-3 RLE compression

2011-08-26T14:27:16Z

CarlKenner: Catacomb usage

[[Commander Keen 1-3]] uses [[RLE compression]] to compress its 320x200 <tt>FINALE.CKx</tt> and <tt>PREVIEWx.CK1</tt> screens. [[Dangerous Dave]] also uses this format to compress its graphics. It was previously used in [[Catacomb|Catacomb I and II]] for the level data (not the graphics). This is one of many possible ways to create a functional RLE system.

The compressed file consists of a UINT32LE giving the decompressed size, followed by the compressed data.

The compressed data is processed in a loop. After each byte is read, its value is used to select an action. If the high bit is set (val & 0x80) then one more than the lower seven bits ((val & 0x7F) + 1) is the number of following bytes passed through unchanged. If the high bit was not set, then that value plus three is the number of times the following byte is repeated.

The next byte is then read and the same procedure followed until there is no more data or the decompressed size in the header has been reached.

An illustration:

80 aa 00 00 81 bb cc 05 dd

This data is interpreted as follows:

80 aa # Copy AA through unchanged
00 00 # Repeat 00 three times
81 bb cc # Copy BB and CC through unchanged
05 dd # Repeat DD eight times

Or as pseudocode:

* Get the initial dword [Final length]
* If [Decompressed data length] < [Final length] then:
*# Read a byte [Value]
*# If the byte value >= 128, then:
*#* Read and output the next [value - 127] bytes
*#* Move forward [Value - 126] bytes and return to step 1.
*# If the byte value < 128, then:
*#* Read the next byte and output it (value + 3) times.
*#* Move forward 2 bytes and return to step 1.

It can be seen that this system is not optimal since two repeating bytes must be treated as non-repeating.

[[Category:Catacomb]]
[[Category:Commander Keen 1-3]]
[[Category:Dangerous Dave]]
[[Category:File Formats]]
[[Category:Compressed Files]]

Catacomb

2011-08-26T14:20:11Z

CarlKenner: Formatting

{{Stub}}

{{GamePage}}

[[Catacomb]] was distributed on the same Gamer's Edge disk as [[Dangerous Dave]].

Catacomb and Catacomb II are top-down Gauntlet-style shooters. They are two of ID software's earliest games.
They are the direct ancestors of most First Person Shooters. They introduced strafing.

They are part of series of 6 games, in two trilogies:
* Catacomb I
* Catacomb II
* [[Catacomb 3-D]]
* [[Catacomb Abyss]]
* [[Catacomb Armageddon]]
* [[Catacomb Apocalypse]]

It evolved like this: [[Catacomb]] + [[Hovertank]] -> [[Catacomb 3-D]] -> [[Catacomb Abyss]] -> [[Wolfenstein 3D]] -> [[Doom]] -> [[Quake]]

== File formats ==

===Game levels===
Game levels are stored in LEVEL1.CAT to LEVEL10.CAT for Catacomb I, and LEVEL1.CA2 to LEVEL30.CA2 for Catacomb II. Note that the number in the level name is not padded, unlike in later games.

Every level is the fixed size 64x64, like the later [[Wolfenstein 3D]]. But there is only one plane of data, and each tile is only one byte.

The first 4 bytes of the level file is the size of the uncompressed data in little-endian format. It is always 4096 (64x64). Following that is the compressed level data, compressed with [[Keen 1-3 RLE compression]]. There is nothing else in the level file.

Here are some possible tile values:
61..7A (lowercase ASCII 'a'..'z') = block with letter A..Z
80 = green floor (purple in CGA)
81 = pink block (white and cyan in CGA)
82 = grey left wall of room (black in CGA)
83 = top wall of room (purple and grey wall in EGA, white bricks in CGA)
84 = bottom right corner of block
85 = top left corner of room
86 = bottom left corner of block
87 = top right corner of block
88 = secret passage
89 = secret passage in left wall
8A = secret passage in top wall

8F = bag pickup (secret)
91 = key pickup (secret)
A2 = bag (not secret)
A3 = scroll
A4 = key pickup
A6 = door
A7 = chest

E7 = portal
E8 = teddy bear
E9 = skeleton
EA = big red/purple monster's top left corner

Levels are in [[Catacomb Map Format]]

===Other===
* Savegames are in [[Catacomb Savegame Format]] ('''Catacomb 2''' only)
* Sounds are in [[Inverse Frequency Sound format]]
* Tile graphics are in [[xGAChars Format]]
* Other graphics are in
** [[xGAPics Format]] for '''Catacomb 2'''
** [[PIC Format]] for '''Catacomb 1'''

In '''Catacomb 1''', all graphics files are stored in the EXE. Their location in the ''uncompressed'' executable is as follows:

Location | Size | Description
-----------+--------+-------------
0x00006100 | 0xB6C0 | EGACHARS.CAT
0x000117D0 | 0x5B60 | CGACHARS.CAT
0x00017997 | 0x4000 | TITLE.PIC
0x0001B997 | 0x4000 | END.PIC

[[Category:Catacomb]]
[[Category:Softdisk]]

Catacomb

2011-08-26T14:18:46Z

CarlKenner: More information