Westwood SHP Format (TS)
The sprite format used in Command & Conquer: Tiberian Sun and Red Alert 2 is a collection of 8-bit frames that all have the same dimensions. The format uses the Westwood RLE-Zero algorithm to compact transparent areas, and uses X and Y offsets to align smaller frame data in the full frame size, but, unlike its predecessors, uses no real data compression like LCW.
An additional 'compression', though applied on game design level rather than on graphics level, is that all separate structure animations in these games are typically stored in separate files. This has the advantages that it both saves space by not including the entire structure graphics in the animation graphics (a problem previously solved by using XOR Delta), and that it gives more freedom for playing multiple looping animations with different amounts of frames on the same structure.
Note that Tiberian Sun and Red Alert 2 are not DOS games. This format is included on this wiki for the sake of completeness in the scope of documenting all indexed graphics formats in the Westwood Studios games, and as logical successor to the Dune II SHP, Lands of Lore SHP and C&C 1 SHP formats.
The file starts with a very simple 8-byte header:
|0x00||UINT16LE||Empty||Always zero. This allows a very simple first check to identify this file type.|
|0x02||UINT16LE||FullWidth||Width of the frames.|
|0x04||UINT16LE||FullHeight||Height of the frames.|
|0x06||UINT16LE||NrOfFrames||Number of frames in the file.|
Frames info table
After the header comes an array with
NrOfFrames entries, where each entry has the following structure:
|0x00||UINT16LE||FrameX||X-offset to align the frame in the full |
|0x02||UINT16LE||FrameY||Y-offset to align the frame in the full |
|0x04||UINT16LE||FrameWidth||Width of the frame image data.|
|0x06||UINT16LE||FrameHeight||Height of the frame image data.|
|0x08||UINT32LE||Flags||A series of bit flags that determine how the game interprets the data. The two flags are HasTransparency (bit 1) and UsesRle (bit 2). Since the RLE compression only collapses transparency, bit 2 should never be enabled if bit 1 isn't.|
|0x0C||BYTE||FrameColor||The first three bytes of this contain R/G/B colour bytes that can be used on the game's minimap. Unlike the game's colour palettes, this uses 8 bits per colour component.|
|0x14||UINT32LE||DataOffset||Offset of the frame's data. If there is no data, this should be 0. In the original files, the data, and thus these offsets, are always aligned to multiples of 8 bytes, though this is technically not necessary. Since the zero-compression algorithm has line lengths in its format, and uncompressed data is exactly |
The flags aren't really decompression options but options to tell the game's blitters how to paint the graphics on the screen. Because of this, it is important to enable the HasTransparency bit if the graphics data contains zero-value bytes, otherwise they may get painted as actual zero-value colours, instead of getting handled as transparency.
Image data decompression
The main way the image data is reduced in size is by saving only a cropped frame, which needs to be realigned in the full frame size using the X and Y offsets. To get the cropped frame, though, the data first needs to be interpreted correctly.
As mentioned, there are two flags that can be set on graphics, and the second should never be enabled if the first one isn't, giving three different types of stored data:
- On fully opaque data, both the HasTransparency and UsesRle flags are disabled.
- On data containing transparency which is not RLE-compressed, the HasTransparency flag is enabled, but the UsesRle flags isn't. Typically, this is only used in case the RLE algorithm gives larger output than the original data, however, the game's mouse cursor graphics are saved without RLE, and will not work correctly if RLE is enabled on them.
- On data containing transparency which is RLE-compressed, both the HasTransparency and UsesRle flags are enabled.
The used compression is the Tiberian Sun variant of the Westwood RLE-Zero algorithm, a flag-based RLE triggering on value 00, which is treated per line, and which has a UINT16LE at the start of each line indicating that line's input data length.
The FrameColor is the average colour of all non-transparent pixels in the frame's image data, when seen in its intended palette, or rather, in the graphics fed into the original encoder used by Westwood. Since reapplying an average-colour algorithm to existing game sprites gives slightly different values than the original ones, it is likely that the original images had 8-bit colour components, and that the process of mapping them to the intended game palette was done by the encoding tool itself.
The main purpose of this colour is to serve as display colour on the game's 'radar' minimap. Most units and structures in the game don't need this, since anything owned by either an AI or human player is displayed on the minimap with the colour of its owner, but it is important to have this for map resources and SHP-based terrain pieces such as bridges.
The original encoder used by Westwood seems to have contained some system to compensate for remappable house colours, however, or had a way to override the house colours to the intended in-game colours, because the frames of the tiberium resource in the game, which is remappable, have green colours set, despite the game using red colours on the remapping range on its colour palette. Likewise, some of the tiberium-based mutated wildlife in the game, which is also typically remapped to green in the game, contains green frame colours that can only be achieved by using green-remapped graphics as source.
It must be noted that, on the tiberium graphics, the game uses a somewhat odd system to change these colours when the tiberium in the game is remapped to a different colour than green. For that reason, it is advised that when editing the tiberium graphics, the original graphics' frame colours are copied exactly, otherwise blue tiberium in the game has a tendency to look strangely purple on the minimap. The TS/RA2 SHP Radar Colour Editor (see Tools) can apply this operation.
Shadow filter area
On the preceding games, unit and structure shadows are typically indicated on the graphics using colour index 4 (though in C&C1 this is technically defined by the game's fading table files), typically coloured bright green #54FC54 on the palettes. Tiberian Sun and Red Alert 2 work differently; the shadows are split off into additional frames behind the normal graphics, and are painted in colour index 1.
Such shadow frames are not a specific part of the format; there are plenty of graphics in the games (like UI graphics) that are not units or structures in the play field that need to cast a shadow, so these frames are not always an expected thing, but if they are present they should be somewhat detectable by the fact there is an even number of frames, and the pixels in the second half of the frames only use palette indices 0 and 1.
Since shadows should never overlap the main graphics, the correct way to combine the frames is to draw the shadows under the main graphics. Most conversion tools convert the shadow data to the classic index 4, for the sake of compatibility, because index 4 is never really used in graphics, and because this index is still set to the clearly visible green colour on most default unit and structure palettes in Tiberian Sun and Red Alert 2.
The following tools are able to work with files in this format.
|Name||Platform||View images in this format?||Convert/export to another file/format?||Import from another file/format?||Access hidden data?||Edit metadata?||Notes|
|Engie File Converter||Windows||Yes||Yes||Yes||No||Has save options to tweak FrameColor.||Generates the FrameColor data from the input frames, and has an option to override the red remap colours in that operation with tiberium-green.|
|XCC Mixer||Windows||Yes||Yes||Yes||No||No||The main conversion tool used by the modding community.|
|OS SHP Builder||Windows||Yes||Yes||Yes||No||No||An actual graphical sprite editor|
|TS/RA2 SHP Radar Colour Editor||Windows||No||No||No||No||Yes||Specific tool for editing the FrameColor data, since most SHP tools have no support for setting it. Can only view the colours set for the SHP frames, not the graphics themselves.|
|OpenRA Utility||Windows, Linux, Mac||No||Yes||No||No||No||Command line tool that can export to png|