Westwood SHP Format (TS)

Westwood SHP Format (TS)

Format type	Tileset
Hardware	VGA
Max tile count	65535
Palette	Shared
Tile names?	No
Minimum tile size (pixels)	1x1
Maximum tile size (pixels)	65535×65535
Plane count	1
Plane arrangement	Linear
Transparent pixels?	Palette-based
Hitmap pixels?	No
Metadata?	None
Supports sub-tilesets?	No
Compressed tiles?	Yes
Hidden data?	Yes
Games	Command & Conquer: Tiberian Sun Command & Conquer: Red Alert 2

The sprite format used in Command & Conquer: Tiberian Sun and Red Alert 2 is a collection of 8-bit frames that all have the same dimensions. The format uses the Westwood RLE-Zero algorithm to compact transparent areas, and uses X and Y offsets to align smaller frame data in the full frame size, but, unlike its predecessors, uses no real data compression like LCW.

An additional 'compression', though applied on game design level rather than on graphics level, is that all separate structure animations in these games are typically stored in separate files. This has the advantages that it both saves space by not including the entire structure graphics in the animation graphics (a problem previously solved by using XOR Delta), and that it gives more freedom for playing multiple looping animations with different amounts of frames on the same structure.

Note that Tiberian Sun and Red Alert 2 are not DOS games. This format is included on this wiki for the sake of completeness in the scope of documenting all indexed graphics formats in the Westwood Studios games, and as logical successor to the Dune II SHP, Lands of Lore SHP and C&C 1 SHP formats.

File format

Header

The file starts with a very simple 8-byte header:

Offset	Data type	Name	Description
0x00	UINT16LE	Empty	Always zero. This allows a very simple first check to identify this file type.
0x02	UINT16LE	FullWidth	Width of the frames.
0x04	UINT16LE	FullHeight	Height of the frames.
0x06	UINT16LE	NrOfFrames	Number of frames in the file.

Frames info table

After the header comes an array with NrOfFrames entries, where each entry has the following structure:

Offset	Data type	Name	Description
0x00	UINT16LE	FrameX	X-offset to align the frame in the full FullWidth*FullHeight image.
0x02	UINT16LE	FrameY	Y-offset to align the frame in the full FullWidth*FullHeight image.
0x04	UINT16LE	FrameWidth	Width of the frame image data.
0x06	UINT16LE	FrameHeight	Height of the frame image data.
0x08	UINT32LE	Flags	A series of bit flags that determine how the game interprets the data. The two flags are HasTransparency (bit 1) and UsesRle (bit 2). Since the RLE compression only collapses transparency, bit 2 should never be enabled if bit 1 isn't.
0x0C	BYTE[4]	FrameColor	The first three bytes of this contain R/G/B colour bytes that can be used on the game's minimap. Unlike the game's colour palettes, this uses 8 bits per colour component.
0x10	UINT32LE	Reserved	Always 0.
0x14	UINT32LE	DataOffset	Offset of the frame's data. If there is no data, this should be 0. In the original files, the data, and thus these offsets, are always aligned to multiples of 8 bytes, though this is technically not necessary. Since the zero-compression algorithm has line lengths in its format, and uncompressed data is exactly FrameWidth * FrameHeight bytes, no end offset is needed.

The flags aren't really decompression options but options to tell the game's blitters how to paint the graphics on the screen. Because of this, it is important to enable the HasTransparency bit if the graphics data contains zero-value bytes, otherwise they may get painted as actual zero-value colours, instead of getting handled as transparency.

Image data decompression

The main way the image data is reduced in size is by saving only a cropped frame, which needs to be realigned in the full frame size using the X and Y offsets. To get the cropped frame, though, the data first needs to be interpreted correctly.

As mentioned, there are two flags that can be set on graphics, and the second should never be enabled if the first one isn't, giving three different types of stored data:

• On fully opaque data, both the HasTransparency and UsesRle flags are disabled.
• On data containing transparency which is not RLE-compressed, the HasTransparency flag is enabled, but the UsesRle flags isn't. Typically, this is only used in case the RLE algorithm gives larger output than the original data, however, the game's mouse cursor graphics are saved without RLE, and will not work correctly if RLE is enabled on them.
• On data containing transparency which is RLE-compressed, both the HasTransparency and UsesRle flags are enabled.

The used compression is the Tiberian Sun variant of the Westwood RLE-Zero algorithm, a flag-based RLE triggering on value 00, which is treated per line, and which has a UINT16LE at the start of each line indicating that line's input data length.

Frame colour

The FrameColor is the average colour of all non-transparent pixels in the frame's image data, when seen in its intended palette, or rather, in the graphics fed into the original encoder used by Westwood. Since reapplying an average-colour algorithm to existing game sprites gives slightly different values than the original ones, it is likely that the original images had 8-bit colour components, and that the process of mapping them to the intended game palette was done by the encoding tool itself.

The main purpose of this colour is to serve as display colour on the game's 'radar' minimap. Most units and structures in the game don't need this, since anything owned by either an AI or human player is displayed on the minimap with the colour of its owner, but it is important to have this for map resources and SHP-based terrain pieces such as bridges.

The original encoder used by Westwood seems to have contained some system to compensate for remappable house colours, however, or had a way to override the house colours to the intended in-game colours, because the frames of the tiberium resource in the game, which is remappable, have green colours set, despite the game using red colours on the remapping range on its colour palette. Likewise, some of the tiberium-based mutated wildlife in the game, which is also typically remapped to green in the game, contains green frame colours that can only be achieved by using green-remapped graphics as source.

It must be noted that, on the tiberium graphics, the game uses a somewhat odd system to change these colours when the tiberium in the game is remapped to a different colour than green. For that reason, it is advised that when editing the tiberium graphics, the original graphics' frame colours are copied exactly, otherwise blue tiberium in the game has a tendency to look strangely purple on the minimap. The TS/RA2 SHP Radar Colour Editor (see Tools) can apply this operation.

Shadow filter area

On the preceding games, unit and structure shadows are typically indicated on the graphics using colour index 4 (though in C&C1 this is technically defined by the game's fading table files), typically coloured bright green #54FC54 on the palettes. Tiberian Sun and Red Alert 2 work differently; the shadows are split off into additional frames behind the normal graphics, and are painted in colour index 1.

Such shadow frames are not a specific part of the format; there are plenty of graphics in the games (like UI graphics) that are not units or structures in the play field that need to cast a shadow, so these frames are not always an expected thing, but if they are present they should be somewhat detectable by the fact there is an even number of frames, and the pixels in the second half of the frames only use palette indices 0 and 1.

Since shadows should never overlap the main graphics, the correct way to combine the frames is to draw the shadows under the main graphics. Most conversion tools convert the shadow data to the classic index 4, for the sake of compatibility, because index 4 is never really used in graphics, and because this index is still set to the clearly visible green colour on most default unit and structure palettes in Tiberian Sun and Red Alert 2.

Tools

The following tools are able to work with files in this format.

Name	Platform	View images in this format?	Convert/export to another file/format?	Import from another file/format?	Access hidden data?	Edit metadata?	Notes
Engie File Converter	Windows	Yes	Yes	Yes	No	Has save options to tweak FrameColor.	Generates the FrameColor data from the input frames, and has an option to override the red remap colours in that operation with tiberium-green.
XCC Mixer	Windows	Yes	Yes	Yes	No	No	The main conversion tool used by the modding community.
OS SHP Builder	Windows	Yes	Yes	Yes	No	No	An actual graphical sprite editor
TS/RA2 SHP Radar Colour Editor	Windows	No	No	No	No	Yes	Specific tool for editing the FrameColor data, since most SHP tools have no support for setting it. Can only view the colours set for the SHP frames, not the graphics themselves.
OpenRA Utility	Windows, Linux, Mac	No	Yes	No	No	No	Command line tool that can export to png