Minimizing Code Bloat for Faster Builds and Smaller Executables

No matter how much code you have, somebody always wants more.  That’s certainly been the case with every game and every engine I’ve worked on.  My good friend Kyle, who has for years been tracking all sorts of wonderful statistics about the development of Despair Engine, recently announced that we’d surpassed 1.5 million lines of principle engine code.  That doesn’t count build systems, tools, tests, middleware, or scripts; that’s 100% Despair-original C++ code the compiler has to wade through every time we build the game.

As if the code we write weren’t bad enough, our engine is particularly adept at generating code.  Those 1.5 MLOC result in a hefty 37 megabyte executable on the Xbox 360.  Kyle estimates that we’ll exceed 2 million lines of code before the end of this console generation.  We already have an executable that can’t fit into main memory on a PS2 or a Gamecube.  By 2014 it won’t fit onto an Xbox or Wii either.  I don’t think we’re in danger of exceeding the capacity of a PS3 or Xbox 360, but the prospect is enough to keep me up at nights.

Looking back at the graphs of our executable size and build and link times, I’m proud to see a few dips punctuating the otherwise consistent upward march.  At least a few of those dips correspond to weeks when I turned my sleepless nights into successful assaults on code bloat.  What’s particularly interesting about those dips is that they don’t correspond to any drops in our code line count.  In my biggest success I reduced the size of our executable by 25% or over 7 megabytes.  At the same time I cut our link times by 19%, and yet I did it all without removing a single line of code!  How?

The first place to start when trimming a codebase is figuring out where to look.  In an engine of 1.5 million lines of code there are a lot of places to hide.  My strategy when dealing with problems like this is to gather as much data as possible into a format that is amenable to analysis.  For code this means extracting symbols from the compiled binaries along with their size, type, source file, and any other details you can get your hands on.  Once you have a dump of all the symbols in your compiled code, it is a pretty simple matter to write a script to collate it and generate whatever statistics you find useful.

There are two methods I use for extracting data from compiled code.  The first is a quick and handy utility called DumpBin.  DumpBin is part of the Visual Studio toolchain and is available for all Microsoft targets.  There are similar tools available for every compiler I’ve encountered.  The gcc equivalents, for example, are objdump and nm.  These tools are useful because they can work with just about any compiled binary.  They target standard binary formats like PE/COFF and ELF, and consequently they work on object files, library files, executables and DLLs.  The ability to extract information from intermediate files in the build process is essential if you’re worried as much about build times as you are about final memory usage.

My preferred way to use DumpBin is to specify the /headers option to extract COMDATs from object files compiled with the /Gy option.  For example:

for /r . %n in (*.obj) do DumpBin /headers “%n” >> all_headers.txt

A portion of the output from such a dump looks like this:

SECTION HEADER #14E
   .text name
       0 physical address
       0 virtual address
       7 size of raw data
   22D6B file pointer to raw data (00022D6B to 00022D71)
   22D72 file pointer to relocation table
       0 file pointer to line numbers
       1 number of relocations
       0 number of line numbers
60501020 flags
         Code
         COMDAT; sym= "public: virtual __thiscall Despair::Reflection::Object::~Object(void)" (??1Object@Reflection@Despair@@UAE@XZ)
         16 byte align
         Execute Read

SECTION HEADER #14F
   .text name
       0 physical address
       0 virtual address
       5 size of raw data
   230B1 file pointer to raw data (000230B1 to 000230B5)
       0 file pointer to relocation table
       0 file pointer to line numbers
       0 number of relocations
       0 number of line numbers
60501020 flags
         Code
         COMDAT; sym= "public: int __thiscall Despair::BroadcasterReceipt::AddRef(void)" (?AddRef@BroadcasterReceipt@Despair@@QAEHXZ)
         16 byte align
         Execute Read

As you can see, the result is an easily parsable list of packaged functions with symbol names and sizes clearly specified.  As great as the /headers dump is, it isn’t perfect.  Data symbols aren’t packaged as COMDATs and although DumpBin can extract data from executables, executables don’t contain COMDATs so there isn’t much to see.

To analyze the contents of an executable, a second approach is needed.  My solution is to reference the debugging information provided by the linker.  With Microsoft’s tools, debugging information is stored in a PDB file that accompanies the executable.  Similar to DumpBin’s /headers dump, the PDB contains information about symbols in the compiled binary.  Unlike the /headers dump, however, the PDB file contains data as well as code symbols, and the information it contains is post-linker optimization which is important if you’re more worried about memory consumption than compile and link times.

There aren’t any standard tools for dumping the contents of a PDB, but there is a DLL provided with the Microsoft toolchain that makes it pretty easy to write your own.  I’ve used the Debug Interface Access SDK included with Microsoft Visual Studio to plumb the contents of an executable and generate symbol tables that can be analyzed alongside a COMDAT dump.

In an upcoming series of articles I’m going to explore the most common causes of C++ code bloat and take a closer look at how you can use DumpBin and the PDB to identify and remove the worst offenders from your code.  When I’m done I’ll share some sample code which you can use to assess the level of code bloat in your own engines.

First up, template overspecialization.