Willus.com Home   |   Archive   |   About  

   CONTENTS

I. BACKGROUND
    1. Overview
    2. The Compilers
    3. Compiler Options
    4. The Programs
    5. Test Hardware
    6. Compiler Issues
    7. Other notes

II. RESULTS
    1. BW1D
    2. BZIP2
    3. CRAFTY
    4. K2PDFOPT (v1.30)
    5. LAME
    6. MESHER
    7. MODEL3D
    8. RESIZER
    9. TRANSCEND
    10. X264
    11. AVERAGE

III. SUMMARY

IV. COMMENTS

  
  
  
Willus.com's 2011 Win32/64 C Compiler Benchmarks:
I. BACKGROUND

2. The Compilers

I used compilers from six different companies/groups/individuals, choosing two commercial ones where I had access to either a free version (Microsoft Visual C/C++ 2010 via Windows 7 SDK) or a 30-day trial version (Intel). I felt I had to include Microsoft and Intel because they are simply the de-facto standard compiler and standard high-performance compiler, respectively, for Windows programming on Intel chips. Intel is used on a vast number of SPEC CPU 2006 results submissions, including the ones with the top results on Intel CPUs. To keep things interesting, I added the latest version of gcc (a pre-release v4.6.3 from mingw64.org) and a version of gcc from the 3.x branch in order to compare the 4.x and 3.x branches. I tried Walter Bright's Digital Mars again, although it is long in the tooth (last release date was in 2004), and I rounded things out with the smallest and fastest compiling compiler of the group, Bellard's Tiny CC. Mainly for me, though, I wanted to see how much progress, if any, has been made by gcc against Intel since gcc is my day-to-day compiler of choice. All of these compilers are discussed more on my Win32 C/C++ Compilers page. I've color coded them in the tables so that they are easily distinguished.
[Update, 4-7-12: I tried Open Watcom (v1.9) this weekend without much success. It failed to compile and/or run correctly three of the first five benchmarks I tried, and those were ones that I considered the easier ones (least code). It has the feel of a compiler that stopped serious development shortly after the introduction of the Pentium Pro CPU, based on the CPU tuning options. On the two benchmarks that compiled and ran (bzip2 and transcend), it scored about the same as Digital Mars in both compile-time and run-time performance. I will say that the setup program is very good and the installation fairly compact (took up only 50 MiB with the options I chose).]

Click on a column heading to sort the rows based on the data in that column.
Company Version Package
Size
(MiB)
Disk
Space
Needed
(MiB)
Free/
Commercial
Description
Intel 2011 v12.1.1.259
Oct 11, 2011
1636 10240 Commercial Big footprint, but big performance. Intel is far and away the most-used compiler for SPEC CPU benchmarks. Frustratingly, even after you install some 40 packages that take up massive amounts of disk space, you're still not done. You also have to install Microsoft's C/C++ compiler. I used a 30-day full-capability version of Intel's compiler along with Microsoft's free 2010 Express version.
Digital Mars v8.52
2004
3 11 Free A very compact and solid offering from Digital Mars, but a bit dated and showing its age in terms of some limitations, incompatibility with newer Windows APIs, and compiled executable performance. The actual compiler executable (dmc.exe) claims that its version number is 8.42n, but the package says it is version 8.52.
Tiny CC v0.9.25
May 29, 2009
1 3 Free The most compact and the fastest compiler of the group--excellent for use in scripts, but not good for high-performance compiled code (it has no command-line optimization options). This is the only compiler in the group that does not also compile C++ code. It also lacks some Windows API support.
Microsoft Visual C/C++ 2010 v16.00
.40219.01
for 80x86/x64
690 1100 Free The reigning PC standard. A compiler benchmark isn't complete without including Visual C. The Windows 7 SDK is freely downloadable at download.microsoft.com (you'll also want VC++ 2010 SP1, and .Net Framework 4 is required) and comes with 32-bit and 64-bit Visual C/C++ 2010. Interestingly, the 64-bit version of the compiler doesn't allow the /arch:SSE2 flag. I don't know if this is because all x64 CPUs have SSE2 and therefore they turn it on by default, or if they simply haven't implemented it yet, but the 64-bit exes don't show as much average performance improvement with MS VC as they do with Intel and gcc. They do have /arch:AVX, but I couldn't test that because I don't have a Sandybridge CPU.
MinGW (gcc 3.4.2) v3.4.2
Sept 6, 2004
57 150 Free Version 3.4.2 of gcc for the PC.       
MinGW (gcc 4.6.3) v4.6.3
Dec 9, 2011
(pre-release)
46 500 Free Version 4.6.3 of gcc for Win32/64--the very latest available for this review and my current compiler of choice for work and home projects.


      
  <<  Previous: Overview

Next: Compiler Options   >>

 
This page last modified
Saturday, 22-Sep-2012 08:54:01 MDT