C Compiler Benchmarks

Edit 1/5/09: I’ve kind of re-done this. I wasn’t happy with it and some of the results were incorrect. I’ve also added two new benchmark programs since the first version of this post.

Well gcc 4.4.0 has been released now (with a few bugs, of course…) and I thought it’d be interesting to do some benchmarks, especially as there are now a few open source compilers out there (gcc, llvm, libfirm/cparser, pcc, ack; I’m not so interested in the latter two just now). Now I don’t have access to SPEC so I cooked up just a few little programs specifically designed to test a compiler’s ability to perform certain types of optimization. Eventually, I might add some more tests and a proper harness; for now, here are the results (smaller numbers are better):

Results

Thankfully, it looks like gcc 4.4.0 performs relatively well (compared to other compilers) in most cases. Noticable exceptions are bm5 (where gcc-3.4.6 does better), bm3 and bm6 (where llvm-2.5 does better, especially in bm6).

llvm-2.5 does pull ahead in a few of the tests, but fails dismally in the 4th, which tests how well the compiler manages a naive memcpy implementation. llvm-2.5 generated code is more than four times slower than gcc generated code in this case, which is ridiculously bad. If it wasn’t for this result, llvm would be looking like the better compiler.

Sadly, none of the compilers I tested did particularly well at optimising bm2. In theory bm2 run times could be almost idential to bm3 times, but no compiler yet performs the necessary optimizations. This is a shame because the potential speedup in this case is obviously huge.

It’s surprising how badly most compilers do at bm4, also.

So, nothing too exciting in these results, but it’s been an interesting exercise. I’ll post benchmark source code soon (ok, probably I won’t. So just yell if you want it).

Mplayer version 1.0… ever?

The guys that develope MPlayer have some serious hangup about doing an actual 1.0 release. I mean, they’ve been through 1.0pre1, pre2, pre3, pre4, pre5, -rc1 (2006-10-22, that’s over 2 years ago) and -rc2 (2007-10-7, over 1 year ago) and still we don’t have a 1.0 release.

Now, there are mailing list posts like this one which say “it’ll be ready when it’s ready” and calling for the use who dared venture forth with the question (of when the next rc or release might be forthcoming) to be the release manager. I mean, look, I understand that these guys are doing a lot of hard work developing MPlayer and it is, after all, a pretty good product by open source standards (I mean, it has some actual documentation for one thing) BUT geez, if you’re not going to actually release 1.0 in some reasonable timeline then why have “release candidates”? Was -rc2 so bad that it can’t be called 1.0? (I mean, that is the point of a “release candidate” right? It is the candidate for the release, yes?) And if -rc2 has problems, wouldn’t it be prudent to at least do another release candidate, i.e. rc3?

Sigh. I guess I’m complaining not so much about the absence of MPlayer 1.0 (although I would certainly like it to arrive) but the fact that these “release candiates” exist and did so for so long without any actual release occurring. I mean, you shouldn’t have a release candidate if you’re not planning (or able) to do an actual release – it kind of gives the wrong impression. (Let’s be clear. The developers aren’t at fault, at least, not most of them. I’m just kind of irked at whoever had the bright idea to bundle a tarball and call it “Mplayer 1.0pre1″ in the first place…)

Xorg server 1.5.3 fails to build… because of glproto

So when I try to build xorg-server-1.5.3, I get:

/bin/sh ../libtool --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I../include -I../hw/xfree86/os-support -I../hw/xfree86/os-support/bus -I../hw/xfree86/common -I../hw/xfree86/dri -I../hw/xfree86/dri2 -I../mi   -DHAVE_DIX_CONFIG_H -Wall -Wpointer-arith -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wnested-externs -fno-strict-aliasing -D_BSD_SOURCE -DHAS_FCHOWN -DHAS_STICKY_DIR_BIT -I/usr/include/freetype2 -I/usr/X11R7/include -I/usr/X11R7/include/pixman-1     -I../include -I../include -I../Xext -I../composite -I../damageext -I../xfixes -I../Xi -I../mi -I../miext/shadow  -I../miext/damage -I../render -I../randr -I../fb -I/usr/X11R7/include -I/usr/X11R7/include -I/usr/include/drm -I/usr/X11R7/include/X11/dri -DXFree86Server -DGLX_USE_TLS -DPTHREADS  -g -O2 -MT glxdriswrast.lo -MD -MP -MF .deps/glxdriswrast.Tpo -c -o glxdriswrast.lo glxdriswrast.c
 gcc -DHAVE_CONFIG_H -I. -I../include -I../hw/xfree86/os-support -I../hw/xfree86/os-support/bus -I../hw/xfree86/common -I../hw/xfree86/dri -I../hw/xfree86/dri2 -I../mi -DHAVE_DIX_CONFIG_H -Wall -Wpointer-arith -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wnested-externs -fno-strict-aliasing -D_BSD_SOURCE -DHAS_FCHOWN -DHAS_STICKY_DIR_BIT -I/usr/include/freetype2 -I/usr/X11R7/include -I/usr/X11R7/include/pixman-1 -I../include -I../include -I../Xext -I../composite -I../damageext -I../xfixes -I../Xi -I../mi -I../miext/shadow -I../miext/damage -I../render -I../randr -I../fb -I/usr/X11R7/include -I/usr/X11R7/include -I/usr/include/drm -I/usr/X11R7/include/X11/dri -DXFree86Server -DGLX_USE_TLS -DPTHREADS -g -O2 -MT glxdriswrast.lo -MD -MP -MF .deps/glxdriswrast.Tpo -c glxdriswrast.c  -fPIC -DPIC -o .libs/glxdriswrast.o
In file included from glxdriswrast.c:49:
glxdricommon.h:32: error: expected ':', ',', ';', '}' or '__attribute__' before '*' token
glxdricommon.h:36: warning: type defaults to 'int' in declaration of '__DRIcoreExtension'
glxdricommon.h:36: error: expected ';', ',' or ')' before '*' token
glxdricommon.h:38: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'systemTimeExtension'
glxdriswrast.c:67: error: expected ':', ',', ';', '}' or '__attribute__' before '*' token

(and a bunch more).

It took me a while to figure this one out. There are conflicting “GL/internal/dri_interface.h” files – one comes from Mesa and the other one from the glproto package (version 1.4.9) which is part of the Xorg distribution. Deleting the latter (at /usr/X11R7/include/GL/internal/dri_interface.h) made the problem go away.

It is a bit strange, though; why does glproto include that header file if it’s not needed – if it is, in fact, problematic?

Comcast is screwing its customers…

Comcast is screwing its customers by blocking “spam” which isn’t spam and also preventing mail servers which are accused of “spamming” from being removed from Comcast’s blacklist. I’ve tried to respond to two support requests from Comcast subscribers recently and in both cases, my reply bounced.

I got this back rom my mail server:

Reporting-MTA: dns; *********

Final-Recipient: rfc822;********@comcast.net
Action: failed
Status: 5.0.0 (permanent failure)
Diagnostic-Code: smtp; 5.4.7 - Delivery expired (message too old) 554-'IMTA06.emeryville.ca.mail.comcast.net comcast ***.***.***.*** Comcast BL004 Blocked for spam.  Please see http://help.comcast.net/content/faq/BL004' (delivery attempts: 0)

(Of course “***” are places where I’ve removed identifying information).

On following the link to http://help.comcast.net/content/faq/BL004, I get a page which suggests I can go to http://www.comcastsupport.com/rbl to get my mailserver unblocked. However, following that link redirects me to http://www.comcastsupport.com/browserupgrade/ which gives me a 404 Not Found.

Way to go, Comcast.

Dbus 1.2.10 won’t compile

I get the following compiling dbus-1.2.10:

dbus-sysdeps-util-unix.c: In function '_dbus_init_system_log':
dbus-sysdeps-util-unix.c:457: warning: implicit declaration of function 'openlog'
dbus-sysdeps-util-unix.c:457: warning: nested extern declaration of 'openlog'
dbus-sysdeps-util-unix.c:457: error: 'LOG_PID' undeclared (first use in this function)
dbus-sysdeps-util-unix.c:457: error: (Each undeclared identifier is reported only once
dbus-sysdeps-util-unix.c:457: error: for each function it appears in.)
dbus-sysdeps-util-unix.c:457: error: 'LOG_DAEMON' undeclared (first use in this function)

(and so on).

The problem seems to be a missing “#include <syslog.h>” in the named file… if I add that in, it’s fine. How the how did this get missed? I mean, it DOESN”T COMPILE. How can you release a package which doesn’t compile?

I wonder if there’s something I’m missing, in particular, some situation (perhaps different ./configure options) which allow compilation without modifying the source.

Where is the Rails bug tracker?

The Ruby-on-Rails website has a link (“Get involved”) with a speech bubble picture, in which the words “bug tracker” (among other things) are contained. However, the link takes you to another page which links to a whole bunch of stuff, none of which is a bug tracker.

Hmm, there seems to be a tracker here:

http://rails.lighthouseapp.com/projects/8994-ruby-on-rails/overview

… but that doesn’t seem to have bug #3231 which was mentioned here:

http://www.ruby-forum.com/topic/39270

… and which I recently stumbled on.

It’s all very chaotic and unsatisfying. Much like Ruby programming…