Monday, June 15, 2009

Carl Worth, performance, GtkPerf

Carl Worth blogged here about Cairo 2D performance measurements:
Various attempts at 2D-rendering benchmark suites have appeared and even become popular. Notable examples are x11perf and gtkperf. My claim is that these tools range from useless to actively harmful when the task is understanding performance of real applications.
...
Unfortunately, the workload of things like x11perf and gtkperf rarely come close to simulating practical workloads.

Should I be offended and sad as someone as respected as Carl blames my precious GtkPerf? Well of course not! Citing myself from GtkPerf website:
I know that bencmarking tools (including GtkPerf) can be fooled and don't give real-life results. Yet, I belive that GtkPerf can be helpful to solve for example this kind of things:
...

So as seen, I fully agree (and have always agreed) that GtkPerf is not the tool to use for getting real-life application performance measurements. But what it is, is a very easy and fast way to get a view of the GTK+/GDK/Cairo performance of your system.

These days I spend my time mostly with Qt and often benchmark the actual frame rate of the application, which with QGraphicsView-based UI is easily done for the whole view using the paintEvent() callback. And keeping the FPS visible when developing your new flashy UI really helps (at least me!) to spot & fix the performance regressions while they happen.

The last thing Carl says is:
The punchline is that we now have an easy way to benchmark 2D rendering in actual, real-world applications. If you see someone benchmarking with only toys like x11perf or gtkperf, go ahead and point them to this post, or the the cairo-perf-trace entry in the cairo FAQ, and insist on benchmarks from real applications.

So pointing: When measuring GTK+/Cairo performance trying to get accurate real-world results, please don't use GtkPerf. Instead use the cairo-perf-trace as noted, thanks!

No comments: