Benchmarking Xdebug

A very popular and widely used PHP extension for debugging and profiling PHP applications is Xdebug. Normally this tool would be used on your development environment, or in rare occasions on some remote environment that is similar to the production environment. However, a little while ago one of my co-workers found a production environment running Xdebug, and we wondered how severe this was influencing performance. Since you are adding extra overhead it is very likely things will run slower. But how significant is this difference. Would it be something worth considering?

Disclaimer
Now I know that debugging a production environment is generally considered unnecessary and a really bad idea. I myself can’t really think of a situation where I would actually want to have Xdebug on my production machine – but that’s not the point. I just had a new question that I wanted to answer: “How much does installing Xdebug affect the performance of PHP?”

The hardware
To be able to test this I first needed to find a proper testing environment. Unfortunately I don’t have any webservers laying around, and testing directly on my macbook would probably be a bad idea – since there are a million processes running that all influence the performance of the machine, and therefore would influence the results. After some pondering I realised that I had an old (well… more like ancient) laptop laying around that would actually suit this purpose quite well. When it comes to speed it doesn’t come anywhere close to a modern production environment – but since we will be looking at relative changes rather than absolute measurements I figured it would be just fine.

The software
I downloaded and installed a fresh new copy of Debian. Once it was installed I decided to take the easy way and install PHP simply by using apt-get install php5. Finally, I installed xdebug using the PECL installer. With just this installed and nothing else, I was convinced my benchmark results would be quite reliable.

Software installed:

  • Debian 5.0.8
  • PHP 5.2.6 with Suhosin-patch
  • Xdebug 2.1.0

The benchmark
Why write a benchmark when they already exist? For testing purposes I decided to use the bench.php script that comes with the PHP source. This script does not run out-of-the-box when Xdebug is installed, because one test (Ackermann function) exceeds the default Xdebug limit of 100 nested function calls. Now I could of course increase this number, but I wanted to test how Xdebug performs without any modifications to the configuration – so I decided to simply skip this one test and just run the remaining ones.

Finally, after a lot of preparation, I was ready to run the script. \o/

The results
I decided to run the script three times with the xdebug extension enabled, and then another three times with the xdebug extension disabled. The results were pretty clear:

No Xdebug
(time in seconds)
Xdebug
(time in seconds)
Run 1 Run 2 Run 3 Run 1 Run 2 Run 3
simple 0.687 0.648 0.681 1.327 1.361 1.367
simplecall 0.813 0.744 0.786 3.581 3.770 3.875
simpleucall 1.138 1.225 1.169 4.439 4.510 4.877
simpleudcall 1.289 1.469 1.336 4.621 4.609 4.747
mandel 2.016 2.259 2.161 3.646 4.028 4.036
mandel2 2.774 2.753 2.766 3.495 4.008 4.215
ary(50000) 0.171 0.175 0.171 0.210 0.224 0.224
ary2(50000) 0.136 0.136 0.136 0.172 0.180 0.184
ary3(2000) 1.363 1.365 1.416 2.133 2.463 2.450
fibo(30) 3.694 3.517 3.538 10.418 12.364 12.943
hash1(50000) 0.212 0.216 0.214 0.515 0.618 0.638
hash2(500) 0.264 0.266 0.266 0.283 0.357 0.342
heapsort(20000) 0.676 0.667 0.671 1.026 1.198 1.196
matrix(20) 0.556 0.552 0.555 0.810 0.960 0.936
nestedloop(12) 1.221 1.208 1.311 2.197 2.570 2.573
sieve(30) 0.786 0.786 0.789 1.105 1.231 1.226
strcat(200000) 0.090 0.093 0.096 0.148 0.173 0.169
Total: 17.886 18.079 18.062
40.126 44.624 45.998
Average: 18.009 seconds
43.583 seconds

So there we go. Without Xdebug it takes about 18 seconds to run the script, and with Xdebug enabled is takes about 44 seconds: 241% of the original time (or: it took 2.41 times as long to run the script).

So what does this mean?
In this situation adding Xdebug slowed down the execution of the benchmark script quite significantly.

So what doesn’t this mean?
This doesn’t mean that your application will slow down with the same relative percentages, nor does it mean that you can compensate for this by buying 2.41x more servers. When we look at the breakdown per test it’s clear to see that the differences per test are quite significant as well. For example: the time needed to run the ‘ary’-test increased with only 27%, while the time needed for the ‘simplecall’ test increased with a whopping 379%.

Apparently the overhead caused by running Xdebug varies and is related to what kind of operations you are running. Calling functions seems to be affected a lot by Xdebug, while the effect of manipulating strings or arrays is less severe.

Test No xdebug Xdebug Percentage
simplecall 0.78 3.74 479%
simpleucall 1.18 4.61 391%
simpleudcall 1.36 4.66 341%
fibo(30) 3.58 11.91 332%
hash1(50000) 0.21 0.59 276%
simple 0.67 1.35 201%
nestedloop(12) 1.25 2.45 196%
mandel 2.15 3.9 182%
strcat(200000) 0.09 0.16 176%
ary3(2000) 1.38 2.35 170%
heapsort(20000) 0.67 1.14 170%
matrix(20) 0.55 0.9 163%
sieve(30) 0.79 1.19 151%
mandel2 2.76 3.91 141%
ary2(50000) 0.14 0.18 131%
ary(50000) 0.17 0.22 127%
hash2(500) 0.27 0.33 123%

A more realistic setup – Testing WordPress with ApacheBench
The numbers from the previous test are quite clear – but they hardly represent a real world example of an application. To get a bit more insight in how enabling or disabling Xdebug would affect an actual real life application I decided to add Apache, MySQL and WordPress into the mix, and benchmark the default homepage of a standard WordPress installation using the benchmark tool ApacheBench (ab), which comes with Apache. Adding these applications means there will be a lot more factors to influence the results – but I was hoping the results would somehow give an indication on how much an average application would be affected by installing Xdebug.

I ran ApacheBench a number of times. Every time it fired a total of 100 requests and every time I changed the number of maximum concurrent requests. I assumed that when Xdebug was enabled the requests would take longer – but because of that it’s also more likely the maximum number of concurrent requests that’s still acceptable is reached sooner. After a couple of test runs I had the following figures:

Max. concurrent requests No Xdebug (req./sec.) Xdebug (req./sec.) Difference (req./sec.) Difference %
1 2,79 2,22 0,57 20,43%
5 2,75 2,16 0,59 21,45%
10 2,73 2,13 0,6 21,98%
15 2,69 2,11 0,58 21,56%
20 2,64 2,1 0,54 20,45%
25 2,61 1,34 1,27 48,66%
30 0,95 - timeout - 0,95 100,00%

or in a pretty graph:

So not only does it take about 20% more time to respond to requests in normal situations, the maximum number of requests your server can handle is also influenced significantly.

Conclusion
So as a conclusion we can say what we actually already suspected, Xdebug influences performance quite a lot. How much exactly is hard to tell and depends on what your application does. Xdebug is a great tool, and pretty much irreplaceable when you are looking for a proper tool to debug or profile an application on your development- or testing environment. Running it on your production machines however is something that would only make sense in very specific situations, as it is typically unnecessary and it slows down your application a lot. Also, make sure you turn off Xdebug on environments that you want to use for benchmarking your application – as it will influence your results.

Edit: Directly after publishing this article Xdebug mentioned on twitter that the current version in SVN is supposed to be a lot faster. I’ll benchmark this new version soon – and let you know if there is any significant different.

  1. Hi!

    Cool benchmark! I’ve recently made a few modifications to Xdebug to speed things up a bit though. They are in SVN. With this, if you turn off the new setting called “xdebug.coverage_enable”, you get a lot of more speed again; but of course code coverage doesn’t work in that case.

    cheers,
    Derick

  2. Thanks for the effort. So here is a question: I’m guessing your installation of xdebug had it enabled for all requests. You can also programatically turn xdebug off and on using xdebug_start_trace() and xdebug_end_trace(). What I’m wondering is this: if you have xdebug installed and active, but are not tracing i.e. you don’t have it enabled globally and you haven’t called xdebug_start_trace(), then what is the performance impact of that? Did you investigate that in your work? I have no doubt that actually doing tracing will slow things down. But I’m less clear if simply installing xdebug will slow things down.

    • If I recall it correctly I didn’t do anything else besides installing the xdebug extension and run the same script again – so I suppose tracing was turned off!

      However, this blogpost is really starting to lose it’s point anyway. Derick already mentioned a newer, faster version back in 2011, so I wouldn’t be surprised if the impact on speed has gotten even smaller since then.

      So I guess I should do what I’ve been planning to do for some time now: do a new benchmark with a newer version of xdebug – and when I do I will make sure so check which settings are turned on or off and what kind of difference that makes :-)

      Thanks for your reply Dave!

      • If all you did was install the extension, it comes down to what the default value for xdebug.profiler_enable is, and that appears to be 0. So your benchmark probably did actually measure what I was interested in, even though the answer isn’t what I’d hoped to hear :-)

        For my purposes, I really am pondering the possibility of always having it installed in production, and conditionally turning on tracing dynamically per request to diagnose hard to reproduce customer issues. If the overhead is anything other than trivial, that probably isn’t a great approach.

        • I’m afraid installing a debugger on a production environment is generally considered a bad idea because of performance reasons. But like I said, things might have changed over the last two years :-)

Leave a Comment


NOTE - You can use these HTML tags and attributes:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>