Reverting a dropped column

In my talks and article I always mention that when it comes to database migrations it is generally a bad idea to rely on undo-patches for rollbacks. As an example I always use the same story: “Imagine you write a database patch that removes a column from a table, and then you write an undo-patch that adds the column back again. Sure, your database schema is now the same as it was before – but the content is gone!” And then I would move on by saying that you probably shouldn’t rely on undo-patches anyway for rollbacks, because it is better to thoroughly test everything before updating so you are absolutely certain that you won’t have to rollback. Also, I recommended to make backups first, and use that to revert database migrations when needed. Easier said than done!

The fact that I am in the luxurious position where this approach usually is an option, doesn’t mean that this is a solution for everyone. For other projects where, for example, larger databases are involved, rolling back by restoring a backup would be considered highly inefficient (and if the database is in use it would actually result in data loss). Being able to rollback patches can be convenient, but reverting dropped columns can’t be done off course.

When I did my talk at DPC10 in Amsterdam however, someone in the audience (sorry, I don’t remember who it was) suggested a very simple but very effective solution. In fact, it was such a simple solution that I couldn’t stand I hadn’t thought of it before: If you rename your column or table first instead of dropping it you can still just use an undo-patch to revert the change. In a later stage, when you are absolutely certain you don’t need to do a rollback anymore, you can simply drop the renamed column or table by writing another patch. For example in a next release.

Reverting a renamed column

As you can see, sometimes the simplest solution is the best. Beware though, as in some situations this approach might still cause problems: Records added to this table between the moment the patch was applied and the moment the undo file was applied won’t have a value in the age column, to name just one. As always when we are talking database version control, it all depends on the specific situation you are in.

I myself still refuse to write undo-patches as most patches create new tables or columns anyway, instead of removing them – and if patches actually do drop tables or columns they probably haven’t been very significant for quite some time anyway. Maybe the main reason is that I have simply never found myself in the situation where an undo-patch would have helped me. This might be different for you though, and if you are looking for a way to rollback dropped columns or tables, renaming them first might be a pretty good idea.

Disclaimer
Now I know that debugging a production environment is generally considered unnecessary and a really bad idea. I myself can’t really think of a situation where I would actually want to have Xdebug on my production machine – but that’s not the point. I just had a new question that I wanted to answer: “How much does installing Xdebug affect the performance of PHP?”

The hardware
To be able to test this I first needed to find a proper testing environment. Unfortunately I don’t have any webservers laying around, and testing directly on my macbook would probably be a bad idea – since there are a million processes running that all influence the performance of the machine, and therefore would influence the results. After some pondering I realised that I had an old (well… more like ancient) laptop laying around that would actually suit this purpose quite well. When it comes to speed it doesn’t come anywhere close to a modern production environment – but since we will be looking at relative changes rather than absolute measurements I figured it would be just fine.

The software
I downloaded and installed a fresh new copy of Debian. Once it was installed I decided to take the easy way and install PHP simply by using apt-get install php5. Finally, I installed xdebug using the PECL installer. With just this installed and nothing else, I was convinced my benchmark results would be quite reliable.

Software installed:

• Debian 5.0.8
• PHP 5.2.6 with Suhosin-patch
• Xdebug 2.1.0

The benchmark
Why write a benchmark when they already exist? For testing purposes I decided to use the bench.php script that comes with the PHP source. This script does not run out-of-the-box when Xdebug is installed, because one test (Ackermann function) exceeds the default Xdebug limit of 100 nested function calls. Now I could of course increase this number, but I wanted to test how Xdebug performs without any modifications to the configuration – so I decided to simply skip this one test and just run the remaining ones.

Finally, after a lot of preparation, I was ready to run the script. \o/

The results
I decided to run the script three times with the xdebug extension enabled, and then another three times with the xdebug extension disabled. The results were pretty clear:

So there we go. Without Xdebug it takes about 18 seconds to run the script, and with Xdebug enabled is takes about 44 seconds: 241% of the original time (or: it took 2.41 times as long to run the script).

So what does this mean?
In this situation adding Xdebug slowed down the execution of the benchmark script quite significantly.

So what doesn’t this mean?
This doesn’t mean that your application will slow down with the same relative percentages, nor does it mean that you can compensate for this by buying 2.41x more servers. When we look at the breakdown per test it’s clear to see that the differences per test are quite significant as well. For example: the time needed to run the ‘ary’-test increased with only 27%, while the time needed for the ‘simplecall’ test increased with a whopping 379%.

Apparently the overhead caused by running Xdebug varies and is related to what kind of operations you are running. Calling functions seems to be affected a lot by Xdebug, while the effect of manipulating strings or arrays is less severe.

A more realistic setup – Testing WordPress with ApacheBench
The numbers from the previous test are quite clear – but they hardly represent a real world example of an application. To get a bit more insight in how enabling or disabling Xdebug would affect an actual real life application I decided to add Apache, MySQL and WordPress into the mix, and benchmark the default homepage of a standard WordPress installation using the benchmark tool ApacheBench (ab), which comes with Apache. Adding these applications means there will be a lot more factors to influence the results – but I was hoping the results would somehow give an indication on how much an average application would be affected by installing Xdebug.

I ran ApacheBench a number of times. Every time it fired a total of 100 requests and every time I changed the number of maximum concurrent requests. I assumed that when Xdebug was enabled the requests would take longer – but because of that it’s also more likely the maximum number of concurrent requests that’s still acceptable is reached sooner. After a couple of test runs I had the following figures:

or in a pretty graph:

So not only does it take about 20% more time to respond to requests in normal situations, the maximum number of requests your server can handle is also influenced significantly.

Conclusion
So as a conclusion we can say what we actually already suspected, Xdebug influences performance quite a lot. How much exactly is hard to tell and depends on what your application does. Xdebug is a great tool, and pretty much irreplaceable when you are looking for a proper tool to debug or profile an application on your development- or testing environment. Running it on your production machines however is something that would only make sense in very specific situations, as it is typically unnecessary and it slows down your application a lot. Also, make sure you turn off Xdebug on environments that you want to use for benchmarking your application – as it will influence your results.

Edit: Directly after publishing this article Xdebug mentioned on twitter that the current version in SVN is supposed to be a lot faster. I’ll benchmark this new version soon – and let you know if there is any significant different.

Of course, I could not possibly do this just by myself. Just imagine what needs to be done: the website needs to be built, the talks need to be selected, the speakers need to be informed, the tickets sales have to be taken care of, the venue needs to be booked, the flights and hotel rooms for the speakers have to be arranged, the social event needs to be organized. Etc. Etc. The list just keeps going. Organizing the conference is a huge task and a lot of people at Ibuildings are helping to organize it. Luckily most members of the team are pretty experienced by now, as they have been involved with organizing the DPC from the very first edition. It is comforting to have such a solid base to rely upon.

I myself will be mainly working together with my colleague Felix de Vliegher. Together we are responsible for what I think is the most fun part of the entire operation: creating the conference schedule, communicating with the community and communicating with the speakers. We will also host the conference, which I think is pretty exciting. Yes I have spoken to groups before. Quite large groups actually. But this time I will be speaking to a fully packed room on the main stage. Cool! I suppose it is much like doing a talk – only this time I will not be talking about database version control, but about what kind of sunglasses we found on the floor of room B and when you will get your food.

At the moment we are very busy selecting the talks and creating the conference schedule. Together with a couple of colleagues we are trying to figure out which talks are the best ones. Not an easy task when you can only pick about 30 talks from 240 great proposals! I really enjoyed reading them all and it is interesting to see all the different subjects and ideas people come up with. It is sad that we will have to disappoint most of the submitters – but having them all would result in a 16 day conference or something, which would not be very realistic either now would it?

You can find the article here:
http://techportal.ibuildings.com/2011/01/11/database-version-control/

The 2011 year calendar hanging next to my desk

And imagine what? It works great!

The reason that this is useful to have in addition to your existing calendar is that whenever the scale of your calendar-view changes, the significance of appointments changes as well. It’s quite easy: when I’m looking at a day-view I want to know exactly at what time what’s going on. I need to go to get a haircut at 3pm. Great, I’ll be there. On month-view however the time isn’t important – just the fact that I’m getting a haircut on that date will suffice. On a year view, seeing all year’s appointments simply would result in one big unusable and unreadable list. It’s far too detailed!

That’s exactly what this calendar next to my desk does: it gives me a filtered view of the year, only showing the appointments that are important on a year’s-view. I don’t mark my tennis lessons there every Friday. Neither do I put down when I’m getting a haircut – I’ve got Google Calendar to do that. The year calendar is for stuff like days off, conferences, deadlines of projects etc. It’s hanging next to my desk, so whenever I want to know something like that, I simply have to turn my head and I see directly what’s going on.

Everybody is different, and especially for these kind of things it’s important to not just do as you’re told, but choose the solution that works best for you. However, the calendar has worked great for me so far – so maybe it will work for you as well. Also, figure out your own standard – use different colors with different meanings. Write stuff next to it. Draw arrows. Maybe use certain shapes for lining-in certain dates. Get creative! It’s analogue, the sky is the limit!

The calendar I use was generated by this pre-historic but still working CGI script: http://cgi.dit.nl/kalender.cgi (dutch). Be sure to mark “Enkel de kalender (printen)” at the bottom, and off course any other options you prefer. English versions available as well and easy to find. You can find an example here: http://www.freeprintablecalendar.net/2011/printcalendar.aspx.

I found it quite shocking to see myself doing my talk. I’m sure we’ve all been there: you record your own voice (for example on you answering machine) and when you hear it back it sounds really awkward. It’s that feeling, but then with video… in a foreign language… 1 hour long. Anyway, I learned a lot from seeing this back, and I can definitely use the video to improve my future talks. For example: I now see I really need to be more aware of where I keep my hands when I’m speaking, and that my English still has this annoying dutch accent going on

Anyway, if you missed the talk or if you want to see it again, or if you simply want to see me speak in front of an audience (hi mom!) this is your chance!

View the video on blip.tv

The slides hard to read on the video because of the low picture quality. Therefor I’d recommend keeping the associated slides next to it and click along with me.

Many thanks to PHPNW, Magma Digital and everybody else involved in organizing this great conference and making and publishing the videos.

My talk is more than just my slides

Photo by Jeroen van Sluijs

First of all, talks are usually more than just the slides. For example: during the most important part of my last talk, my slides showed a picture of a slice of pizza for about 3 minutes. By browsing through someones slides you might get a general idea on what the talk is about, but you’re missing out on the details – and probably the most important message the speaker is trying to get across. It gets even better when the speaker uses more “Presentation Zen”-like techniques. Thijs Feryn‘s talk “PHP through the eyes of a hoster” is a fantastic talk, but if you would download his slides you would mainly just end up with a bunch of beautiful, but meaningless photographs (and some keywords).

That however is the way slides should look! They should confirm and strengthen what the speaker is telling, not tell the story themselves. Therefore sharing your slides then is not only pointless, but whenever somebody is going to see your slides it’s likely that the viewer will get a crooked view of what you were trying to tell in the first place.

Sharing slides is a gesture
I share them however, and most speakers do. My main reason is that people can browse through the slides again afterwards, and remember what I told them. If I did a good job the pizza-slide will then remind the viewer of the used metaphor and what I was trying to tell when this slide was on the screen. Even people that didn’t see my talk might still get a clue about the contents by looking at the slides. They will probably miss the main clue, but at least they will see a couple of mentioned tools that are worth giving a try – and if that makes you happy that’s just great! In such a case I’m glad you at least found some benefit in the work I’ve done.

Sharing slides is a gesture though. Something extra the speaker does especially for you. Not sharing slides is not “evil”, it’s normal. There are plenty of reasons why a speaker wouldn’t upload his slides. Maybe he thinks the slides themselves shouldn’t be viewed because the viewer would miss out on so much background information and explanations that it makes the talk look plain and stupid. Maybe there’s a reason like copyright restrictions on used photographs, or maybe the speaker doesn’t want to share his slides because he wants to do the same talk somewhere else next month and he doesn’t like it when people in the audience are already reading his slides before he has even started the talk.

Either way, it’s the speaker’s choice and not something that should be taken for granted. Sharing slides is a little bit extra a speaker did for you, and worth saying “thank you” for. Speakers don’t owe you anything, they usually don’t get paid (worse: they often even have to pay for their own expenses just to be at the conference) and they’ve usually put many hours into preparing their talks. All just for you! So next time, be grateful for the efforts they’ve put into it and give them a beer, instead of bitching about speakers that for whatever reason chose not to share their intellectual property with the entire world.

Zwolle, Amsterdam, Manchester and Barcelona

Soon after that I got to repeat the same talk at other places in Europe. During these events I met some great people, had a lot of fun, saw cities I had never been before and I learned a lot as well. I tried to improve my talk based on the feedback I got, and sure enough most people seemed to like it!

In total, I did the talk four times now:

• PFCongres, Zwolle (The Netherlands)
• DPC, Amsterdam (The Netherlands)
• PHPNW10, Manchester (UK)
• PHP Barcelona, Barcelona (Spain)

Enough!

I think four is enough. I assume that the majority of the European PHP community has visited at least one of the events above, and had the chance to see the talk (and besides that I’m getting a bit tired of repeating the same story over and over again ). It’s time for something new! I don’t exactly know what it will be, but I have several ideas on new talks so hopefully I can make at least one of those ideas into a talk, and start submitting to the calls for papers at the different conferences really soon!

Article on database version control

As a last hoorah I’m planning to write an article on database version control, which will probably be published either here or on techportal. Missed the talk? Looking to refresh your memory? Or downloaded the slides and you’re still wondering what the heck I was talking about during the slide with the picture of a sunken boat? No worries! Just keep an eye on this site and you’ll be able to read all about it soon.

But for now: Farewell database version control talk! You’ve been a great friend!

Photo by Nelson Piedra

Well f*ck that. Of course, whenever you regularly meet a group of people you will make some new friends, especially when you share a common interest. You might enjoy drinking beers with them or do some coding on their projects, because you think it’s a great project and you want to help them out. However, you will also make friends when you go out to the pub. Hell, you will probably even make friends at the weekly meeting for people with ingrown toenails – which is great, sure! But saying ingrown toenails are a great thing because “you meet such great people” just seems a bit awkward to me.

Don’t get me wrong: I love the sharing, partying and beer drinking as well, and I wouldn’t want to miss it. The most interesting people you meet on conferences are the people you meet on the social events afterwards, but it’s all a consequence, not a cause. Saying you are part of the community because of it, is like saying to your date that you went through the whole process of “having sex and all” because you like smoking the cigarette at the end so much.

So besides the hippy arguments, the PHP community also has some great, more down-to-earth things to offer for companies, developers and everybody else involved. Below I just listed a couple of arguments I could come up with by brainstorming for ten minutes. I’m pretty sure I might have missed one or two, but it should at least give you an idea.

Contributing to open source projects
On first sight, contributing to open source projects might look like charity. You and your noble steed have come to save the day, and with all your goodness you fix a bug in the project. Hooray! But contributing has some other upsides as well. First of all, it gives you a bit more renown and might make your resume look more interesting. How great would it be if you could put “regular contributor to the Zend Framework project” on there? By contributing you would probably learn a lot as well, just by looking into other people’s code and writing documentation or tests for it.

Or if you’re using an open source product but you need to fix or extend it for your own purpose, giving back the changes you made means you can now keep updating to newer versions of the product. Your changes will be in it, so you won’t be stuck with the same version for the rest of your life because you added some custom hacks and an update would overwrite those changes.

Open sourcing your own projects
This one is rather simple; open sourcing your own project means that – if you do it right – people will start contributing to it. When your company has developed a somewhat useful tool, open sourcing it will probably mean that other people will start using the tool as well. Eventually, some people might come and fix some bugs in it, improve the tool and build some new features for it. Your tool gets enhanced, extended, better – for free!

Conferences and user group meetings

Events like conferences are a great place to learn and to get inspired, but can also be used for recruitment and networking. People might learn from the talks, or from talking to each other. Talks can also be inspirational and work as a trigger for somebody to start investigating a technique or a tool he didn’t think of before.

If you do it right, speaking at such events will give you and the company you’re representing some great renown. Tens, maybe hundreds of potential colleagues, clients and people who might hire you see that you, and the company you’re representing, really know what it’s all about. As a company it’s a great way to show that you take software development seriously, and that your office is a great place to work. The speaker gets a bit more fame, and can add another interesting line on his resume.

I did a little research on this subject by starting a poll about a week ago. The results were somewhat surprising, and they are probably hopelessly unreliable because of the group that answered the poll Still, it gives you a bit of an idea:

67% goes there to talk to interesting people

58% goes there for the talks (which I had expected to be more, since that’s the main thing conferences have to offer – the stuff they’re selling)

53% goes there for the hugging and beer drinking

25% goes there to speak (you might see now what I meant by hopelessly unreliable )

24% goes there to promote themselves

16% goes there to promote their company, brand or product

8% goes there to recruit new developers

19% selected “other”. Answers included “I go there to convert all over to ruby” and one voter goes to PHP conferences to “be away from the missus”

The community online
Whether it’s on IRC or on some forum, there are some great resources out there for getting and sharing information, pretty much for the same reasons as the speaking and attending talks part: the person answering a question get a bit more renown, the person asking the question has his problem solved and can continue. Also, answering questions often forces you to do a little research yourself, how did this work again exactly? Somebody else might give an alternative answer to the question that would work just as well, but you hadn’t thought about that solution yet. You can really learn a lot from answering other people’s questions!

The hippy thing
And then of course, there’s the hippy thing. A great network of people that help each other out, give each other advice, do an open source project together and then drink some beer. This post is to show that it’s not the only thing that matters – but of course it is an important reason why it’s a lot of fun to be a part of the PHP community.

Step 1: tell NetBeans where your tests are

Open the project preferences and enter the directory containing your unittests in the field ‘Test folder’.

If you did this right, you will notice that NetBeans doesn’t show this folder anymore in the projects tree, but instead it shows a separate tree: underneath the tree “Source files” there’s now also a tree called “Test files”:

Step 2: tell NetBeans where PHPUnit is

Go to Netbeans’ general preferences, open the tab ‘PHP, and then the tab ‘Unit Testing’. Here, enter the path to PHPUnit:

Step 3: tell NetBeans how to run the tests

Finally, we need to tell NetBeans how to run our tests. This can be done by pointing it to the phpunit.xml, in the Project Properties dialog (PHPUnit menu):

That’s it! You can now run your unittests using NetBeans!

Running unittests

In the ‘run’ menu a new option has appeared. Just underneath ‘Run Project’ option there’s a new option, ‘Test Project’. When you select this option NetBeans will run your unit tests and show the results:

As you can see something went wrong. It will not surprise you that clicking on the failed test will open the right unittest in your editor, showing you the line that failed so you can start figuring out what went wrong. The Test Results tab is sometimes a bit cryptic as to what went wrong, so it might help to open the Output dialog as well (menu Window -> Output -> Output), as the raw output of phpunit is captured here.

Code coverage

NetBeans can also capture the code coverage while you’re testing. To do this you first need to enable this by right-clicking the project and then selecting “Code Coverage” -> “Collect and Display Code Coverage”. Please note that you have to got the xdebug extension installed before you can do this, otherwise PHPUnit won’t run.

Once enabled you can run phpunit again just as you did earlier, but when it’s done running it will show in your source files which code was tested and which code was not, by marking it either green or red:

Upsides

There are probably a lot more things you can do with the NetBeans – PHPUnit integration. I haven’t got time yet to investigate everything that’s possible, but I think that this already is quite nifty. Opening my iTerm and typing ‘phpunit’ usually isn’t that much of a hassle, but generating, opening and browsing the code coverage HTML can sometimes be quite time consuming. Now, I can just press ^ F6 and the tests are run and the tested/untested lines are marked green and red in my editor. Great!

Downsides

There are downsides. First of all, you need to be able to run PHPUnit from the OS your IDE is on – so probably that means that you need to be running your webserver locally. Personally I like to run my webserver on a vmware image that represents the production environment, and I share the folder where my project is in using a hgfs mount. This means I couldn’t use PHPUnit that I installed on my vmware image, but I had to install it again on Mac OSX. This not only means extra work, but it also means the environment I’m running my tests on it not longer representative to the production environment.

Before updating your production environment, I think it’s a good idea to first run your tests once more on a more representative environment. Nevertheless the NetBeans – PHPUnit integration saves me quite some time when writing tests, because basically everything is happening in the same screen.