My test setup is evolving. I now need to play with buildbot and git integration to get some level of automated build and test management. The only examples of git integration I could find with buildbot and git were the meta buildbot instance and this blog post. The prior’s config has not been published yet (though it appears to be grabbing changes via 4 hour scheduler) and the later is a bit confusing. So I thought I’d document my walk through this little situation since I needed to iron out how it all would work.
Side note: One of continuing issues I have with using buildbot is that while the documentation is great, there are not enough examples of usage for the various version control systems. Having had to set it up for perforce, subversion and now git, it’d be nice to see what folks had previously done. Me thinks we need a cook book for that.
Now this is a test setup to iron out the kinks of a basic work flow. These instructions are a guide for getting started and the setup would need a lot more tuning for a production environment. This post assumes the following:
- that you have an instance of the buildbot server up and running.
- That you have atleast one buildbot slave attached to the buildbot server.
- that you have public git repo you administrate.
- That you have ssh keys setup so that you don’t have to enter passwords when you do a pull to your slave machine from the git repo.
If you don’t have a buildbot instance, go set one up using these instructions. If you don’t have a test setup for a public git repo, take a look at this previous entry in this blog (which also has info on setting up ssh keys).
Here we go:
- In the buildbot/contrib directory, you will find git_buildbot.py. Copy this to the remote server your public git repo is being hosted on. You need to make sure that python and twisted are installed on this host, since the script uses the twisted frame for network interactions. Make sure to edit master variable in the script to point to the server hostname and listening port of your buildbot instance. Example:
- On the remote server, create a post-receive hook script file in your public repo and make sure it’s executable. You would do this under .git/hooks and point the script to run the git_buildbot.py script with arguments. Some like this:
- Once that’s done, you’ll need to edit your buildbot instance’s master.cfg file to add a PBChangesource statement. This will have buildbot listen for the changes being sent by the git_buildbot.py script. Do this:
- Next we’ll add a scheduler to catch the changes. Make sure that you specify the branch as master, otherwise your changes will get ignored by the scheduler:
- Now we’ll add a build factory that will get executed when the changes occur. For simplicity sake, we’ll just pull (or fetch in the case of buildbot) the latest revision of the code. Note that the repourl we’re using is prefaced by ssh. This is why you’ve previously setup your ssh keys between your slave and the git repo:
- Restart your buildbot server and make sure your slave is attached.
- Checkout, edit, commit and push a file in the code base your using to test this setup. You should see the following:
/path/to/script/git_buildbot.py $1 $2 $3
from buildbot.changes.pb import PBChangeSource
c['change_source'] = PBChangeSource()
c['schedulers'] = 
test_schedule = Scheduler(name="test", branch="master", treeStableTimer=0, builderNames=["test-builder"])
test_factory = factory.BuildFactory()
- Buildbot gets the change notice from the post-receive hook.
- Buildbot hands the change off to the builder.
- The buildslave runs the job and successfully gets the code update.
And that’s it. Just a reminder, make sure your hook set to be executable on the git public repo. This tripped me up for a couple of minutes until I reread the docs, so it’s probably a good idea to do a bit more reading on hooks in git. They’re pretty much the same as subversion, only just more of them.