Posts Tagged ‘buildbot’
There are a number of tools for checking whether your Python code meets a coding standard. These include pep8.py, PyChecker and PyLint. Of these, PyLint is the most comprehensive and is the tool which I prefer to use as part of my buildbot checks that run on every commit.
PyLint works by parsing the Python source code itself and checking things like using variables that aren’t defined, missing doc strings and a large array of other checks. A downside of PyLint’s comprehensiveness is that it runs the risk of generating false positives. As it parses the source code itself it struggles with some of Python’s more dynamic features, in particular metaclasses, which, unfortunately, are a key part of Django. In this post I’ll go through the changes I make to the standard PyLint settings to make it more compatible with Django.
This line disables a few problems that are picked up entirely. W0403 stops relative imports from generating a warning, whether you want to disable these or not is really a matter of personal preference. Although I appreciate why there is a check for this, I think this is a bit too picky. W0232 stops a warning appearing when a class has no __init__ method. Django models will produce this warning, but because they’re metaclasses there is nothing wrong with them. Finally, E1101 is generated if you access a member variable that doesn’t exist. Accessing members such as id or objects on a model will trigger this, so it’s simplest just to disable the check.
These makes the output of PyLint easier to parse by Buildbot, if you’re not using it then you probably don’t need to include these lines.
Apart from a limited number of names PyLint tries to enforce a minimum size of three characters in a variable name. As qs is such a useful variable name for a QuerySet I force this be allowed as a good name.
The last change I make is to allow much longer lines. By default PyLint only allows 80 character long lines, but how many people have screens that narrow anymore? Even the argument that it allows you to have two files side by side doesn’t hold water in this age where multiple monitors for developers are the norm.
PyLint uses the exit code to indicate what errors occurred during the run. This confuses Buildbot which assumes that a non-zero return code means the program failed to run, even when using the PyLint buildstep. To work around this I use a simple management command to duplicate the pylint program’s functionality but that doesn’t let the return code propagate back to Builtbot.
from django.core.management.base import BaseCommand from pylint import lint class Command(BaseCommand): def handle(self, *args, **options): lint.Run(list(args + ("--rcfile=../pylint.cfg", )), exit=False)
At work we recently set up Buildbot to replace an in-house continuous integration tool that never really took off. I’ve used Buildbot before, in a hobby capacity, but using it my day job has really brought home to me how important not only testing is, but testing continuously.
Buildbot is pretty simple to explain. Every time you make a commit to your source control system this triggers buildbot to run a series of steps over your code. What these steps are is totally configurable by you. Typically though, you’ll configure it to check out your code, compile it and run unit tests.
I’m a big fan of test driven and issue driven development. With these two methodologies when you want to make a change you record the issue in your bug tracking software, then write a test that detects the deficiency in your code. Once you’ve done that you can start actually making the change that you wanted to make in the first place. By writing down the issue you’re forced into thinking about the change that you want to make, and creating the test ensures that you know when it’s complete.
TDD and IDD are useful things to aim for, but in reality you’ll probably fall short sometimes. Not everything can be unit tested easily, or you might be under significant time pressures that force you to cut corners. One of the projects I manage has a test suite that takes 25 minutes to run. If I’m getting phone calls urging me to fix a problem as soon as possible then I can’t wait for the tests to run before I commit the change.
Because builtbot runs without user intervention you don’t need to remember to do anything. Buildbot is also a nag. If you introduce a bug with a change you quickly get an email letting you know that there is a problem, and if you set those emails up to go to your team then you also have peer pressure to get that problem fixed.
I spend most of my time building Django based sites. For these sites I set up six steps for Buildbot to follow. The first step is the checkout step, then a step to copy a settings file into place. Django stores settings in a Python file and these settings need to be configured for each server that the site is run on. I typically have a file settings_local.py.buildbot stored in the repository and this step copies this file to settings_local.py so the following commands will run with the correct settings. The next step is to download and install the dependencies of the site. I’m going to go into more detail on how I manage settings and dependencies in a future post, but Buildbot just runs the correct commands.
The first three steps that Buildbot runs are just configuration steps. The first of the real test is a simple compilation step. It’s rare that a change gets committed without it ever being run, but it’s still worthwhile to check that the code compiles. It is possible that when merging changes from different branches errors can creep in. Fortunately Python makes it easy to check that all the files in a directory structure compile using the compileall module. This can be run using the command python -m compileall -q which return an exit code of zero if all the files compile, and nonzero if there is an error. This means that if there is an error Buildbot will detect it, and mark the build as failed.
The next step I run is the unit tests. This where the bulk of the testing occurs. I’m not going to go into details on how to write unit tests, but assuming you have them this assures you that your tests pass after every commit. Because Buildbot can be easily configured to build on several different machines on each commit you can easily test that a change you made on a Python 2.5 machine also works on a Python 2.4 machine.
The final step I run is PyLint. PyLint checks that your code complies with a set of coding standards and simple checks. If your unit tests don’t have a 100% coverage (and frankly, whose does?) then PyLint will hep detect that there are no obvious flaws in your code. PyLint takes a bit of time to configure, especially as it doesn’t like some of the metaclass magic that Django uses, but once you have a configuration file written PyLint is an excellent final check.
Hopefully this has given you a flavour of what Buildbot can do for you, and if you’re in a team or are building a cross-platform piece of software then Buildbot really needs to be there, watching your back.