Integrating Python and Javascript with PyV8

Scania 500/560/580/620 hp 16-litre Euro 3/4/5 V8 engineA hobby project of mine would be made much easier if I could run the same code on the server as I run in the web browser. Projects like Node.js have made Javascript on the server a more realistic prospect, but I don’t want to give up on Python and Django, my preferred web development tools.

The obvious solution to this problem is to embed Javascript in Python and to call the key bits of Javascript code from Python. There are two major Javascript interpreters, Mozilla’s SpiderMonkey and Google’s V8. Unfortunately the python-spidermonkey project is dead and there’s no way of telling if it works with later version of SpiderMonkey. The PyV8 project by contrast is still undergoing active development.

Although PyV8 has a wiki page entitled How To Build it’s not simple to get the project built. They recommend using prebuilt packages, but there are none for recent version of Ubuntu. In this post I’ll describe how to build it on Ubuntu 11.11 and give a simple example of it in action.

The first step is make sure you have the appropriate packages. There may be others that are required and not part of the default install, but there are what I had to install.

sudo aptitude install scons libboost-python-dev

Next you need to checkout both the V8 and PyV8 projects using the commands below.

svn checkout v8
svn checkout pyv8

The key step before building PyV8 is to set the V8_HOME environment variable to the directory where you checked out the V8 code. This allows PyV8 to patch V8 and build it as a static library rather than the default dynamic library. Once you’ve set that you can use the standard Python commands to build and install the library.

cd v8
export PyV8=`pwd`
cd ../pyv8
python build
sudo python install

In future I’ll write more detailed posts about how to use PyV8, but let’s start with a simple example. Mustache is a simple template language that is ideal when you want to create templates in Javascript. There’s actually a Python implementation of Mustache, but let’s pretend that it doesn’t exist.

To start import the PyV8 library and create a JSContext object. These are equivalent to sub-interpreters so you have several instance of your Javascript code running at once.

>>> import PyV8
>>> ctxt = PyV8.JSContext()

Before you can run any Javascript code you need enter() the context. You should also exit() it when you are complete. JSContext objects can be used with with statements to automate this, but for a console session it’s simplest to call the method explicitly. Next we call eval() to run our Javascript code, first by reading in the Mustache library and then to set up our template as a variable.

>>> ctxt.enter()
>>> ctxt.eval(open("mustache.js").read())
>>> ctxt.eval("var template = 'Javascript in Python is {{ opinion }}';")

The final stage is to render the template by dynamically created some Javascript code. The results of the expressions are returned as Python objects, so here rendered contains a Python string.

>>> import random
>>> opinion = random.choice(["cool", "great", "nice", "insane"])
>>> rendered = ctxt.eval("Mustache.to_html(template, { opinion: '%s' })" % (opinion, ))
>>> print rendered
Javascript in Python is nice

There’s much more to PyV8 than I’ve described in this post, including calling Python code from Javascript but unfortunately the V8 and PyV8 documentation is a bit lacking. I will post some more of my discoveries in future posts.

Photo of Scania 500/560/580/620 hp 16-litre Euro 3/4/5 V8 engine by Scania Group.

Crowd Sourcing Mapping

GPS Status app - HTC Desire by avlxyzRecently Google announced that they were making their crowd sourcing mapping tools available to users in the United States. This tool lets uses edit Google Maps, adding businesses and even roads, railways and rivers. This raises interesting questions about whether wisdom of the crowd can be applied to data that requires a high degree of accuracy.

Open Street Map has been doing this since 2004, and has put together an amazing resource of free map data, but only recently has Google begun to allow people to edit its maps for large parts of the world.

Accurate mapping data is terribly important. While the majority of Google Maps queries are likely to be “how do I get from my house to my aunt’s?” some are much more important. A war was almost caused when the border between Nicaraguan and Costa Rica was incorrectly placed. While a war is a little far-fetched, it’s not hard to imagine how a mistake on map could cost someone’s life in a medical emergency.

Originally Open Street Map required budding cartographers to get out with their GPS devices and manually record their position. With the explosion of satellite* mapping information creating maps just involves sitting at your webbrowser and clicking. But how accurate are the satellite images? It’s easy to find a road on Google Maps that disagrees with the imagines underneath it. It’s not so easy to work out which one is correct. The height and angle of the terrain, the location of the plane and other processing errors may make the satellite imagery not line up with GPS data.

Like most crowd sourced data sources Google’s Map Maker and Open Streetmap have a voting process so vandalism will be weeded out, but for people sitting at their computers looking at the same, possibly misleading photos, it won’t improve the accuracy of the changes.

Without knowing more about the collection process for the satellite imagery it’s hard to know what level of accuracy they have. Presumably it’s pretty good, but we need to decide how accurate maps need to be. When you’re looking a country level map even a kilometer here or there doesn’t matter too much, but get down to the level of a walker and suddenly centimeters become important.

What’s more important, masses of mapping information or accurate maps? I’m not sure, but I think it’s probably the latter.

If you want to see the mass of data that is being created by the crowd, Map Maker Pulse is a fascinating and hypnotic site to visit.

The final issue is that of licensing. Open Street Map is very clear that any time you spend improving their maps is rewarded by your work being released under an open source license. Google’s terms of service require you to given them a license to do whatever they like to your work. They do not need to make your changes available for others to use. Google is asking you to do high precision work for free and then taking that work and locking it inside in Google Maps, and that doesn’t seem like a fair trade to me.

* Hopefully no one is actually using imagery from a satellite for mapping, but rather photographs taken from a plane.

Photo of GPS Status app – HTC Desire by avlxyz.

Can the entrance barrier ever be too low?

Stop Sign by thecrazyfilmgirlYesterday Google announced a new feature for Google Code’s Project Hosting. You can now edit files directly in your browser and commit them straight into the repository, or, if you don’t have commit privileges, attach your changes as a patch in the issue tracker.

If you’re trying to run a successful open source project then the key thing you want is more contributors. The more people adding to your project the better and more useful it will become, and the more likely it is to rise out of the swamp of forgotten, unused projects to become something that is well known and respected.

It’s often been said that to encourage interaction you need to lower the barrier so that people can contribute with little or no effort on their part. Initially open source projects are run by people who are scratching their own itches, and producing something that is useful to themselves. Google’s intention with this feature is clearly to allow someone to think “Project X” has a bug, I’ll just modified the code and send the developers a patch. The edit feature is very easy to find, with a prominent “Edit File” link at the top of the screen when you’re browsing the source code so Google have clearly succeeded in that respect.

Editing a file on Google Code

My big concern here is that committing untested code to your repository is right up there at top of the list of things that programmers should never, ever, do. I like to think of myself as an expert Python programmer, but I’ll occasionally make simple mistakes like missing a comma or a bracket. It’s rare that anything beyond a trivially small change will work perfectly first time. Only by running the code do you pick up these and ensure that your code is at least partially working.

I’m all for making it easy to contribute, but does contributing a large number of untested changes really help anyone? I’m not so sure. Certainly this feature is brilliant for making changes to documentation where all you need to do is to read the file to know that the change is correct, but it seems a long way from best-practice for making code changes.

Perhaps I should be thinking about this as a useful tool for sketching out possible changes to code. If you treat it as the ability to make ‘pseudo-code’ changes to a file to demonstrate how you might tackle a problem it seems to make more sense, but open source has always lived by the mantra ‘if you want it fixed, fix it yourself’.

I suppose I should worry about getting my pet open source project to a state where people want to contribute changes of any quality, and then I can worry about making the changes better!

Photo of Stop Sign by thecrazyfilmgirl.

Google Sync For iPhone

Google released an excellent new tool for automatically syncing your iPhone’s contacts and calendars and GMail and Google Calendar. Although, as is usually the case with Google, the service is marketed as being in Beta it seems to be working very well. Add or edit a contact either on in GMail or on your iPhone and within seconds it’s been synced across. Even contact photos are synchronized. The same is true of calendar entries.

Rather than implement a new syncing application for the iPhone Google have just linked your contacts and calendars to a Microsoft Exchange server. Apple implemented push services from Exchange services in the 2.2 firmware to help market the phone as business capable. Now though, anyone who uses GMail can benefit from this technology.

Push services are useful because rather than your phone checking at a predetermined interval, probably to find that there has been no change, changes are ‘pushed’ to you. This means that not only are changes that do happen reflected much more quickly on your phone, and you don’t waste battery life or network traffic checking unnecessarily. Unfortunately it doesn’t support email syncing, but surely that’s only a matter of time?

Visit to get your iPhone set up.