Andrew Wilkinson

Random Ramblings on Programming

Posts Tagged ‘github

Accessing FitBit Intraday Data

with 40 comments

JoggingFor Christmas my wife and I brought each other a new FitBit One device (Amazon affiliate link included). These are small fitness tracking devices that monitor the number of steps you take, how high you climb and how well you sleep. They’re great for providing motivation to walk that extra bit further, or to take the stairs rather than the lift.

I’ve only had the device for less than a week, but already I’m feeling the benefit of the gamification on FitBit.com. As well as monitoring your fitness it also provides you with goals, achievements and competitions against your friends. The big advantage of the FitBit One over the previous models is that it syncs to recent iPhones, iPads, as well as some Android phones. This means that your computer doesn’t need to be on, and often it will sync without you having to do anything. In the worst case you just have to open the FitBit app to update your stats on the website. Battery life seems good, at about a week.

The FitBit apps sync your data directly to FitBit.com, which is great for seeing your progress quickly. They also provide an API for developers to provide interesting ways to process the data captured by the FitBit device. One glaring omission from the API is any way to get access to the minute by minute data. For a fee of $50 per year you can become a Premium member which allows you do to a CSV export of the raw data. Holding the data, collected by a user hostage is deeply suspect and FitBit should be ashamed of themselves for making this a paid for feature. I have no problem with the rest of the features in the Premium subscription being paid for, but your own raw data should be freely available.

The FitBit API does have the ability to give you the intraday data, but this is not part of the open API and instead is part of the ‘Partner API’. This does not require payment, but you do need to explain to FitBit why you need access to this API call and what you intend to do with it. I do not believe that they would give you access if your goal was to provide a free alternative to the Premium export function.

So, has the free software community provided a solution? A quick search revealed that the GitHub user Wadey had created a library that uses the urls used by the graphs on the FitBit website to extract the intraday data. Unfortunately the library hadn’t been updated in the last three years and a change to the FitBit website had broken it.

Fortunately the changes required to make it work are relatively straightforward, so a fixed version of the library is now available as andrewjw/python-fitbit. The old version of the library relied on you logging into to FitBit.com and extracting some values from the cookies. Instead I take your email address and password and fake a request to the log in page. This captures all of the cookies that are set, and will only break if the log in form elements change.

Another change I made was to extend the example dump.py script. The previous version just dumped the previous day’s values, which is not useful if you want to extract your entire history. In my new version it exports data for every day that you’ve been using your FitBit. It also incrementally updates your data dump if you run it irregularly.

If you’re using Windows you’ll need both Python and Git installed. Once you’ve done that check out my repository at github.com/andrewjw/python-fitbit. Lastly, in the newly checked out directory run python examples/dump.py <email> <password> <dump directory>.


Photo of Jogging by Glenn Euloth.

Written by Andrew Wilkinson

December 30, 2012 at 1:22 pm

Where GitHub (Possibly) Went Wrong

with 6 comments

8 Forks by bitzceltWhile on my delayed train this morning I was listening to episode 80 of the excellent Stack Overflow podcast. In this episode Jeff Atwood was complaining to Joel Spolsky about his problems with GitHub.

GitHub is a social coding site, along the same lines as Sourceforge or Google Code, but focused entirely on the distributed version control system Git. Where GitHub differs from the other project hosting sites, and where I think Jeff’s confusion comes from is that with GitHub the primary structure on their site is that of the developer, not of the project. They treat every developer as a rock star, who is bigger than the projects that they work on.

GitHub makes it incredibly easy to take a codebase, make your own changes and to publish them to world. What GitHub fails to do is to encourage people to collaborate together to push one code base forward. What I’m not suggestion is that branching is a bad idea. Branching code is a useful coding technique which can be used to separate in-development features from other changes until the code has stabilised again. What GitHub focuses on is the changes that an individual developer makes, not the changes required for a particular feature.

dewy branch by calliopeWhen a developer creates a copy of some code of GitHub they get a wiki and an issue tracker as well. This further confuses matters because not only do you have trouble knowing which git tree is the correct one to pull from, but you also don’t know where to report bugs or go to for documentation.

Google Code seems to be in a better position for combining distributed version control with project management. They have an excellent wiki and issue tracker, and give each project a straightforward and simple homepage. You can also use Mercurial, which is similar to git, as your version control system. All that they need to do is allow developers to publish their own changes, but in a markedly separate section to the core code of the project.

I can see how GitHub is nice for developers, but in any mildly successful open source project the number of users vastly outweighs the number of developers. It seems crazy to me to make your primary web presence suited only for the minority of people who are involved with the project.


Photo of 8 Forks by bitzcelt.

Photo of dewy branch by calliope.

Written by Andrew Wilkinson

January 27, 2010 at 1:21 pm