I have two similar data collection projects, both of which collect time-series data and serve it up with graphs and such. The first one I did uses postgres, and manually (in code, not postgres) manages averaging, data at various resolutions, and a few other complicated issues that are inherent in time-series data. For the second project, I went with RRD instead of postgres, and it vastly simplified data management (other than having to write the RRD integration code for javascript myself). Now, I’m working on refactoring these projects, because it makes sense to move to the first project over to RRD and then there’s no reason to have the graphing bits separate.

Step 1: Extract Graphing Code
To start with the 2-project refactoring, I created a new (third) project: Timewise. This node.js project periodically polls a set of configurable endpoints, each of which serve up a single XML page of RRD’s built-in serialization. The XML gets read in and deserialized to a new RRD file, and this vastly simplifies the postgres replication hacks I had in place for Power Hungry. For now, the project consists only of code I yanked out of Hot or Not, but and right now that’s the only project it serves. This refactoring, though, sets up Power Hungry for the refactoring (to RRD) that is coming its way.

Published by

Patrick Schless

I'm a (mostly) ruby/rails developer, living in Chicago. I work at Braintree, and I have lots of side-projects.

Leave a Reply

Your email address will not be published. Required fields are marked *