Scraping the Senate

i am excited about this. i am really excited about this. this frelling rox.

The United States government and the Semantic Web are a perfect match: imagine all of those senators and representatives, each query-able by age, party affiliation, bills proposed, committee membership, and voting record. For the last few years, I've wanted to collect as much data on the U.S. government as I could, convert it to RDF, and build a site and a web service that make it possible to explore that data. This will be my goal over the next year, and I'll document my progress here on XML.com.

I am aware that I am reinventing the wheel with this project. Several other sites attempt to map the government, most notably the The Open Government Information Awareness project. Wikipedia also has solid, cross-linked information on the current U.S. government, among many other sites. But I still think it's a worthy undertaking because I'm curious to see whether the promise of the Semantic Web holds true.

Does creating a Semantic Web of data make it easier to analyze and explore that data in new ways? In addition to testing the Semantic Web concept, if all goes well, I'll have a nicely organized map of the U.S. government, structured using publicly available ontologies, available in a single, reliable format (RDF), which anyone can incorporate into their own Semantic Web projects. It seems worth trying.

worth trying? more like worth its weight in frelling gold. the equivalent project is the amazing british site TheyWorkForYou.com, which is best described in its own words:

Everything MPs say in the House of Commons is recorded in a document called Hansard. TheyWorkForYou.com helps make sense of this vital democratic resource and, crucially, allows you to add your own comments and links to the official transcripts of Parliament.

The ability to trawl through everything said on the floor of the congress, to search persons involved in government whether congressperson or introduced speaker, to stick our grubby democratic fingers into the guts of the machine - and make comments - is something we have needed now for, oh, say, the entire length of the existence of the republic. i am well and truly excited, i am.

now hurry up & publish it!