Congrats: 500pearlstreet.com launched!

Our friend Walt launched 500Pearlstreet.com earlier this week. It’s a blog that will be focused on covering white collar crime in the news as well as exploring other topics related to white collar crime. We’re excited to see where Walt takes the blog and wish him the best of luck!

Technically, 500Pearlstreet.com is running Drupal 7 with a couple of of pretty neat modules:

  • The theme is a standard sub-theme of the excellent Omega responsive theme. The amount of configuration possible through the Omega UI is really impressive and a welcome change from how difficult theming was in Drupal 6.
  • In-context links and images are being automatically powered by Zemanta which is pretty neat.
  • OpenCalais is also running in the background providing semantic tagging capabilities. These aren’t exposed yet but hopefully will become useful when there is more content.

Anyway, it should be a good read so drop it into your readers!

Guestimating the election with twitter

We were sitting around tonight and decided to whip something together to leverage twitter to get some real time election information.

It is ugly and open to bias but we’re hopping it might show something interesting.

We’re also planning to take snapshots of the map and assemble a time lapse for Wednesday.

See the map live at: http://election.setfive.com

Update at 4.40 EST:

So we’ve captured about 6000 tweets and the map is basically all blue. Just to clarify – we never intended this to be a serious vizualization or estimation of how the election is progressing. The project was soley meant to be a fun peak at how information spreads across Twitter.

Anyway, a couple of people have been asking about our methodoly so I’ll try and explain a bit.

We are using the Twitter Search API to run searches that we thought would indicate that someone just voted or intends to vote for either John McCain or Barrack Obama. Next, we apply some heuristics to the tweets to make sure they really are “just voted” tweets. If the tweet passes through the heuristics we record it for whichever candidate and then record the “from_user_id” to ensure a single user can’t blow up the vote totals.

In order to geolocate a user we are using the twittervision API I get the impression that the twittervision API just scrapes user profiles but I can’t verify this. We probably could have avoided using their API and just scraped ourselves but one less thing to deal with at 4am is always good.

The graph colors are calculated by taking the larger vote total (red vs blue) and then determining in percent, how much larger this is than the total number of votes for that state:

$totalScore=$state[‘redscore’]+$state[‘bluescore’];
$percentage=($state[‘redscore’]-$state[‘bluescore’])/$totalScore;

Anyway, there are defitley other entertaining things to do with twitter – we just haven’t thought of them yet. – Ashish