Author: Matt Daum

A graduate of Tufts University, Matt worked at Boston Technologies and Tufts University before starting Setfive Consulting. A connoisseur of daytime television and sitcoms, Matt is an avid outdoorsman, beer brewer, and certified scuba diver. Connect with him on LinkedIn.

Importance of Having Technical Knowledge Onboard

By Matt Daum | 3 Comments on Importance of Having Technical Knowledge Onboard

Today I want to take the chance to stress the importance of having someone onboard that knows some technology. Through our experience we’ve often had clients who were mislead or taken advantage of by other technology firms. Whether it is being given poor advice or being overcharged for simple tasks, we’ve seen it quite a bit. In many of these situations a person with some technological knowledge would have saved them money and time.

What can a person with technical knowledge do?

  • Know what questions to ask – “Do you provide documentation for your code? Do you use a framework? Is your code MVC compliant? etc.”
  • Know when you are being taken advantage of – We’ve seen in the past many clients purchased servers and functionality they just don’t need. A technical person can make sure that you aren’t purchasing five servers when one will do, that you aren’t overpaying for simple hosting, etc.
  • To help choose your development firm. It is very difficult to tell whether a firms code is good. If you do not have knowledge in the area, good code and bad code appear to be the same. Choosing the development firm for your project can make or break the project. If the firm chosen produces messy, inefficient code, it dooms your product before it is even launched.
  • To monitor code quality. Once you’ve picked your development firm, it is important to check in now and then to make sure the quality of the code is of the caliber you are paying for. It is crucial to catch poor code in the beginning, to make sure the firm doesn’t continue to use the same coding style.

What if you don’t want to have a technical person on staff full time? Hire a consulting firm. Many firms provide assistance with choosing your development team and to monitor most of the technical aspects of the product. We offer these services.

Moral: Having technical knowledge onboard prevents your company from being taken advantage of and will often pay for itself in saved time and a quality delivered product.

Guestimating the election with twitter

By Matt Daum | 12 Comments on Guestimating the election with twitter

We were sitting around tonight and decided to whip something together to leverage twitter to get some real time election information.

It is ugly and open to bias but we’re hopping it might show something interesting.

We’re also planning to take snapshots of the map and assemble a time lapse for Wednesday.

See the map live at: http://election.setfive.com

Update at 4.40 EST:

So we’ve captured about 6000 tweets and the map is basically all blue. Just to clarify – we never intended this to be a serious vizualization or estimation of how the election is progressing. The project was soley meant to be a fun peak at how information spreads across Twitter.

Anyway, a couple of people have been asking about our methodoly so I’ll try and explain a bit.

We are using the Twitter Search API to run searches that we thought would indicate that someone just voted or intends to vote for either John McCain or Barrack Obama. Next, we apply some heuristics to the tweets to make sure they really are “just voted” tweets. If the tweet passes through the heuristics we record it for whichever candidate and then record the “from_user_id” to ensure a single user can’t blow up the vote totals.

In order to geolocate a user we are using the twittervision API I get the impression that the twittervision API just scrapes user profiles but I can’t verify this. We probably could have avoided using their API and just scraped ourselves but one less thing to deal with at 4am is always good.

The graph colors are calculated by taking the larger vote total (red vs blue) and then determining in percent, how much larger this is than the total number of votes for that state:

$totalScore=$state[‘redscore’]+$state[‘bluescore’];
$percentage=($state[‘redscore’]-$state[‘bluescore’])/$totalScore;

Anyway, there are defitley other entertaining things to do with twitter – we just haven’t thought of them yet. – Ashish

Joins, for better and for worse

By Matt Daum | 1 Comment on Joins, for better and for worse

Recently I’ve been having one hell of a time with SQL Joins. It isn’t that I don’t understand how to use them, its that they have came to haunt me in completely opposite instances.

First: An application I had written a while ago that was suppose to be kept small, and very simple which continually was enlarged and became very big was running slow. I upgrade the person to our faster server believing it was just that our server that is meant for static sites couldn’t handle it. While it did greatly reduce the load time of the site(1/6) it was still slow for me. I looked into it and found that after all the alterations of the application which we had done, that one page had 1600 queries at its current state.

Before you think down upon our development abilities and our code efficiency let me put everything in perspective. The project was started as a very simple CRUD only database, that we did basically free for the a client who was a friend. Shortly after it continually was being expanded upon, and the friend could only afford minimal costs, so he asked us to just “hack” it together as it was a prototype. Well about a month later after development, we had possible one of the most hacked together prototypes for him. He never wanted to redo it from scratch so we could architect it correctly due to the cost. For the prototype, it worked fine and very fast. However the prototype turned into a beta test for the friend. He added much more information to the database, and the inefficient queries began to show. Now the application is loaded with information, and the inefficient queries are terrible. We have told him that if he plans to use this we should rewrite it from scratch as many other cool features could be used.

So back to the problem. Well we attempted to add JOINs to the program which should cut the number of queries by at least 1/10th. However when I added the left joins via Criteria/Propel I found that it was adding the joins twice. The reason they needed to be left joins is because the foreign keys were not required,

Propel and Primary keys

By Matt Daum | Leave a Comment on Propel and Primary keys

Today I was writing an administrator backend for a project we have. I had the code:

$old_feeds=RssFeedRelationPeer::doSelect($c);
if(count($old_feeds)>0){
   foreach ($old_feeds as $old) $old->delete();
}

I was trying to remove some old foreign constraints before I deleted the main object, however, for the longest time the “$old” objects were not deleting, but no errors were being thrown. I did the usual debugging, added a die() statement inside the count, it was going there. I added a die statement in the foreach, it went there. I thought, “It must not be pulling the right ‘old’ objects.” I then added a $old->getName(); to see exactly what objects it was going through-they were the right ones. It made no sense. I next tired after the $old->delete();to add a $old->save();. The error I received was “You cannot save an object that has been deleted.” This didn’t make sense, since the object was still in the database. It hit me then, I had this problem in the past. Propel hates tables without primary keys. I quickly just added a simple primary key to the model, and the delete statements worked.

Moral: If dealing with Propel and you are getting some unexpected behavior, with zero errors, check to see if you have a primary key; it may save you hours of head banging.