Blog

Recently we found ourselves having a really weird problem on a project: Every time a page was loaded it seemed a bunch of different Javascript functions were being called multiple times and making many widgets on the page break and we couldn’t figure it out. Some of the functions were our own code, some part of the package we were using. After a while we narrowed it down to that all the functions in the

$(document).ready(...);

we’re being called twice. We had never seen this. After about an hour of removing javascript files and just headbanging, and many thanks to Ashish, we found the root cause. We had written in a quick hack for a late proof of concept to string replace on the entire HTML of a page a specific string. We did it this way:

$('body').html($('body').html().replace(/{REPLACETEXT}/i, "More important text"));

Basically we used a regex to parse the entire HTML tree and then replace it with the updated text. Unknowingly this caused the document ready event to be triggered again(though now it makes sense), causing many widgets to get extra HTML.

Let this save you some headbanging.

Wow its been awhile!

We’ve been insanely busy over the last month or so. We launched Setfive Ventures and are anxiously anticipating the launch of both WeGov and OmniStrat in the immediate future. There are also a handful of internal project that should be rolling out before Christmas. Get Excited.

Anyway, the jQuery UI Dialog class is pretty sweet. Basically, it provides a class to display a modal dialog box from a regular old DOM element (a div, span, or whatever.)

One of the thing that isn’t explained well (or at all?) in the documentation is that you can create a dialog with on the fly HTML! I found this out after posting on the Google Group asking why this feature didn’t exist (it does. Ashish fail.)

So if you want to create a dialog with on the fly HTML all you need to do is:

$("<p>Hello World!</p>").dialog();

Pretty sweet.

Preface: This is a bad idea with untested code. If you deploy it on a production server bad things will happen.

A few weeks ago I was trolling the Internet and ran across an interesting piece over at John Resig’s blog about Javascript WebWorkers. Basically, WebWorkers are a draft recommendation that allow you to run Javascript functions on a background (non-UI thread) thread. Essentially, they would allow you to do long running computations without hanging the browser’s UI. Pretty neat. Problem is that they are currently only available in Firefox 3.5+, Safari 4, and Chrome 3.0

In my never ending quest to use every buzzword at least once I decided to try and implement a compatibility layer to bring support for WebWorkers to other browsers. The plan was to use Java6’s new embeded Javascript interpreter (it’s just Rhino) to execute the WebWorker code server side and then pipe the output back to the client. Again, this is really just a proof of concept.

There are three parts to the rig: the client Javascript library, a Flex/AS3 application for streaming client to server communication, and a Java application that uses Rhino to execute the Javascript.

Client Javascript

The client Javascript detects the user’s browser and then will define a “Worker” object if the user’s browser doesn’t support WebWorkers. The new “Worker” object uses the Flex application to pass messages back and forth to the server and calls the user’s onmessage function when data arrives from the server.

I sniped the browser detection code from Quirksmode and it seems to work fairly well. The rest of the code is below:

BrowserDetect.init();

var sfWebWorkers = new Array();

var SF_WORKER_SERVER = "192.168.1.102";

var SF_WORKER_PORT = "9999";

var sfWwConduitIsLoaded = false;

function sfWebWorkersRecieveData(msg){

  var obj = $.evalJSON( msg );

  var e = new Object();

  e.data = obj.data;

  sfWebWorkers[ obj.sfWebWorkerId ].onmessage( e );

}

function sfWebWorkersSWFReady(isReady){

  sfWwConduitIsLoaded = true;

}

if(!((BrowserDetect.browser == "Firefox" && BrowserDetect.version == "3.5")

	    || (BrowserDetect.browser == "Safari" && BrowserDetect.version == "4")) ){

  $(document).ready( function(){

    var params = "{\"allowscriptaccess\": \"always\"}";

    var vars = "{\"server\": \"" + SF_WORKER_SERVER + "\"" 

                + ", \"port\": \"" + SF_WORKER_PORT + "\"}";

    $("body").append( "


" );                

    $("body").append( "" );

  });

  var Worker = function(fileName){ 

                 this.messages = new Array();

	               this.fileName = fileName;

	               this.id = sfWebWorkers.length;

	               this.isLoaded = false;

	               sfWebWorkers.push( this );

	               var pathToFile = "http://" + window.location.hostname 

	                                 + ":" + window.location.port + "/" + fileName;

                 var myId = this.id;

                 var loadWorker = function(){

                    if( sfWwConduitIsLoaded ){

                      sfWebWorkers[ myId ].isLoaded = true;

                      getFlashMovie("sfWebWorker").sendDataToFlash( 

                          $.toJSON( { message_type: 1, id: myId, message: pathToFile } ) );

                    }else{

                        window.setTimeout( function(){ loadWorker(); }, 500 );

                    }

                 };

                 loadWorker(); 

	             };

    Worker.prototype.postMessage = 

	    function(data){

	     var myId = this.id;

	     var isLoaded = this.isLoaded;

	     var sendData = function(data){

	       if( sfWwConduitIsLoaded ){

            var e = new Object();

            e.data = data;

            getFlashMovie("sfWebWorker").sendDataToFlash( 

                          $.toJSON( { message_type: 2, id: myId, message: $.toJSON(e) } ) );

         }else{

            window.setTimeout( function(){ sendData(data); }, 500 );

         }

	     };

	     sendData(data);

	   };

  }

function getFlashMovie(movieName) {

  var isIE = navigator.appName.indexOf("Microsoft") != -1;

  return (isIE) ? window[movieName] : document[movieName];  

}

Flex/AS3 Application

The Flex application is basically a dumb conduit between the server and the client. All it really does is pass messages between the Java on the server and the Javascript on the client.

The trickiest part of getting this to work was Adobe’s insane rules for allowing their Socket classes to connect to servers. In order for the client to successfully connect to the server you need to serve a XML policy file from port 843. Additionally, this file can’t be served by a HTTP server but must be a custom server that only spits back the file along with a null carriage return. A detailed description of this abortion is here http://www.adobe.com/devnet/flashplayer/articles/socket_policy_files.html

This really posses two problems. One, you need to be running some random “policy file server” for Flex sockets to be of any use. And two, since 843 is a privileged port, this server can’t be started by a regular user.

The most interesting parts of the ActionScript are probably the snippets that call out to Javascript functions:

ExternalInterface.call("sfWebWorkersSWFReady", true);

Java Server

The most complicated part of this whole thing is probably the Java application that actually executes the WebWorker Javascript. All the communication between the Flex and Java is done entirely with JSON. The server basically does the following:

1. Listen for connections from the Flex and accept them when they come in.
2. When a message comes in – it can either be a request to create a new web worker or a postMessage() event containing some data for an existing worker.
3. If it’s a request for a new worker, the server will download the Javascript file and then execute it inside a Rhino container.
4. Otherwise if Flex passed a postMessage() message the server will forward that data to the running web worker.
5. The other event that happens is that a web worker can send messages back to the Flex.

Anyway, I tested this on IE7+ and it seemed to work decently well. Per the warning on top I don’t want to leave this running on a live server anywhere.

If you want to get it to actually run, do the following:

1. Download the zip of all the sources here.
2. Start the JAR in WebWorkerServer/WebWorkerServer.jar with java -jar WebWorkerServer.jar 9999
3. On the top of web/sfwwcompat.js change the IP address or the server to where your Java server is located (localhost if you want)
4. Open web/wwsha1.html in IE or Chrome 2.0 and you should see stuff happen.

What’s in the box:

• web/ contains the Javascript and a demo.
• WebWorkerConduit/ contains the Flex applicaiton.
• WebWorkerServer/ contains the Java server.

Credits: I borrowed the WebWorker SHA1 implementation from John Resig who adapted it from Ray C Morgan.

So here is another crazy idea. Instead of executing the WebWorker code on the server, would it be possible to dynamically make the WebWorker code re-entrant using setTimeout() on the client where loop structures exist?

Over the past few weeks we had the opportunity to work with FanFeedr to put together some widgets for their sports news platform. Previously, FanFeedr had been using Sprout to build their widgets but this required someone to hand build a Flash widget for every “resource” on FanFeedr (there are a lot). In addition, since the Sprout widgets are Flash they aren’t easily crawled by search engines.

Our widgets are different. They allow FanFeedr to generate widgets on the fly for any of their pages and allow users to customize the color schemes. Check out a widget builder for the NY Yankees here.

Basically, our widget builder works by allowing users to customize the size and colors used in the widget. This data is serialized as a JSON object and then base64 encoded so that it can be sent to the “generator” on the server. Then, the server unpacks the payload and builds a widget according to the data specified in the JSON object. In addition, our embed code includes a noscript tags so that search engines pick up the links in the widget as well.

Anyway, working with FanFeedr was a great experience and we hope to continue our relationship moving forward. Go build yourself a widget!

A couple of months ago John Resig posted on his blog about a “new” way for users to pick time.

The component is a jQuery plugin called timepickr and I thought it was particularly neat. Anyway, I finally needed to write a form which had a time component so I figured I’d drop in the timepickr jQuery plugin.

It didn’t look like there was a Symfony Forms widget for it so I whipped one up. You can grab it here.

The only issue with timepickr is that it introduces a ton of dependencies. It requires jQuery, jQuery-ui, jQuery.utils, jQuery.string, and ui.dropslide. Additionally, it needs the dropslide css as well as the timepickr css.

The form widget assumes that you will include jquery-ui by yourself since everyone usually has different naming conventions. It will include the other JS files and CSS files for you.

You can download a package with all of the JS and CSS you need here. Again this DOES NOT include the jQuery-ui stuff. YOU have to include that yourself in your Symfony project as well as on the page that you deploy this widget.

To use it, just drop it in your Symfony project somewhere where classes get autoloaded (projectdir/lib works) and then instantiate a widget with new sfWidgetjQueryTimepickr() . It currently will support all of the timepickr options passed in as widget options (on the constructor). Full documentation for timepickr is here.

Have fun picking time.

10/12/2009 UPDATE: Added fixes from Marcel Chastain

UPDATE: It looks like the version that got uploaded was missing a * in the trigger URL! That might be the issue everyone is having.


UPDATE: Video of the process: fbimport

Facebook’s API + FBConnect is great but it has some severe limitations. Notably, it doesn’t expose all the functionality available on the Facebook  site. Tonight in particular, I wanted to be able to copy a dump of my friends’ names and phone numbers off the site to load into a fresh cell phone. Unfortunately, looking at the API this isn’t possible.

Never fear – Greasemonkey provides enough of a hook into Firefox that it would be possible to write a UserScript to accomplish this.

Continuing beyond this point is probably against the Facebook TOS and will probably severely void your warranty.

You have been warned.

The following describes how to use this userscript to extract your Facebook “Phonebook”. It produces of a CSV of your friends’ names and phone numbers. Fair warning – this is a rough prototype and does almost no error handling. Also, since the “Phone” field is a free text field I can’t promise people will have formatted their numbers in any sane fashion. But either way it’s a good start to revering lost numbers.

So here is what you need to do to use the script:

1. Install Greasemonkey – https://addons.mozilla.org/en-US/firefox/addon/748

2. Follow these instructions to install the script – http://userscripts.org/about/installing

Edit: The script is also on Userscripts at http://userscripts.org/scripts/show/43681

3. Navigate over to http://m.facebook.com/friends.php? (You’ll have to login)

4. Answer yes to the prompt and sit back – the script will move through your phonebook and eventually dump you a CSV of the results.

5. Copy/Paste the CSV wherever you want.

6. Un-install the Greasemonkey script.

So that’s it, one less walled garden to worry about. And hopefully one less “I lost my cellphone!” event/group on facebook!

The script:

Facebook Phonebook Exporter

// ==UserScript==
// @name           fb phonebook
// @namespace      setfivefb
// @description    Extract name/phonenumber from fb phonebook. Navigate to http://m.facebook.com/friends.php? and switch the tab to "Phonebook". The script will extract your friends' name/phone and present a CSV at the end.
// @include        http://m.facebook.com/friends.php?*
// ==/UserScript==

// lets make sure we are actually looking at a phonebook page
var nodesSnapshot = document.evaluate('//div[@class="nopad"]/div[@class="nd header"]/small', document, null, XPathResult.ORDERED_NODE_SNAPSHOT_TYPE, null );
var desc = nodesSnapshot.snapshotItem(2).childNodes;

for(var i=0; i < desc.length; i++){
  // figure out what page we are on
  if(desc[i].nodeName == "B" && desc[i].firstChild.textContent != "Phonebook"){
    return;
 }
}

// get the pagination link
var allLinks = document.getElementsByTagName("a");
var nextLink = "";

nodesSnapshot = document.evaluate('//div[@class="pager"]', document, null, XPathResult.ORDERED_NODE_SNAPSHOT_TYPE, null );
var pagerLinks = nodesSnapshot.snapshotItem(0).childNodes;

// get the stuff back from the cache
var arr = GM_getValue("setfivefb");
if(arr == null){
  arr = new Array();
}else{
  arr = new String(arr).split("^");
}

// we are on the first page so reset our name/phone cache
// and present a conformation dialog
if( new String(pagerLinks[0].nodeName) == "#text" ){
  arr = new Array();
  var res = confirm("Do you want to extract your phonebook?");
  if(!res)
   return;
}

for(var i=0; i < pagerLinks.length; i++){

  if(pagerLinks[i].innerHTML == "Next"){
  nextLink = "http://m.facebook.com" + new String(pagerLinks[i].getAttribute("href"));
  break;
}
}

// grab all the names+phones off the page
nodesSnapshot = document.evaluate('//table[@class="results"]//tr', document, null, XPathResult.ORDERED_NODE_SNAPSHOT_TYPE, null );
var obj;
var ns;
var t;
var hasMatch = false;

for ( var i=0 ; i < nodesSnapshot.snapshotLength; i++ )
{
  ns = nodesSnapshot.snapshotItem(i).childNodes[1].childNodes;
  obj = new String();
  hasMatch = false;

  for(var j=0; j  0){

  if(t.indexOf(":") != -1 && ns[j].childNodes[1] && !hasMatch){
  obj += "," + ns[j].childNodes[1].textContent;
  hasMatch = true;
}else{
  obj = t;
}

}
}

if(obj.indexOf(",") == -1)
  obj += ",NaN";

   arr.push(obj);
}

// throw the array into our temporary storage spot and push the page forward
GM_setValue("setfivefb", arr.join("^"));

if(nextLink != "")
  window.location = nextLink;
else{
  // display the results of this mess somewhere
  var str = new Array();
  var tmp;

  for(var i=0; i < arr.length; i++){
  str.push(arr[i]);
}

document.body.innerHTML = str.join("");
}

One of our clients came to us a few days ago asking us if we could build something to aggregate several RSS feeds and then display it on their site.

Easy enough. Except the caveats were 1) This had to be done on a shoe string budget and 2) We had no permission to script on the server side (only client side scripting allowed kids).

We put our heads together and realized that Yahoo Pipes would provide the functionality to easily combine several RSS feeds, sort them, filter them, ect. And best part? It’s all built and all free. Pipes also provides the functionality to export a pipe as a JSON feed (as opposed to RSS).

So that takes care of problem 1 and almost fixes problem 2. Of course the remaining problem is getting around the cross domain XHR restrictions on the JSON. I did some poking around the Yahoo Pipes documentation and it turns out you can add a _callback parameter to the URL and bang it wraps the JSON in a JS callback!

Issues solved. The final part was to mix in a little jQuery to load the JSON+callback feed into a <script> tag and then display it.

Total lines of code: 27 lines of javascript.

EDIT. Now with code! rsspipes

The other day one of my friends jokingly mentioned that he thought strippers were being “marginalized” as profession. I thought this was funny, so I leveraged the powers of the internet to prove to him that in fact strippers are less marginalized then say podiatrists.

Enter amimarginalized.com

Anyway, with all of the hub-dub about Javascript being the next “big thing” I decided to give appjet a whirl. Appjet allows developers to develop and deploy server side Javascript applications. Additionally, appjet recently released “appjet in a jar” which lets users deploy the appjet platform on a different server.

Getting the appjet platform up and running was really simple. I already had Java 6 installed on one of our servers so all I had to do was download the appjet jar and fire it up with “java -jar appjet.jar”

In general, getting things moving was pretty easy with appjet. It also just “feels” natural to write Javascript for the client and the server. Switching between client and server is done with a comment directive:

/* appjet:client */
/* appjet:server */

appjet also provides a special directive for CSS styles:

/* appjet:css */

The other feature I really liked about appjet is its support for printing HTML elements. Appjet comes with a tag library that makes creating HTML tags particularly simple. The syntax looks something like this:

print(

FORM({action:”/”, method:”get”, onsubmit: “javascript: return submitJob()”},
SPAN({id: “job-label”}, “Job Title: “),
INPUT({text:”text”, name:”jobField”, id: “jobField”}),
INPUT({type:”hidden”, name:”isPost”, id: “isPost”, value: “1″}),
INPUT({type: “submit”, value: “Submit”})
)
);

And boom you have a XHTML compliant form.

The other neat feature, (from the appjet docs) is that “You can also treat a tag like a JavaScript array (it has all the same methods) and add to it programmatically.” so something like this works:

var list = UL();

list.push(LI(“One item”));
list.push(LI(“Two item”));

["Red","Blue"].forEach(
function(color) {
list.push(LI(color+” item”));
}
);

printp(“The following “+list.length+” items may be of interest.”);
print(list);

I was also really impressed with the appjet persistent storage system. The platform lets you persist arbitrary Javascript objects in “collections” that can then be iterated, filtered, and sorted. On amimarginalized.com I have about 800 elements loaded up and it seems to perform reasonably well.

The one issue I have with the storage library is that there isn’t any way to just load a bunch of data into the system. You can only load data from inside an appjet script file. The problem I ran into is that I hit the Java 64kb file size limit pretty quickly. It would be really awesome if the JAR had some functionality to load up say a file full of JSON objects.

All and all, using appjet has been a positive experience. It was really easy and fun to build an “easy” app using it. I’d really like to know some more about how appjet is put together but the documentation is sparse. The only information I could find was the logos on the download page. They suggest that appjet is composed off AppJet, Rhino, Jetty and of course Java. I’d be interested to know if there are any plans to expose JVM libraries to appjet code. It seems like this would allow the platform to quickly gain extensive library support – including the JDBC.

Anyway, the amimarginalized.com site uses “advanced” algorithms to determine how marginalized your career is. It just counts the number of results on Yahoo BOSS for the job you enter and compares it to a set of about 800 saved jobs.

You can download the source code here.