Blog

Iterating over Symfony Forms for Custom Output

June 19th, 2009 by Matt Daum

Recently we were working on a project in which we needed to switch from forms auto-formatting themselves ( <?php echo $form;?>) to allow for much more customization in the output.   While there is a Symfony Forms for Designers chapter in the forms documentation, it doesn’t help much for iterating over a form object and customizing the output.  There is a simple foreach($form as $field) that will iterate over every field in the form.  The problem with this is when you have embedded forms and you want to do something different with the formatting on them.  This will iterate over all the fields and you will not know when the field is the embedded form or not.  So we came up with the following which works:

foreach($form as $field)
{
  // If this is true, then that means its an embedded form.
  if(get_class($field)=='sfFormFieldSchema')
  {
    foreach($field as $f)
    {
      // Don't want to see hidden field labels
      if(!$f->isHidden())
        echo $f->renderLabel()." ";
      echo $f->renderError();
      echo $f->render();
    }
  }
  else
  {
    if(!$field->isHidden())
        echo $field->renderLabel()." ";
    echo $field->renderError();
    echo $field->render();
  }
}

This will work fine for a form with as many as single embedded forms.  It allows you to easily use the same view for multiple forms and be able to customize their embedded forms easily.  If you have only one form that you will be doing this with, and are worried about performance we recommend then not using a foreach loop and doing it by hand, this will save you on performance as you will not have as many if statements in each iteration.


Extracting text from PDFs without pdftotext

June 16th, 2009 by Ashish Datta

For a recent project, I had to extract the text out of a PDF so that I could save it into a database table.

Normally, I would of used the popular pdftotext program but it wasn’t available in the particular environment I was working in. I contact support and they advised that the XPDF package has several X windows dependencies and that’s why they had not installed it. Fair enough.

I poked around a bit and found Apache’s PDFBox library. I downloaded the package and looked at the examples. Sure enough there was a program called “ExtractText” that did exactly what I wanted.

Using ExtractText is similar to pdftotext – just pass in the PDF file and the text comes back. Awesome.

Anyway, hats off to Ben Litchfield who wrote the ExtractText example. I rebuilt the ExtactText.java file as a standalone project and packaged it as a JAR.

I’ve attached the JAR and Eclipse project if anyone wants a copy of either.

The JAR

The Eclipse Project


Loading Different Javascript/CSS Files in Different Environments

June 9th, 2009 by Matt Daum

Today we were working on minifying our Javascript and CSS files for a site we are launching tomorrow.  We came across that we didn’t want to have to get the repository out of sync and update view.yml to only include our single CSS style sheet and one javascript file when working between developement environment and production.  What we wanted was the following in view.yml the javascripts: and stylesheets: parameters to change depending on what environment you are loading, however using prod: and dev: in this didn’t work like app.yml does.  An example of this would be:

all:
  title: My site
dev:
  stylesheets:[styleone.css,styletwo.css,stylethree.css]
  javascripts:[one.js,two.js,three.js]
prod:
  stylesheets:[style.min.css]
  javascripts:[main.min.js]

This way we could have them load in production much quicker and reduce load time, however still easily debug the files in development environments.  After looking around a bit we couldn’t find any standard current solution so we came up with the following.  In app.yml we defined two variables for the dev and production environments – javascript_files and css_files.  Here you put the list of the files you wanted to loaded in each environment.  Then in view.yml it now looks like:

all:
  title: My Title
  stylesheets: [<?php echo sfConfig::get('app_css_files');?>]
  javascripts: [<?php echo sfConfig::get('app_javascript_files');?>]

The configuration variables get their value depending on the environment you are running in so it loads the proper files.  With this configuration we can easily debug in development environment and still have minified versions of the CSS and Javascript in production environement.   Hopefully this will save some of you time and make your life easier.


Taking Your Application International

May 27th, 2009 by Matt Daum

Many clients develop their applications with only one language in mind.  Recently one of our clients after we had developed two different applications for them decided that the applications needed to be translated into seven different languages.  At first the client said, “We’ll just make seven copies of it, and update each one separately.”  While this may at first seem the be a quick simple solution, think of the long term affects of this.  First, if the application is large, you are going to be wasting much space.  Second, and most important, using this approach you are going to be stuck trying to maintain seven different copies of the same application; every update for each each application will have to be made seven times.  Not only is this error prone, but it is inefficient.

Our solution?  Use a common practice called internationalization (i18n) and localization (l10n).  i18n and l10n is text translation (from page content to form labels to error message) and localizing of content ( displaying dates, currency, numbers, etc. in a specific format).  For many applications this is not an easy process, and often could require one to go back and rewrite much of the code.  However, we use Symfony which makes the task much easier.  Symfony allows you to use dictionary files and the database to handle this.  Symfony can scan your entire project looking for specific markup(__(”Text here”)) and pull the strings out into a simple XML file which you can enter the translations for.  The file looks like the following:

<?xml version=”1.0″ encoding=”UTF-8″?>
<!DOCTYPE xliff PUBLIC “-//XLIFF//DTD XLIFF//EN” “http://www.oasis-open.org/committees/xliff/documents/xliff.dtd”>
<xliff version=”1.0″>
<file source-language=”en_US” target-language=”en_US” datatype=”plaintext” original=”messages” date=”2009-05-28T17:27:02Z” product-name=”messages”>
<header/>

<body>

<trans-unit id=”1″>
<source>My String To Be Converted</source>
<target/>
</trans-unit>

</trans-unit>
</body>
</file>
</xliff>

With file all you need to do is to modify the <target /> to <target>My Translated Text</target>  When the string “My String To Be Converted”  is output in the application it will be converted to “My Translated Text”.  You’d save this file for example as messages.es.xml if it was the Spanish translation file.

To read more on internationalization with Symfony visit http://www.symfony-project.org/book/1_2/13-I18n-and-L10n.  We use Symfony because of situations like this it allows us to quickly adapt our products to our customer needs.

Always think ahead when developing your applications.  Planning for the future work on an application can save you hours of rewriting.


Monkeys and shakespeare: genetic algorithms with Jenes

April 30th, 2009 by Ashish Datta

The other night at a bar, we started talking about evolution which somehow sparked a discussion about the law of large numbers and the probability that humanity is just a cosmic fluke. Eventually, someone brought up the “monkeys on a typewriter” argument which caused uproar among the philosophers in the group.

This morning, I decided to see what Wikipedia had to say about monkeys and typewriters and eventually stumbled across an article about the “Weasel program” which Richard Dawkins wrote to demonstrate “random variation and non-random cumulative selection in natural and artificial evolutionary systems.” Basically, it simulates the monkeys on a typewriter to produce a line from Hamlet. At this point, I was hooked – I wanted to make one.

I’d experimented with genetic algorithms in a class I took at Tufts and I’ve been increasingly curious since the “evolving Mona Lisa” code got out on the web.

Anyway, I decided to use the Jenes library to whip up some code to “evolve” strings. The Jenes library is absolutely fantastic. It is easy to setup, easy to use, and the documentation is well written and easy to follow.

My implementation is online at: http://setfive.com/evolve.php

And it evolves Dawkin’s Hamlet line in about 3 seconds – link

The code to run the genetic algorithms is written in Java and uses a Jetty container to accept and processes HTTP requests. Using an embedded Jetty container proved to be seamless and the application server seems to running pretty smoothly.

A zip file containing an Eclipse project for the code is available here.

Additionally, a self contained JAR for the server is available here . Start it with java -jar wordga-jetty.jar

As always, questions and comments are welcome.


MS SQL starts on wrong port

April 13th, 2009 by Ashish Datta

Over the past few days we’ve been working with one of our clients to develop an enterprise search solution for one of their databases. Due to various historical reasons, the database is on MS SQL and has to stay that way. No worries right? Fail.

I decided to move forward with Solr because of its DataImportHandler feature and its ability to easily expose search results in various formats (JSON, XML, PHP serialized objects, etc).

For some reason I can’t seem to get MS SQL server to open on the port it is configured to. This proved particularly hairy to debug because I couldn’t tell if my JDBC DSN binding was failing, the JDBC driver was failing, or if the DB server was actually mucking it up.

The JDBC URL I’m using is:

url="jdbc:sqlserver://localhost:1170;databaseName=dbName;User=test;Password=*****"

To try and narrow things down, I used the Eclipse Data Tools package to let me use a JDBC driver from within Eclipse to connect to the server. Using this, I could clearly see that the exception being thrown was that the MS SQL server was not accepting connections on port 1433.
As far as I can tell MS SQL is set up correctly.

sql-server-1

sql-server-2

I checked the MS SQL start up log and it was showing this:

Server is listening on [‘any’ 1170]

I tried using 1170 in Eclipse and Solr and bang everything fell into place.

What is weird is that the configuration is showing that the server should be running on 1433.
If anyone has any idea why this is happening I’d love to know for future reference.

I replicated this behavior on a production machine. In both instances I was using MS SQL Server 2008 Express. On the dev box I was running Windows XP SP2+ and the production box was running Windows Server 2003.


sfWidgetjQueryTimepickr – Symfony timepickr widget

April 6th, 2009 by Ashish Datta

A couple of months ago John Resig posted on his blog about a “new” way for users to pick time.

The component is a jQuery plugin called timepickr and I thought it was particularly neat. Anyway, I finally needed to write a form which had a time component so I figured I’d drop in the timepickr jQuery plugin.

It didn’t look like there was a Symfony Forms widget for it so I whipped one up. You can grab it here.

The only issue with timepickr is that it introduces a ton of dependencies. It requires jQuery, jQuery-ui, jQuery.utils, jQuery.string, and ui.dropslide. Additionally, it needs the dropslide css as well as the timepickr css.

The form widget assumes that you will include jquery-ui by yourself since everyone usually has different naming conventions. It will include the other JS files and CSS files for you.

You can download a package with all of the JS and CSS you need here. Again this DOES NOT include the jQuery-ui stuff. YOU have to include that yourself in your Symfony project as well as on the page that you deploy this widget.

To use it, just drop it in your Symfony project somewhere where classes get autoloaded (projectdir/lib works) and then instantiate a widget with new sfWidgetjQueryTimepickr() . It currently will support all of the timepickr options passed in as widget options (on the constructor). Full documentation for timepickr is here.

Have fun picking time.


GIT Ignores and Symfony

March 22nd, 2009 by Matt Daum

We use GIT for version control at Setfive.  However often with our Symfony projects creating all the ignore files in all the plugins model/base’s and other locations throughout the project get very tiresome.  Since on some larger projects you can end up with 10-20 plugins, having to create an ignore for the autogenerated model, forms, and filters takes a long time.  Today we really quickly just wrote a plugin that allows you to quickly just run symfony util:generate-ignores git --add-ignores and it will automatically place the .gitignore file throughout your project in the correct locations and add them into your next commit.  You can also just have it place them throughout the project, but not add them to the next commit if you drop the --add-ignores option.  The plugin also accepts “cvs” instead of git for cvs based projects.

The reason you do not want the base, om, etc. directories in your repository is because every time a person rebuilds the model they will be updating those files(many times just changing the timestamp at the top of the autogenerated files), which causes uncessarily large commits.

You can get the plugin via http://www.symfony-project.org/plugins/sfSCMIgnoresTaskPlugin or install it via symfony plugin:install sfSCMIgnoresTaskPlugin.

If you have any questions/requests shoot us an email.


Use Greasemonkey to extract your Facebook Phonebook

March 6th, 2009 by anonymous

UPDATE: It looks like the version that got uploaded was missing a * in the trigger URL! That might be the issue everyone is having.

UPDATE: Video of the process: fbimport

Facebook’s API + FBConnect is great but it has some severe limitations. Notably, it doesn’t expose all the functionality available on the Facebook  site. Tonight in particular, I wanted to be able to copy a dump of my friends’ names and phone numbers off the site to load into a fresh cell phone. Unfortunately, looking at the API this isn’t possible.

Never fear – Greasemonkey provides enough of a hook into Firefox that it would be possible to write a UserScript to accomplish this.

Continuing beyond this point is probably against the Facebook TOS and will probably severely void your warranty.

You have been warned.

The following describes how to use this userscript to extract your Facebook “Phonebook”. It produces of a CSV of your friends’ names and phone numbers. Fair warning – this is a rough prototype and does almost no error handling. Also, since the “Phone” field is a free text field I can’t promise people will have formatted their numbers in any sane fashion. But either way it’s a good start to revering lost numbers.

So here is what you need to do to use the script:

1. Install Greasemonkey – https://addons.mozilla.org/en-US/firefox/addon/748

2. Follow these instructions to install the script – http://userscripts.org/about/installing

Edit: The script is also on Userscripts at http://userscripts.org/scripts/show/43681

3. Navigate over to http://m.facebook.com/friends.php? (You’ll have to login)

4. Answer yes to the prompt and sit back – the script will move through your phonebook and eventually dump you a CSV of the results.

5. Copy/Paste the CSV wherever you want.

6. Un-install the Greasemonkey script.

So that’s it, one less walled garden to worry about. And hopefully one less “I lost my cellphone!” event/group on facebook!

The script:

Facebook Phonebook Exporter


// ==UserScript==
// @name fb phonebook
// @namespace setfivefb
// @description Extract name/phonenumber from fb phonebook. Navigate to http://m.facebook.com/friends.php? and switch the tab to "Phonebook". The script will extract your friends' name/phone and present a CSV at the end.
// @include http://m.facebook.com/friends.php?*
// ==/UserScript==

// lets make sure we are actually looking at a phonebook page
var nodesSnapshot = document.evaluate(’//div[@class="nopad"]/div[@class="lnav header"]/small’, document, null, XPathResult.ORDERED_NODE_SNAPSHOT_TYPE, null );
var desc = nodesSnapshot.snapshotItem(0).childNodes;

for(var i=0; i < desc.length; i++){
// figure out what page we are on
if(desc[i].nodeName == “B” && desc[i].firstChild.textContent != “Phonebook”){
return;
}
}

// get the pagination link
var allLinks = document.getElementsByTagName(”a”);
var nextLink = “”;

nodesSnapshot = document.evaluate(’//div[@class="pager"]‘, document, null, XPathResult.ORDERED_NODE_SNAPSHOT_TYPE, null );
var pagerLinks = nodesSnapshot.snapshotItem(0).childNodes;

// get the stuff back from the cache
var arr = GM_getValue(”setfivefb”);
if(arr == null){
arr = new Array();
}else{
arr = new String(arr).split(”^”);
}

// we are on the first page so reset our name/phone cache
// and present a conformation dialog
if( new String(pagerLinks[0].nodeName) == “#text” ){
arr = new Array();
var res = confirm(”Do you want to extract your phonebook?”);
if(!res)
return;
}

for(var i=0; i < pagerLinks.length; i++){

if(pagerLinks[i].innerHTML == “Next”){
nextLink = “http://m.facebook.com” + new String(pagerLinks[i].getAttribute(”href”));
break;
}
}

// grab all the names+phones off the page
nodesSnapshot = document.evaluate(’//table[@class="results"]//tr’, document, null, XPathResult.ORDERED_NODE_SNAPSHOT_TYPE, null );
var obj;
var ns;
var t;
var hasMatch = false;

for ( var i=0 ; i < nodesSnapshot.snapshotLength; i++ )
{
ns = nodesSnapshot.snapshotItem(i).childNodes[1].childNodes;
obj = new String();
hasMatch = false;

for(var j=0; j 0){

if(t.indexOf(”:”) != -1 && ns[j].childNodes[1] && !hasMatch){
obj += “,” + ns[j].childNodes[1].textContent;
hasMatch = true;
}else{
obj = t;
}

}
}

if(obj.indexOf(”,”) == -1)
obj += “,NaN”;

arr.push(obj);
}

// throw the array into our temporary storage spot and push the page forward
GM_setValue(”setfivefb”, arr.join(”^”));

if(nextLink != “”)
window.location = nextLink;
else{
// display the results of this mess somewhere
var str = new Array();
var tmp;

for(var i=0; i < arr.length; i++){
str.push(arr[i]);
}

document.body.innerHTML = str.join(”
“);
}


Gmail fail

March 1st, 2009 by Ashish Datta

Looks like someone at gmail forgot to make sure the system won’t display negative time for future timestamped emails. I’m not sure where/how this email got a bad timestamp but gmail is displaying it as being sent in the future!
See it!