LimeSurvey with load balancers, fixing the user sessions.

For a client we’ve been working with recently it came to our attention that they needed more frontend servers to keep up with the traffic for their surveys. They use LimeSurvey which is powerful open source survey platform. We set the client up in the cloud to scale as necessary with a load balancer in front. This is when we noticed the problem that LimeSurvey doesn’t work well when a user is bouncing between different frontend servers. LimeSurvey keeps all the user’s session attributes on the local server and not in the database. After googling around for a while, we found other people also had this problem before, and no one had really solved it. We figured we would.

We didn’t feel like doing a ton of extra work to reinvent the wheel in terms of storing the session in the database. We snagged most of the code straight from the Symfony “storage” module which handles it’s session management if you want to store the user sessions in a database. After a quick few modifications, we got it up:

<?php
/**
 * Taken from parts the Symfony package "storage" module
 * @author Ashish Datta ashish@setfive.com
 * @author Matt Daum matt@setfive.com
 * Setfive Consulting, LLC                                                     
 */                                                     

class setfiveSession {
                      
  private $options = array();
  private $sessionIdRegenerated;
                                
  public function initialize()  
  {                             
    $this->options =  array('db_table' => 'session',
                            'db_id_col'   => 'sess_id',
                            'db_data_col' => 'sess_data',
                            'db_time_col' => 'sess_time');
                                                          
    session_set_save_handler(array($this, 'sessionOpen'), 
                             array($this, 'sessionClose'),
                             array($this, 'sessionRead'), 
                             array($this, 'sessionWrite'),
                             array($this, 'sessionDestroy'),
                             array($this, 'sessionGC'));    
                                                            
    // start our session                                    
    // session_start();                                     
  }                                                         
                                                            
  /**                                                       
   * Closes a session.                                      
   *                                                        
   * @return boolean true, if the session was closed, otherwise false
   */                                                                
  public function sessionClose()                                     
  {                                                                  
    // do nothing                                                    
    return true;                                                     
  }                                                                  
                                                                     
  /**                                                                
   * Opens a session.                                                
   *                                                                 
   * @param  string $path  (ignored)                                 
   * @param  string $name  (ignored)                                 
   *                                                                 
   * @return boolean true, if the session was opened, otherwise an exception is thrown
   *                                                                                  
   * @throws <b>Exception</b> If a connection with the database does not exist or cannot be created
   */                                                                                              
  public function sessionOpen($path = null, $name = null)                                          
  {                                                                                                
    // we're assuming LimeSurvey all ready has db settings and opened a db connection              
    return true;                                                                                   
  }                                                                                                
                                                                                                   
                                                                                                   
  /**                                                                                              
   * Destroys a session.                                                                           
   *                                                                                               
   * @param  string $id  A session ID                                                              
   *                                                                                               
   * @return bool true, if the session was destroyed, otherwise an exception is thrown             
   *                                                                                               
   * @throws <b>Exception</b> If the session cannot be destroyed.                                  
   */                                                                                              
  public function sessionDestroy($id)                                                              
  {                                                                                                
    // get table/column                                                                            
    $db_table  = $this->options['db_table'];                                                       
    $db_id_col = $this->options['db_id_col'];                                                      

    // cleanup the session id, just in case
    $id = $this->db_escape($id);           

    // delete the record associated with this id
    $sql = "DELETE FROM $db_table WHERE $db_id_col = '$id'";

    if ($this->db_query($sql))
    {                         
      return true;            
    }                         

    // failed to destroy session
    throw new Exception(sprintf('%s cannot destroy session id "%s" (%s).', get_class($this), $id, mysql_error()));
  }                                                                                                               
                                                                                                                  
                                                                                                                  
  /**                                                                                                             
   * Cleans up old sessions.                                                                                      
   *                                                                                                              
   * @param  int $lifetime  The lifetime of a session                                                             
   *                                                                                                              
   * @return bool true, if old sessions have been cleaned, otherwise an exception is thrown                       
   *                                                                                                              
   * @throws <b>Exception</b> If any old sessions cannot be cleaned                                               
   */                                                                                                             
  public function sessionGC($lifetime)                                                                            
  {                                                                                                               
    // get table/column                                                                                           
    $db_table    = $this->options['db_table'];                                                                    
    $db_time_col = $this->options['db_time_col'];                                                                 

    // delete the record older than the authorised session life time
    $lifetime = $this->db_escape($lifetime); // We never know...    
    $sql = "DELETE FROM $db_table WHERE $db_time_col + $lifetime < UNIX_TIMESTAMP()";

    if (!$this->db_query($sql))
    {                          
      throw new Exception(sprintf('%s cannot delete old sessions (%s).', get_class($this), mysql_error()));
    }                                                                                                      

    return true;
  }             
                
                
  /**           
   * Reads a session.
   *                 
   * @param  string $id  A session ID
   *                                 
   * @return string      The session data if the session was read or created, otherwise an exception is thrown
   *                                                                                                          
   * @throws <b>Exception</b> If the session cannot be read                                                   
   */                                                                                                         
  public function sessionRead($id)                                                                            
  {                                                                                                           
                                                                                                              
    // get table/column                                                                                       
    $db_table    = $this->options['db_table'];                                                                
    $db_data_col = $this->options['db_data_col'];                                                             
    $db_id_col   = $this->options['db_id_col'];                                                               
    $db_time_col = $this->options['db_time_col'];                                                             

    // cleanup the session id, just in case
    $id = $this->db_escape($id);           

    // delete the record associated with this id
    $sql = "SELECT $db_data_col FROM $db_table WHERE $db_id_col = '$id'";

    $result = $this->db_query($sql);

    if ($result != false && $this->db_num_rows($result) == 1)
    {                                                        
      // found the session                                   
      $data = $this->db_fetch_row($result);                  

      return $data[0];
    }                 
    else              
    {                 
      // session does not exist, create it
      $sql = "INSERT INTO $db_table ($db_id_col, $db_data_col, $db_time_col) VALUES ('$id', '', UNIX_TIMESTAMP())";
      if ($this->db_query($sql))                                                                                   
      {                                                                                                            
        return '';                                                                                                 
      }                                                                                                            

      // can't create record
      throw new Exception(sprintf('%s cannot create new record for id "%s" (%s).', get_class($this), $id, mysql_error()));                                                                                                              
    }                                                                                                               
                                                                                                                    
  }                                                                                                                 

  
  /**
   * Writes session data.
   *                     
   * @param  string $id    A session ID
   * @param  string $data  A serialized chunk of session data
   *                                                         
   * @return bool true, if the session was written, otherwise an exception is thrown
   *                                                                                
   * @throws <b>sfDatabaseException</b> If the session data cannot be written       
   */                                                                               
  public function sessionWrite($id, $data)                                          
  {                                                                                 
    // get table/column                                                             
    $db_table    = $this->options['db_table'];                                      
    $db_data_col = $this->options['db_data_col'];                                   
    $db_id_col   = $this->options['db_id_col'];                                     
    $db_time_col = $this->options['db_time_col'];                                   

    // cleanup the session id and data, just in case
    $id   = $this->db_escape($id);                  
    $data = $this->db_escape($data);                

    // update the record associated with this id
    $sql = "UPDATE $db_table SET $db_data_col='$data', $db_time_col=UNIX_TIMESTAMP() WHERE $db_id_col='$id'";

    if ($this->db_query($sql))
    {                         
      return true;            
    }                         

    // failed to write session data
    throw new Exception(sprintf('%s cannot write session data for id "%s" (%s).', get_class($this), $id, mysql_error()));                                                                                                               
  }                                                                                                                 

/**
   * Regenerates id that represents this storage.
   *                                             
   * @param  boolean $destroy Destroy session when regenerating?
   *                                                            
   * @return boolean True if session regenerated, false if error
   *                                                            
   */                                                           
  public function regenerate($destroy = false)                  
  {                                                             
    if ($this->sessionIdRegenerated)                            
    {                                                           
      return;                                                   
    }                                                           

    $this->sessionIdRegenerated = true;
    $currentId = session_id();         
    $newId = session_id();             
    $this->sessionRead($newId);        

    return $this->sessionWrite($newId, $this->sessionRead($currentId));
  }                                                                    
                                                                       
                                                                       
  /**                                                                  
   * Counts the rows in a query result                                 
   *                                                                   
   * @param  resource $result  Result of a query                       
   * @return int Number of rows                                        
   */                                                                  
  protected function db_num_rows($result)                              
  {                                                                    
    return mysql_num_rows($result);                                    
  }                                                                    

  /**
   * Extracts a row from a query result set
   *                                       
   * @param  resource $result  Result of a query
   * @return array Extracted row as an indexed array
   */
  protected function db_fetch_row($result)
  {
    return mysql_fetch_row($result);
  }

  /**
   * Executes an SQL Query
   *
   * @param  string $query  The query to execute
   * @return mixed The result of the query
   */
  protected function db_query($query)
  {
    return @mysql_query($query);
  }

  /**
   * Escapes a string before using it in a query statement
   *
   * @param  string $string  The string to escape
   * @return string The escaped string
   */
  protected function db_escape($string)
  {
    return mysql_real_escape_string($string);
  }

}

$sfSessionHandler = new setfiveSession();
$sfSessionHandler->initialize();

This requires you create a table in your MySQL database called session. Here is a dump of the create statement for the table:

CREATE TABLE IF NOT EXISTS `session` (
  `id` int(11) NOT NULL auto_increment,
  `sess_id` varchar(255) NOT NULL,
  `sess_data` text NOT NULL,
  `sess_time` datetime NOT NULL,
  PRIMARY KEY  (`id`)
) ENGINE=MyISAM  DEFAULT CHARSET=latin1 AUTO_INCREMENT=1;

Basically this uses PHP’s session_set_handler function to manipulate how the PHP retrieves, updates, and stores the user’s session. The final touches were to include this class where the user sessions are started in LimeSurvey. We found them in the index.php and sessioncontrol.php files. Include our file from above just before the session_start(); in the code in those two files. In admin/scripts/fckeditor.265/editor/filemanager/connectors/php/upload.php include the file before the first include. Lastly we need to update a couple locations where it does session_regenerate_id and replace it with $sfSessionHandler->regenerate(). You can find these edits in the following three files: admin/usercontrol.php on line 128, admin/login_check.php line 65, and index.php at lines 207 and 215. You should be up and running now, let us know if you have any problems.

FanFeedr Widgets Are Live!

Over the past few weeks we had the opportunity to work with FanFeedr to put together some widgets for their sports news platform. Previously, FanFeedr had been using Sprout to build their widgets but this required someone to hand build a Flash widget for every “resource” on FanFeedr (there are a lot). In addition, since the Sprout widgets are Flash they aren’t easily crawled by search engines.

Our widgets are different. They allow FanFeedr to generate widgets on the fly for any of their pages and allow users to customize the color schemes. Check out a widget builder for the NY Yankees here.

Basically, our widget builder works by allowing users to customize the size and colors used in the widget. This data is serialized as a JSON object and then base64 encoded so that it can be sent to the “generator” on the server. Then, the server unpacks the payload and builds a widget according to the data specified in the JSON object. In addition, our embed code includes a noscript tags so that search engines pick up the links in the widget as well.

Anyway, working with FanFeedr was a great experience and we hope to continue our relationship moving forward. Go build yourself a widget!

internOwl Launched!

Today we are proud to unveil  http://internowl.com">internOwl.  internOwl is a site for students to research internships and find them.  As the site grows students will be able to gain invaluable insight into the quality of different internships around the country.   Currently the site is being launched with a focus on targeting Massachusetts’ students.  We are excited to see how it performs.

If you are a student in the Amherst or Northampton area you can get a FREE burrito via the following url: http://www.internow.com/bueno">http://www.internowl.com/bueno

We hope you all enjoy and there will be more updates about the site to follow as well as the technology used behind the site!

FOSS Saturday: sfFbConnectGuardPlugin - sfGuard meets FB Connect

I was slaving over a hot keyboard all Friday!

But at last it is done - FBConnect for sfGuard.

Get it here http://www.symfony-project.org/plugins/sfFbConnectGuardPlugin

A detailed explanation of how to install it and use it is on the Symfony site.

Anyway, the plugin basically just introduces a new table to keep track of Facebook IDs <—> sfGuardUserIds

Here’s a fun nugget. One of the problems with using FB Connect is that you can’t mug a user’s email address from Facebook. Obviously this is a smart move on Facebook’s part but it makes life hard for my Nigerian spammer friends. If you want to snag a user’s email address (or anything else for that matter) while still using Facebook Connect here’s a sketch of how to do it.

Everything is the same except you can’t use Facebook’s FBML to render the FB Connect button. What you want to do instead is trigger the “connect” event by hand. Here is basically how we do it:

  1. The user requests to sign up.
  2. We pop up a Lightbox using Thickbox
  3. We ask the user for their email address and verify that is valid and unique via AJAX in the background.
  4. The validation routing sets an attribute on the user using setAttribute() that contains the entered email address.
  5. We close the Lightbox and initiate a Facebook Connect request with FB.Connect.requireSession
  6. In our createFbUser() method we get the attribute back and save it with the new user

Bam. Got the user’s email address and logged them in via FB Connect.

FOSS Fridays: sfSCMIgnoresTaskPlugin version 1.0.3 released - Doctrine Supported

In the past we have always used Propel as our main ORM for our Symfony projects. Recently with Doctrine becoming the default ORM for Symfony 1.3 we decided we should make sure our plugin supports Doctrine. For those who don’t know about the plugin, it automatically creates ignores for your Source Code Management(SCM) if you use Git or CVS. When you have a large project it gets tiring to create all the different ignores for all the bases, logs, configuration files and such.

Let us know if you find any problems with Doctrine support or have any additional suggestions for the plugin.

More information on the plugin can be found on the Symfony Plugins site: http://www.symfony-project.org/plugins/sfSCMIgnoresTaskPlugin

You can download the most recent pear package manually at: http://plugins.symfony-project.org/get/sfSCMIgnoresTaskPlugin/sfSCMIgnoresTaskPlugin-1.0.3.tgz

or you can install it via:

./symfony plugin:install sfSCMIgnoresTaskPlugin