LimeSurvey with load balancers, fixing the user sessions.

For a client we’ve been working with recently it came to our attention that they needed more frontend servers to keep up with the traffic for their surveys. They use LimeSurvey which is powerful open source survey platform. We set the client up in the cloud to scale as necessary with a load balancer in front. This is when we noticed the problem that LimeSurvey doesn’t work well when a user is bouncing between different frontend servers. LimeSurvey keeps all the user’s session attributes on the local server and not in the database. After googling around for a while, we found other people also had this problem before, and no one had really solved it. We figured we would.

We didn’t feel like doing a ton of extra work to reinvent the wheel in terms of storing the session in the database. We snagged most of the code straight from the Symfony “storage” module which handles it’s session management if you want to store the user sessions in a database. After a quick few modifications, we got it up:

<?php
/**
 * Taken from parts the Symfony package "storage" module
 * @author Ashish Datta ashish@setfive.com
 * @author Matt Daum matt@setfive.com
 * Setfive Consulting, LLC                                                     
 */                                                     

class setfiveSession {
                      
  private $options = array();
  private $sessionIdRegenerated;
                                
  public function initialize()  
  {                             
    $this->options =  array('db_table' => 'session',
                            'db_id_col'   => 'sess_id',
                            'db_data_col' => 'sess_data',
                            'db_time_col' => 'sess_time');
                                                          
    session_set_save_handler(array($this, 'sessionOpen'), 
                             array($this, 'sessionClose'),
                             array($this, 'sessionRead'), 
                             array($this, 'sessionWrite'),
                             array($this, 'sessionDestroy'),
                             array($this, 'sessionGC'));    
                                                            
    // start our session                                    
    // session_start();                                     
  }                                                         
                                                            
  /**                                                       
   * Closes a session.                                      
   *                                                        
   * @return boolean true, if the session was closed, otherwise false
   */                                                                
  public function sessionClose()                                     
  {                                                                  
    // do nothing                                                    
    return true;                                                     
  }                                                                  
                                                                     
  /**                                                                
   * Opens a session.                                                
   *                                                                 
   * @param  string $path  (ignored)                                 
   * @param  string $name  (ignored)                                 
   *                                                                 
   * @return boolean true, if the session was opened, otherwise an exception is thrown
   *                                                                                  
   * @throws <b>Exception</b> If a connection with the database does not exist or cannot be created
   */                                                                                              
  public function sessionOpen($path = null, $name = null)                                          
  {                                                                                                
    // we're assuming LimeSurvey all ready has db settings and opened a db connection              
    return true;                                                                                   
  }                                                                                                
                                                                                                   
                                                                                                   
  /**                                                                                              
   * Destroys a session.                                                                           
   *                                                                                               
   * @param  string $id  A session ID                                                              
   *                                                                                               
   * @return bool true, if the session was destroyed, otherwise an exception is thrown             
   *                                                                                               
   * @throws <b>Exception</b> If the session cannot be destroyed.                                  
   */                                                                                              
  public function sessionDestroy($id)                                                              
  {                                                                                                
    // get table/column                                                                            
    $db_table  = $this->options['db_table'];                                                       
    $db_id_col = $this->options['db_id_col'];                                                      

    // cleanup the session id, just in case
    $id = $this->db_escape($id);           

    // delete the record associated with this id
    $sql = "DELETE FROM $db_table WHERE $db_id_col = '$id'";

    if ($this->db_query($sql))
    {                         
      return true;            
    }                         

    // failed to destroy session
    throw new Exception(sprintf('%s cannot destroy session id "%s" (%s).', get_class($this), $id, mysql_error()));
  }                                                                                                               
                                                                                                                  
                                                                                                                  
  /**                                                                                                             
   * Cleans up old sessions.                                                                                      
   *                                                                                                              
   * @param  int $lifetime  The lifetime of a session                                                             
   *                                                                                                              
   * @return bool true, if old sessions have been cleaned, otherwise an exception is thrown                       
   *                                                                                                              
   * @throws <b>Exception</b> If any old sessions cannot be cleaned                                               
   */                                                                                                             
  public function sessionGC($lifetime)                                                                            
  {                                                                                                               
    // get table/column                                                                                           
    $db_table    = $this->options['db_table'];                                                                    
    $db_time_col = $this->options['db_time_col'];                                                                 

    // delete the record older than the authorised session life time
    $lifetime = $this->db_escape($lifetime); // We never know...    
    $sql = "DELETE FROM $db_table WHERE $db_time_col + $lifetime < UNIX_TIMESTAMP()";

    if (!$this->db_query($sql))
    {                          
      throw new Exception(sprintf('%s cannot delete old sessions (%s).', get_class($this), mysql_error()));
    }                                                                                                      

    return true;
  }             
                
                
  /**           
   * Reads a session.
   *                 
   * @param  string $id  A session ID
   *                                 
   * @return string      The session data if the session was read or created, otherwise an exception is thrown
   *                                                                                                          
   * @throws <b>Exception</b> If the session cannot be read                                                   
   */                                                                                                         
  public function sessionRead($id)                                                                            
  {                                                                                                           
                                                                                                              
    // get table/column                                                                                       
    $db_table    = $this->options['db_table'];                                                                
    $db_data_col = $this->options['db_data_col'];                                                             
    $db_id_col   = $this->options['db_id_col'];                                                               
    $db_time_col = $this->options['db_time_col'];                                                             

    // cleanup the session id, just in case
    $id = $this->db_escape($id);           

    // delete the record associated with this id
    $sql = "SELECT $db_data_col FROM $db_table WHERE $db_id_col = '$id'";

    $result = $this->db_query($sql);

    if ($result != false && $this->db_num_rows($result) == 1)
    {                                                        
      // found the session                                   
      $data = $this->db_fetch_row($result);                  

      return $data[0];
    }                 
    else              
    {                 
      // session does not exist, create it
      $sql = "INSERT INTO $db_table ($db_id_col, $db_data_col, $db_time_col) VALUES ('$id', '', UNIX_TIMESTAMP())";
      if ($this->db_query($sql))                                                                                   
      {                                                                                                            
        return '';                                                                                                 
      }                                                                                                            

      // can't create record
      throw new Exception(sprintf('%s cannot create new record for id "%s" (%s).', get_class($this), $id, mysql_error()));                                                                                                              
    }                                                                                                               
                                                                                                                    
  }                                                                                                                 

  
  /**
   * Writes session data.
   *                     
   * @param  string $id    A session ID
   * @param  string $data  A serialized chunk of session data
   *                                                         
   * @return bool true, if the session was written, otherwise an exception is thrown
   *                                                                                
   * @throws <b>sfDatabaseException</b> If the session data cannot be written       
   */                                                                               
  public function sessionWrite($id, $data)                                          
  {                                                                                 
    // get table/column                                                             
    $db_table    = $this->options['db_table'];                                      
    $db_data_col = $this->options['db_data_col'];                                   
    $db_id_col   = $this->options['db_id_col'];                                     
    $db_time_col = $this->options['db_time_col'];                                   

    // cleanup the session id and data, just in case
    $id   = $this->db_escape($id);                  
    $data = $this->db_escape($data);                

    // update the record associated with this id
    $sql = "UPDATE $db_table SET $db_data_col='$data', $db_time_col=UNIX_TIMESTAMP() WHERE $db_id_col='$id'";

    if ($this->db_query($sql))
    {                         
      return true;            
    }                         

    // failed to write session data
    throw new Exception(sprintf('%s cannot write session data for id "%s" (%s).', get_class($this), $id, mysql_error()));                                                                                                               
  }                                                                                                                 

/**
   * Regenerates id that represents this storage.
   *                                             
   * @param  boolean $destroy Destroy session when regenerating?
   *                                                            
   * @return boolean True if session regenerated, false if error
   *                                                            
   */                                                           
  public function regenerate($destroy = false)                  
  {                                                             
    if ($this->sessionIdRegenerated)                            
    {                                                           
      return;                                                   
    }                                                           

    $this->sessionIdRegenerated = true;
    $currentId = session_id();         
    $newId = session_id();             
    $this->sessionRead($newId);        

    return $this->sessionWrite($newId, $this->sessionRead($currentId));
  }                                                                    
                                                                       
                                                                       
  /**                                                                  
   * Counts the rows in a query result                                 
   *                                                                   
   * @param  resource $result  Result of a query                       
   * @return int Number of rows                                        
   */                                                                  
  protected function db_num_rows($result)                              
  {                                                                    
    return mysql_num_rows($result);                                    
  }                                                                    

  /**
   * Extracts a row from a query result set
   *                                       
   * @param  resource $result  Result of a query
   * @return array Extracted row as an indexed array
   */
  protected function db_fetch_row($result)
  {
    return mysql_fetch_row($result);
  }

  /**
   * Executes an SQL Query
   *
   * @param  string $query  The query to execute
   * @return mixed The result of the query
   */
  protected function db_query($query)
  {
    return @mysql_query($query);
  }

  /**
   * Escapes a string before using it in a query statement
   *
   * @param  string $string  The string to escape
   * @return string The escaped string
   */
  protected function db_escape($string)
  {
    return mysql_real_escape_string($string);
  }

}

$sfSessionHandler = new setfiveSession();
$sfSessionHandler->initialize();

This requires you create a table in your MySQL database called session. Here is a dump of the create statement for the table:

CREATE TABLE IF NOT EXISTS `session` (
  `id` int(11) NOT NULL auto_increment,
  `sess_id` varchar(255) NOT NULL,
  `sess_data` text NOT NULL,
  `sess_time` datetime NOT NULL,
  PRIMARY KEY  (`id`)
) ENGINE=MyISAM  DEFAULT CHARSET=latin1 AUTO_INCREMENT=1;

Basically this uses PHP’s session_set_handler function to manipulate how the PHP retrieves, updates, and stores the user’s session. The final touches were to include this class where the user sessions are started in LimeSurvey. We found them in the index.php and sessioncontrol.php files. Include our file from above just before the session_start(); in the code in those two files. In admin/scripts/fckeditor.265/editor/filemanager/connectors/php/upload.php include the file before the first include. Lastly we need to update a couple locations where it does session_regenerate_id and replace it with $sfSessionHandler->regenerate(). You can find these edits in the following three files: admin/usercontrol.php on line 128, admin/login_check.php line 65, and index.php at lines 207 and 215. You should be up and running now, let us know if you have any problems.

javascript - $(document).ready getting called twice? Here's why.

Recently we found ourselves having a really weird problem on a project: Every time a page was loaded it seemed a bunch of different Javascript functions were being called multiple times and making many widgets on the page break and we couldn’t figure it out. Some of the functions were our own code, some part of the package we were using. After a while we narrowed it down to that all the functions in the

$(document).ready(...);

we’re being called twice. We had never seen this. After about an hour of removing javascript files and just headbanging, and many thanks to Ashish, we found the root cause. We had written in a quick hack for a late proof of concept to string replace on the entire HTML of a page a specific string. We did it this way:

$('body').html($('body').html().replace(/{REPLACETEXT}/i, "More important text"));

Basically we used a regex to parse the entire HTML tree and then replace it with the updated text. Unknowingly this caused the document ready event to be triggered again(though now it makes sense), causing many widgets to get extra HTML.

Let this save you some headbanging.

PHP: Adding variables in the current scope

So earlier today I was working on a Facebook App and wanted to use “partials” in a similar fashion as Symfony’s partials. At this point, I realized I had no idea how Symfony placed variables into the current execution scope when you do things like

include_partial("somePartial", array("foo" => $foo, "bar" => $bar));

A bit of digging led me to the extract() function in PHP.

From the documentation, “extract — Import variables into the current symbol table from an array”.

Pretty neat.

Cool kid stuff: Sizzle for PHP!?

Every now and then, I’ll end up having to scrape HTML pages for some content. I know, I know, there’s like a bazillion different ways to do this, but I really like doing it in PHP so I can jack right into Symfony. Usually, I just get down and dirty with the PHP DOM and use XPath to select nodes within the document. The problem with this is that the XPath sucks and the PHP implementation sucks…alot.

But hold on, we know a selector engine that doesn’t suck! The jQuery selector engine, called Sizzle is probably one of the best CSS/DOM selector engines to use. Turns out there is a PHP port! Enter phpQuery

At its root, phpQuery is a port of the jQuery selector syntax to PHP. Additionally, phpQuery includes dozens of the jQuery traversal methods like next(), prev(), find(), and so on. It also implements the CSS3 filters like :first, :last, :eq, ect.

Anyway, if you’re tired of suffering through the PHP XPath implementation and dig jQuery then you should definitely give phpQuery a whirl.

Happy Holidays!

Happy holidays everyone! Hope everyone had an awesome Christmas and is getting excited for a fun New Years Eve and then a great 2010.

Anyway, since the sun never sets on the Setfive empire I was actually doing some coding earlier when I ran across an interesting little problem. What I was looking to do was “match” a string input against a set of acceptable strings. The caveat was that the inputs might have spelling mistakes or typos. For example, an input might be “onnlinee ad” matching against [“online ad”, “video”, “news”, “online”] with the goal of matching “online ad”.

Unfortunately, you can’t simply iterate over the two strings matching letters because a single wrong letter will cause you to miss all of the rest. Remembering back to some old engineering courses I found my way over to the Hamming distance article on Wikipedia. From there, I made my way over to the Levenshtein distance article which proved extremely useful.

So, at this point I figured I wanted to minimize the Levenshtein distance and that would be my matching string. Fortunately enough, PHP has a built in function to calculate Levenshtein distances! levenshtein() The Levenshtein distance works pretty well for what I was looking to do. In addition, PHP has another built in function - similar_text() for comparing two strings. similar_text will return the number of matching characters in the two input strings.

Anyway, the only thing to be aware of is that both these functions have really bad running times. similar_text clocks in at O(n^3) where n is the length of the longest string and levenshtein runs at O(m*n) where m and n are the lengths of the input strings.

Well that’s it for now. Happy string comparing.