Over the past month we’ve been working with Her Campus (HerCampus.com) to help them with issues they were having. When we started talking with Her Campus, we learned that they had different types of issues ranging from some Drupal based ones to actual server level problems. They were having some trouble keeping up to traffic demands, and if a traffic spike occurred their current infrastructure wasn’t sufficient.

After looking at their setup, we noted quite a few areas in which we could improve performance. The old setup was a fairly standard setup, a frontend server using Apache to handle HTTP requests and then a second server which was their MySQL database server. The servers were a 4 gig and 8 gig server respectively.

It was clear Apache was adding unneeded overhead, and wasn’t the best solution for them. We revamped their setup significantly. We switched them to Nginx + PHP-FPM. We immediately saw great improvement on from this change alone. However we wanted to get them to a single server, and to be able to handle traffic spikes with a single server.

We look forward to helping Her Campus with their continuing expansion of their site and user base!

Recently one of our clients decided to white label their product.  With that we had to setup the server to use multiple IPs as the application requires you communicate over SSL and we needed a SSL per domain.  We did not want to buy a UCC(a multiple domain)  SSL certificate as right now it wasn’t required for the small number of white labels. After we added the additional IP we had the issue that the application which connects to off site MySQL servers, was sometimes going over the new IPs and then getting denied accessed.

We knew the solution was with iptables so after some digging and testing, we came up with the following command.  This command we use will redirect all traffic that is not over port 443 (in this example) to go out over the ‘YYY.YYY.YYY.YYY’ address that is about to go out over the XXX.XXX.XXX.XXX ip.

iptables -t nat -A POSTROUTING -p tcp ! --dport 443 -s XXX.XXX.XXX.XXX -j SNAT --to-source YYY.YYY.YYY.YYY

We didn’t see any examples of this clearly defined (after a quick google that is), on the web, so hopefully this will save you time from having to read through the iptables documentation.

I was working on an application earlier today that required allowing a user to upload a large file (several hundred MB) which would eventually be stored on Amazon S3. After reviewing the requirements, I realized it made sense to just upload the file directly to S3 instead of having to first stage the file on a server and then use PHP to push the file to S3.

Amazon has a nice walk through of using a plain HTML form to upload a file directly to S3 here.

I had all ready been using SWFUpload to upload files to the server so I decided to look into using it to uploading directly to S3. After some head banging, I finally got it to work – here’s the quick n dirty.

1. Download SWFUpload 2.5
2. Get SWFUpload ready to use in your project. Copy the SWF file somewhere accessible and include their swfupload.js Javascript file. More info here
3. Setup an S3 bucket. You’ll need to set the policy to allow uploads from your own user (its the default).
4. Place a crossdomain.xml file in the root of your S3 bucket. This file “authorizes” flash player to upload files into this host. The content of the file is below.
5. Initialize the SWFUpload object (example below).
6. Before beginning the upload, you need to set the appropriate postParams in the SWFUpload object. This is really the “magic” of this process. Example is below.
7. Start the upload with startUpload()

Thats it! It’s pretty straight forward once you have things going. As an FYI, you can put SWFUpload into “debug” mode by adding debug: true as a property to the initialization object. You can also debug the responses from Amazon by using a packet sniffer like Wireshark.

crossdomain.xml

You probably want to make this file a little less permissive. More details here. Also note, there are differences in the implementation of the file between various versions of Flash player.

Initialize SWFUpload

        var swfu = new SWFUpload(
                       {  flash_url: "/assets/swfupload.swf",
                          flash9_url: "/assets/swfupload_fp9.swf",
                          file_size_limit: "1000 MB",
                          file_types: "*.*",
                          debug: false,
                          upload_url: "http://your-bucket.s3.amazonaws.com",
                          button_placeholder_id : "SWFUploadButton",
                          button_image_url : "/assets/select_filesbtn.png",
                          button_width: '112',
                          button_height: '33',
                          button_cursor : SWFUpload.CURSOR.HAND,
                          http_success : [201, 303, 200], /* Amazon returns a 303 on success */
                          file_post_name: "file", /* Amazon expects the file data to be in a input named "file"

                          file_queued_handler: function(f){

                            // track the filenames so you can upload them later
                            cachedUploadFiles[ f.index ] = f.name;
                          },
                          upload_complete_handler: function(e){ uploadSWFFile( ); },
                          upload_start_handler: function(e){
                                  // reset the progress bar
                        	  $("#progressBar").progressbar( 'value', 0 );
                          },
                          upload_error_handler: function(e){

                          },
                          upload_progress_handler: function(f, c, t){
                             // update the progress bar as the process continues
                             $("#progressBar").progressbar( 'value', Math.ceil( ( c/t ) * 100 ) );
                          }
        });

Set SWFUpload postParams

The HMAC signature MUST be calculated on the server because it uses your S3 secret. You MUST keep that value secret in order to maintain the security of your S3 buckets. I’m using Don Schonknecht’s S3 PHP library to calculate the HMAC signatures but you could just as easily do it in straight PHP.

/* In PHP */
$encodedPolicy = json_encode( array(
              "expiration" => "2011-4-22T13:54:23.000Z",
              "conditions" => array(
                  0 => array( "acl" => "public-read" ),
                  1 => array( "bucket" => "your-bucket" ),
                  2 => array( "x-amz-meta-sig" => 'some meta signature to ensure authentic requests'),
                  3 => array( "redirect" => $'URL to redirect a success request (its doesnt matter)' ),
                  4 => array( "key" => "the S3 key for the file (the S3 filename)" ),
                  5 => array( "Filename" => "The original filename of the file. THIS IS IMPORTANT." )
              ),
            )
);

$encodedPolicy = base64_encode( $encodedPolicy );
$s3 = new S3( sfConfig::get("app_amazon_s3_id"), sfConfig::get("app_amazon_s3_secret") );
list($dist, $hmacSignature) = explode(":", $s3->__getSignature( $encodedPolicy ));

/* END PHP */

            var swfConfig = {
                    'AWSAccessKeyId': 'your amazon ID',
                    'acl': 'public-read',
                    'key': 'the S3 key for the file (the S3 filename)',
                    'policy': '<?php echo $encodedPolicy?>',
                    'signature': '<?php echo $signature'?>,
                    'redirect': 'URL to redirect a success request (its doesnt matter)',
                    'x-amz-meta-sig': 'some meta signature to ensure authentic requests',
            };

            // this line sets the post params so that SWFUpload will send the additional fields when it uploads the file.
            $.swfu.setPostParams( swfConfig );

Earlier today we launched Boston Built in collaboration with our friends at Bocoup, UpStatement, and SignedOn

Basically, Boston built allows you to “represent” by either adding a graphic logo or a 1×1 tracking pixel to your site via Javascript. Then, the bostonbuilt.org site will pick you up and list your favicon along side the other sites that are including the tracking code.

BostInnovation has a nice write up and a poll at The ‘Built in Boston’ initiative.

Happy Friday!

Check out Menu Attributes

It will allow you to add attributes like target=”_blank” or nofollow to your navigation menu items.

Recently we’ve launched a new site for a client called Purpose Beverages: http://trackyourimpact.com . We’ve received great feedback from users so far. The site uses a wide range of technologies. It is built on symfony and uses the Apostrophe CMS to manage the main parts of the site. It integrates with a SMS provider to allow you text into it to find out more about your purchase.

Tēvolution is a new brand of tea on the market that does good with each purchase. Every time someone buys it they done a specific amount(for example 25 cents) to a specific charity. In order to find out how large of a donation and what charity your bottle goes to you can actually text the code found on the bottle to the website, or you can login in on your phone browser or regular browser and enter the code. You will find out how much and to whom you just donated money to!

Right now Tēvolution is just coming to the market so keep your eyes peeled for it! It’s a great product that does good!

I was looking around earlier for a jQuery plugin to allow me to use jQuery UI’s dialog() widget to popup a confirm dialog. Didn’t have any luck finding one so I whipped something up. It’s not pretty but it works. You could even go as far as to overload window.confirm() but thats probably a bad idea.

jQuery.confirm = function(options){

	var opts = jQuery.extend( { message: "", ok: function(){}, cancel: function(){ } }, options );

	jQuery("

"
				+ opts.message + ""
				+ "


").dialog({
					autoOpen: true,
					modal: true,
					autoOpen: false,
					resizable: false,
					draggable: false,
					title: "",
					width: "400px",
					buttons: {
					    "Cancel": function(){
							opts.cancel.call( this );
						},
						"Ok": function(){
					    	opts.ok.call( this );
					    }
					}
	}).dialog("open");

};

// use it 

		jQuery.confirm( {
			"message": "Are you sure?",
			"ok": function( ){

				$(this).find(".loader:first").show();

				// do stuff
                               $(this).dialog("close");
			},
			"cancel": function( ){
				$(this).dialog("close");
			}
		});

Well D7 has been out for a little while now and we finally got a chance to use it on a site this week.

Anyway, this site is one of the heavier Drupal sites we’ve done and it involved loading ~200+ nodes of data just to set things up. This presented two problems, how to batch load data and then how to load custom content types with several custom fields.

The Drupal module documentation has example code for adding a node with drupal_exeucte here but it doesn’t deal with how to set custom fields on your content type. On top of this, drupal_execute has been renamed to drupal_form_submit in Drupal 7 and the function signature has changed a bit.

Anyway, I dug around a bit and finally managed to get this working. You’ll obviously need Drush installed for the following code to work but you could rip it out and use it outside a Drush command. I was looking to basically replicate the “load-data” task from Symfony so that I could seed my Drupal database with Nodes at any point so I chose to make this a Drush command.

Here’s what you need:

- You’ll need a module to hold the Drush task. I used Module Builder to generate my scaffolding.
- Create a file named [modulename].drush.inc in your module directory
- Here is the code I’m using for [modulename].drush.inc Replace “cm” with the name of your module:

/**
* Implementation of hook_drush_command().
*/

function cm_drush_command() {

        // callback is the function that will be called when the command is executed
	$items['load-rep-data'] = array(
	    'callback' => 'cm_load_rep_data',
            'description' => 'Loads the representative data.',
            'bootstrap' => DRUSH_BOOTSTRAP_DRUPAL_FULL,
	);

	return $items;
}

function cm_load_rep_data( ){
   // you need this to autoload the functions to create nodes
   module_load_include('inc', 'node', 'node.pages');

  // do some business logic to load your data from where it is
  foreach( $arr as $res ){

    // define the custom node type
    $node = array('type' => 'representative');

   // set up the form array
   $form_state = array();

   // set the title of the node
   $form_state['values']['title'] = $res["title"];

   // set a custom field that is a text type
   $form_state['values']['field_first_name']['und']['0']['value'] = $res["first_name"];

   // set a long text field and enable full_html - NOTE you'll need to allow anonymous users to use this for Drush to work
   $form_state['values']['body']['und']['0']['format'] = 'full_html';
   $form_state['values']['body']['und']['0']['value'] = $res["bio"];

  // set some custom select fields
  $form_state['values']['field_house_committees']['und'][ 0 ] = 34;
  $form_state['values']['field_house_committees']['und'][ 0 ] = 37;

  // can't leave this out or the form wont save
  $form_state['values']['op'] = t('Save');

  // actually try and "submit" the form
  drupal_form_submit('representative_node_form', $form_state, (object)$node);

  // printing from Drush is easy
  drush_print( $res["title"] );
  }
}

Thats about it.

With Firebug, it’s really easy to see the field names and the values that you can set by just looking at a form to create whatever type of node you want.

drupal_form_submit can also be used to “submit” any other type of form in Drupal.

An open question is how to “fill out” an ImageField field via the command line since nothing is actually going to be uploaded.

On a project I’m working on I came across the following problem: we had a email field that we needed to be unique in our system, but we also made sure that it matched a confirm email field. A snippet of our form looks like this:

  public function configure()
  {
     $this->setWidgets( array(
        'email'         => new sfWidgetFormInputText()
        'confirm_email' => new sfWidgetFormInputText()
      ));

      $this->setValidators(array(

      'email'=>         new sfValidatorAnd(
                          array(
                              new sfValidatorEmail( array('required' => true) ),
                              new sfValidatorDoctrineUnique(
                                  array('throw_global_error' => true, 'model' => 'sfGuardUser', 'column' => 'username'),
                                  array('invalid' => 'Sorry! A user with that email address already exists.')
                              )
                          )),
        'confirm_email' => new sfValidatorEmail( array('required' => true) )

      ));

      $this->validatorSchema->setPostValidator(
        new sfValidatorSchemaCompare('password', '==', 'confirm_password')
      );
  }

When we submitted an email that was already in the system we got back two errors:

• Sorry! A user with that email address already exists.
• Invalid.

For a while I thought is there some extra validator somewhere that I left on? Where is this invalid coming from? It ended up being due to the way the validators work. If a validator throws an error it doesn’t return that validator’s value. So by the time it gets to the sfValidatorSchemaCompare post validator the value of `email` is null and `confirm_email` has the value you input, thus the seemingly extra ‘Invalid’ message.

This can be fixed easily with a sfValidatorCallback instead of the sfValidatorSchemaCompare. Here is the fix:

  public function validateConfirmEmail( $validator, $values ){

    if($values['email']&&$values['email']!=$values['confirm_email'])
    {
      throw new sfValidatorError($validator, 'Please confirm your email, currently they do not match!.');
    }

    return $values;
  }

This way if the email is blank it doesn’t both making sure that the `email` matches the `confirm_email`. You don’t need to worry about a person just passing two blank emails as the earlier validator(the sfValidatorEmail requires it to be there and valid).

If you are getting an extra validation error, check your postValidators and how the values get to them.

I was using cURL in PHP to POST some data to a URL earlier tonight and ran into this problem.

With VERBOSE on cURL was erroring with the following error:

"Received problem 2 in the chunky parser"

After some Googling it turns out this is a problem with how some servers respond with chunked encoding.

A simple fix for this is to set the HTTP version cURL is using to 1.0:

curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_0 )

It’s not pretty but hey it works!