Use Exim and pipes to replicate MailGun routes

Over the last month or so I’ve been working with a friend of mine on a little side project (more on that later). During the course of the project, we ended up using Mailgun to receive email which was then passed to a PHP script using Mailgun routes.

Mailgun was pretty easy to setup and things were great but unfortunately in the last day or so our free app started creeping over the 200 message/day limit on the Mailgun free tier. Since the app is currently free, we couldn’t really justify paying the $19/mon minimum for the first Mailgun tier so I started exploring self-hosted options.

After a big of Googling, the first thing that caught my eye was the LamsonProject. From my understanding, Lamson basically provides a SMTP server which binds to a MVC application framework. After checking out the docs and the code it seemed like Lamson would allow you to build a RESTful application that responded to SMTP messages instead of HTTP requests. It certainly seems like a cool project but seemed like a bit overkill for what I needed to do.

Going further down the rabbit hole, I found out that the Exim allows messages to be delivered to a regular *nix process via a pipe and that the configuration was supposedly relatively straightforward under Ubuntu. Given that, I decided to give it a shot, it turned out to be relatively easy to setup but here is a quick rundown.

Step 1: Setup Exim

By default, Ubuntu systems are configured with sendmail and only setup to receive local mail. You’ll need to remove sendmail, install Exim, and then configure it.

To do this you’ll roughly need to run the following:

sudo apt-get remove sendmail
# for some reason the dependencies aren't being installed correctly
# see http://ubuntuforums.org/showthread.php?p=10411454
sudo apt-get install exim4-base exim4-config
sudo apt-get install exim4
sudo dpkg-reconfigure exim4-config

Next, follow the guide on https://help.ubuntu.com/8.04/installation-guide/hppa/mail-setup.html to configure Exim.

Step 2: Configure Pipes and Aliases

Per https://answers.launchpad.net/ubuntu/+source/mailman/+question/120447 it turns out something isn’t enabled in the default Ubuntu configuration of Exim. To solve this, I added the following

.ifndef SYSTEM_ALIASES_PIPE_TRANSPORT
SYSTEM_ALIASES_PIPE_TRANSPORT = address_pipe
SYSTEM_ALIASES_USER = Debian-exim
SYSTEM_ALIASES_GROUP = daemon
.endif

Into the bottom of /etc/exim4/conf.d/transport/30_exim4-config_address_pipe

After you edit an Exim setting, you’ll need to always run the following for the settings to take effect:

sudo update-exim4.conf.template -r
sudo update-exim4.conf
udo service exim4 restart

Next, you’ll need to enable the catch-all router configuration. This was surprisingly difficult to track down how to do:

Per, http://pontus.ullgren.com/view/Setting_SMTP_server_development_test_server_running_Ubuntu you need to create a file at /etc/exim4/conf.d/router/950_exim4-config_catchall containing

catch_all:
   debug_print = "R: catch_all for $local_part@$domain"
   driver = redirect
   data = ${lookup{*}lsearch{/etc/aliases}}
   # NOTE: I added this line for the catch all pipes to work
   pipe_transport = address_pipe

Finally, add a catch-all alias that pipes to a script to your /etc/aliases file. Mine looks like:

*: "|/home/ubuntu/exim4.php"

Step 2: The PHP script

I did this with PHP since my original Mailgun script was in PHP but anything should work. Key notes, the script needs to be accessible AND executable by the Exim user. Also, you’ll need the appropriate shebang to make the script work without a named interpreter.

Your script is going to receive raw SMTP email so you’ll need to parse that out to do anything meaningful with it. Thankfully there are plenty of libraries to do this. I ended up using MailParse along with PHP MimeMail Parse to abstract the nitty gritty into some cleaner object oriented code.

The part of my script that accepts and parses the email looks like:

From there, you’d be free to do anything you wanted with $msgInfo, just like on the receiving end of a Mailgun route.

Anyway, as always let me know if you have any feedback, questions, or comments.

AJAX Request Slow With PHP? Here’s Why

Recently I was working on a project where we had a page which loads tons of data from numerous sources. I decided after a while that we wanted to AJAX each section of data so that the page would load a bit quicker. After splitting up the requests and sending them asyncronously, there was little improvement. I thought at first it may be due to the fact we were pinging a single API for most of the data multiple times, that wasn’t it. Maybe it was a browser limit? Nope was still far below the 6 requests most allow. I setup xdebug and kcachegrind and to my surprise it was the session_start() that was taking the most time on the requests.

I looked around the web for a while trying to figure out what in the world was going on. It turns out that PHP’s default session_start will block future session_starts for the same session until the session is closed. This is because the default method uses a file on the filesystem which it locks until you close it. If you want more information on this and how to close it you can read a bit more here.

We switched over to database based sessions and it fixed it. In symfony 1.4 the default session storage uses the file system, however switching over to sfPDOSessionStorage is very easy and quick.

Facebook: How-to force users to LIKE page

With Facebook’s move to deprecate FBML for tabs the documentation around how to make a “please Like! before…” has become much more choppy and inconsistent. Anyway, I recently found myself in a position where I needed to make this happen so here goes.

With in-line FBML deprecated, the only way to accomplish this without using a third party branded solution is to create a Facebook iframe app. Here are the steps you need to take to get something up using PHP and the Facebook PHP SDK.

1. Create a new Facebook Application at https://developers.facebook.com/apps

2. Configure your new Facebook App the enable “Website” and “Page Tab”. You’ll need to enter a valid URL for the following fields:

  • Site URL
  • Page Tab URL
  • Secure Page Tab URL

You’ll also want to use a HTTPs URL since Facebook sessions default to HTTPs by default and your iframe will be marked insecure if its over vanilla HTTP. For this walk through, lets assume were using https://www.setfive.com/fb/index.php? as the URL.

3. Now, you’ll want to add your new App to a Facebook Page. The easiest way to do this is to use this URL https://www.facebook.com/dialog/pagetab?app_id=YOUR_APP_ID&next=YOUR_URL replacing YOUR_APP_ID and YOUR_URL with your App ID and then a URL that is derived from your endpoint (or even just your endpoint). When you load that URL, you’ll be prompted to add your app to a page – select the page you want and submit the form.

4. The final piece is throwing together the actual PHP script. You’ll need the Facebook PHP SDK available on GitHub – https://github.com/facebook/php-sdk. Clone that and then this is the PHP script you’ll need:

And thats it! Now you’ll be able to gate content from non-fans while growing the fanbase of your Facebook Page.

Drop any questions in the comments.

Fixing blank CCK Location fields in Views

Recently, we inherited a Drupal 6 site via a client of ours and ran into a pretty irritating bug with the Location module.

The site had been configured to allow users to create profiles using Node Profile along with the Location to allow users to input their street addresses.

Anyway, the issue was that when we created a View that included Location fields the fields were always rendering as blank even when we confirmed there was data in the database. A bit of poking around lead to this issue.

It turns out that due to an optimization in CCK or Views that the tables that have the data for the location fields are not getting JOIN’ed in when the view is executed. Unfortunately, the patch provided on the issue doesn’t work on the latest 3.x release of the Location module.

The fix that worked for us is #14 (copied below)

/**
* Preprocess hook for location().
*/
function yourtheme_preprocess_location(&$variables) {
  if (!isset($variables['location']['name']) && isset($variables['location']['lid'])) {
    $variables['location'] = array_merge($variables['location'], location_load_location($variables['location']['lid']));
    template_preprocess_location($variables);
  }
}

Basically, you’ll need to add the above snippet to a template.php file in your theme and change the name to reflect the theme you’re using. What this function does is basically pre-process the location fields to pull in the data so that the View will work properly.

Anyway, enough blogging it’s football time.

Adding a task/command in Symfony2

I recently took the Symfony2 plunge and started working on a little fun side project (more on that later).

Anyway, this particular project involves sending out daily text messages using the rather awesome Twilio API so I decided to use a Symfony2 task for this. The documentation on how to actually add your own task is a bit sparse so I figured I’d share.

The process is actually pretty straight forward:

  1. In your bundle create a directory named “Command” (without the quotes).
  2. Create a file that extends ContainerAwareCommand
  3. Create a protected function configure – “protected function configure()” to allow you to configure the name of your task and add any options or arguments you might need.
  4. Create a protected function execute – “protected function execute(InputInterface $input, OutputInterface $output)” to actually do whatever needs to be done.
  5. Thats it! Now you can run app/console and you’ll see your task.

Here is the code for mine: