nginx: Using auth_request to secure vhosts

One of our clients recently had a unique use case. They had a Wiki site where they wanted to restrict viewing of posts to only their app’s authorized users. Picture something like a SaaS app where the Wiki site had proprietary content that our client only wanted paying users to access.

The two obvious options to implement this would be:

  • Create a Wiki user for each authorized user – this has the downside that we’d need to maintain two accounts, figure out how to keep users logged into both, and deal with synchronizing account data.
  • Modify the Wiki’s application code to authorize the users in some fashion – this is problematic because it would make upgrading the Wiki software difficult.

Turns out there’s a third option which is much smoother! Nginx has a directive called auth_request which allows nginx to authorize access to a resource based on a 2nd HTTP request.

The way it works is:

  • Your SaaS app is setup at platform.setfive.com where users are authenticated by a Symfony application.
  • You configure your Symfony application to send a cookie back with a wildcard domain of “.setfive.com”
  • Your wiki is running at wiki.setfive.com and configured to authorize requests to platform.setfive.com/is-authenticated
  • Now, when users request wiki.setfive.com their browser will send your Symfony authentication cookie, nginx will make a request to platform.setfive.com/is-authenticated, and if they’re authenticated they’ll be granted access to your wiki.

The nginx config for this is pretty straightforward as well. One thing to note is this module is not standard so on Ubuntu you do need to install the nginx-extras package to enable it.

PHP: Geocoding with MaxMind and nginx

Earlier this week, one of our adtech clients reached out asking if we could setup IP based geocoding for one of their applications. At a high level, what the application basically does is serve as a backend for an advertising pixel which is embedded on the sites of various publishers. When users are visiting a publisher’s site, the JS pixel makes a HTTP request to our backend from the client’s browser, receives a data payload from the backend, and then does various computations on the frontend.

What our client was looking to do was add the geocoding data into the payload that is returned by the backend. We’ve had some success with the MaxMind database in the past so we decided to investigate using that solution here as well. Initially, we implemented the geocoding using the static MaxMind database along with PHP and memcached to cache the “warmed” MaxMind PHP object. Unfortunately, using PHP presented significant performance issues at the scale we were serving requests. At an average of 20,000 requests/minute, the additional load introduced by the PHP processes serializing and deserializing the MaxMind objects would have ultimately been prohibitively expensive, even across 3 frontends.

So what’s the alternative? Turns out, there’s actually an nginx module that leverages the MaxMind database to make the geocoding data available as CGI parameters. Effectively, this lets you access the geocoding variables for the client’s IP address directly from the $_SERVER variable in PHP. Here’s how you set it up:

Depending on what your setup is, you’ll just need to enable the geoip module in ngnix by following the directions here. Once you have ngnix recompiled with the module on, you’ll need to add the configuration parameters specified into your “server” block. The final configuration step is to add the variables that you want exposed as CGI parameters. All together, you’ll need to end up making these modifications to your config files:

Reload or restart ngnix. Then, to access the variables in PHP you can just grab them out of the $_SERVER variable like:

That’s about it. From our tests, adding the geo module has a negligible effect on performance which is awesome. Of course, your mileage may vary but it’ll certainly be faster than using PHP directly.

Scaling HerCampus.com

Over the past month we’ve been working with Her Campus (HerCampus.com) to help them with issues they were having. When we started talking with Her Campus, we learned that they had different types of issues ranging from some Drupal based ones to actual server level problems. They were having some trouble keeping up to traffic demands, and if a traffic spike occurred their current infrastructure wasn’t sufficient.

After looking at their setup, we noted quite a few areas in which we could improve performance. The old setup was a fairly standard setup, a frontend server using Apache to handle HTTP requests and then a second server which was their MySQL database server. The servers were a 4 gig and 8 gig server respectively.

It was clear Apache was adding unneeded overhead, and wasn’t the best solution for them. We revamped their setup significantly. We switched them to Nginx + PHP-FPM. We immediately saw great improvement on from this change alone. However we wanted to get them to a single server, and to be able to handle traffic spikes with a single server.

We ended up doing the following:

  • Switch from Apache -> Nginx+PHP-FPM
  • Update all MyISAM tables to INNODB tables, and upgrade their MySQL to 5.1
  • Tune MySQL settings to fit their requirements
  • Update several tables adding indexes, reducing query time from in one case 34 seconds to 0.02 seconds.
  • Add the Boost Module to their setup
  • Update several of the view queries to be better written, added caching to each query.
  • Use ImageCache and sub-domains to load assets

After these updates we were able to move them from their two servers (8 gig and 4 gig) to a single server(4 gig). We have also reduced load times significantly. Their server loads dropped from 4-5 on average to 0.25. Recently they had an article on the Huffington Post and didn’t have any problems handling the 4x traffic spike they saw. At points we were seeing according to ChartBeat over 600 people on the site at once. The single server handled this without problems.

The updated infrastructure will give them a savings of about 75% from their previous setup. It also gives the users on the site a much faster and reliable experience.

We look forward to helping Her Campus with their continuing expansion of their site and user base!