Author: Ashish Datta

A few days ago, I was browsing my Feedly dashboard and ran across this AdWeek post describing how big retailers are gearing up to poach their competitors customers this holiday season. The article goes into some specifics, but the idea is basically that brands are planning to monitor Twitter for relevant conversations and then “at” message potential customers with special offers, product details, or even local store inventory information.

So imagine @MikeBruins65 from Boston tweeting “Wtf! @BestBuy offering 25% off all 4K TVs in-store…except nothing in stock.” and then @target replying “Cheer up @MikeBruins65! We have 4K TVs in-stock in Everett, MA! Grab coupons at http://bit.ly/target-4k-ma”. Since these brands are certainly leveraging powerful tools like Radian6 or even the full Twitter Firehose, it seems like it would be straightforward for them to execute strategies like this around high value markets. But what about as an individual, could you employ a similar strategy to make a few bucks?

Amazon Associates Links

The most obvious, least risky, and least lucrative approach would be to monitor Twitter for tweets that sounded like they were from frustrated buyers and then message them Amazon associates links for the product they're looking for. Looking at Amazon's fee structure, you'd want to target high margin categories with moderately expensive products and then hopefully end up doing a decent amount of volume. So imagine searching for Tweets from users frustrated that they can't checkout on a small eCommerce site, finding the product they're searching for on Amazon, and then Tweeting them the link to buy with your Associates link.

Dropshipping

More risky and potentially more upside. I'm not entirely sure how feasible this would be, but I think the idea would be to use a SaaS eCommerce platform like Shopify to setup an eCommerce shop and then dynamically list items which you'll later dropship. The challenge would be two fold, using Twitter to identify which previously obscure items are starting to trend and then figuring out how to introduce enough margin so that you end up profiting on the sale. It might be feasible though, with the explosion of small, boutique eCommerce sites it might be possible to negotiate a "I'll buy 400 for 50% off!" type deal quickly enough to introduce a profitable sale. The bigger challenge would probably be identifying these items as they start trending, but that could be solved by….

Pinterest

Recent member of the billion dollar boys club and frequent target of "haters", it's current traction and latent purchase intent potentially make it the perfect place for affiliate marketing. Beyond that, the wealth of potential gift pins and the follower/repin graph might hold the key to identifying relatively obscure products right before they begin to go viral. Anyway, I don't have any concrete ideas on how you could leverage Pinterest but it definitely seems like the ingredients for success are there.

Totally coincidentally, this article just came across TechCrunch - A Pin On Pinterest Is Worth 25% More In Sales Than Last Year, Can Drive Visits & Orders For Months

Anyway, are any of these actually feasible? Who knows, but I’d love to hear any other ideas.

Last week, LinkedIn published an indepth technical explanation of how their new LinkedIn Intro mobile product works on iOS. What Intro does is basically display LinkedIn data about your contacts directly in your email client - similar to what Rapportive did for gmail. It’s a cool app but the implementation details LinkedIn shared ignited an Internet firestorm, especially among the startup/hacker crowd.

How Intro works is it basically modifies the users normal iOS email client so that it connects through a LinkedIn proxy server instead of interacting with their webmail provider directly. What this does, is allow LinkedIn to dynamically modify a user’s email before it reaches their mail client, depending on if the user is connected to the sender on LinkedIn. From a IT security standpoint, introducing a third party that would sit between a user and the mail server they’re connecting to undoubtedly introduces a new attack vector but what really caught my interest was how LinkedIn was achieving this. In order to smoothly update the user’s proxy settings, LinkedIn is using a iOS feature known as Configuration Profiles.

I’m not familiar with the iOS SDK or APIs so this was the first time I’d heard about Configuration Profiles. In short, what they allow an app to do is install a set of settings on an iOS device - from email and web proxy settings to additional credentials and SSL keys. Configuration profiles are typically used in enterprise environments to allow a company’s IT department to quickly configure the settings on an employee’s iOS device. When provisioning a new device, IT would basically use the configuration profile to install things like a VPN, internal credentials, etc. So what’s the problem?

Well according to the LinkedIn post and comments from users that have used profiles before, the user experience of installing a profile which radically alters your iOS system settings is surprisingly unassuming. As a user, you click through a couple of prompts and boom, all of a sudden Safari is using a proxy server to fetch websites. So what nefarious things could you do by routing iOS mobile traffic through a proxy server? Unsolicited injected display advertising.

On the desktop web, unscrupulous extension developers have been monetizing their install base by injecting display ads into the browsing experience of their users for years. From companies like Bonzi Buddy to newer companies like PageRage, the model is tried, true, and profitable. However, on mobile there isn’t an obvious opportunity to inject ads and get access to the rapidly growing number of mobile web impressions. It seems like using configuration profiles would be the perfect vector to change this. Crapware iOS developers could quietly prompt their users to install a configuration profile to get access to “hot new features” and then surreptitiously start injecting display ads into websites on the proxy server.

I’m not familiar enough with iOS development to speak to how easy developing an app like this would be or if it would get past the app store approval process, but if it’s feasible someone is certainly going to do it. If anyone is familiar with an app already doing this, I’d love to know about it.

Last week, we were looking to setup one of our Symfony2 projects to use a master/slave MySQL configuration. We’d looked into using the MasterSlaveConnection Doctrine2 connection class, but unfortunately it doesn’t really work the way you’d expect. Anyway, the “next best” way to set up master/slave connections seemed to be creating two separate EntityManagers, one pointing at the master and one at the slave. Setting up the Doctrine configurations for this is pretty straightforward, you’ll end up with YAML that looks like:

# Doctrine Configuration
doctrine:
    dbal:
      connections:
        default:
          driver: %database_driver%
          host: %database_host%
          port: %database_port%
          dbname: %database_name%
          user: %database_user%
          password: %database_password%
          charset: UTF8
        slave:
          driver: %database_driver%
          host: %database_slave_host%
          port: %database_slave_port%
          dbname: %database_slave_name%
          user: %database_slave_user%
          password: %database_slave_password%
          charset: UTF8

    orm:
        auto_generate_proxy_classes: %kernel.debug%
        default_entity_manager: default
        entity_managers:
          default:
            connection: default
            mappings:
              SetfiveBundle:
                dir: Entity
              FOSUserBundle: ~
          slave:
            connection: slave
            mappings:
              SetfiveBundle:
                dir: Entity
              FOSUserBundle: ~

At face value, it looked like everything was working fine but it turns out they weren’t - the FOSUserBundle entities weren’t getting properly setup on the slave connection. Turns out, because FOSUserBundle uses Doctrine2 superclasses to setup it’s fields there’s no way to natively use FOSUserBundle with multiple entity managers. The key issue is that since the UserProvider checks the class of a user being refreshed, you can’t just copy the FOSUserBundle fields directly into your entity:

<?php
// From https://github.com/FriendsOfSymfony/FOSUserBundle/blob/6fc28e41805868c6c635de12266c40595c2e7ce8/Security/UserProvider.php

    public function refreshUser(SecurityUserInterface $user)
    {
        if (!$user instanceof User && !$user instanceof PropelUser) {
            throw new UnsupportedUserException(sprintf('Expected an instance of FOS\UserBundle\Model\User, but got "%s".', get_class($user)));
        }

        if (null === $reloadedUser = $this->userManager->findUserBy(array('id' => $user->getId()))) {
            throw new UsernameNotFoundException(sprintf('User with ID "%d" could not be reloaded.', $user->getId()));
        }

        return $reloadedUser;
    }

So how do you get around this? Turns out, you need to add a custom UserProvider to bypass the instance class check. My UserProvider ended up looking like:

<?php

namespace Setfive\AcmeBundle\Security;

use FOS\UserBundle\Security\UserProvider as BaseUserProvider;
use Symfony\Component\Security\Core\User\UserInterface as SecurityUserInterface;

class UserProvider extends BaseUserProvider {

    /**
* Constructor.
*
* @param UserManagerInterface $userManager
*/
    public function __construct($userManager)
    {
        parent::__construct($userManager);
    }

    public function refreshUser(SecurityUserInterface $user)
    {

        if (null === $reloadedUser = $this->userManager->findUserBy(array('id' => $user->getId()))) {
            throw new UsernameNotFoundException(sprintf('User with ID "%d" could not be reloaded.', $user->getId()));
        }

        return $reloadedUser;
    }

}

And then the additional YAML configurations you need are:

# services.yml

setfive.security.userProvider:
   class: Setfive\AcmeBundle\Security\UserProvider
   arguments: [@fos_user.user_manager] 


# security.yml

providers:
  fos_userbundle:
    id: fos_user.user_provider.username
  setfive_user:
    id: setfive.security.userProvider

The last step is copying all the FOSUserBundle fields directly into your User entity and update it to not extend the FOSUserBundle base class. Anyway, that’s it - two EntityManagers and one FOSUserBundle.

Since going live early this month, Healthcare.gov has been universally panned for being unstable, unusable, and close to an outright disaster. As the publicity of the problems have intensified, it’s become clear that the project has was mismanaged and probably sloppily developed. The team certainly faced daunting challenges like going from 0 to full capacity overnight but with a reported budget of over $50 million dollars and three years to execute it’s hard to have too much sympathy. Anyway, without knowing the project guidelines, business constraints, and other considerations it would be irresponsible to armchair quarterback and predict technical decisions that would have performed any better. However, on the non-technical side there are some strategic and architectural decisions that are disappointing.

The Team

Like any large project, the success of the project is ultimately driven by the strength of the team behind it. The experience, structure, and mindset of the team building the project will be the key drivers for what tradeoffs are made, how things are architected, and of course how development is executed. Unfortunately for us, according to the Washington Post, CGI Federal the company that built healthcare.gov is both a relative newcomer to US government IT consulting and also has a spotty track record of building large, commercial web applications.

I’ll be the first to admit that I have no background in government procurement, but it strikes me as odd that the Obama administration didn’t handpick the team that was going to build the software for their watershed legislation. Most notably, since the sweeping democratic win in 2008 has largely been attributed to the Obama campaign’s exceptional technology resources. Brad Feld actually suggested bringing in Harper Reed who served as the CTO of the campaign to fix the problems, but why wasn’t he leading the project from the outset?

Transparency (or lack thereof)

For an administration that campaigned on transparency (lets ignore NSA/drones), the lack of transparency surrounding healthcare.gov has been severely dissapointing. The lack of information has left media outlets reporting outlandish claims like that the site is 500 million lines of code or that Verizon has been brought in to help triage the situation. Apart from technical details, the White House has also remained completely silent on visitor and enrollment statistics even though they clearly have them since the site is running a slew of analytics and monitoring services. Leaving the public in the dark has driven rampant speculation and left everyone wondering "What if they aren't releasing data because its THAT bad".

Another area where healthcare.gov has been dissapointingly secretive is the actual source code of the project. Although some people, including Fred Wilson, are recommending open sourcing the code as a potential triage measure the code really should of been available from the outset. Even if the proprietary licensed technology had remained private, tax payer dollars financed the site and open sourcing it from the outset would have added some much needed oversight and accountability. Looking at the site, its using several open source libraries anyway including Twitter Bootstrap, jQuery, BackboneJS, and apparently violating the OSS license for jQuery.Datatables.

No Plan B

One of the accepted truths in software engineering is that shit is going to break and that bugs will crop up in even the most heavily tested applications. Mix in things like legacy integrations and 3rd party APIs and issues almost become a guarantee. Despite this, looking at the rollout of healthcare.gov it's clear that the team had no contingency plans for how to handle various error scenerios. As pieces of the application started to fail, they didn't have any way to mitigate poor user experiences, losing data, or simply showing "fail whale" style error messages. And although bad at launch, the situation still seems just as bad 3 weeks later - with their band aid to simply boot users once they've reached their concurrent session limit.

Building a high traffic web app that interfaces with legacy systems, is highly available, and is also easy to use is undoubtedly still difficult but the failures of healthcare.gov haven’t just been technological but procedural and organizational as well. Hopefully someone has the experience and political clout to right the ship before the website’s troubled launch defines the healthcare law it was built to implement.

Earlier this week, one of our adtech clients reached out asking if we could setup IP based geocoding for one of their applications. At a high level, what the application basically does is serve as a backend for an advertising pixel which is embedded on the sites of various publishers. When users are visiting a publisher’s site, the JS pixel makes a HTTP request to our backend from the client’s browser, receives a data payload from the backend, and then does various computations on the frontend.

What our client was looking to do was add the geocoding data into the payload that is returned by the backend. We’ve had some success with the MaxMind database in the past so we decided to investigate using that solution here as well. Initially, we implemented the geocoding using the static MaxMind database along with PHP and memcached to cache the “warmed” MaxMind PHP object. Unfortunately, using PHP presented significant performance issues at the scale we were serving requests. At an average of 20,000 requests/minute, the additional load introduced by the PHP processes serializing and deserializing the MaxMind objects would have ultimately been prohibitively expensive, even across 3 frontends.

So what’s the alternative? Turns out, there’s actually an nginx module that leverages the MaxMind database to make the geocoding data available as CGI parameters. Effectively, this lets you access the geocoding variables for the client’s IP address directly from the $_SERVER variable in PHP. Here’s how you set it up:

Depending on what your setup is, you’ll just need to enable the geoip module in ngnix by following the directions here. Once you have ngnix recompiled with the module on, you’ll need to add the configuration parameters specified into your “server” block. The final configuration step is to add the variables that you want exposed as CGI parameters. All together, you’ll need to end up making these modifications to your config files:

http {

  # .... rest of your configs

  # You can download these files from http://dev.maxmind.com/geoip/legacy/geolite/
  # These files need to be in the same directory as your config file or just use absolute paths
  geoip_country GeoIP.dat;
  geoip_city    GeoLiteCity.dat;

}

server {
  # ...rest of your configs
 
 # make the variables available in PHP 
 fastcgi_param GEO_COUNTRY_CODE $geoip_city_country_code;
 fastcgi_param GEO_CITY $geoip_city;
 fastcgi_param GEO_DMA $geoip_dma_code;
 fastcgi_param GEO_LATITUDE $geoip_latitude;
 fastcgi_param GEO_LONGITUDE $geoip_longitude;
}

Reload or restart ngnix. Then, to access the variables in PHP you can just grab them out of the $_SERVER variable like:

<?php

foreach( array("GEO_COUNTRY_CODE", "GEO_COUNTRY_CITY", "GEO_DMA", "GEO_LATITUDE", "GEO_LONGITUDE") as $key){
  echo $_SERVER[ $key ] . "\n";
}

That’s about it. From our tests, adding the geo module has a negligible effect on performance which is awesome. Of course, your mileage may vary but it’ll certainly be faster than using PHP directly.