Author: Ashish Datta

I’ll preface this by saying that I know just enough about machine learning to be dangerous and get myself into trouble. That said, if anything is inaccurate or misleading let me know in the comments and I’ll update it. Last April Amazon announced Amazon Machine Learning, a new AWS service aimed at developers to help them build and deploy machine learning solutions. We’ve been excited to experiment with AWS ML since it launched but haven’t had a chance until just now.

A bit of background

So what is “machine learning”? Looking at Wikipedia’s definition machine learning is ‘is a subfield of computer science that evolved from the study of pattern recognition and computational learning theory in artificial intelligence. In 1959, Arthur Samuel defined machine learning as a “Field of study that gives computers the ability to learn without being explicitly programmed”.’ That definition in turn translates to using a computer to solve problems like regression or classification. Machine learning powers dozens of the products that internet users interact with everyday from spam filtering to product recommendations to Siri and Google Now.

Looking at the Wikipedia article, ML as a field has existed since the late 1980s so what’s been driving its recent growth in popularity? I’d argue key driving factors have been compute resources getting cheaper, especially storage, which has allowed companies to store orders of magnitude more data than they were 5 or 10 years ago. This data along with elastic public cloud resources and the increasing maturity of open source packages has made ML accessible and worthwhile for an increasingly large number of companies. Additionally, there’s been an explosion of venture capital funding into ML focussed startups which has certainly also helped boost its popularity.

Kicking the tires

The first thing we need to do before testing out Amazon ML was to pick a good machine learning problem to tackle. Unfortunately, we didn’t have any internal data to test with so I headed over to Kaggle to find a good problem to tackle. After some exploring I settled on Digit Recognizer since its a “known problem”, the Kaggle challenge had benchmark solutions, and no additional data transformations would be neccessary. The goal of the Digit Recognizer problem is to accept bitmap representations of handwritten numerals and then correctly output what number was written.

The dataset is a modified version of the Mixed National Institute of Standards and Technology which is a well known dataset often used for training image processing systems. Unlike the original MNIST images, the Kaggle dataset has already been converted to a grayscale bitmap array so individual pixels are represented by an integer from 0-255. In ML parlance, the “Digit Recognizer” challenge would fall under the umbrella of a classification problem since the goal would be to correctly “classify” unknown inputs with a label, in this case a 0-9 digit. Another interesting feature of the MNIST dataset is that the Wikipedia provides benchmark performance for a variety of approaches so we can have a sense of how AWS ML stacks up.

At a high level, the big steps we’re going to take are to train our model using “train.csv”, evaluate it against a subset of known data, and then predict labels for the rows in “test.csv”. Amazon ML makes this whole process pretty easy using the AWS Console UI so there’s not really any magic. One thing worth noting is that Amazon doesn’t let you select which algorithm will be used in the model you build, it selects it automatically based on the type of ML problem. After around 30 minutes your model should be built and you’ll be able to explore the model’s performance. This is actually a really interesting feature of Amazon ML since you wouldn’t get these insights with visualizations “out of the box” from most open source packages.

Performance

With the model built the last step is to use it to predict unknown values from the “test.csv” dataset. Similar to generating the model, running a “batch prediction” is pretty straightforward on the AWS ML UI. After the prediction finishes you’ll end up with a results file in your specified S3 bucket that looks similar to:

1,6,0,4,7,3,5,8,9,2
1.544274E-10,3.736493E-6,1.298402E-4,3.529298E-8,2.738585E-7,1.814797E-5,3.520103E-6,7.861468E-6,1.17829E-6,9.998354E-1
2.196675E-11,7.322348E-6,9.969799E-1,3.165914E-10,1.485307E-5,4.171782E-6,2.970602E-3,6.487699E-6,1.930486E-7,1.643579E-5
1.345541E-3,1.968907E-4,1.209908E-5,7.132479E-2,1.572009E-3,7.926991E-4,3.921966E-2,2.677833E-2,8.559376E-1,2.820424E-3
8.210548E-6,2.666948E-3,6.711699E-3,5.835555E-2,2.407767E-1,3.312858E-5,1.107812E-3,5.861267E-4,6.673768E-1,2.237697E-2
1.205649E-3,1.128586E-2,4.179959E-4,4.942253E-5,2.8488E-4,7.751592E-1,6.43877E-3,8.812359E-3,1.996288E-4,1.961462E-1

Because there are several possible classifications of a digit the ML model generates a probability per classification with the largest number being the most likely. Individual probabilities are great but what we really want is a single digit per input sample. Running the input through the following PHP will produce that along with a header for Kaggle:

<?php

$lines = explode("\n", file_get_contents("results_test.csv"));
$header = str_getcsv($lines[0]);

unset($lines[0]);

echo "ImageId,Label\n";
$imageId = 1;

foreach($lines as $ln){
    $ln = str_getcsv($ln);
    for($i = 0; $i < count($ln); $i++){
        $ln[$i] = (float) $ln[$i];        
    }
        
    $maxIndex = array_search(max($ln), $ln);
    
    echo $imageId . "," . $header[$maxIndex];
    
    if($imageId < count($lines)){
        echo "\n";
    }
    
    $imageId += 1;    
}

And finally the last step of the evaluation is uploading our results file to Kaggle to see how our model stacks up. Uploading my results produced a score of 0.91671 so right around 92% accuracy. Interestingly, looking at the Wikipedia entry for MNIST a 8% error rate is right around what was academically achieved using a linear classifier. So overall, not a bad showing!

Takeaways

Comparing the model’s performance to the Kaggle leaderboard and Wikipedia benchmarks, AWS ML performanced decently well especially considering we took the defaults and didn’t pre-process the data. One of the downside of AWS ML is the lack of visibility into what algorithms are being used and additionally not being able to select specific algorithms. In my experience, solutions that mask complexity like this work great for “typical” use cases but then quickly breakdown for more complicated tasks. Another downside of AWS ML is that it can currently only process text data that’s formatted into CSVs with one record per row. The result of this is that you’ll have to do any data transformations with your own code running on your own compute infrastructure or AWS EC2.

Anyway, all in all I think Amazon’s Machine Learning product is definitely an interesting addition to the AWS suite. At the very least, I can see it being a powerful tool to be able to quickly test out ML hypothesis which can then be implemented and refined using an open source package like skit-learn or Apache Mahout.

Picture the scene, it’s Friday night, you’ve got friends over and everyone wants to listen to some great music. The problem is everyone wants to jam to something different and you’re not thrilled to sit by your laptop all night. Enter, the TxtyJukebox.

TxtyJukebox lets you setup an event which creates you a unique number which users can text in song requests to. As TxtyJukebox receives song requests, it searches YouTube for music videos and then places the videos into your event’s queue. And then if you hook up TxtyJukebox to a TV you’ll be able to jam to videos on a big screen with big room sound. But wait, there’s more! If you have a Chromecast you can connect TxtyJukebox to your Chromecast via our app. The Chromecast app will launch from within http://jukebox.setfive.com/ so there’s nothing to download or setup.

So how does TxtyJukebox work under the hood? Well sit tight, technical details lay ahead. The webapp itself is a standard Symfony2 app along with the usual suspects - Bootstrap, Underscore, and a sprinkling of jQuery. Along with that, we’re using Twillio’s REST API to handle SMS along with a “webhook” from Twillio to the webapp to recieve messages. In addition, we’re leveraging the YouTube API to search and load videos which are then loaded into an iframe. Finally, the Chromecast app is HTML/CSS/JS powered by jQuery and underscore.

Building TxtyJukebox was a lot of fun and we’re thrilled that it’s been positively received. An awesome surprise was that Ryan over at Makeusof.com found it and incldued it in his post of How to Share Music from Multiple Devices to a Chromecast. As always, let us know if you have any questions or comments.

For the last few months we’ve been working on a Spring Boot project and one of the more challenging aspects has been wrangling Spring’s security component. For the project, we were looking to authenticate users using a custom HTTP header that contained a token generated from a third party service. There doesn’t seem to be a whole lot of concrete examples on how to set something like this up so here’s some notes from the trenches. Note: I’m still new to Spring so if any of this is inaccurate, let me know in the comments.

Concretely, what we’re looking to do is authenticate a user by passing a value in an X-Authorization HTTP header. So for example using cURL or jQuery:

:~$ curl -H "X-Authorization: $some_secret_token" http://localhost/user

$.ajax({
    url: 'http://localhost/user',
    headers: { 'X-Authorization': '$some_secret_token' }
});

In addition to insuring that the token is valid, we also want to setup Spring Security so that we can access the user’s details using “SecurityContextHolder.getContext().getAuthentication()”. So how do you do this? Turns out, you need a couple of classes to make this work:

• An Authentication Token: You need a class that extends AbstractAuthenticationToken so that you can let Spring know about your authenticated user. The UsernamePasswordAuthenticationToken class is a pretty good starting point.

• The Filter: You’ll need to create a filter to inspect requests that you want authenticated, grab the X-Authentication filter, confirm that it’s a valid token, and set the corresponding Authentication. Since we only want this to run once per request you can extend the OncePerRequestFilter class to set this up. You can see an example class below:

  import java.io.IOException;
  import javax.servlet.FilterChain;
  import javax.servlet.ServletException;
  import javax.servlet.http.HttpServletRequest;
  import javax.servlet.http.HttpServletResponse;
  
  import org.springframework.security.core.Authentication;
  import org.springframework.security.core.context.SecurityContextHolder;
  import org.springframework.web.filter.OncePerRequestFilter;
  
  public class DemoAuthenticationFilter extends OncePerRequestFilter {
  
      @Override
      protected void doFilterInternal(HttpServletRequest request,
              HttpServletResponse response, FilterChain filterChain)
              throws ServletException, IOException {
          
          String xAuth = request.getHeader("X-Authorization");
          
          // validate the value in xAuth
          if(isValid(xAuth) == false){
              throw new SecurityException();
          }                            
          
          // The token is 'valid' so magically get a user id from it
          Long id = getUserIdFromToken(xAuth);
          
          // Create our Authentication and let Spring know about it
          Authentication auth = new DemoAuthenticationToken(id);
          SecurityContextHolder.getContext().setAuthentication(auth);            
          
          filterChain.doFilter(request, response);
      }
  
  }

• An Authentication Provider: The final piece is a class that extends AuthenticationProvider which handles retrieving a JPA entity from the database. By implementing an AuthenticationProvider instead of doing the database lookup in the filter, you can keep your filter framework agnostic by not having to autowire in a JPA repository. My implementation looks similar to:

  import org.springframework.beans.factory.annotation.Autowired;
  import org.springframework.security.authentication.AuthenticationProvider;
  import org.springframework.security.core.Authentication;
  import org.springframework.security.core.AuthenticationException;
  import org.springframework.stereotype.Component;
  import com.pearson.reader.error.UnknownUserException;
  import com.pearson.reader.models.User;
  import com.pearson.reader.repositories.UserRepository;
  
  @Component
  public class DemoAuthenticationProvider implements AuthenticationProvider {
  
      // This would be a JPA repository to snag your user entities
      private final UserRepository userRepository;
      
      @Autowired
      public DemoAuthenticationProvider(UserRepository userRepository) {
          this.userRepository = userRepository;
      }    
      
      @Override
      public Authentication authenticate(Authentication authentication) throws AuthenticationException {
          
          DemoAuthenticationToken demoAuthentication = (DemoAuthenticationToken) authentication;        
          User user = userRepository.find(demoAuthentication.getId());
          
          if(user == null){
              throw new UnknownUserException("Could not find user with ID: " + demoAuthentication.getId());
          }
          
          return user;
      }
  
      @Override
      public boolean supports(Class<?> authentication) {
          return DemoAuthenticationToken.class.isAssignableFrom(authentication);
      }
  
  }

And finally, the last step is to wire this all up. You’ll need a class that extends WebSecurityConfigurerAdapter with two ovveridden configure methods to configure the filter and the authentication provider. For example, the following works at a bare minimum:

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.annotation.authentication.builders.AuthenticationManagerBuilder;
import org.springframework.security.config.annotation.method.configuration.EnableGlobalMethodSecurity;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;
import org.springframework.security.config.annotation.web.servlet.configuration.EnableWebMvcSecurity;
import org.springframework.security.web.authentication.www.BasicAuthenticationFilter;

@Configuration
@EnableWebMvcSecurity
@EnableGlobalMethodSecurity(securedEnabled = true)
public class SecurityConfigDemo extends WebSecurityConfigurerAdapter {
    
    @Autowired
    private DemoAuthenticationProvider demoAuthenticationProvider;
    
    @Override
    protected void configure(HttpSecurity http) throws Exception {        
                
        http
        .authorizeRequests()
            .antMatcher("/user")
                .addFilterBefore(new DemoAuthenticationFilter(), BasicAuthenticationFilter.class)                
        ;
                        
    }
    
    @Override
    public void configure(AuthenticationManagerBuilder auth) throws Exception {        
        auth.authenticationProvider(demoAuthenticationProvider);        
    }    
    
}

And then finally to access the authenticated user from a controller you’d do:

        Authentication auth = SecurityContextHolder.getContext().getAuthentication();
        User user = (User) auth.getPrincipal();

Anyway, hope this helps and as mentioned above if there’s anything inaccurate feel free to post in the comments.

One of the more opaque concepts about AngularJS is the process that converts a chunk of HTML from a template into “Angularized” HTML which is then inserted into the DOM. During this conversions, custom directives are replaced with their corresponding HTML content, Angular directives like ng-repeat are processed, and any event handlers of interest are wired up. As it turns out, Angular’s $compile service is what’s responsible for making the magic happen. OK great, but why is this interesting or important? Because leveraging the $compile service directly lets you take dynamic content and process it to enable Angular directives and behaviors.

Since examples are always helpful, here’s an admittedly contrived one that we’ll walkthrough. Imagine that we’re building a WordPress slideshow plugin and we want to support custom themes for individual slides. So in our plugin, a user would be able to modify the HTML that displays a slide, we’d save it to the database, and then retrieve that template when we render the slides. For arguments sake, let’s assume the “default” template for the slideshow looks something like this:

<div ng-class="slide.isActive ? 'active slide' : 'slide'">
  <slide-image ng-src="{{ slide.imageSrc }}"></slide-image>
  <div class="description" ng-bind="slide.description"></div>
  <starbar config="slide.starConfig"></starbar>
</div>

As you can see, we’ve got a few directives and by default we’re displaying some description. Generally, we could set this up by creating a “slide” directive that looks something like:

angular.directive('slide', function(){
    return {
      restrict: 'E',
      scope: {slide: '='},      
      templateUrl: 'slide.html',
    };
});

Great, nothing to crazy but with this setup there’s no way to supply dynamic HTML from our database to use in the template. In order to allow a custom template you’d just need to modify the directive to look something like:

.directive('slide', function($sce, $compile){
  return {
    restrict: 'E',
    replace: true,
    scope: {slide: '='},
    template: "<div></div>",
    link: function(scope, el, attr){
      angular.element(el).html(scope["slide"].template);
      $compile(el)(scope);
    }
  };
})

And then you’d be able to use it with:

<slide slide='config'></slide>

/** Javascript **/
$scope["config"] = {
  "template": "<h3 style='color: red; font-weight: bold' ng-bind='slide.description'></h3>",
  "description": "hello world!",
};

The key difference is that in the modified directive the template is inserted into the directive’s element using “angular.element(el).html(scope[“slide”].template);“ and then finally the $compile service is invoked to process the regular HTML to get Angular magic.

Anyway, as always questions or comments welcome!

Labor day has come and gone so summer is officially over. We sat down with our intern Phil to chat about his time interning at Setfive.

Favorite Part About Interning At Setfive?

My favorite part about interning at Setfive was being introduced to so many different programming tools, and having the ability to increase my programming skill set. This summer I learned about PHP, the Symfony 2 Framework, MYSQL, I improved my JavaScript skills, learned some Angular.js, and even learned how to write unit tests. I was exposed to so many new things that everyday was fun and no two days were ever the same.

The environment here encouraged questions, and allowed me to ask and receive answers to anything I wanted to know more about. Some of the guys would even go out of their way to send me related documentation about something if they felt that they couldn’t confidently answer it themselves.

Working under the guys here was an incredible experience, I was given the freedom to make mistakes and figure out problems on my own, but at the same time was given sufficient structure to make consistent progress. It was awesome to have the comfort of knowing I had a smart, qualified person to guide me in the right direction if I ever got too stuck on any one problem.

Most important thing that you learned?

The most important skill that I learned was definitely an improved conceptual understanding of MVC, and that while sometimes using this pattern slows down your programming, in the long run it helps you create readable, modular code.

I also learned that installation is just the worst.

Most Memorable Moment?

The most memorable moment of the summer was the first time we used the Txty Jukebox in the office. It didn’t quite work the first time around, however, watching people use and get enjoyment out of something that I helped to create was something that I’ will never forget.

Where do you want to go from here?

From here I definitely want to continue building custom applications. I’ve spent the last part of the summer teaching myself objective-c, and the skills that I’ve learned here will definitely help me make the transition into developing iOS applications.

Top 5 Things To Eat

1. Buffalo Soulja - Darwins (Only available on Thursdays)
2. Mango Bubble Tea - Dosa Factory
3. Steak Sammy - Orinoco
4. Burger topped with shortrib meat – Charlies Beer Garden
5. Chicken Pad Thai - Thelonious Monkfish
6. Honorable Mention: Cuban Sammy - Plough and Stars