Author: Matt Daum

A graduate of Tufts University, Matt worked at Boston Technologies and Tufts University before starting Setfive Consulting. A connoisseur of daytime television and sitcoms, Matt is an avid outdoorsman, beer brewer, and certified scuba diver. Connect with him on LinkedIn.

Symfony2 Entity Translations and Sonata Admin

By Matt Daum | 1 Comment on Symfony2 Entity Translations and Sonata Admin

We’ve worked on a number of projects which require the UI to be translated using the standard Symfony2 translator and YAML files. Recently we came into a few projects which also required different entities to have certain fields translated. Most of our applications we build use Sonata Admin for the admin backend so making sure we could integrate with it nicely was important. Looking around on Google and Stackoverflow it was clear that there are several different ways to go about getting your entities translated from the Gedmo Translatable, KnpLabs Translatable, to A2LiX I18n. Many of the packages have different takes on the “proper” way translation should be setup for the project. There are other nuances between each package such as supporting a fallback locale.

In the end we settled on using the KnpLabs Translatable bundle as it ticked all the boxes we wanted in functionality including fallback locale and a nice integration with the form (more on that later in this post). Installing the bundle follows the standard add it to composer and enable it in the kernel. From there setting up an entity was pretty straight forward:

Taking a look at how it actually works. First in the main entity you use the Translatable trait. Then in the translation entity (which is your original Entity with the name Translation appended to it) you add what fields you want translated, as well as the Translation trait. From there you can do something like $entity->translate(‘en’)->getName(). In our case we had a fairly large application already built and having to go through everywhere to update it to $entity->translate(…)->getXXX() would of been a huge pain and time waste. Luckily there is a fairly easy way to get around this. Using PHP’s magic __call method you can intercept all the calls so that it will go through the translations automatically:

The reason that it checks if arguments were passed in is that the Symfony2 property accessor doesn’t support passing arguments. We wanted to use it though when no arguments were passed since twig would otherwise first try on entity.name a call of “$entity->name()” which would fail as no name exists. You could wrap a few checks to make sure the method exists, instead since majority of our gets from twig do not pass any parameters we opted to just use the property accessor if no arguments were passed. This fixed the problem of {{ entity.name }} in Twig causing an error that the callback doesn’t exist and causing a 500. We ended up making our own Translatable trait which included this special __call override.

The final piece of advice on getting the translations working is when you add new translations to make sure you call $entity->mergeNewTranslations(). If you don’t you’ll be confused on why for it seems that none of your translations are being saved. This is documented, I just had looked over it first.

Now our second goal was a nice integration with Sonata Admin and any other forms we needed to use the translatable fields on. Luckily the A2LiX Translation Form Bundle already existed and we went forward with using it. Using the bundle was very easy. It was a simple as installing it, configuring it(just indicating what locales you want to use), and then updating the different form fields/admin setups. One thing to note is in the documentation it uses $form->add(‘translations’,’a2lix_translations’) as a bare minimum use case. At first, like me, you may think that the “translations” field is one of your field names. In fact that is used to load all translatable fields from your entity. It drops it into a nice tabbed input box. If you want to customize the field types and other options you can pass an array of options to set each field up in terms of labels, field type, etc. All in all it was really a huge time saver to have this bundle and was very easy to use from both a developer and user standpoint.

For the most part this is how we went about enabling translations on different entities in our application. In my next post I’ll write up the steps we used to migrate all the data from our existing entities to the new translations.

Update: My post on how to migrate your data to translatable entities is now available.

Making Doctrine and Symfony Logged Queries Runnable

By Matt Daum | 3 Comments on Making Doctrine and Symfony Logged Queries Runnable

On many of our projects we use Gearman to do background processing.  One of problems with doing things in the background is that the web debug toolbar isn’t available to help with debugging problems, including queries.  Normally when you want to see your queries you can look at the debug toolbar and get a runnable version of the query quickly.  However, when its running in the background, you have to look at the application logs to see what the query is.  The logs don’t contain a runnable format of the query, for example they may look like this:

Problem is you can’t quickly take that to your database and run it to see the results. Plugging in the parameters is easy enough, but it takes time. I decided to quickly whip up a script that will take what is in the gist above and convert it to a runnable format. I’ve posted this over at http://code.setfive.com/doctrine-query-log-converter/ . This hopefully will save you some time when you are trying to debug your background processes.

It should work with both Doctrine 1.x/symfony 1.x and Doctrine2.x/Symfony2.x. If you find any issues with it let me know.

Good luck debugging!

Are You a Developer? Know a Developer? We’re hiring!

By Matt Daum | Leave a Comment on Are You a Developer? Know a Developer? We’re hiring!

As we continue to expand in 2015 we’re looking to add another developer to our team.  Currently we’re seeking a junior level engineer to join us!  A few attributes of a person that we’re looking for:

  1. 1-2 years of experience with MVC based frameworks (we use Symfony2)
  2. 1-2 years of real world experience
  3. Comfortable talking directly with clients, no account managers here!
  4. Works well in a team environment, but also self-managed.

A few of the perks:

  1. Flexible hours
  2. 100% paid health care (PPO)
  3. 401(k) with matching
  4. Company outings

For some more detailed information on the job please visit the posting.  If you are, or know, a developer who is looking for a new opportunity lets connect!

High Performance With Netty and Aerospike

By Matt Daum | 1 Comment on High Performance With Netty and Aerospike

Recently we’ve been working with one of our clients to build application for use with AppNexus.  We were faced with a challenge which required a bunch of different technologies to all come together and work together.  Below I’ll try to list out how we approached it and what additional challenges we faced.

First came the obvious challenge:  How to handle at least 25,000 requests per second.  Our usual language of choice is PHP and knew it was not a good candidate for the project.  Instead we wanted to do some benchmarks on a number of other other languages and frameworks.  We looked at Rusty/Nginx/Lua, Go, Scala, and Java.  After some testing it appeared that Java was the best bet for us.  We initially loaded up Jetty.  We knew that this had a bit more baked in than we needed, but it was also the quickest way to get up and running and could be migrated away from fairly easily.    The idea overall was to keep the parsing of the request logic separate from the business logic.  In our initial tests we were able to get around 20,000 requests a second using Jetty, which was good, but we wanted better.

Jetty was great at breaking down the incoming HTTP requests to easily work with, it even provided an out of the box general statistics package.  However, we didn’t need much heavy lifting on the HTTP side, what we were building required very little complexity on with regards to HTTP protocol.   Jetty in the end was spending too many CPU cycles for what we needed.  We looked to Netty next.

Netty out of the box is not as friendly as Jetty as it is much lower level.   That said, it wasn’t too much work to get Netty up and running responding to HTTP request.  We ported over most of the business logic from our Jetty code and were off to the races.  We did have to add our own statistics layer as Netty didn’t have an embedded one for what we were looking for.  After some fine tuning with Netty we were able to start to handle over 40,000 requests per second.  This part of the puzzle was solved.

On our DB side we had heard great things about Aerospike in terms of performance and some of its features.  We ended up using this on the backend.  When we query Aerospike we have the timeout set at 3ms.  We’ll get around one or two request timeouts per second, or about 0.0025% of the time we’ll timeout, not too shabby. One of the nice features of Aerospike is the XDR function of the enterprise version.  With this we can have multiple Aerospike clusters which all stay in sync from a master cluster.  This lets us load our data onto one machine, which isn’t handling all the requests, and then it is replicated to the machines which are handling all the requests.

All in all we’ve had a great experience with the Netty and Aerospike integration.  We’re able to consistently handle around 40,000 requests a second with the average response time (including network time) of 4ms.

Thoughts and Suggestions on Airline Communication

By Matt Daum | 1 Comment on Thoughts and Suggestions on Airline Communication

It’s that time of the year that Setfive has retreated to warmer environments to focus on internal team building, communication, and management. I write this post as we fly past Florida on JetBlue to the Caymans and wanted to reflect on the struggle it has been to get to this point.

As we all know, weather in the Northeast the past month or so has not been very forgiving to anyone, especially airlines. Noone controls the weather so only best efforts can be made to work around it. Before I dive into some thoughts on how airlines may be able to improve the experience for their passengers, I want to give a up-to-date accounting of everything that has occurred thus far to show what I am drawing some of these suggestions from, I’ll try to keep this somewhat short and concise.

Thursday I realize that the majority of our team has a 45 minute layover in Newark which is already tight  and with the recent weather unlikely to be enough time.  I call up United Airlines and ask if we can move the group up to a flight that leaves on Sunday (the same day as originally planned) but about an hour earlier.  I’m told it would be 200 a person if we wanted to make the change, but there is plenty of room.  I suggest that we’re trying to just make sure we make our flight and United only has one flight out of Newark to the Caymans on Sunday so if we missed it, we’d be stuck.  I’m assured we’ll be there in plenty of time.

Saturday I decide to tweet “@united we have developers flying down tmrw morn. w/45 min layover, there is an earlier flight to have 1.5hr layover, can move them up”  to see if I have better luck.  United is very responsive and quickly look through options via direct messages.  The earlier flight is full and all other routes don’t look promising.  I understand they can’t make seats appear and understand we’re hoping for the best.  Saturday night at 9:55PM we get a notification from United our flight is delayed 20 minutes due to “Crew availability”.

We arrive Sunday morning and talk with an United service rep to see what we can do about our situation.  We’re told that we still may make the connection with only 10 minutes now at the layover with our delay, but they have already double booked us on a backup flight to fly to Miami, stay the night, and fly out Monday morning to the Caymans.  After another delay sets us back now an hour, we go back to the same rep and see if there are other routes we can take, like directly to Miami and catch a flight that night to the Caymans.  Unfortunately all the seats straight there are full, but the rep says we have our Miami flight on “backup” and that we may be able to catch the last flight to the Caymans and to just board our delayed Newark flight.

We arrive in Newark and talk to a service rep to get our backup boarding passes.  We find out that there is no booking for us at all, in fact the system couldn’t find any good alternatives so the automatic rebooking didn’t even work.  At this point the rep in Newark says the rep in Boston never booked us on anything.  Newark rep proceeds to try to get us on a later Miami flight, can’t get us confirmed, and then says we’ll rebook you tomorrow morning on a flight with an hour layover which you have to re-checkin for another airline and then will get to the Caymans.  I brought up the last time the hour layover didn’t work so hot, but the rep said she wasn’t concerned we could always get another flight to the caymans later that day or the next day.

At this point we decide that the Jetblue direct flight from JFK is worth the extra money to have less of not arriving for another day at the Caymans.  I call up United and try to get any sort of refund but am told our Boston-Newark flight was the majority of the cost so no refund would be given.  We try to get our luggage from United which they say wait for at least an hour and it should come out on that belt.  2 hours later we find out that 2 of the 3 bags we’re waiting for are now Miami bound and they’ll try to figure it out in Miami what to do with them.

Looking back on this we all thought the biggest problem was the lack of communication and accountability.  The group of us all felt that if there had been clear communication (or in some cases any communication) that much of the stress and problems would have been mitigated.  The other problem is each time we did anything, we started having to double check that it was actually done.

Here are a some suggestions for possible improvements that I think may help all airlines (and possibly other industries) work better with their customers.  Some of these may already exist at some airlines or not be feasible to do in some situations; I just wanted to get some thoughts out and see what people think.

  1. Have a clear, non-technical way which all communication can be documented and viewed.  On several of my calls with United I asked them to make sure they made notes on my record to document what was discussed.  Out of all those requests, I have not had one rep say they saw any log of any previous calls I made to them.
  2. Increase mediums for communication.  United did a great job of responding quickly via Twitter, which also satisfies my first suggestion, however it would be great if there was a live chat for customers without Twitter.  The live chat also provides easy documentation of everything that was discussed and who it was with.
  3. Increased accountability.  One of the more frustrating parts of any situation is when you are told one thing, but find out from another representative that it is not the case.  This seems to happen a lot at larger companies.  Even when you can prove you were told one thing, the other representative’s answer is usually “they shouldn’t have said that”.  I can’t imagine what our clients would think if you talked to one of our developers and were told one thing, but then told something completely differently from another developer.  Aside from better training to prevent these situations, I would think the company would try to “make it up” in one fashion or another to the customer.  Anytime I feel that we may have crossed communication here or misled someone, I make sure to do everything and anything within my power to make sure that the client is satisfied.  I’m not saying the airline should dwell out free flights, but things that have a very small cost could make difference in the customers eyes, for example free access to the lounge.
  4. Callbacks.  Some companies I’ve noticed have started to do this.  You can call up, leave a number and it will ring you back when you are about to be connected with a representative.  I imagine this should be more efficient for all parties.  As a customer I no longer have to wait listening hold music for an hour or even worse have a dropped call after 35 minutes of holding.  Often I’ll put the phone down and forget about it, and come back later to someone saying “This is the last time I will ask can you hear me?”   On the airline side, representatives will have very few, if any, calls where the person has left for the moment and isn’t ready when they are taken off hold.    If a user doesn’t answer, it could even try back 5 minutes later before removing them from the queue.

I understand our troubles with the airline getting down are no more important or different than the thousands of other’s that had problems.  I’m more interested in seeing what ways we can try to improve the experience overall for both customers and airlines alike.  I know running a business is never as simple as it seems and some of these suggestions may be implemented behind the scenes.  However, there is always room for improvement.

What do you think?  How can airlines improve the customer relations experience?