PHP: Is PHP losing popularity?

I was on Quora earlier this week and ran across a question asking Why is PHP losing popularity? along with the elaboration being:

I've been keeping my eye on job boards and tech media, and it seems that the new trend is up with Node.js and down with PHP, Ruby staying about the same. I know every tool has its purpose, but most web applications could be built in any language and framework and fare all the same. PHP is surely scalable, fast enough, safe enough, and heavily supported. Facebook has been happy enough to keep it around, and they have a lot to own up to.

The top two answers basically declare that PHP is on its way out because developers have more options (Ruby, Python NodeJS, etc.) and because the PHP ecosystem is “standing still”. It’s pretty clear that both of these answers are incomplete, if not outright wrong but then where does the truth lie?

Since this isn’t high school there isn’t a universal metric for how to evaluate the popularity of a given programming language. People tend to use things like search trends, job salaries, or the number of questions tagged on StackOverflow. These KPIs are fine for vanity comparisons but they don’t really reveal anything about the velocity, evolution, or enthusiasm for a language or framework. Since we’re concerned specifically with PHP, it’s easier to pick out specific examples that demonstrate its continuing popularity.

Drupal and WordPress

Although treated with disdain by developers, both applications power hundreds of millions of websites. According to Wikipedia, "WordPress is used by more than 18.9% of the top 10 million websites as of August 2013." and Drupal "is used as a back-end system for at least 2.1% of all websites worldwide". Which, to put in perspective, are both absolutely staggering numbers. On top of that, Automatic and Acquia, the companies commercially backing WordPress and Drupal, have been raising money and growing at a phenomenal pace.

So what does that mean for PHP? Velocity. With two well funded commercial companies actively developing, selling, and supporting PHP software there will be an increasing number of PHP sites coming online every day.

Symfony, Doctrine, and Zend

In the last few years, all 3 projects have completely overhauled their architectures and rewritten their code bases to incorporate from their respective "version 1s". A total rewrite is a heroic feat for any project, let alone a popular open source project with thousands of users. All three rewrites have had a positive effect on the PHP community as a whole, including powering new frameworks (Symfony2, Laravel, etc.), enabling consolidation (Drupal 8 powered by Symfony Components), and of course pushing developers to evaluate PHP for new projects.

For PHP, this certainly signals a continuing evolution and a willingness of the community to learn, adapt, and evolve as the web changes.

The Language / PHP Internals

Unfortunately, this is one facet that strongly negatively affects the perception of the PHP ecosystem. Looking at PHP the language, plenty has been written bemoaning the inconsistencies, "wtfs", and generally bizarre paradigms that the language constructs introduce. Although things have certainly gotten better, a lot of the same issues people were complaining about in PHP4 still exist today with no roadmap for them to be resolved.

Related to the language, is the PHP internals mailing list where core devs discuss language changes and generally how PHP will evolve. Most recently, core dev Anthony Ferrara outlined the major problems with the internals mailing list and generally why he feels the project is in trouble.

The issues with the PHP the language and its apparent lack of real evolution clearly affect the enthusiasm for the ecosystem. Outsiders look in and ask why we’re dealing with a “shitty” language while insiders are stuck defending PHP by pointing to features it got in 5.4 while meekly dodging the fact that there’s still no real unicode support.

So is PHP becoming less popular? Almost certainly not. In the last few years, dozens of interesting new tools and frameworks have been built and at least two VC funded companies have built successful business on software powered by PHP. Unfortunately, the perception of PHP the language as a ghetto is still persistent and the internals team seems to have no plans to change it.

isset(), empty(), is_null() - What's the difference?

I came across an articleon phpro.org about the difference between isset, empty and isnull methods that I found it informative so I’m going to summarize and re-post it here.

There are often times where you need to check for empty or null values or if a variable is set. It’s pointed out that in many circumstances the wrong function is used to make these assertions. The code may end up working; however, in some cases using the wrong function returns a value that programmer didn’t expect and leads to errors.

Actions speak much louder than words so I’ll cut to the example. The example script tests the following functions and operators:

  • isset()
  • empty()
  • is_null()
  • ==
  • ===

against the following values:

  • no value set
  • null
  • zero
  • false
  • numeric value
  • empty string

and builds out a comparison table (see below) of the results. The notice above the table is because the isset() function is trying to check a variable that has not been initialized(Not Set).

I’ll be honest, I never knew passing a zero into the empty() function returns a true!

Anyways, the chart below ends up becoming a useful reference guide as well. The code used to produce the table is at the bottom of this post.

The Code

<?php
    /*** turn on error reporting ***/
    error_reporting( E_ALL );

    /*** an array of test values ***/
    $values = array( $var, null, 0, false, 100, '');
    echo '<tr>';
    echo '<td>isset()</td>';
    foreach( $values as $val )
    {
        echo '<td>';
        var_dump( isset( $val ) );
        echo '</td>';
    }
    echo '</tr>';

        echo '<tr>';
    echo '<td>empty()</td>';
        foreach( $values as $val )
        {
                echo '<td>';
                var_dump( empty( $val ) );
                echo '</td>';
        }
    echo '</tr>';

        echo '<tr>';
        echo '<td>is_null()</td>';
        foreach( $values as $val )
        {
                echo '<td>';
                var_dump( is_null( $val ) );
                echo '</td>';
        }
        echo '</tr>';

        echo '<tr>';
    echo '<td>==</td>';
        foreach( $values as $val )
        {
                echo '<td>';
                var_dump( $val == false  );
                echo '</td>';
        }
    echo '</tr>';

        echo '<tr>';
    echo '<td>===</td>';
        foreach( $values as $val )
        {
                echo '<td>';
                var_dump( $val === false );
                echo '</td>';
        }
        echo '</tr>';
?>
</tbody>
</table>

WordPress: Modify Native Calendar Widget for Event Post Types

Recently I was working on WordPress website and the client’s theme had an existing “Event” custom post type. The client wanted a way to calendarize the event start date and display with a widget.

There is a calendar widget built into WordPress; however, it has a couple shortcomings for what the client was trying to do. First, it calendarises all posts regardless of type (and in this case all we want is event posts). Secondly, it also calendarises posts based on date posted rather than a field such as event date. As an added constraint, the non-profit client did not have a budget for building a custom plugin from scratch to fill the need.

So I went digging through the PHP script that powers the native calendar widget and found a way to modify the plugin while only having to bill about a half hour of custom development. It actually took me longer to write this post!

It’s not the most ideal or elegant solution but it saved the client a bunch of money by spending almost no time on this. Couple drawbacks without further modifications 1) the widget won’t display multi-day events properly 2) The calendar widget now only works with event post types (i.e., can’t calendarize post dates of normal blog posts).

Here’s what it ends up looking like:

The Code

PHP

The file I had to modify is located in the Wordpress 3.5 installation in wordpressroot>wp-includes>general-template.php

In order to know which days on the calendar to highlight as links to posts, the WordPress devlopers built a query that pulls the day numbers of the posts that were posted in the current month.

 <?php
  // Get days with posts
	$dayswithposts = $wpdb->get_results("SELECT DISTINCT DAYOFMONTH(post_date)
		FROM $wpdb->posts WHERE post_date >= '{$thisyear}-{$thismonth}-01 00:00:00'
		AND post_type = 'post' AND post_status = 'publish'
		AND post_date <= '{$thisyear}-{$thismonth}-{$last_day} 23:59:59'", ARRAY_N);
	
  if ( $dayswithposts ) {
		foreach ( (array) $dayswithposts as $daywith ) {
			$daywithpost[] = $daywith[0];
		}
	} else {
		$daywithpost = array();

There’s a couple issues here as it relates to getting this to work for events as opposed to regular posts. First, the query is pulling post_type equal to “post” so we’ll need to change that “event” so only events are being pulled. Secondly, the original query is selecting posts based on post date in the current month. For events, when we’re looking at a calendar, we don’t care when it was posted, but rather the start date of the event.

To deal with these issues, I had to rewrite the query. Unfortunately, the event start date field was one of the custom fields added for the event post type so it resides in the wp_postmeta table as opposed to wp_posts so you have to join the wp_postsandwp_postmeta` tables to get access to all the required fields. The query returns the event start day of the month (dom) , post id (post_id) , url of the event (guid) and event title (post_title). You’ll see soon why I pulled more fields than just the day numbers like before. Also, I changed the return value to an array of objects instead of numerical array per my own preferences. The new query looks something like this:

<?php

// Get days with posts
  $dayswithposts = $wpdb->get_results("SELECT (FROM_UNIXTIME(`wp_postmeta`.`meta_value`,'%d')) as dom , 
    `wp_postmeta`.`post_id` , `wp_posts`.`guid` , `wp_posts`.`post_title` 
		FROM $wpdb->postmeta 
		LEFT JOIN  `wp_posts` ON  `wp_postmeta`.`post_id` =  `wp_posts`.`ID` 
		WHERE  `wp_postmeta`.`meta_key` =  'event_startdate'
		AND  `wp_posts`.`post_status` =  'publish'
		AND  `wp_postmeta`.`meta_value` >= UNIX_TIMESTAMP(  '{$thisyear}-{$thismonth}-01 00:00:00' ) 
		AND  `wp_postmeta`.`meta_value` <= UNIX_TIMESTAMP(  '{$thisyear}-{$thismonth}-{$last_day} 23:59:59' )", OBJECT);

The next block of the original code (see gist below) executes a second query almost identical to the first, but instead of pulling just day numbers like before, it pulls the post id, post titles and day numbers. The block following the query goes through each post and puts the titles into an associative array indexed by the day of the month. The array of post titles ($ak_titles_for_day) will used to display the post titles on their corresponding day cell in the calendar when it builds out the calendar table.

  <?php
  
  $ak_titles_for_day = array();
	$ak_post_titles = $wpdb->get_results("SELECT ID, post_title, DAYOFMONTH(post_date) as dom "
		."FROM $wpdb->posts "
		."WHERE post_date >= '{$thisyear}-{$thismonth}-01 00:00:00' "
		."AND post_date <= '{$thisyear}-{$thismonth}-{$last_day} 23:59:59' "
		."AND post_type = 'post' AND post_status = 'publish'"
	);
	if ( $ak_post_titles ) {
		foreach ( (array) $ak_post_titles as $ak_post_title ) {

				$post_title = esc_attr( apply_filters( 'the_title', $ak_post_title->post_title, $ak_post_title->ID ) );

				if ( empty($ak_titles_for_day['day_'.$ak_post_title->dom]) )
					$ak_titles_for_day['day_'.$ak_post_title->dom] = '';
				if ( empty($ak_titles_for_day["$ak_post_title->dom"]) ) // first one
					$ak_titles_for_day["$ak_post_title->dom"] = $post_title;
				else
					$ak_titles_for_day["$ak_post_title->dom"] .= $ak_title_separator . $post_title;
		}
	}

Use of two queries seemed a bit duplicative to me since they are so similar so I got rid of the second one and pulled the additional fields in the first query. Next I modified the foreach to record both the event title and event page url for every event post (this is so we can display a list of event links when a user hovers over a calendar day). I also consolidated the foreach code with the loop that follows the first query. Based on the results form the first query, two arrays are built. The first array, $dayswithpost, is simply an array containing all the day numbers of the month that have events to be used when building out the calendar table. The $ak_titles_for_day array, as mentioned above, is an associative array indexed by day numbers. It stores the event title and event url under the corresponding day number index and will be used to display to the user when they hover over a day number on the calendar that has event starting dates.

<?php  
  if ( $dayswithposts ) {
		foreach ( $dayswithposts as $daywith ) {
			
			$daywithpost[] = $daywith->dom;
			
			$post_title = esc_attr( apply_filters( 'the_title', $daywith->post_title, $daywith->post_id ) );

			if(empty($ak_titles_for_day)){
					$ak_titles_for_day[$daywith->dom]= array ();
					$ak_titles_for_day["$daywith->dom"][] = array('title'=>$daywith->post_title,'url'=>$daywith->guid);
				}else{
					$ak_titles_for_day["$daywith->dom"][] = array('title'=>$daywith->post_title,'url'=>$daywith->guid);
					}
		}
	} else {
		$daywithpost = array();
	}

Lastly, the code to build out the table/calendar had to be modified. The original block of code compares the day number of the calendar cell being generated to the $dayswithposts array to determine if any posts are listed for that day. If the day number has corresponding posts in the $dayswithposts array, an anchor tag is wrapped around the day number and href generated using the built in WordPress function get_day_link(). This function returns the url of an archive page for all posts on a given day. This works nicely for days with multiple posts because you can see all the posts by navigating to that daily archive url which is a built in WordPress template. The post title(s) for days with posts are also inserted into the title attribute of the day number link from the values stored in the $ak_titles_for_day array so the user will see the post titles when they hover over a calendar day link.

<?php

$calendar_output .= '<td>';
  	if ( in_array($day, $daywithpost) ) // any posts today?
				$calendar_output .= '<a href="' . get_day_link( $thisyear, $thismonth, $day ) . '" title="' . esc_attr( $ak_titles_for_day[ $day ] ) . "\">$day</a>";
		else
			$calendar_output .= $day;
	$calendar_output .= '</td>';

For events, a single anchor wrapped around the calendar day number would not work because clicking on the day would just take you to the archive page for events posted on that day not the start date of actual event day. Therefore the code needs to be modified so that a user is able to click on each individual event title and have it link to the events page url instead of an archive page.

Instead of wrapping the day number in an anchor tag, I switched it to a span so it can be selected later by css/js. Also, I created a <div> containing event titles and links for all the events for a given day and inserted it after the calendar number </span>. In the CSS, I set the <div> to display:none, and position:absolute and positioned it relative to the <td> it resides in. Then using jQuery I set a hover on both the <span> and the <div> so when a user hovers over a day with posts, you can display the <div> with the event links and titles.

<?php

$calendar_output .= '<td>';
  		if ( in_array($day, $daywithpost) ){ 
					$title_div = "<div>";
					foreach($ak_titles_for_day[$day] as $ak_title){
						$title_div .= "<a href='".$ak_title['url']."' title='".$ak_title['title']."'>".$ak_title['title']."</a><br/>";
						}
					$title_div .= "</div>";
				$calendar_output .= "<span>$day</span>".$title_div;
			}
		else{
			$calendar_output .= $day;
		}
		$calendar_output .= '</td>';

JS and CSS

$('#wp-calendar span').hover( function () {
  		$(this).next().show();
		  },
		  function () {
			$(this).next().hide();
		  });
		
$('#wp-calenda div').hover( function () {
			$(this).show()
		  },
		  function () {
			$(this).hide();
		  });

PHP: Does "big-o" complexity really matter?

Last week, a client of ours as us to look at some code that was running particularly slowly. The code was powering an autocompleter that searched a list of high schools in the US and returned the schools that matched and an identifying code. We took a look at the code, and it turns out the original developers had implemented a naivete solution that was choking up since the list had gotten to ~45k elements and I imagine they had only tested with a dozen or so. During the process of implementing a slicker solution, we decided to benchmark a couple of different approaches to see how much the differences in “big-o” complexity really mattered.

The Problem

What we were looking at was the following:

  • There is a CSV file that looks something like:
ID, STATE, SCHOOL NAME
2,NMSC DEPT OF ED & SVCS,IL
3,MY SCHOOL IS NOT LISTED DOMEST,NY
4,MY SCHOOL IS NOT LISTED-INTRNT,NY
8,DISTRICT COUNCIL 37 AFSCME,NY
20,AMERICAN SAMOA CMTY COLLEGE,AS
81,LANDMARK COLLEGE,VT

With data for about 45k schools.

  • On the frontend, there was a vanilla jQuery UI autocompleter that passed a state as well as “school name part” to the backend to retrieve autocomplete results.
  • The endpoint basically takes the state and school part, parses the available data, and returns the results as a JSON array.
  • So as an example, the function accepts something like {state: “MA”, query: “New”} and returns:
[
  {name: "New School", code: 1234}.
  {name: "Newton South", code: 1234},
  {name: "Newtown High", code: 1234},
]

The Solutions

In the name of science, we put together a couple of solutions, benchmarked them by running them 1000 times and calculating the min/max/average times, and those values are graphed below. Each of the solutions is briefly described below along with how they’re referenced in the graph.

The initial solution that our client had been running read the entire CSV into a PHP array, then searched the PHP array for schools that matched the query. (readMemoryScan)

A slightly better approach is doing the search “in-place” without actually reading the entire file into memory. (unsortedTableScan)

But can we take advantage of how the data is structured? Turns out we can. Since we’re looking for schools in a specific state whose name’s start with a search string we can sort the file by STATE then SCHOOL NAME which will let us abort the search early. (sortedTableScan)

Since we’re always searching by STATE and SCHOOL NAME can we exploit this to cut down on the number of elements that need to be searched even further?

Turns out we can by transforming the CSV file into a PHP array indexed by state and then writing that out as a serialized PHP object. Another detail we can exploit is that the autocompleter has a minimum search length of 3 characters so we can actually build sub-arrays inside the list of schools keyed on the first 3 letters of their name (serializednFileScan).

So the data structure we’d end up creating looks something like:

{
...
  "MA": {
  ...
   "AME": [...list of schools in MA starting with AME...],
   "NEW": [...list of schools in MA starting with NEW...],
  ...
  },
  "NJ": {
  ...
   "AME": [...list of schools in NJ starting with AME...],
   "NEW": [...list of schools in NJ starting with NEW...],
  ...
  },
  "CA": {
  ...
   "AME": [...list of schools in CA starting with AME...],
   "NEW": [...list of schools in CAA starting with NEW...],
  ...
  },
...
}

The results

Running each function 1000 times, recording the elapsed time between results, and calculating the min / max / and average times we ended up with these numbers:

test_namemin (sec.)max (sec.)average (sec.)
readMemoryScan.662.690.673
unsortedTableScan.532.547.536
sortedTableScan.260.276.264
serializednFileScan.149.171.154

And then graphing the averages gets you a graphic that looks like:

The most interesting metric is how the different autocompleters actually “feel” when you use them. We setup a demo at http://symf.setfive.com/autocomplete_test/ Turns out, a few hundred milliseconds makes a huge difference

The conclusion

Looking at our numbers, even with relatively small data sets (<100k elements), the complexity of your algorithms matter. Even though the actual number differences are small, the responsiveness of the autocompleter between the three implementations varies dramatically.

Anyway, so long story short? Pay attention in algorithms class.

Symfony2: Logging users in programatically with Doctrine2 and FOSUserBundle

Recently I was doing a fairly common task on Symfony2, logging in a user programatically. Often applications do this on registration, via auto login links, complex login forms, etc. This time I was using an auto login link that expires that users get via email. I came across the issue that it seemed the first time the page loaded I was logged in properly but then as soon as I redirect or navigated anywhere I was logged out.

Here is the basic workflow we were using:

  1. Create an auto login link, basically just an entity which had an expiration date and a special url hash
  2. When clicked forward to action which gets the related entity, and then retrieves the user.
  3. Login the user programatically
  4. Redirect to whatever we want them to see

The issue was somewhere between step 3 and 4 something was amiss, if I eliminated step 3 the profiler toolbar showed I was properly logged in as expect, as soon as I redirect it showed me as unauthenticated. Here is the code for the most part:

<?php

    /**
     * @Route("/login-token/{token}/{userHash}", name="participant_autologin_token")
     * @ParamConverter("autologinToken", class="MyBundle:AutologinToken")
     * @Template()
     */
    public function autoLoginAction(Request $request, AutologinToken $autologinToken, $userHash){
        
        $csrf_token = $this->container->get('form.csrf_provider')->generateCsrfToken('authenticate');
        $now = new \DateTime();
        
        if( $autologinToken->getExpiresAt() != NULL && $autologinToken->getExpiresAt() < $now ){
            return array("csrf_token" => $csrf_token, "last_username" => "", "nextUrl" => $autologinToken->getUrl());
        }        
        
        if( $autologinToken->getUser()->getAutologinToken() != $userHash ){
            return array("csrf_token" => $csrf_token, "last_username" => "", "error" => "Sorry! There is a problem with your link. Please contact support.");    
        }
        
        $user = $autologinToken->getUser();

        $providerKey = $this->container->getParameter('fos_user.firewall_name');
        $token = new UsernamePasswordToken($user, null, $providerKey, $user->getRoles());
        $this->get('security.context')->setToken($token);
        
        $event = new InteractiveLoginEvent($request, $token);
        $this->get("event_dispatcher")->dispatch("security.authentication", $event);

        return $this->redirect( $autologinToken->getUrl() );     
    }

Fairly simple, used a ParamConverter to convert the incoming request to an entity. After a while of troubleshooting, I noticed that the ‘$user’ in this case wasn’t an actual Entity, it was a ProxyClass that Doctrine2 had generated. I had read that Doctrine2 ProxyClasses when serialized don’t properly bring over some of their attributes, namely the ID. This caused an issue with FOSUserBundle as the UserProvider looks up the user by their ID. Since the ID was blank this kept causing it to not find my user on the next page load.

There are a number of ways you can fix this, two that come to mind is to override the ‘refreshUser’ method of the UserProvider to look up by username as that is properly serialized from Proxy objects. Instead, as this was only for this one action and I wanted to be more efficient I switched the query to do a join to the user from the get go. This means when you do getUser Doctrine will return the actual Entity and not a Proxy class. Here is my update annotation:

<?php
    /**
     * @Route("/login-token/{token}/{userHash}", name="participant_autologin_token")
     * @ParamConverter("autologinToken", class="MyBundle:AutologinToken", options={"repository_method" = "findByTokenJoinWithUser"})
     * @Template()
     */
    public function autoLoginAction(Request $request, AutologinToken $autologinToken, $userHash){
    ....
    }

For more on how to use joins and entity repository specific consult the current manual.

Good luck!