Category: Propel

Adding ORDER BY FIELD to Propel Criterias

By Ashish Datta | 11 Comments on Adding ORDER BY FIELD to Propel Criterias

Every now and then, we use Sphinx to provide full text searching in MySQL InnoDB tables. Sphinx is pretty solid. It’s easy to set up, pretty fast, and easy to deploy.

My one big issue with Sphinx has always been making it play nice with Symfony, specifically Propel. The way Sphinx returns a result set is as an ordered list of [id, weight] for each document it matched. As outlined here the idea is to then hit your MySQL server to return the actual documents and use “ORDER BY FIELD(id, [id list])” to keep them in the right order that you received the list.

The problem is, Propel Criteria objects provide no mechanism to set an ORDER BY FIELD. This is an issue because if you drop Criterias you loose Propel Pagers which generally adds to a lot of duplicated code and is honestly just not very elegant.

Anyway, after some thought I came up with this solution.

If you read through the definition of “Criteria::addDescendingOrderByColumn()”:

All it really does is add the second part of the ORDER BY clause to an array which then gets joined up to build the final SQL. Because of this, you can actually just add an element onto the orderByColumns array which will cause Propel to execute an ORDER BY FIELD SQL statement.

To make the magic happen, I sub-classed Criteria and then added a addOrderByField() function to let me add a field to order by as well as a list to order by.

8/8/12: Update per Simon’s comment below

Also add this function to make sure your ORDER BY FIELD columns get cleared:

To use it, do something like this:

And thats about it. Since sfCriteria is a sub-class of Criteria the code works seamlessly with existing PropelPagers and anything else that expects a Propel Criteria.

FOSS Fridays: MacGyvered Key/Value in Symfony

By Ashish Datta | 3 Comments on FOSS Fridays: MacGyvered Key/Value in Symfony

On a project we’re currently working on, we arrived at a situation where our client had a loose and very fluid idea of the information he wanted to store about certain objects in his application. We didn’t specifically know the number of fields or the format of the data. Continually modifying the schema would of been painful so I wanted to try something different.

Since the data is more or less non-relational (it only relates to the object that owns it), what I really wanted was an ad-hoc key/value store. But I didn’t want to break Propel’s ORM abstractions. I still wanted to be able to do:

$company->getMission();

With the new system.

Turns out you basically can. Here’s how it works:

  1. Add a “dynamic_field” table to your schema. (definition is below)
  2. Override the __call(), hydrate(), and save() functions in Propel model file that you want to MacGyver.
  3. Pray.

Definition of the dynamic_field table:

So the idea is we want to basically build a Propel Behaviour to capture any undefined get/set calls and “get” the data out of the dynamic_field table or “set” the data by storing the value into the table. Since the table stores the model class and model id, the “keys” only have to be unique by model (just like Propel normally works).

Here is the code you need to add to the model file:

The code captures any undefined get/set calls and then deals with them appropriately. It won’t serialize the fields until the save() call (just like regular Propel objects). I also overloaded the hydrate() function so that the object will fetch all of its dynamic fields in one shot, as opposed one query per get.

Using the modified objects is exactly like regular Propel objects, the changes are entirely transparent except that you can get/set anything you want.

For example:

Will work even though there is no “vision” column on the company table. Magic.

There is one big problem with this trick though. Because of the Propel class hierarchy, there isn’t any way to introduce this code in one file and have other objects inherit the changes. You have to manually copy it to any model file that you want to enable it for.

sfPropelPager and GROUP BY criteria

By Ashish Datta | 2 Comments on sfPropelPager and GROUP BY criteria

So for one reason or another (actually a few bad ones) I ended up having to use a Criteria object looking like this:

It makes SQL that looks something like this: 

“SELECT tag_id, tag_model, id, COUNT(*) AS the_count FROM sf_tagging WHERE tag_id IN (1,2,3) GROUP BY taggable_model, taggable_id HAVING the_count > int”

The query sucks but whatever it works.

My issue came when I tried to use it with a sfPropelPager. I set up the pager per usual but for some reason the results that were coming back weren’t correct. For some reason, the COUNT being returned by the sfPropelPager was completely wrong. It turns out the offending lines are here in sfPropelPager.class.php :

For whatever reason, sfPropelPager clears the GROUP BY clauses before it calculates the COUNT for a criteria object. I’m not sure why it does this – but it certainly is unexpected and breaks my query in particular.

There are a handful of posts about this on the Symfony forums and it looks like the Propel people know about the issue to.

The solution to this is to use the setPeerCountMethod() from sfPropelPager. The setPeerCountMethod() function allows you to specify a custom COUNT() method inside the peer for your Criteria. I went ahead and added a new function to put the GROUP BY columns back in:

This solution works but it is extremely rigid. Since the custom count function has to be static you’d really be out of luck if you had variable columns or other dynamic requirements.

I’d love to know if someone has a cleaner/better/more elegant solution for this.

Propel and Primary keys

By Matt Daum | Leave a Comment on Propel and Primary keys

Today I was writing an administrator backend for a project we have. I had the code:

$old_feeds=RssFeedRelationPeer::doSelect($c);
if(count($old_feeds)>0){
   foreach ($old_feeds as $old) $old->delete();
}

I was trying to remove some old foreign constraints before I deleted the main object, however, for the longest time the “$old” objects were not deleting, but no errors were being thrown. I did the usual debugging, added a die() statement inside the count, it was going there. I added a die statement in the foreach, it went there. I thought, “It must not be pulling the right ‘old’ objects.” I then added a $old->getName(); to see exactly what objects it was going through-they were the right ones. It made no sense. I next tired after the $old->delete();to add a $old->save();. The error I received was “You cannot save an object that has been deleted.” This didn’t make sense, since the object was still in the database. It hit me then, I had this problem in the past. Propel hates tables without primary keys. I quickly just added a simple primary key to the model, and the delete statements worked.

Moral: If dealing with Propel and you are getting some unexpected behavior, with zero errors, check to see if you have a primary key; it may save you hours of head banging.