Doctrine Archives - Page 2 of 3 - {5} Setfive

Earlier this week we were repeatedly getting notifications about a “usort() Array was modified by the user comparison function” warning for one of our new Symfony2 projects. The weird thing was the sort function was relatively straightward and looked something like:

Obviously not modifying the array. Anyway, Daum dug up this StackOvervlow thread which suggested that using introspection methods silently modify the array and trigger the warning but I’m clearly not using any of those either.

After some more poking around, we ran across a Doctrine bug specifically mentioning the usort issue. It turns out, because of how Doctrine’s lazy loading functionality works if the usort callback function causes Doctrine to lazy load it’ll silently modify the array and cause that warning. Great, so how do you fix it? It’s actually pretty straightforward, you just need to force the lazy loading before sorting the collection. I ended up with something like:

Anyway, fun fact of the day. Questions and comments always welcome.

Last week, one of our projects hit a pretty odd limit that I’d never expected to reach. The project is an analytics platform that allows admins to “pull” data from another, third party application. To accomplish this, the application allows admin users to dynamically add and remove columns from SQL tables and then dynamically chart these columns. Because of this, one of the tables had gotten over 350 columns which had all been created dynamically at runtime.

Anyway, things were working fine until last week when the application started throwing the following fatal error: “Fatal error: Uncaught exception ‘Doctrine_Table_Exception’ with message ‘Invalid expression found: ‘ in /usr/share/php/symfony/plugins/sfDoctrinePlugin/lib/vendor/doctrine/Doctrine/Table.php:2746″ Looking at the error, I noticed a warning was actually getting thrown right before the fatal error: “Warning: preg_replace(): Compilation failed: regular expression is too large at offset 32594 in /usr/share/php/symfony/plugins/sfDoctrinePlugin/lib/vendor/doctrine/Doctrine/Table.php on line 2745″ Looking through the code of Table.php, its clear that because “preg_replace” fails the $expression is subsequently blank which causes Doctrine to throw an error. I wanted to see how bad the regex was so I updated the Table.php to dump the expression. Here is what Doctrine was trying to run:

/(lsc_calculated_promoterscore_delta_innovativeness_formula|Lsc_calculated_promoterscore_delta_innovativeness_formula|lsc_calculated_promoterscore_delta_innovativeness_order|lsc_calculated_promoterscore_delta_favorability_formula|Lsc_calculated_promoterscore_delta_favorability_formula|Lsc_calculated_promoterscore_delta_innovativeness_order|Lsc_calculated_promoterscore_delta_favorability_order|

[Lots of columns...]

|Grid_mac_total|GridTotalTotal|GridTotalWinpc|Grid_total_mac|GridWindowsMac|grid_both_mac|grid_mac_ipad|audience_type|GridBothWinpc|GridBothTotal|Grid_both_mac|GridTotalIpad|Audience_type|WindowsosFy13|Grid_mac_ipad|grid_mac_mac|Grid_mac_mac|Program_type|program_type|AudienceType|GridMacTotal|GridBothIpad|GridTotalMac|GridMacWinpc|venue_child|ProgramType|GridMacIpad|Venue_child|GridBothMac|Program_id|program_id|VenueChild|GridMacMac|Fiscalyear|fiscalyear|is_locked|Is_locked|ProgramId|IsLocked|Country|country|venue|Venue|os|OS|id|Os|Id)(Or|And)?/

Looking at the php.net documentation for preg_replace and preg_match neither actually mention a hard limit on the size of a regex that can be compiled. Obviously there is a limit though and I imagine it must depend on the underlying RegExp library that your PHP is compiled against so it might be platform dependent.

As for solutions for this problem? The best solution for an extreme case like this is probably to just manually fill those in with real methods in the Doctrine_Table classes:

On one of our projects that we use multiple connections that are defined at run time we recently were generating reports that required us to change a specific connection multiple times in a single run. We noticed that even though we would define a new connection, it would not throw any errors but just continue to use the originally defined connection. Here is how we were doing the connections:

If you called the code above once, it would connect properly to the given DSN. However if you then called it a second time with a new DSN, it would not error and would simply just remain connected to the first DSN. After hunting around a bit it was the problem that Doctrine wasn’t assigning the new connection as the old connection was still open. To get around this we updated the code to the following:

You need to first check to see if the Doctrine Manager has the connection, as if you try to get a connection that doesn’t exist, it will throw an exception.

Hope this saves you some time!

We’ve been working on a project in which a bunch of the tables had composite primary keys. Often we wanted to embed these tables in other forms. ahDoctrineEasyEmbeddedRelationsPlugin is a great plugin for managing embedded forms with Doctrine and Symfony. It lets you easily let users add multiple new relations and delete previous ones. However, it didn’t really support composite keys. We also wanted to be able to expose the primary keys rather than unsetting them from the form(so that the relation is automatically declared). This was a problem as we needed to be able to select at least one of the primary keys. Here is an example where we wanted to let the user pick one of the primary keys:

User:
  columns:
     first_name:
        type: string(16)
Product:
   columns:
      name:
         type: string(32)
UserProduct:
   columns:
      user_id:
         type: integer
      product_id:
         type: integer
        price:
          type: decimal
    relations:
       User:
          local: user_id
          foreign: id
       Product:
         local: product_id
         foreign: id

In this case there are Products and Users. A Users can have a products and specify how much each costs. When you are adding products to a user, you would want to be able to select which product you are adding in the embedded form. This is where our problem was, with the plugin as of version 1.4.4 you couldn’t tell it not to unset the primary keys when it embedded the form. We did a checkout of the plugin from SVN and modified it a bit. It now has and additional parameter: newFormUnsetPrimaryKeys . This will make it so the plugin will not unset the primary keys on a new form if you set it to true (it defaults to false).

We also found that the plugin had hard coded a couple of places findOneById which required the primary key to be called `id`. We’ve updated this to use the Doctrine method getIdentifierColumnNames() to get the primary keys.

We only applied to be developers on this script, so are currently waiting on it to be packaged and released, however if you want our updated just do a svn checkout of the plugin and you will be all set!

Let us know if you have any questions on it!

In April I wrote about using Doctrine with multiple connections for specific table models. This worked really well, except Symfony’s web debug toolbar would not show any SQL queries that were not defined in the databases.yml file. This of course made it quite difficult to debug many queries, as the query logs show the queries before the parameters are inserted, for example:

SELECT a.id, a.title FROM posts a WHERE id = ?

The web debug toolbar however shows them with the parameters in place. This makes it a ton quicker to debug as you can see the parameters in place, as well as copy and paste the query straight into the SQL client to see the raw results. After a few months and a project becoming much more complex it was necessary to see the queries. I looked up how sfWebDebugPanelDoctrine gets the queries and found:

So the sfDatabaseManager would manage all the connections and return which ones to pull queries off of. I looked at it a bit and saw that it has setDatabase which sets the databases it has registered. Since it requires you give it a sfDatabase as a parameter I had to update the way we connect the databases a bit so we could pass them to it. The new version is below:

Now you will now see the queries in the web debug toolbar.

Category: Doctrine

Symfony2: usort() Array was modified by the user comparison function

Doctrine 1.2: To many columns causes findBy* to fail

Changing a Doctrine connection with the same name in single instance

ahDoctrineEasyEmbeddedRelationsPlugin and Composite Primary Keys