What Symfonic Drupal means

Submitted by Larry on 24 October 2011 - 10:49pm

Earlier today, Dries committed a patch that adds two Symfony2 Components to Drupal: ClassLoader and HttpFoundation.

On its face it's a fairly simple patch; the new code in it is maybe a dozen lines. But it's an important part of a larger shift within Drupal to better embrace the modern web, on the server as well as the client.

The new face of PHP

The PHP development community has long been highly fractured. Everyone had their own CMS, their own framework, and their own applications. Even when developers started rallying around a few market leaders (Drupal included), there was no real interoperability. Attempts at a common framework either suffered from the standards problem, morphed into all-or-northing frameworks, or ran into naming collisions with everyone else, making them almost useless.

That has changed in recent years thanks to three key developments.

  1. First, Object-Oriented architecture has firmly taken hold in the PHP world. While not the solution to every problem, OOP does support library mix-and-match design much more easily than procedural code.
  2. Second, PHP 5.3 introduced code-level namespaces, allowing different libraries to co-exist without clobbering each other on a class or function name.
  3. Third, some of the leading PHP projects collectively agreed on how to leverage namespaces. That convention, dubbed PSR-0 (for PHP Standard Reference #1, because computer people always start counting at 0), specifies a common pattern for naming and organizing namespaced classes. The result is that it's now possible to leverage PHP's autoload capabilities across a range of projects in a consistent and predictable manner.

And all of those changes passed Drupal by. Drupal 7 shipped as a mostly procedural, non-namespaced framework with a somewhat fragile custom autoload system, the Registry. And, as a result, very little ability to benefit from the changes happening in the wider PHP community.

That's actually rather sad for me personally, given that Drupal was a leading project in the last great PHP collaborative effort, GoPHP5. Sadly we never kept that momentum going.

Pushing Drupal forward

The Web Services and Context Core Initiative for Drupal 8 is working on major changes to Drupal, refactoring it from a page-oriented CMS to a request/response-oriented REST server. That's no small task, and one that could benefit from a leg up from the wider open source community.

For that reason, several people began recommending that Drupal start to embrace the changes happening in the wider development community, first with a PHP 5.3 base and now with PSR-0 classes. By adopting the existing standard for classes, making our autoloading simpler and less error prone, and open the door to considerable performance improvements. By converting our core code over to easily-lazy-loaded classes, we can greatly reduce Drupal's memory footprint by not bootstrapping code we don't need. That makes response time faster, which is critical for any form of web service.

Much to my surprise, there was very little resistance.

That also means that we could now leverage existing code like HttpFoundation that we don't need to write ourselves, saving time and effort. That too met with little resistance.

Drupal is back in the PHP game.

Why Symfony?

The PHP library world has been changing rapidly in recent years. Once upon a time, there was only one really popular component library, PEAR, which had quasi-official status but was still a pain to install on many shared hosts. More recently, a number of popular frameworks have grown up, including Zend Framework, CodeIgnighter, and others, specifically casting off the PHP 4 legacy of earlier years. The Symfony framework, a Rails-y pseudo-MVC application framework, decided to go that route as well and completely reinvented itself for version 2. That included spinning off many low-level systems as free-standing components.

Zend and Symfony were both parties to the original PSR-0 standard, making it possible for (properly-written) code to be mixed and matched between the two.

When Drupal was looking for a 3rd party HTTP library to adopt rather than writing our own, we looked at two that seemed to have the best market share behind them: Zend 2 and Symfony2. We knew that whichever one we adopted we stood a very good chance of adopting more components from, so it was perhaps a more important decision than those involved realized at the time. After a comparison, we settled on Symfony for a number of reasons.

  1. Symfony's component libraries, or at least HttpFoundation, appeared to be richer and more self-contained, allowing us to pick and choose what we use.
  2. Zend Framework has a Contributor License Agreement, which Symfony does not. While I fully understand the reasons why Zend has a CLA, and I've even toyed with the idea of one for Drupal in the past, it raises the barrier to entry for Drupal developers to contribute back upstream. We wanted to ensure that we didn't just grab-and-go, but formed a partnership with whatever project we decided to leverage.
  3. Speaking of partnerships, the quick feedback and cooperation from Symfony's project lead, Fabien Potencier, and Lukas Smith carried a great deal of weight.

Even since then, Fabien and Lukas have made themselves available for help, insight, and other advise. Drupal folks have even started to submit code back to Symfony, too, for minor cleanup as well as feature fixes. That's exactly what I was hoping to see happen.

Next steps

Now that we have the ClassLoader and HttpFoundation in core, there's a lot of work yet to do.

  • The HTTP handler for the Context API is not quite finished, but hopefully will be soon. That lets us leverage the HttpFoundation in a consistent, clean, and powerful way to help control how Drupal responds to page request.
  • HttpFoundation includes a session-handling system, but leveraging it requires better decoupling our user and session systems. Which is something we should be doing anyway.
  • There's an open discussion of where else we could leverage Symfony2 components. There are actually a number of viable possibilities.
  • Now that we have adopted PSR-0 for core, we need to start refactoring the /includes directory to leverage it. That means cleaning up and converting core subsystems to object-oriented designs that we can then lazy-load without effort. That should help performance and code clarity.
  • So far, we're just using PSR-0 for core classes. What about modules? That's actually a much harder question, but it's one we will need to figure out sooner rather than later.
  • And oh yeah, we should update our coding standards documentation. That's on my list.

We have a lot of work ahead of us, but this is the beginning of Drupal no longer being an isolated island in the PHP world. If you want to help Drupal remain the dominant PHP CMS, let's roll up our sleeves and get to work.

I'll see you in the issue queues.

I was watching the issues around Symfony components every week and I'm really excited about this step. We need more OOP in core and pluggable core architecture if we want Drupal to stay innovative and competitive.

I hope these changes are first step forward to make Drupal better platform.

Happy to see that Drupal star is raising again!

willvincent (not verified)

25 October 2011 - 10:15am

While this is probably all for the best. I'm not terribly excited at the prospect of morphing core into something vastly different.

How is this going to affect module development? How much of that will change?

I've spent several years working with Drupal, specifically working on custom & contributed module development, and this kind of talk scares me that a lot of the learning I've done will have been wasted as it seems likely that a LOT is going to change.

With every release of Drupal there are a great number of things to 'relearn' but those are typically fairly small changes to syntax, or a different function name or set of functions. A good example would be hook_node_load, _view, _save, etc instead of all of that going through hook_nodeapi.

That change was logical and easy to understand.

The new DB API, while much more flexible is a lot more obtuse to work with when you're used to writing SQL queries. Having to wrap my head around the OO database methods has been a significant challenge, and I still have to refer to documentation for nearly every db interaction I do.

If more (most?) of core becomes more complex to understand I don't know how much longer I'll stick with Drupal.. And that thought really just makes me sad.

This is a terrible argument for not pushing forward, if you ask me. Drupal is lagging behind and if you want to catch up there's no compromise to severely breaking and rebuilding the platform properly. Then it might stand a chance five, ten years from now.

Don't get me wrong but if you want easy-to-use at the cost of half-assed-software-architecture there's an even more popular PHP system for that.

I agree that the changes coming down the pipeline are major. However, they are both nothing new, and necessary.

Complexity is in the eye of the beholder. I find DBTNG far easier to grok than, say, FAPI or the Render API. FAPI was completely alien when it was introduced in Drupal 4.7, and required rewriting, well, almost every line in Drupal. It was a far more disruptive change than DBTNG was, for instance, and it wasn't all smiles and roses. However, it did make Drupal far more flexible, secure, and powerful, and is one of the watershed points in Drupal's history.

Most of the changes under discussion are part of the WSCCI initiative anyway. They're not Symfony-specific. Adopting some Symfony components along the way is simply a way to get the job done faster, and with less effort.

That leg up is important, because Drupal is going to fall behind in the new, fourth age of the web unless we leapfrog past our current limitations. That's what the Web Services initiative is all about. Converting Drupal from the best of the last-generation of CMS (page centric) to the best of the next-generation of CMS (request-centric). That does mean fundamental changes, yes, but they're necessary and unavoidable if we want Drupal to continue to succeed.

Naturally we want to keep complexity down along the way, but that's as I said a subjective question. I find OOP code easier to follow than procedural as often as not.

I don't want to sound extreme, but right now I don't know how to clearly express my concerns without putting it into an extreme —

While you're obviously right with "Change happens", I am heavily concerned about the amount and extent of changes attempted in the WSCCI initiative, with regard to the amount and diversity of people working on it.

Currently, I've the impression that a couple of like-minded people are working on WSCCI, and there's almost no one who disagrees and challenges that group.

I'm following all issues in the sandbox myself, but have to admit that most of what is being discussed is somewhat beyond my horizon.

Major decisions are being made. They'll have a critical impact on Drupal's future. Loosely following the issues, it seems like there have been a few concerns that got quickly shot down with "We already decided on that." I'm concerned whether that is a healthy environment.

I can only applaud @chx for his occasional "spikes" in the queue, throwing in a different perspective, although some or most of them may not be very clearly formulated. Anyway, I think we badly need more of these.

Lastly, I've the impression that WSCCI has way too many objectives. Unlike the other initiatives, this one seems to attempt to "Change Everything That Comes Along Our Way" in a kinda obfuscated way. I mean, introducing context and plugins/classes are two things, but replacing various plugins/subsystems with entirely new things "while being there" is completely unacceptable, if you'd ask me.

With the current scope of things being done, the initiative should have been called "Rewrite Drupal core", which may sound like fun to you, but definitely not for others/outsiders. As such, all issues, code, discussions, patches, and commits should have happened in the Drupal core queue and repository. I am, however, concerned and criticizing the scope itself though.

I'm aware that this comment isn't really helpful or productive. However, in talking to various people at random occasions, I've seen an increasing need to formulate these concerns in a (hopefully) clear way.

All of that being said — huge thanks for your hard work on Drupal, Larry!

Thanks,
sun

I have stated this in the issue queue:

1.eaton points out the scatteredness of context.
2. we decided to solve the problem. It's a solvable problem. It'd make Drupal much more awesome and simpler. Rare treat, simplicity in Drupal.
3. Suddenly instead of focusing on the context problem we have a full blown rewrite of Drupal at hand.

Ever since that happened, every single talk, Butler or WSCCI had a complete vision complete focusing on a complete Drupal rewrite (see http://london2011.drupal.org/sites/default/files/WSCCI%20Update.pdf title, introduction, first meaningful slides already talk about layout) instead on focusing on a great context API. We still don't have one.

However, this seems to be Dries endorsed and as a consequence I have switched my involvement to damage control only (I am checking every RTBC patch for example in the core) and I do not plan to contribute much to Drupal 8 if this is the direction we want to take. I have no other way to protest.

bmcguire (not verified)

27 October 2011 - 11:21am

Larry this is great news.

I have been working in both the Java world and PHP/Drupal world and have wondered why in Drupal we didn't make use of such libraries. In Java we leverage Spring which is becoming more and more prominent in most all of the java application being created today.

Symphony2 seems to be a very similar concept and is a great addition to the Drupal platform. We have leveraged it in some of our custom modules and it has been very helpful for creating an API and allow use to swap out different implementations depending on the requirements.