Built from source.

Zend Framework has been attracting more and more attention from the PHP community lately, and while it lacks certain things (like code generation) that other frameworks (like Rails) have implemented to great effect, Zend Framework 2.0 is slowly taking shape and it looks like it will be the framework of choice for startups and enterprises alike. (Yes, it will even have code generation.)

But despite having several “enterprise-ready” components, I’ve found that one in particular is not: Zend_Search_Lucene, Zend Framework’s native PHP implementation of Apache Lucene, written in Java.

Don’t get me wrong; Zend_Search_Lucene is great for a small site or blog. However, from extensive personal experience, it is not appropriate for a site with a medium or large index. I think this should be noted upfront in the documentation.

Against my better judgment, the company I work for migrated our previous search solution to Zend_Search_Lucene. On pretty heavy-duty hardware, indexing a million documents took several hours, and searches were relatively slow. The indexing process consumed vast amounts of memory, and the indexes frequently became corrupted (using 1.5.2). A single wild card search literally brought the web server to its knees, so we disabled that feature. Memory usage was very high for searches, and as a result requests per second necessarily declined heavily as we had to reduce the number of Apache child processes.

We have since moved to Solr (a Lucene-based Java search server) and the difference is dramatic. Indexing now takes around 10 minutes and searches are lightning fast. What a difference a language makes.

Not one month after I wrote about the future of PHP (June 2008), I was quite happily proven wrong.

For my part, I’d like to see first-class functions and closures included in the language. [...]

But none of that will happen, because PHP is a language in decline. Not a decline in usage—it will only continue to expand its reach—but in the addition of innovative features from other languages. There will be no need to evolve; most of the agitators for change will have moved on.
—Me

It’s always been in my nature to own up to it when I’m proven wrong, so consider this my mea culpa. PHP has always been a klugey language, borrowing from other languages and implementing those ideas in somewhat endearingly clunky ways. I naively believed that that dynamism was coming to an end, but as you can see from the link above, that’s demonstrably not the case. Despite that, I still think there is a slow but steady “brain drain” from the ranks of the top tier of PHP developers—I’ve seen it first hand in the last few years and the overall trend should make PHP developers at least a little uncomfortable.

Anyway, the closure implementation coming in PHP 5.3 is, like namespaces, a little clunky.

function replace_in_array($search, $replacement, $array) {
    $map = function ($text) use ($search, $replacement) {
        if (strpos($text, $search) > 50) {
            return str_replace($search, $replacement, $text);
        }
        return $text;
    };
    return array_map ($map, $array);
}

Yeah, you have to manually link the variables to make them available to the closure. Not ideal, but neither is having to manually specify your scope in JavaScript (via Function.apply()).

There are a few of other differences in PHP 5.3’s implementation of closures and other languages:

• First, like other functions, they have access to the global scope with the global keyword. Do yourself a great big favor and just avoid doing that.
• Second, you can choose which variables are linked by reference and which are not.
• Finally, you can declare an anonymous function static if it’s declared in a class but doesn’t use an instance of that class for anything. If you have a large object, this would prevent the closure from retaining a reference to that instance (and therefore, its memory footprint) after it has outlived its usefulness. This will probably be the least understood aspect of PHP closures for most developers.

Currying is now possible as well. Ryan Timmons wasted no time in writing a method for doing just that:

function curry($function, $argument) {
    return function() use ($function, $argument) {
        $arguments = func_get_args();
        array_unshift($arguments, $argument);
        return call_user_func_array($function, $arguments);
    };
}

Between this, namespaces, late static binding, and a bundled packaging method (ext/phar), the next version of PHP is looking more like a major release instead of a minor one.

PHP 5.3 is scheduled for final release in October.

On and off for the last two or three months, Jurriën Stutterheim and I wrote Zend_Paginator, the pagination component for Zend Framework. Yesterday it was officially released as part of Zend Framework 1.6.

This was our first contribution to the framework, and was very much a collaborative working relationship. I first created a proposal in about half an hour on Christmas Eve 2007 for how I thought a pagination component should work after being dissatisfied with existing solutions. It contained some good ideas about flexibility, but on its own it was half an idea, at best. So I promptly forgot about it.

Then Jurriën came along and created a proposal based on his Zym_Paginate component, which was part of Zym Framework, itself an extension of Zend Framework.

At the urging of Zend, we merged our proposals. We refined our ideas, incorporating aspects from each of our proposals in addition to brand new thoughts. Then I wrote a new component from scratch based in part on Jurriën’s existing work. He wrote the excellent unit tests (with 100% code coverage, not always easy to achieve!), and I wrote the DocBook documentation. I also can’t overstate the positive impact the community had, particularly from Bryce Lohr who prompted us to decouple and reorganize major portions of the component.

Anyway, I’m very happy with how it turned out. There are still a few issues, most of them minor—and because of the component’s flexible design, there are almost always workarounds. They should be fixed in time for 1.6.1.

Here’s some praise for the component:

Upcoming Zend_Paginator in 1.6 is absolutely brilliant. Wasted time be gone.
—Joakim Nygård

The new Zend Framework 1.6 release candidate includes a Zend_Paginator class, which is an excellent thing to have around because I know I’ve re-invented that wheel on every site I’ve developed. My only criticism of the new Zend_Paginator is it offers a daunting amount of possibilities.
—Chris Beer

This really is a beautifully crafted component, very elegant, very classy.
—David Mintz, via e-mail

I’ve seen dozens of different paginators in the last couple of years, and to be honest, this one is by far the best. You got everything right! I can’t wait for this component to hit the incubator.

It’s great, well done guys.
—Federico Cargnelutti

Jurriën Stutterheim posted an interesting article on the future of PHP recently (“PHP… what to say?”). In it he argues that PHP shouldn’t try to maintain complete backwards compatibility in the next release, saying,

“PHP 6 will make or break PHP as a language. For PHP to make it, it needs a clear vision of what it wants to be. Trying to maintain compatibility with ancient and mostly poorly written scripts can’t be part of this vision.”

The language can’t advance, he says, if it’s chained to outdated yet popular open-source projects.

But here’s the problem. If you’re intent on being the lingua franca of the Web, backwards compatibility is critical. It’s part of that compatibility-stability-security triumvirate that users love.

You see this conflict elsewhere in the tech industry. Microsoft’s greatest strength and biggest weakness is backwards compatibility. Users love it because decade-old programs still run in Windows. If you’re, say, a medical transcription company that uses an application written in the late ’90s, your company is going to stay with Windows indefinitely, because no external factor forces you to update that application. And Microsoft isn’t going to do anything to jeopardize that.

This is the same wall that PHP is headed toward. I view it as a wall. Others might not; it’s certainly not a bad place to be at the moment. On the plus side, it means PHP developers have more opportunities afforded to them because of the popularity of the language. The downside is that they can look left and right and see niche languages innovating in compelling ways. The more “cosmopolitan” PHP programmers inevitably develop language envy and move on, but these are exactly the kind of developers PHP should fight to retain.

It doesn’t have to be this way. PHP could flourish as a smaller language. This is the tack that Apple has taken, and it seems to work well for the company and its users. Apple is one of the most innovative companies in the entire tech industry owing in no small part to this strategy.

For my part, I’d like to see first-class functions and closures included in the language. An object-oriented API ($file->read(1024)) alongside the procedural functions (fread($resource, 1024)) would also be welcome.

But none of that will happen, because PHP is a language in decline. Not a decline in usage—it will only continue to expand its reach—but in the addition of innovative features from other languages. There will be no need to evolve; most of the agitators for change will have moved on.

There is one hope for PHP, however: Zend Framework. I think if it can gain a foothold among developers, some of these trends can be reversed. Hopefully (for PHP) everyone hasn’t moved on to Ruby by then.

A coworker recently asked me what the difference was, functionally, between PHP’s create_function() function and traditional closures that you might find in languages with first-class functions, like Ruby or JavaScript. You can pretty easily illustrate this with a couple of examples.

First, a bit about closures. The idea with closures is that you can cleanly and readably pass around a bit of logic as an object, and any references that that object makes to variables in the surrounding scope must persist until that object is done with them.

So here’s an example in JavaScript:

<script type="text/javascript">
function getGreeter(name)
{
    return function(salutation) {
        alert(salutation + ', ' + name);
    }
}

var greeter = getGreeter('Eddy');
greeter('Hello');   // Hello, Eddy
greeter('Howdy');   // Howdy, Eddy
greeter('Bonjour'); // Bonjour, Eddy
</script>

Here’s the closest equivalent in PHP:

<?php
$code = '$name, $salutation', 'print $salutation . ', ' . $name;';
$greeter = create_function($code);
$greeter('Eddy', 'Hello');
// etc.

And that’s a callback, not a closure. In JavaScript the garbage collector reclaims the memory used by the anonymous “greeter” function… but in PHP functions get declared and stay declared, so every time you call create_function(), you increase the memory usage.

It gets worse. This is basically what PHP does internally:

<?php
function create_function($args, $code)
{
    // create a random $functionName
    eval('function ' . $functionName . '($args){$code}');
    return $functionName;
}

Yeah, the entire thing is evaluated. So not only does it not get garbage collected, but it has all the problems of eval()—it’s slow, difficult to debug, and uncacheable by bytecode caches like APC. Problems that closures don’t have in other languages.

It’s why you can do something like this…

<?php
$code = 'print "I print repeatedly.\n"; } print "I print once.\n"; if (false) {';
$function = create_function('', $code);
call_user_func($function);
call_user_func($function);
call_user_func($function);

// I print once.
// I print repeatedly.
// I print repeatedly.
// I print repeatedly.

…and why you should never use create_function().