In testing iPhoto ’09 for my full review, I plowed through more than 30,000 photos using over 40 identified faces, mostly human. Here’s how iPhoto’s face detection and recognition works—and doesn’t work:
Face Detection
Apple says it uses facial detection to determine the existence of faces, and then facial recognition to separate one person from the other. The problem is, that first step is far from a catch-all:
• It suffers from the typical face-detection problem of an incomplete picture—it won’t pick up all faces turned to the side, revealing just one eye. But far worse than that, it has a very hard time picking up faces tilted to either side, even if they’re otherwise perfectly clear and symmetrical. In other words, if the year is 2029 and Apple’s deathbots are coming for you, cock your head to the side, and they’ll just truck on by.
• The other problem iPhoto’s face detection suffers from is overconfidence. Sure, it makes sense that it picks up the faces of Mount Rushmore (well, at least Lincoln’s) but it’ll invent eyes and a nose from any old rumpled curtain or wood grain when it wants to.
Face Recognition
Once you get past the detection, the recognition kicks on. Some people are already saying it’s crap, but it’s remarkable when it wants to be. How do I know it’s working? Because on many occasions I have seen it correctly identify faces in frames in the background of a shot. But recognition has some hang-ups of its own:
• Once you name a person and click on their face on the corkboard, iPhoto suggests lookalikes, which you approve or reject by clicking. The first round of suggestions are mainly nonsense—iPhoto needs a lot of data to work correctly. Confirm the identity of your subject 10 or 20 times then click Done so the system can recalibrate.
• In the first round or two, recognition errs on the side of inclusion: A bald guy with a beard and glasses won’t just bring up similarly described gents, but will attract everybody who is bald, everybody who wears glasses and everybody with a beard. In my experiences, the images with the closest resemblance appear first, but as you scroll down, there are more and more random guesses. When it comes to babies, good luck—in those early rounds, iPhoto thinks all babies look alike. Again, you approve the suggested photos that are of the same person, and reject the ones that are not.
• Once you’ve done a round or two greenlighting more accurate shots of the person, it’s important to reject ones that are not. If you leave them there, iPhoto will keep on suggesting them. I found that, if two people look kinda alike, it pays to identify them both, and go back and forth between them confirming more and more shots, so iPhoto learns faster who belongs where.
• Baldness, hair color and facial hair are all strong indicators for iPhoto. If your friend has a goatee or some kind of fancy moustache, don’t ever let him shave it off. Likewise, if your mom switches hairstylists and starts getting a different dye job, she may as well don full hunter’s camouflage. For some reason, iPhoto had an easier time discerning the blondes than the brunettes.
• People wearing glasses cause iPhoto to suggest matches of other people wearing glasses, in some cases people who are otherwise comically different. But iPhoto seems to have a lot of trouble with glasses in general, and can’t always grasp the glasses-wearer well enough to confidently suggest more of that same person. Sunglasses are obviously a problem for recognition, but people wearing sunglasses are often suggested for anyone wearing glasses, as if it was all the same to iPhoto.
• There are two kinds of recognition dead-ends you can encounter, where iPhoto won’t suggest any more photos for a person, even though you know they’re out there, and where iPhoto suggests an endless supply of random faces as potential matches, clearly unable to narrow it down further. Both are infuriating, and require you to go out and identify photos manually in hopes of jogging its memory.
• Pets are not guaranteed to work with recognition. I’m not going to slander the good people of MacLife by calling BS, but seriously, I can’t get neither cat nor dog to be recognized in any way by iPhoto, and I don’t believe it’s possible. You can identify them yourself, of course, but the reason it doesn’t pull up suggested shots containing the same furry animal is that it’s not looking at furry animals.
The Faces system is technically a time saver even when the recognition is not up to snuff, because by batching the more-or-less appropriate pictures together, you can tag them a lot faster than you otherwise could. There are plenty of user interface problems that I will address in the actual review, but in the meantime, I will leave you with this: After heavy testing for half a day, iPhoto became shockingly good at identifying my face. I can only imagine that, given more calibration and identified content, it will be better and better. My biggest fear is how many photos aren’t clearing the first hurdle—face detection—and are therefore left completely out of the system. Who would have thought that the recognition would be easier to nail than the detection? Not me. [iPhoto ’09]
