Face recognition has been one of the most requested digiKam features, and the latest version of the photo management application provides this functionality.
As the name suggests, the face recognition functionality can be used to find photos containing faces and attach face tags to persons in photos. This lets you quickly locate all photos of a specific person using digiKam filtering capabilities.
Tagging faces in digiKam is a rather straightforward procedure. Open the photo you want in the preview pane, press the Add a Face Tag button, draw a rectangle around a face on the photo, enter the face tag (e.g., the person’s name), and press Confirm.
Tagging faces manually can be a daunting proposition, especially if you have a considerable number of photos of people. Fortunately, digiKam can do the donkey job of automatically identifying faces for you. Expand the People sidebar, and press the Scan collections for faces button. In the Scanning Faces window tick the Detect and recognize faces check box. By default, digiKam scans all collections and tags, but you can limit the scan operation to certain albums and tags. To do this, press the Options button and select the albums and tags you want from the Search in drop-down list in the Albums section. While at it, you can tweak the face detection parameters in the Parameters section. Press then the Scan button and let digiKam do its job. Once the scan is completed, you should see all photos containing faces. You can then go through the scanned photos to fix face tags and remove incorrectly identified images.
Tech writer covering Linux and open source software
Scribbles and Snaps 
One of the best features that picasa has is the auto-learning and recognizing the already identified faces.
Does Digikam has the same feature?
Well, it works somewhat but the detection rate is very low. Even after training several thousand photos only a few are correctly identified, almost all are wrong. Manually tagging turns out to be considerably faster and much more useful (and accurate). So right now it’s no big help.
I hop those bugs are squished before 2.0final.
Prevention of mis-detecting non-faces as faces is quite good, though.
tagging faces manually takes a long time…face recognition and tagging means it should automatically detect like faces….
One thing I haven’t heard addressed from Picassa to Digikam to every other facial recognition implementation I’ve ever read about – how does it deal with aging? My photo collection spans about 20 years. Would the software get confused when I tag both the 10 year old me and the 28 year old me as the same person?
I’ve found the easiest way to do that is to create essentially two people for yourself: young you, and now you.
@Eric Mesa: it depends on the face recognition algorithm used, which is not precised in the article….
Is it Viola-Jones inspired or something else?
Digikam rely on OpenCV for this and their haarcascades data. More technical detail from opencv on general implementation of face detection in applications can be found here:
http://opencv.willowgarage.com/wiki/FaceDetection
Ok, then it’s Viola-Jones. Thanks!
So, what we’ll have is a system that is very good at pointing out to us that there are faces in the photograph that we are looking at, but if we let it tag those faces it will identify nearly all of those faces incorrectly and we’ll have to start over again and do it manually. Is that about right?
Commenters are wondering how well it will handle ageing. The answer is that it will handle ageing just as well as any other recognition challenge, it will fail quite miserably.
I’m sorry if I sound like I’m being negative. I’m not trying to be. I’m just trying to point out that what is a fine and laudable achievement for the developers will, at this early stage, just be a bitter disappointment to many expectant users. Digikam 2.0 will probably be better off without a very ineffective implementation of many users’ most wished-for feature. In its current immature state, it will only drag down the perceived quality of the whole product for little real gain. Wait until it is useful and then wow us with it. Don’t rush it out and lose face. (sorry!)
Come on Bob, the world isn’t black and white. There are far better options than to never release features until they are 100% perfect.
For example, in this case I think it would be great if digiKam could integrate some kind of automatic user feedback system to allow the developers to learn more about the particular situations in which the algorithms work well or not at all.
Of course the world is not, as you write, black and white; I did not claim that that face recognition had to be 100% perfect before it could be released. However, I think this comment by “nn”, “Well, it works somewhat but the detection rate is very low,” suggests that we are dealing with a rather dark shade of grey. “nn” did not give us a percentage, but I would take “very low” to mean less than 10% (please correct me, “nn”). Is 10% good enough? 90% false positives? I don’t think so. I think it would need to be well over 50% to avoid being anything other than an annoyance if using it for tagging, though the threshold could be lower if it were used for searching. At present, face recognition does not sound like it is ready for the big time and will disappoint far more than it will delight.
I’m no expert on face recognition, but I have read comments from the developers on the matter. They did not seem confident that face recognition would ever be very effective in Digikam. It is a very hard problem to solve. On the other hand, face detection sounds like it is working well, so perhaps it should be put to other uses, such as better automatic red-eye reduction, or easier controls for improving skin tone, or selective sharpening, or other such features. It would be a shame if all the good work on the face detection side–which I do appreciate–were wasted by only applying it to ineffective face recognition and tagging.
Ok, I did a kind of “controlled” test, using 1.9.9git20110406 from 11.4 suse.
500 images, 50 of those had no people in them, all others had (at least one). Interestingly, there was no misdetection in the images without any people, but in two others parts of the floor were “recognized” as a face. 180 images had only one person in them and all their faces were detected. 200 images had two persons, but in only 70 of them both were recognized, in all others cases one was missing. The remaining 70 images had at least three people in them. In only one case all (four) faces were recognized, 38 images with two detected faces, 20 had only one face found and 11 none at all.
I can’t see any pattern in that. Group shots nearly always turn up one or two faces correctly, but not those I’d expect it to find.
Now, I manually corrected the first 150 images, added the face tags and assigned them to the correct people. The remaining 300 produced 21 correct face recognitions, all others were off. Way off, sometimes.
Sorry, I didn’t put down the exact numbers, as it was getting late. Interestingly, large single-person portraits seem to be a problem. While you’d expect them to be the easiest ones, they’re almost always wrong. Pairs seem to work quite well and only a few cases with both being wrong. Those faces recognized in group shots are mostly right, but rarely all are found.
So, I’d take a guess and say the algorithm is intended and optimized for picking and identifying people out of larger groups, which is ok as it’s the most troublesome to do manually. Portraits are easy and quick to tag yourself, even in thumbnail view, so they didn’t seem to be a major design goal.
Large, single person portraits probably aren’t found due to a bug in OpenCV (to do with wrong indices in scaling the Haar filters). There’s always a trade-of between finding many faces and not finding anything else. The author of facelib has opted for finding as little non-faces as possible, while my own code on
http://www.svenutcke.de/sw/Face/
(which, by the way, can export to digikam 🙂 goes the opposite way and finds many non-faces (how many many depends on how often you photograph persons versus non-persons – expect to see non faces approaching the same percentage as non-persons). Which means it will find most all (frontal) faces.
May be worth a try…
Sven
I would like to add that by default, the Face Suit mentioned above will try to find (nearly) _all_ frontal faces. As a side effect, it will find a LOT of crud (false positives) and take a long, LONG time to do so. I like that, but you might not (it really is a LOT of crud, and a LONG time – as in an order of magnitude more time, 30 times as much crud, but only 4 times as many faces, compared to digikam’s default, when tested on a week worth of real images – and only about 15% more relevant faces, i.e. faces of people you actually meant to photograph).
In order to approximate digikam’s behaviour, you might want to give the “–scale=4” parameter (or calculate an even better scale using something like 2\sqrt{#Pixels/1MP}, which will give you 4 for a 4MP Camera, 6 for a 9MP camera, and 8 for a 16MP camera). This is then much more likely to only find relevant faces (as opposed to all faces somewhere in the background too). This will also bring the speed up to par with digikam, while still finding more faces and providing better localisation (i.e. the bounding box around the face), which in turn is crucial for better recognition results.
PS: as the libface used in digikam uses exactly the same algorithm for finding faces, it should theoretically be possible to tune both programs to return exactly the same result – differences probably only reflect mine and Aditya’s (or the digikam developers’?) different preferences of missing as few faces as possible (mine) versus having as few false positives as possible (Aditya) – I might add that I still miss faces, and at least on my sample set, Aditya’s approach still contains 55% false positives (by comparison: The Face Suit will give you nearly 90% false positives on those images with my presets, but “only” 60% with a scale of 5).
My biggest wish for digikam is that it could load thumbnails just as fast as picasa on my hardware.
If I have 300+ photos in one directory, thumbnail-loading is way to slow. In Picasa I can scroll down without having to wait for the thumbnails to load.
Why is it that Picasa can do this but not digikam? Secret magic?
Your not utilizing the MySQL database. It’s virtually instant with that combination.
Face recognition is a nice idea, but it would actually be more useful to me if it was “shape” or “design” recognition. My wife is a quilter. It would be incredibly useful to be able to lasoo a quilt design, give it a name, then tell the software to go out and find that in all my photos. Why is everyone so fixated on faces instead of the more general and useful concept of shape recognition?
I made a rebuild of Digikam 2.0 beta 4 available for Maverick which use OpenCV 2.2 instead of OpenCV 2.1 that comes with Ubuntu 10.10 and 11.04 (Haven’t built it for Natty yet). OpenCV 2.2 have been rewritten and have some big changes but I’m not sure if it will work better than 2.1 in any way regarding to face detection/recognition in Digikam. If you use a amd64 installation of Ubuntu 10.10 then OpenCV 2.2 is built with more optimisation options (CPU support of SSE, SSE2 and SSE3). OpenCV 2.2 also supports OpenEXR but I don’t know if that matters in Digikam. It would be interesting to see benchmarks if OpenCV 2.2 is faster than OpenCV 2.1 if it scans plenty of images.
Otherwise you won’t notice much changes in Digikam as this only is backend stuff.
From what I heard, OpenCV 2.2 may not always be faster than 2.1. It depends on what functions are used, and I didn’t play with face recognition. A benchmark would be nice indeed.
OpenCV 2.3 is scheduled for release in may. Not sure what it have up its sleeves but it might be interesting to follow it up. One thing that I think will have matured in that release is CUDA support so you also can use you (Nvidia) GPU to help your CPU during heavy calculations. There is experimental CUDA support in 2.2 if you activate that during build which I haven’t done in my packages.
I quickly checked the svn logs, and I noticed a lot of stuff around porting existing algorithms to GPU… and one of them is Face Recognition 😉
Hopefully, it will enable a fast implementation of the algorithm. As you said, OpenCV 2.3 is scheduled for May, let’s hope to see this in Digikam soon after.
As Digikam use the version of OpenCV it’s built against it’s more up to the maintainer of packages which version of OpenCV Digikam uses (or if you build it yourself). Hopefully OpenCV 2.3 is compatible with 2.2 so it would be more a matter of rebuildning Digikam when OpenCV 2.3 is out… 🙂
For me it would be a god sent if I could build digikam without face recognition – I have no use whatsoever for it (100,000 images in my database + and maybe 100 with people in it)… So please make this feature – as good as it may be for some – optional and not mandatory!
I agree with bob and nn
The face recognition is very poorly recognise faces and it seems more like random tagging.
If that will not be greatly improved until the final release it is better to hide the face recognition feature for latter release and keep only the face detection with easy UI for manually tagging.
Where is the data (list of persons per photo) stored? Is it nice and portable, maybe a photo.png.persons for photo.png?
There is an option in digikam to store all tags inside the image files in the keyword field. I think the tags for face recognition are just regular tags, so they would be stored this way too if the option was set. I don’t know of any software besides digikam that can recognize more than one keyword tag in an image, though.
digiKam crash instantly after clicken the Scan-button. Error:
digikam: symbol lookup error: digikam: undefined symbol: _ZN10KFaceIface25OpenCVCascadeDataDirExistEv
Any idea what’s wrong?
Thanks
What is that, Digikam no starting;
QSqlDatabasePrivate::removeDatabase: connection ‘ConnectionTest’ is still in use, all queries will cease to work.
digikam: symbol lookup error: digikam: undefined symbol: _ZN11KExiv2Iface14AltLangStrEdit15setLinesVisibleEj
Are there some easy hotkeys for the facial tagging process? I find the “check” and “minus” buttons extremely tiny, and difficult to click on. Also, I find it difficult to switch from entering a name to clicking the confirm button without some very precise mousework. Does anyone else find this process awkward, or am I doing it wrong?