The business and culture of our digital lives,
from the L.A. Times

Category: Facial recognition

CES 2012: Sesame Street Kinect shows promise of TV voice, gesture control [Video]

At the 2012 Consumer Electronics Show in Las Vegas, we saw a bit of a scramble by TV makers such as Samsung and LG to show off what they working on or releasing in the coming year that would allow us to control our TVs using voice, gesture and facial recognition.

Many technology pundits and analysts have said these sorts of announcements, which also took place at last year's CES, are in response to rumors that Apple is working on an "iTV" that will offer a new way of controlling a TV and maybe even how we pay for or watch channels and TV shows.

But as many video-game lovers out there know, TV voice recognition, gesture controls and facial recognition are already here in the form of Microsoft's Kinect motion-sensing camera, which is an accessory to the Xbox 360 home gaming console.

However, Kinect is just getting started, and currently has a small number of apps. And it's still a device that sells for about $150 and requires an Xbox 360, which starts at $200. Make no mistake, there will be a cost of entry to the future of TV.

At the 2012 Consumer Electronics Show TV makers such as Samsung and LG showed off TVs with voice, gesture and facial-recognition control, but such controls are already here in the form of Microsoft's Kinect motion-sensing cameraAt CES 2012, Microsoft showed off a bit of what the future may hold for Kinect, the Xbox and TV with demonstrations of its latest Kinect-enabled app for the Xbox, called Sesame Street Kinect (you can see our demonstration of the app in a video atop this article).

Sesame Street Kinect is what it sounds like, episodes of the long-running children's program tailored to use the Kinect camera. And what Kinect can do is really impressive.

Since 1969, children around the world have sat in front of TVs repeating back the alphabet, colors, words and numbers to characters on Sesame Street (I did it when I was a child). Until Sesame Street Kinect, which is set to release later this year at an unannounced price, the characters on the screen couldn't respond to the viewer's actions. Now, to a limited extent, they can.

The demonstration we saw featured the Grover, Elmo and Cookie Monster characters prompting viewers to interact by either saying certain words or moving in certain ways.

For example, we took part in a demonstration in which Grover drops a box of coconuts and asks that the viewer pick them up and throw them back to him.

I At the 2012 Consumer Electronics Show TV makers such as Samsung and LG showed off TVs with voice, gesture and facial-recognition control, but such controls are already here in the form of Microsoft's Kinect motion-sensing cameraf the viewer stands up and moves in the way that they would throw an imaginary coconut (don't throw a real coconut unless your trying to break your TV) then Grover catches each one in his box, even reacting to how hard the Kinect interprets the viewer's throw to be.

The experience was a lot of fun for a room of four adults, and I imagine kids will enjoy this sort of thing too. Jose Pinero, am Xbox spokesman, said a similarly interactive app from National Geographic is coming this year as well.

Although Microsoft has sold more than 66 million Xbox consoles and more than 18 million Kinect cameras, the tech giant realizes it has something bigger than just video games on its hands with Kinect.

Both Kinect and Xbox Live are headed to Windows 8 later this year. Hopefully, that will mean more interactive "two-way TV" apps like Sesame Street Kinect, and more apps related to media outlets such as ESPN and National Geographic.

There are also rumors that the company is working to get Kinect built directly into TVs, which would very likely place Xbox Live and Kinect in direct competition with Google TV and Apple's expected entry into the TV market. That's a living-room showdown I'd like to see.


Kinect is coming to Windows, but are TVs next?

Samsung tablet + Kinect + motorized skateboard = wear a helmet

Kinect-enabled dressing room lets you change clothes without having to take clothes off

-- Nathan Olivarez-Giles in Las Vegas

Nathan Olivarez-Giles on Google+

Photos: Sesame Street Kinect in action. Credit: Armand Emamdjomeh / Los Angeles Times

British riots: Police use facial-recognition technology


Britain's police are using facial-recognition technology to help them track down those who have taken part in the riots that struck London this week.

The facial-recognition technology, which belongs to London's Metropolitan Police agency, Scotland Yard, was first going to be used at the Summer Olympic Games of 2012, which are set to take place in London, according to the Associated Press.

But the civil unrest that began as a protest in the neighborhood of Tottenham in response to police fatally shooting 29-year-old Mark Duggan last week, and quickly and violently spread to other parts of the city, was enough for law enforcement to put the technology into practice ahead of schedule, Chief Constable Andy Trotter of the British Transport Police told the AP.

Officers have been sending photographs of suspects to Scotland Yard to be run through the agency's face-matching software, but Trotter said the technology wasn't a major part of how police would identify those who took part in the riots, the AP report said.

"There's a mass of evidence out there," Trotter said in the report. "The public are so enraged that people who wouldn't normally come forward are helping us -- especially when they see their neighbors are coming back with brand new TVs."

A Scotland Yard officer, who also spoke anonymously in the AP report, said that the facial-recognition technology "would only be used to help identify those suspected of serious crimes, such as assault, and that in most cases disseminating photographs to the general public remains a far cheaper and more effective way of finding suspects."

By the middle of the day on Friday, nearly 1,700 people were arrested in connection with the unrest, many of whom were taken into custody on suspicion of burglary, theft and arson, while some faced murder charges.

Trotter told the AP that 16,000 police would remain on active duty on London streets over the weekend in an effort to end the riots.


Twitter, BlackBerry Messenger cited as fueling London riots

London riots: Violence may be contagious in London, but so is altruism

London riots: Ban on Twitter, BlackBerry Messenger, other social networks considered

-- Nathan Olivarez-Giles

Photo: Police officers lead away a riot suspect after a raid on a property in Pimlico, London. Credit: Reuters

Researchers show power of Facebook facial-recognition software

Faces of Facebook: In a nutshell

Facebook has come under a lot of heat for its facial-recognition software, in which the social networking site has been automatically enrolling its more than 750 million users.

But Facebook has made it clear that the software, which automatically tags people in photos, isn't going anyway anytime soon. In fact, facial-recognition software is growing and is being used and further developed by Facebook, Google, Apple and the U.S. government.

On Friday Carnegie Mellon University researcher Alessandro Acquisti showed off his research, funded in part by the U.S. Army, on how facial-recognition technology can be used with Facebook profile photos to match names and other identification data to pictures.

Acquisti presented his findings, alongside fellow researchers Ralph Gross and Fred Stutzman, at the Black Hat Technical Security Conference in Las Vegas, according to tech website Cnet, which reported on the group's presentation. 

The researchers set up a computer webcam on the Carnegie Mellon campus and asked people to volunteer to have their pictures taken, Cnet said.

Those photos were then cross referenced with a database the team built of about 25,000 Facebook profile photos (all Facebook user names and photos are publicly shared with the world afterall), the report said.

The researchers found that facial recognition software identified 31% of the students by name, Cnet said.

Acquisti then demonstrated an app for Apple's iPhone that can "take a photograph of someone, pipe it through facial-recognition software, and then display on-screen that person's name and vital statistics," the report said.

"Facial visual searches may become as common as today's text-based searches" and that has "ominous risks for privacy," Acquisti said in the Cnet report. 

"What we did on the street with mobile devices today will be accomplished in less intrusive ways tomorrow," he said in the report. "A stranger could know your last tweet just by looking at you."

In yet another demonstration, about 6,000 profile photos and names from a dating site were cross referenced with 277,978 Facebook profile photos and names and "about 1 in 10 of the dating site's members -- nearly all of whom used pseudonyms -- turned out to be identifiable," Cnet said.

In a draft of the researcher's presentation, posted online and titled Faces of Facebook: Privacy in the Age of Augmented Reality, they even said they've been able to use profile photos and facial-recognition software to get details such as birthdate and social security number predictions.


Facebook's facial recognition violates German privacy laws, officials say

Facebook data privacy questioned by Denmark, Finland, Norway and Sweden

Facebook under scrutiny for face-recognition feature from privacy group, lawmakers

-- Nathan Olivarez-Giles

Image: Screenshot of a slide in a draft of the Carnegie Mellon University presentation at the Black Hat Technical Secuity Conference in Las Vegas titled Faces of Facebook: Privacy in the Age of Augmented Reality. Credit: Alessandro Acquisti, Ralph Gross and Fred Stutzman /Carnegie Mellon University

Facebook's facial recognition violates German privacy laws, officials say

Facebook facial recognition software

Facebook could reportedly face legal action and fines if it doesn't suspend or make major changes to its facial recognition software in Germany.

Facebook's facial recognition technology, which automatically enrolls all Facebook users, is used to identify and tag people in photos uploaded to the social network, which has more than 750 million users worldwide.

Users can opt out of the feature, so they can't be automatically tagged in photos that Facebook identifies them in, but doing so is a bit of a complicated process (lucky for you, we've explained the process step by step).

The facial recognition feature and Facebook's decision to not ask users about using the feature before enrolling them in it has sparked an outcry from privacy groups and politicians.

On Wednesday, German authorities were the latest to come out against the technology, saying that it violates privacy laws in that country and across Europe, according to a report by the Guardian in England.

"Hamburg's data protection official has written to Facebook to demand it stops running the facial recognition programme on German users and deletes any related data," the Guardian reported, adding that Facebook could face fines up to about $430,000 if the feature is not disabled in Germany.

If Facebook wants to keep the feature in place it must change it, German officials told the Guardian, saying that the world's most widely used social network "must ensure that only data from persons who have declared consent to the storage of their biometric facial profiles be stored in the database," the Guardian said.

"If the data were to get into the wrong hands, then someone with a picture taken on a mobile phone could use biometrics to compare the pictures and make an identification," Johannes Caspar, Germany's data protection supervisor told the Guardian.

Caspar said Facebook's system could be used by oppressive governments to spy on members of the opposition and that "the right to anonymity is in danger," the report said.

Facebook officials were unavailable to comment on the report Wednesday.


Facebook's new 'Expected: Child' tag sparks outcry

Facebook tells users how to opt out of facial recognition

Facebook under scrutiny for face-recognition feature from privacy group, lawmakers

-- Nathan Olivarez-Giles

Image: A screen shot of Facebook's facial recognition software at work. Credit: Facebook

Facebook tells users how to opt out of facial recognition

Facebook facial recognition

Facebook has begun running ads to show users how to opt out of a controversial facial-recognitition feature for photographs after Connecticut Atty. Gen. George Jepsen raised privacy concerns.

Jepsen said in a written statement that "Facebook has made significant changes that will provide better service and greater privacy  protection to its users."

I am still trying to figure out what these "significant changes" are. 

In a letter to Facebook in June, Jepsen said Facebook should ask users to opt into the feature rather than forcing them to opt out of it. Facebook has not done that.

What has changed? Facebook is running ads that direct users to a link where they can learn how to disable the feature in their privacy settings. I showed you how to disable the feature last month.

 In an e-mailed statement, Tim Sparapani, Facebook's director of public policy, said the "collaboration" with Jepsen "means that people across the country using Facebook will be more aware of our personalized privacy settings."

"Tag suggestions" uses facial recognition software to help speed up the process of labeling friends in photos. How it works: The software scans uploaded photos, compares faces in those photos with friends in other photos, then suggests a "tag" for the person in the photo.

Jepsen also said Facebook has assured him that when users opt out, all of the data collected is deleted. Again, that's what Facebook has said all along. 


Here's how to opt out of Facebook's facial-recognition feature

Connecticut AG challenges Facebook over facial recognition photo tagging feature

Facebook under scrutiny for face-recognition feature from privacy group, lawmakers

-- Jessica Guynn

Connecticut AG challenges Facebook over photo tagging feature that uses facial recognition


Connecticut's attorney general does not want Facebook to automatically suggest the names of users in photographs without their permission.

George Jepsen has joined lawmakers and privacy watchdogs in objecting to how Facebook has rolled out the photo-tagging feature, which uses facial recognition technology to identify people.

He called the lack of an opt-in process "troubling" and said it could expose users to "unwelcome attention and loss of privacy."

Jepsen said he wants to meet with Facebook officials.

Facebook says it has been in touch with Jepsen's office and will answer any question he has.

The company has been gradually rolling out the feature since December. It points out that any user who does not want the feature can disable it.

"Millions of people have used it to add hundreds of millions of tags," a company spokesman said in a statement. "The fact that we've had almost no user complaints suggests people are enjoying the feature and are finding it useful."


Here's how to opt out of Facebook's facial-recognition feature

Facebook under scrutiny for face-recognition feature from privacy groups, lawmakers

Consumers Union pushes Facebook to do more to keep out underage users, protect teens

-- Jessica Guynn

Image: A screenshot of Facebook's facial recognition feature in action. Credit: Facebook

Facebook photo-sharing app in development for Apple's iPhone? [Updated]

TechCrunch Facebook photo app

Facebook may be building a photo-sharing app for smartphones and tablets, in a bid against social networking apps from Instagram, Hipstamatic, Path and even Twitter.

The Palo Alto-based social network is the world's most widely used, with an estimated user base of more than 600 million, and it's also the most popular photo-sharing website, with more than 100 million photos uploaded daily.

According to the website TechCrunch, Facebook is developing an app for Apple's iOS operating system (used on the iPhone, iPod Touch and iPad) that would allow users to take and edit photos, then share the shots on Facebook.

In Facebook fashion, users would also be able to "like" photos from friends, tag people they know in pictures, check-in to locations, comment on shots and group photos into albums, TechCrunch said, adding that the site was given screenshots of the app from an unnamed source at the social network.

The screenshots don't make it clear whether Facebook is building a standalone photo-sharing app, a Web-based HTML 5 app, or simply adding the new photo taking and sharing functions to the pre-existing Facebook iOS app, TechCrunch said, adding that maybe they're doing all three.

Facebook officials were unavailable for comment on the report on Wednesday.

[Updated 4:11 p.m.: A Facebook spokeswoman emailed along a company statement that neither confirmed nor denied the TechCrunch report, stating: "We're constantly working on new features and enhancements to our products but have nothing new to announce at this time."]


Twitter announces photo-sharing service

How to opt out of Facebook's facial-recognition feature

Facebook under scrutiny for face-recognition feature from privacy group, lawmakers

-- Nathan Olivarez-Giles

Image: A screenshot of TechCrunch's report on a rumored Facebook photo-sharing app under development. Credit: TechCrunch

Facebook under scrutiny for face-recognition feature from privacy group, lawmakers [Updated]

Facebook's new facial-recognition feature is getting some unwelcome recognition from a prominent privacy group and lawmakers in the U.S. and European Union.

The Electronic Privacy Information Center, based in Washington, D.C., said Wednesday that it plans to file a complaint with the Federal Trade Commission raising concerns over Facebook's new "tag suggestions" feature which allow users to identify people across multiple photos at once using facial-recognition software.

163475_10150118904661729_20531316728_7373784_7246884_n "Obviously we're not going to comment in detail until we file whatever were going to file," said John Verdi, senior council at EPIC. "But, we think the facial recognition feature raises real questions about what sort of data Facebook is collecting from its users and from its users' photographs.

"And it also raises questions about what Facebook does with this user data once it collects it and who else is accessing that data after it's collected."

Facebook announced that it was planning to roll out the new facial-recognition feature across its entire social network -- which has more than 500 million worldwide users -- on Tuesday in a company blog post. The feature was first announced, for testing, in December.

"When you or a friend upload new photos, we use face recognition software -- similar to that found in many photo editing tools -- to match your new photos to other photos you're tagged in," said Justin Mitchell, a Facebook engineer in the blog post announcing the feature's roll-out. "We group similar photos together and, whenever possible, suggest the name of the friend in the photos."

If a Facebook user doesn't want their name to be suggested in the new feature, they can opt out, Mitchell said, explaining the process in the blog post:

You will be able to disable suggested tags in your Privacy Settings. Just click "Customize Settings" and "Suggest photos of me to friends." Your name will no longer be suggested in photo tags, though friends can still tag you manually.

But the fact that Facebook's facial-recognition feature is opt-out and not opt-in and Facebook "changed user privacy settings to automatically turn on a new facial recognition feature that detects a user's face in an image or photo" was a point of criticism for Mass. Rep. Edward J. Markey, who is the co-chairman of the Congressional Privacy Caucus.

"Requiring users to disable this feature after they've already been included by Facebook is no substitute for an opt-in process," Markey said in a statement. "If this new feature is as useful as Facebook claims, it should be able to stand on its own, without an automatic sign-up that changes users' privacy settings without their permission."

Facebook officials weren't available Wednesday to comment on the criticism of the site's facial-recognition feature.

A group of privacy regulators in the European Union also said on Wednesday that they would launch a probe into the new Facebook feature to "measure for possible rules violations," according to a report from Bloomberg.

"Tags of people on pictures should only happen based on people's prior consent and it can't be activated by default," Gerard Lommel, a Luxembourg member of the Article 29 Data Protection Working Party, said in the Bloomberg report.

Introducing such a feature without that consent "can bear a lot of risks for users" and officials at the E.U. are planning to "clarify to Facebook that this can't happen like this," Lommel said, according to the report.

[Updated 2:39 p.m.: A Facebook spokeswoman emailed a statment to the Technology Blog stating that the company should have made users more aware of the new feature before it was widely released.

"We launched Tag Suggestions to help people add tags of their friends in photos; something that's currently done more than 100 million times a day. Tag Suggestions are only made to people when they add new photos to the site, and only friends are suggested. If for any reason someone doesn't want their name to be suggested, they can disable the feature in their Privacy Settings.

When we announced this feature last December, we explained that we would test it, listen to feedback and iterate before rolling it out more broadly. We should have been more clear with people during the roll-out process when this became available to them. Tag Suggestions are now available in most countries and we'll post further updates to our blog over time."]


Consumers Union pushes Facebook to do more to keep out underage users, protect teens

Google's Eric Schmidt says he didn't push hard enough for deal with Facebook

Facebook hires former Bush aides as Washington lobbyists

-- Nathan Olivarez-Giles

Image: A screenshot of Facebook's facial recognition feature in action. Credit: Facebook

Viewdle brings automatic photo tagging to smart phones

Viewdle screenshot
There's something Orwellian about machines that can identify people by looking at their faces. In fact, the facial-recognition software being developed by Viewdle, a San Francisco-based start-up, has its roots in technology created for the surveillance-happy government of the former Soviet Union. But Viewdle has found broader and far more consumer-friendly uses for its wares.

The company has been conducting an invitation-only trial of desktop-computer software that automatically identifies faces in photos and videos, then "tags" the files with this information. At the Consumer Electronics Show last week, Viewdle demonstrated an application that can do the same with a smart phone camera, automatically and in real time. That made Viewdle the first company to show "real visual computing that works in a mobile device," Chief Executive Laurent Gil claimed.

The company, which is selling the technology to other companies in the mobile-phone industry, expects versions of the app to be available by the end of the year.

Viewdle automates the labor-intensive process of tagging photos and uploading them to Facebook. That sort of convenience will appeal most to young people who are "all about sharing the moment" and documenting their lives through their mobile-phone cameras, Gil said.

The app determines identities by comparing the faces seen by the camera's lens with images previously stored on the phone. That's a twist on the usual practice of trying to match faces against a vast online database. It also uses a comparison technique -- morphing images to try to match unknown faces with known ones -- that's faster and requires less computing horsepower than the conventional practice of comparing multiple features of the unknown face against those in identified images, Gil said.

For the app to work, users have to build up a database on their phones of tagged photos that Viewdle can use as a reference. The software makes the task easier, though, by logging into Facebook and collecting information from the photos they've tagged there.

Facial recognition is just the first step for Viewdle, whose morphing technique can be used to recognize any physical object seen by a camera -- stairways, cars, animals, you name it. Other potential uses include augmented-reality applications for gaming and commerce, said Jason Mitura, Viewdle's chief product officer. For example, a retailer could add virtual displays to its store stocked with its online-only inventory, with interactive 3-D images of those products visible through a smart phone camera lens.

-- Jon Healey

Healey writes editorials for The Times' Opinion Manufacturing Division.

Facebook facial recognition software suggests photo tags

Profile Facebook users may find their names attached to more photos in coming weeks as the social networking site rolls out a new feature that uses facial recognition software to suggest photo tags.

The software matches faces in newly posted photos to those in previously tagged pictures and suggests the names of friends pictured in the new photo, according to Facebook's blog. U.S. Facebook users will begin to receive tag suggestions over the next few weeks.

Users can prevent their names from popping up in tag suggestions by disabling the suggested tag feature in their privacy settings. To do so, click "Customize Settings" and "Suggest photos of me to friends." Friends will still be able to manually tag users who disable the suggested tag feature.

Some commenters called the new concept creepy, but Facebook says it's just trying to make users' lives easier.

"While tags are an essential tool for sharing important moments, many of you have said tagging photos can be a chore," the Facebook blog said.

The social networking site has already launched a group-tagging feature that allows users to type one name and apply it to multiple photos of the same person.


Cartoon profile pictures on Facebook to come down Monday

Facebook makes profiles more personal

-- Abby Sewell

Photo: The home page for Facebook founder Mark Zuckerman on Oct. 18. Credit: Karen Bleier/AFP/Getty Images


Recommended on Facebook


In Case You Missed It...


How to Reach Us

To pass on technology-related story tips, ideas and press releases, contact our reporters listed below.

To reach us by phone, call (213) 237-7163


Andrea Chang
Armand Emamdjomeh
Jessica Guynn
Jon Healey
W.J. Hennigan
Tiffany Hsu
Deborah Netburn
Nathan Olivarez-Giles
Alex Pham
David Sarno