Technology for Recognizing the Flow of People from Surveillance Camera Video with Concern for Privacy

Video images taken by surveillance cameras involve privacy issues

In recent years, an increasing number of surveillance cameras are being installed on streets and almost anywhere in cities. Surveillance cameras are used primarily for preventing crime and disasters, measuring, and recording. Recently, there are growing expectations concerning the use of videos from surveillance cameras capturing pedestrian flows to relieve congestion when there are events, to devise operating plans for public transportation systems, and to guide evacuations during disaster emergencies.

Typically, the following technologies are used to extract information on the flow of people: 1) extract a person-like shape from an image from a surveillance camera, and then 2) using multiple cameras, determine whether a person detected by one camera is the same person detected by other cameras. However, as the resolution of surveillance cameras has improved, it has become possible to clearly recognize a person’s face, which raises privacy issues when third parties have information on a person’s movements, such as “what the person was doing at a particular time and place.”

Recognizing the flow of people using images that do not identify individuals

Fujitsu Laboratories has developed the industry’s first technology that allows highly accurate detection of people’s movements and can recognize the flow of people from low-resolution imagery incapable of distinguishing faces.

The technology extracts head-shaped candidates, focusing on the fact that, even in low-resolution images, the head remains to be a defining human feature. Based on their proximity to the camera, head shapes are extracted, beginning with the closest person, and the image of people further away is updated to compensate for partially obscured areas of a person’s body. This makes it possible to independently recognize the heads and torsos of many, overlapping people.

Human features extraction from low-resolution imagery (Left: input image; Middle: First-round detection; Right: Detection results)

Also, by selectively extracting only the distinctive colors of a person’s clothing, a single detected person’s movements between multiple cameras can be recognized, which enables reliable person matching using low-resolution images.

Person matching using low-resolution images

Improving customer service while respecting customers’ privacy

This technology makes it possible to use surveillance camera images that respect the privacy of the people whose images are being recorded. It allows the movement paths of people to be analyzed without identifying their faces, which contributes to improving customer services—such as effectively arranging and displaying merchandise in stores—reducing the waiting time by predicting congestion at cash registers.

Fujitsu Laboratories plans to conduct more practical testing to improve detection accuracy, and aims for commercial implementation during fiscal 2016.