Comprehensive video content analytics increases effectivity

Increasingly, municipalities and corporations are getting more value out of their new and existing video surveillance (CCTV) systems by implementing AI-powered video content analytics software. These solutions detect, classify, and identify objects (people, vehicles, animals, etc.), and then index them, thereby transforming video footage into data that is searchable, actionable and quantifiable. Video intelligence software can help security and law enforcement teams rapidly locate and track vehicles or persons-of-interest, such as a missing person or a suspect leaving a crime scene, by distilling information about persons or vehicles from hours of footage. Some robust platforms even offer fully integrated facial recognition and license plate recognition (LPR) functionalities to further empower users when searching for specific people or vehicles; however, not all face or license plate recognition software solutions are both effective and versatile enough to provide value across the various challenging scenarios and conditions in a real-world video surveillance network.

Download Facial Recognition Report

Facial recognition and license plate recognition

Face recognition based on Deep Learning is able to extract unique identity features from a face image input and match them against a gallery of reference features (e.g., a watchlist) to determine the identity of the individual in query. The recognition process can occur automatically when a new object is detected, and a human operator can assess and confirm any resulting match. Operators may create watchlists of faces, using external image sources or digital images extracted from video.

License plate recognition works in much the same way and also allows for watchlist configuration based on an uploaded list of vehicle plates or manually entered license plate numbers.

Operators can search for faces or plate numbers either in real-time (alerting when a face or plate that appears in video exists on a watchlist – or if it doesn’t exist on a watchlist in cases like the validation of authorized access), or after an event while conducting a forensic review of video footage.  This improves situational awareness and response times in critical, time-sensitive situations to accelerate investigations and time-to-target.

Considering needs for constrained or “in the wild” recognition

When it comes to selecting video content analysis solutions or standalone facial or license plate recognition technology, organizations should consider whether they need LPR and face recognition for “in the wild” or constrained scenarios. Some video analytics solutions have face recognition and LPR functions that are purpose-built for constrained scenarios, such as identifying a face at a terminal to allow access to an office/room inside a building, or identifying a license plate when vehicles stop at the entrance or exit to a parking garage. In constrained recognition scenarios, the following factors are at play:

• A specialized camera is usually used — the camera has a single purpose and is positioned in a dedicated way.
• The camera is in a controlled (constrained) environment where the object’s behaviors are somewhat predictable, such as a vehicle slowing or stopping at an entrance or exit of a parking lot, or a person passing through a turnstile.
• The scene conditions (such as illumination) are controlled and optimal.

Other face recognition and LPR solutions are built for “in the wild” recognition, to more accurately locate a person or vehicle across a wide network of existing or non-dedicated surveillance cameras that are located indoors or outdoors, in challenging settings such as highway poles, streetlamps, or buildings. Depending on an organization’s video content analysis objectives and constraints, it may need a solution that is designed especially for “in the wild” surveillance scenarios.

Appearance similarity can complement face recognition

“In the wild” face recognition and LPR solutions can encounter challenging conditions, such as sub-optimal camera position, lighting, or weather, yet they can still provide important information to augment simple forensic and real-time video intelligence. However, given those environmental challenges, when faces or license plates cannot be identified using facial recognition or LPR on some cameras, an appearance similarity function is an important adjunct capability.  In addition, there might be privacy or legal considerations that might prohibit facial recognition or LPR from being used – making appearance similarity the only way to effectively identify a person or vehicle in video. 

Appearance similarity may be based on predefined object attributes such as gender, hats, clothing colors, etc., and can also be based on Deep Neural Network feature extraction – the process of extracting identity features from a person or vehicle, which is similar to facial recognition.

Operators using traditional forensic video analytics can use search criteria mentioned above such as gender and clothing color – but this will probably not be definitive or discriminative enough to, for example, locate a missing person on a university campus teeming with people.  In such cases, where facial recognition or license plate recognition is not an option (because of the way the cameras are set up, or privacy regulations), a deep learning-based appearance similarity-matching engine is one of the most effective tools for finding occurrences of specific persons and vehicles in video.

Advantages of comprehensive video content analytics

Organizations that are vetting video analytics solutions should strongly consider comprehensive and scalable solutions that offer appearance similarity search alongside face and license plate recognition. Point solutions that are dedicated to LPR or face recognition have limitations in some use cases, whereas comprehensive video content analysis solutions enable organizations to flexibly address varying use cases and to maximize their investments in video surveillance networks. With an extensible video content analysis solution that includes face and license plate recognition (among other capabilities), organizations can enjoy the benefits of a robust point solution, as well as additional capabilities to more fully leverage their investment in video surveillance – such as appearance similarity search.