This featured article was originally published on page 76 of the September 2022 Issue of Security Journal Americas Magazine.
By Lizzi Goldmeier
Director of Marketing
BriefCam
While video analytics solutions are important to most people who utilize video surveillance, understanding how the technology works and what it is capable of beyond the realm of security is key to leveling up the value of your assets. Video analytics is an artificial intelligence technology built on the underlying capabilities of object detection and recognition which drive sophisticated video analytics for diverse business and security use cases. Understanding these vital components of video analytics will empower you to see the limitless opportunities that this tool places at your fingertips so you can transform your video surveillance investments from cost centers into revenue generating investments.
What is object detection and recognition?
Object detection and identification are crucial components of video surveillance analytics and power the tools that bring value to your video surveillance suite. Simply put, object detection is the ability to detect objects in video. This capability, however, is far from simple, and the result is the powerful ability to quickly identify specific objects and track them, from frame to frame, through hours of footage.
Furthermore, the ability to extract objects from video and differentiate them from the backdrop against which they are detected, enables more sophisticated video analytic activity, such as forensic search, real-time alerting, or VIDEO SYNOPSIS® technology – which displays all extracted video objects simultaneously, so the full activity of an hours-long scene can be viewed all at once, rather than in linear time. This advanced video analysis provides valuable insights and impactful applications for a diverse set of users far beyond what most people think of when they think of surveillance or video analytics.
How does it work?
Object detection – and subsequent object recognition – is made possible by the same type of artificial intelligence that we encounter daily through common tools like voice recognition or automatic translation services. Video analytics not only leverages artificial intelligence but relies heavily on deep learning as well. Deep learning is an AI-backed discipline by which computers learn, through exposure to data, how to execute tasks such as identifying or recognizing an object throughout video. Therefore, for deep learning to be successful, you need vast amounts of data to be processed and annotated in order to train the network until it is capable of repeating what it has been trained to do. The more data generated the higher the accuracy of the combined analysis will be.
Let’s look at an example. In terms of video analytics, the training data, which is the video itself, is broken down into individual frames to extract every item or object and train the system’s deep learning neural networks to associate specific objects with their classification and attributes. If investigators are looking for a missing child wearing pink pants, they can leverage video analytics to search video evidence for objects that the system can recognize as people wearing pants that are pink. The user would filter his or her search based on these attributes, and the video analytics software – having been exposed to large data sets of people, pink objects, and pants – would be able to answer a query against actual footage: ‘identify all instances of people wearing pink pants.’
In order to detect matches, the deep learning algorithm needs to know what pink is and will need to have been fed thousands of data examples of this color. This is repeated ad infinitum for as many objects as are required.
Object tracking
It is not enough for an object to be detected, it must also be tracked. For any video search – whether as part of a law enforcement investigation or a way to gain deeper business intelligence – it is important to be able to accurately follow an object from the time it enters the scene until the time is leaves the scene, regardless of obstructions or crowds.
Once the object is detected and tracked, descriptive information (or metadata) can be applied to create a vast structured database of classified objects. This step is what allows the technology to quickly examine hours of footage to identify a specific object – or, in our example, the pink pants.
When is it used?
Object detection and recognition can be used before, during, and after an event occurs. While reviewing capabilities allow for the investigation of an event that has already occurred, real-time alerting can also be enabled, which allows for response to an event as it is happening based on an identified object of interest. There are obvious applications here for security and law enforcement, but object detection and recognition have other far-reaching applications that are often less obvious. For example, a real-time alert can be set for a specific camera at the entrance of a construction site to ensure that everyone who passes is wearing a hard hat. If someone enters not wearing the designated object, in this case an orange hard hat, an alert is sent to operators who can then respond in real-time to ensure construction site compliance and safety standards are maintained.
Object detection in the form of people counting can also be leveraged to optimize maintenance and facility operations. For instance, bathrooms in shopping malls are traditionally maintained by preset schedules. However, this model is unable to account for increased use during peak hours, or for some bathrooms experiencing heavy traffic due to their easily accessibly location while other bathrooms experience little to no traffic throughout the day. Therefore, property managers can set thresholds that will send alerts, and consequently activate maintenance personnel, after a designated number of people have entered the restroom. This maintenance model not only supports efficient operations and staffing but also enhances the guest experience by ensuring high standards of consistent cleanliness.
Improving elevator wait times and office space optimization are additional ways video analytics can make an unexpected impact. A hotel, for instance, can use video analytics to recognize long queues and dwell times, and can even visualize the extracted data into dashboards to determine elevator traffic patterns. This actionable intelligence empowers operational management to align traffic flow by optimizing the elevator bank configurations. Furthermore, video analytics can be applied to understand space usage in an office building. For example, if a 10-person meeting room is only being used by 3-4 people at a time daily, the company may decide to reconfigure their space into smaller meeting rooms, so their employees have more consistent access to meeting spaces.
Empowered transformation
Object detection, recognition, and tracking are important underlying technological capabilities for video content analytics. Not only do they drive sophisticated analysis and present valuable actionable data, but the use case possibilities reach far beyond security application into the world of business intelligence. Understanding how video analytics works will impower you to transform your surveillance investments into a revenue generating tool that will create cross-silo impact no matter the field you are in.