From VIDEO SYNOPSIS® to Smart Synopsis: Getting the Most out of Video Analytics

When I joined BriefCam^® in 2011, we had an ambitious challenge: how to take the groundbreaking  VIDEO SYNOPSIS capability developed by Prof. Shmuel Peleg at the Hebrew University of Jerusalem to the next level. How to turn it from a technology that enables reviewing hours of video in minutes into a commercially-viable technology that could enable viewing weeks or even months of video in a short time.

Enter Smart Synopsis. The addition of the possibility to search for specific objects.

This addition of sophisticated, and often complex search capabilities requires full object tracking and accurate segmentation, and is definitely a game changer. The importance of full tracking is twofold: it enables tracking all interesting objects in a scene, as well as the tracking the full trajectory of an object throughout a scene.

This is particularly useful to:

• Understand the significance of behavior, such as the flow of customers in a store, where people tend to loiter (and the impact on security and sales), as well as interactions between people.
• Achieve a single alert per event. This is important since most tracking analytics are frame-based (they try to detect objects in a specific frame) with moderate localization (they only have a rough estimation of where the object is located – the bounding box), meaning that they may deliver multiple alerts per event or miss following events such as tailgaters.

Accurate segmentation is also essential in the development of Smart Synopsis, in order ensure that synthetic video rendering looks fully realistic. This is achieved by deploying a tight-fitting mask in each frame (pixel level localization).

Smart Synopsis delivers the full tracking and accurate segmentation that is essential for most mission-critical applications. It resolves object occlusions, and therefore can be used in high-level processes such as action recognition, object identification and object recognition. So, when basing video analytics on Smart Synopsis, finding what you are looking for becomes an even faster and more accurate process. Full context can be retained with the original, full-length video always remaining a click away, while reviewing the synopses generated.

Furthermore, Smart Synopsis integrates a methodology that uses mosaic images together with rich metadata for video indexing, thereby enabling the inclusion of objects from different times in static synopsis images.

More on how things can get even smarter, next week.