Gathering the Power of Smart Converged Storage to Build a Safe City − Developing a Video Surveillance Solution with Smart Converged Storage
Background: Intelligent video analysis becomes the norm
The Chinese economy continues to develop steadily after many decades of rapid growth thanks to the many policies put in place to stimulate progress. With this progress, the Chinese government has also placed much focus on strengthening social stability by leveraging science and technology to help law enforcement work smarter as Safe City rollouts continue nationwide. As part of this focus, ever-increasing numbers of surveillance cameras are being deployed to make communities and metropolises safer, resulting in exponential growth in image data. Given the ineffectiveness of conventional manual approaches to processing these images and the time and labor needed to complete these tasks, the video surveillance component of Safe City requires intelligent video analysis capabilities.
Intelligent video analysis helps operators identify events, attributes, and patterns of behavior through video analysis of monitored environments, and generates alerts on any anomalies detected. In Safe City, intelligent video analysis is mainly used to enhance video image processing, image quality, behavior detection, classification of indexed video data, and extraction and query of key objects (such as people and vehicles) in the data. With this technology, vast amounts of redundant and useless information can be filtered out, thus reducing much of the need for intensive manual monitoring and its associated costs.
Pain point: Restrictions encumbering deployment of intelligent video analysis
The technologies and devices going into intelligent video surveillance have been optimized over the last ten years, and now it is widely used for video analysis and recognition. Although even wider application will come in the future, it is still hampered by many technical obstacles.
Lack of a universal platform
The current intelligent video analysis technologies fall into two categories: front-end solutions and back-end solutions. Back-end solutions based on intelligent video processors introduce intelligent analysis into front-end cameras and video recorders, and are mainly used for transportation management and behavior recognition. Back-end solutions intelligently analyze the video data aggregated onto storage nodes. The most typical practice nowadays is to revamp hardware devices with computing capabilities into dedicated platforms to deliver intelligent analysis functions. However, each of the platforms is designed to address a specific need, and a large number of platforms are needed if diversified video analysis is required. In addition, as the intelligent algorithms and technologies are becoming more sophisticated, these platforms are becoming harder to upgrade and maintain while collaboration between these platforms is becoming more problematic, which means these platforms are incapable of meeting the huge surge in video analysis needs. Fortunately, with the development of software-defined technologies and the wider acceptance of openness and integration in the public security arena, many software products have been developed for intelligent video surveillance and are answering the need for a common platform. These improvements are creating the approaches needed to apply intelligent video analysis on a broader scale.
Need to process and store enormous amount of surveillance data
Continued expansion of the Safe City Project requires a vast amount of surveillance cameras. The area under the jurisdiction of each local police station has about 100 lanes of 1080P HD cameras deployed on average, and this number will increase to 500 lanes in the near future. 100 TB in storage capacity will be required for each sub-district if the video images captured by the 500 lanes of cameras are retained for 30 days. Let's assume that the police station needs to investigate around ten crimes at the same time and 30 cameras have recorded some sort of evidence, event, or object helpful to the investigation. Let's further assume that the videos captured by these cameras must be analyzed for 3 hours and retained for 90 days. In this case, 120 TB of storage capacity must be available. Furthermore, the analysis of each crime must be completed within 60 minutes and cannot affect ongoing video recording and retrieval, which magnifies the need for tiered data storage, elastic capacity expansion, and concurrent read/write performance.
Limited computing capabilities fail to address parallel analysis needs
Intelligent video analysis involves a series of complicated algorithms and formulas, and CPU processing capability becomes the bottleneck to overall computing performance. The previous isolated devices must be combined into a system to deliver concurrent analysis capabilities and improve efficiency.
Figure1: Huawei's converged storage and intelligent analysis solution
The solution to the problems lies in convergence
Having thoroughly analyzed the needs in the Safe City Project, Huawei designed a two-layer storage architecture. On the access layer, SAN storage devices are deployed for local police stations and communities to receive video streams. On the aggregation layer, storage cloud devices are deployed for public security bureaus to aggregate the video streams. This architecture also converges two types of intelligent analysis services to deliver a variety of functions, including data storage, intelligent analysis, and application archiving.
The access layer incorporates video recording and analysis storage systems as well as leading-edge intelligent analysis service software based on kernel-based virtual machine (KVM). It not only addresses the basic needs in data recording and playback, but also uses advanced functions like quality of service (QoS) to allocate CPU computing resources to intelligent analysis services. This layer prioritizes memory and bus resources to mission-critical and real-time services, and allocates other resources to the other non-real-time services to maximize resource utilization. In addition, real-time video streams can be analyzed and scheduled, and any exceptions occurring during the analysis process are identified, while archived video records can be intelligently retrieved and associated to provide evidence for the criminal investigation. In this way, this layer perfectly converges the capabilities of video storage and analysis.
The aggregation layer adopts a distributed cloud storage architecture to store and analyze the vast amount of video data. Every storage node on the layer is leveraged to condense the video content, identify attributes in the footage, and extract valuable data. After removing the useless content, useful summaries are categorized, retrieved, and scheduled on a universal IT platform. The data in the summaries can then be used to search for clues and provide evidence to help solve crimes. The aggregation layer achieves the convergence of video storage, analysis, and archiving, thereby significantly improving the video analysis and concurrency capabilities as well as the efficiency of the criminal investigation.
In addition to the impressive convergence performance in video storage and analysis, this two-layer storage architecture also delivers the following technological advantages.
Convergence of hardware and software on a universal IT platform
Compared with conventional platforms that can only address specific needs, the universal IT platform provides intelligence-rich analysis capabilities and can better serve the application purposes of criminal investigations. Adding to the appeal, these universal platforms also require less hardware, which in turn reduces the physical footprint, conserves power, and lowers construction and management costs.
Elastic capacity expansion
The access layer adopts Huawei T series SAN storage devices, which can be stacked for capacity expansion and provide access and analysis capabilities for up to 2000 lanes of video. The aggregation layer employs a distributed cloud storage system that can be scaled up to simultaneously process 100,000 read and write requests and provide 40 PB in storage capacity. All the devices and systems on the two layers can be utilized and expanded on demand to address video storage requirements both now and in the future.
In addition, a variety of storage protocols are supported as the T series support SAN while the cloud storage devices support NAS and HDFS, fulfilling diversified storage requirements in intelligent video analysis and Big Data.
Efficient and parallel analysis capabilities
With such cutting-edge technologies as video stream consolidation, intelligent prefetch, and I/O passthrough, Huawei achieves an efficiency in video image retrieval 24 times higher than the industry average, and increases the read/write ratio to 4:1, demonstrating the viability of the solution's ability to handle the copious number of read requests in intelligent video analysis. The two-layer architecture for video storage and analysis allows video data to be stored and analyzed in a distributed manner while being managed centrally. Under pressure to concurrently process thousands of video streams, system workloads are evenly distributed to achieve a balance between capacity and performance. The Hadoop Big Data platform archives the original footage, videos and images generated during the analysis process, as well as videos and images associated with crimes separately, which helps the public security sector manage video data in an orderly fashion and locate the needed information efficiently.
Intelligent reliability assurances
The powerful backup and reliability capabilities of Huawei storage devices provide all-around reliability assurances for video data. Such top-notch functions as disaster recovery for important cameras, N+1 to N+4 data protection, file-level data protection, node redundancy, and QoS assurance make Huawei storage devices immune to single points of failure and ensure continuity in recording, playback, and intelligent analysis.
Huawei also delivers enhanced flexibility in data lifecycles to resolve the previous pain points in inflexible and short video retention periods. With the revolutionary fade-in and fade-out functions, the level of importance of certain types of data gradually degrades as the data ages, and when the threshold is reached, the data will be discarded if the data is not categorized as higher-level or critical data. These functions allow for certain types of surveillance data to be retained longer and ensure highly reliability throughout the data lifecycle.
Gathering the Power of Smart Converged Storage to Build a Safe City
Huawei's two-layer architecture for video storage and analysis improves performance, capacity, reliability, and management capabilities across the board. By leveraging its technical advantages in cloud storage and backup, intelligent storage of video streams, and integration of IT utilities, Huawei is enhancing the intelligence level of its storage devices and extracting new value. With to the drive to deliver elastic, efficient, and reliable video analysis solutions for customers, Huawei is committed to developing world-class technologies for application in the video surveillance field.