Profile: Central Video Data Storage in Public Security Bureaus
As HD video surveillance technologies become more sophisticated as Safe City continues to expand, demands for higher video resolutions are also increasing. An increased number of cameras are deployed for the project, capturing vast amounts of data, which magnifies the pressure on data storage. The amount of surveillance data that needs to be stored for a medium-sized city deploying Safe City averages tens of petabytes every year while that for a large city averages hundreds of petabytes. In addition to the huge data storage requirement, extracting the most value from the data generated by enabling sharing among systems and users is a critical to Safe City success.
With the increased threat from terrorism and its impact on social stability in recent years, accelerating Safe City rollouts has become all the more important. The vast majority of video surveillance systems in these projects are now producing digital signals instead of simulated signals, adopting a dedicated video surveillance network to transfer data captured by the HD cameras. A full spectrum of intelligent technologies are also introduced into the system, covering such areas as facial recognition and behavior analysis. These technologies dramatically enhance the capabilities of the video surveillance system. Driven by the thrust from Big Data, video surveillance systems are migrating to cloud platforms to store the ever-increasing amounts of data. Big Data analytics helps enable law enforcement "nip crime in the bud" with early intervention – one of the main objectives in Safe City.
Figure 1 Application scenarios of the Big Data platform in video surveillance
Recognizing the video analysis and Big Data analytic needs of the public security sector both now and for the future, Huawei develops many innovative technologies in cloud computing, cloud storage, virtualization, parallel databases, and data mining, and employs an open and expansive data platform to replace the previous scattered service platforms and to address the storage and analysis needs for structured, semi-structured, and unstructured data.
Huawei leads the industry with video surveillance cloud storage solution
The Huawei video surveillance cloud storage solution is built on a Big Data storage platform and is specifically designed to store and analyze the vast amounts of data produced by video surveillance applications. The innovative symmetric distributed architecture allows the system to provide impressive capacity and a complete compatibility with both Oracle structured data and Hadoop Map/Reduce semi-structured data. The Huawei-proprietary data redundancy algorithms provide data protection mechanisms of different redundancy levels to generate one to four duplicates for a piece of data based on its priority level, significantly reducing risk of data corruption and loss.
Figure 2 System architecture of the Huawei video surveillance cloud storage solution
Virtual and unified resource pool
The Huawei video surveillance cloud storage solution shatters the technical restrictions in conventional video surveillance storage devices by employing a 10GE IP network to connect storage nodes and a Huawei-proprietary storage operating system to centrally manage storage resources. All video surveillance services like data storage, geographic information, and personnel/vehicle track analysis can access the unified resource pool of the Huawei solution through such data interfaces as NFS/CIFS (for unstructured data), SQL (for structured data), and Map/Reduce (for semi-structured data). The resource pool functions as a powerful data platform for efficient data sharing and utilization.
Parallel processing of structured data
The Huawei video surveillance cloud storage solution employs the massively parallel processing (MPP) technology to upgrade the conventional central database system to a distributed parallel database system (Wushan-SQL), significantly accelerating database processing capabilities. The distributed nodes can collaborate to respond to billions of write and query requests on structured data. In the video surveillance Big Data systems of the future, each video image will contain descriptive information on the personnel, vehicles, location, and other attributes in each frame, which will generates copious amounts of structured data. These next-gen video surveillance Big Data platforms must deliver distinctive performance in processing structured data. Built upon the MPP technology, the Huawei video surveillance cloud storage solution is ready to meet the challenges in the future of video surveillance.
Parallel processing of unstructured data
With the increasing degree of Big Data applications in video surveillance, conventional storage systems are choked by the following performance bottlenecks:
- Performance and capacity cannot be expanded at the same time.
- System management becomes complicated after expansion.
- Conventional RAID groups cannot deliver sufficient reliability.
The Big Data storage system in Huawei's video surveillance cloud storage solution delivers the needed scalability, performance, reliability, and ease of management to resolve all these pain points. Its core component for handling unstructured data, the Wushan distributed file system, integrates three logical layers (file systems, volume manager, and RAID) into the software layer, and then builds an intelligent file system that covers all nodes in the storage system. The system can process up to 5 million operations per second (OPS) with a maximum bandwidth of 200 GB/s, meeting the storage and analysis requirements of 1080p HD video data.
Parallel processing of semi-structured data
Safe City video surveillance systems must process complex and diversified surveillance data, and semi-structured data is a substantial part of that processing. The Hadoop system is the ideal solution for storing and utilizing semi-structured data. By optimizing the open-source Hadoop codes, Huawei developed a proprietary Hadoop system, the FushionSight Hadoop, and introduced this system into the video surveillance cloud storage solution. This system delivers excellent performance, reliability, stability, and all the processing and other capabilities required in the Safe City rollout.
Intelligent and automated management
The larger the system, the more important simplified management becomes. The Huawei video surveillance cloud storage solution is capable of processing huge amounts of complicated services, and achieves near-effortless system management through such automated functions as software pre-installation, dynamic software and hardware recognition, plug-and-play devices, and fast service provisioning.
The solution is also equipped with a unified management system to centrally manage nodes, monitor resource status, and generate alarms. In this way, users can perform management operations via PCs, smart phones, and tablets at anytime and from just about anywhere.
The Big Data storage system in Huawei's video surveillance cloud storage solution incorporates the storage, analysis, and retrieval functions for mass data and provides tailored data redundancy technologies at the network layer to build a virtual resource pool, facilitating resource management and maintenance. With the brilliant convergence of cloud computing, Big Data analytics, and massive storage, this solution is ideal for any Safe City rollout, both now and well into the future.