Scalability issues for mass multimedia storage systems

Pegler, D. and Hutchison, D. and Shepherd, D. (1996) Scalability issues for mass multimedia storage systems. IEE Colloquium (Digest) (206). pp. 1-2. ISSN 0963-3308

Abstract

The storage and bandwidth requirements of digital video and audio exceed those that can be supported by conventional file servers. Despite the emergence of new compression algorithms capable of providing extremely high compression ratios, there is still a challenge to provide optimised storage services capable of storing 1000s of hours of multimedia data and providing simultaneous access to hundreds and potentially thousands of clients. A number of prototype multimedia storage devices have been designed and built to address the real-time demands of digital video and audio1,9. These systems use RAID (Redundant Arrays of Inexpensive Disks) technology to increase the bandwidth and storage capacities of single disks. Unfortunately, RAID is non-scalable10 (i.e. disks cannot be incrementally added to provide higher bandwidth and storage facilities) and can prove to be a bottleneck in the system. This server bottleneck only becomes significant when the aggregate disk bandwidth of RAID increases beyond either the network interface or the server CPU / bus speed. Once this occurs the available storage and bandwidth can only be increased by introducing more RAID servers and introducing a load balancing policy where client requests can be shared across multiple server instances. This paper describes a Scalable Multimedia Storage Architecture (SMSA) that supports wide area storage, storage server scalability allowing the addition of extra storage nodes, and maximised available data streams through the use of the load balancing techniques of network striping / file replication. It also allows for the storage of multi-resolution data produced by scalable compression techniques to match the Quality of Service requirements of heterogeneous clients.