Multimedia: Computing, Communications, and Applications

Ralf Steinmetz and Klara Nahrstedt

Prentice Hall, 1995

Introduction

Multimedia means, from the user's perspective, that computer information can be represented through audio and/or video, in addition to text, image, graphics and animation. For example, using audio and video, a variety of dynamic situations in different areas, such as sport or ornithology lexicon, can often be presented better than just using text and image alone.

The integration of these media into the computer provides additional possibilities for the use of computational power currently available (e.g., for interactive presentation of huge amounts of information). Furthermore, these data can be transmitted through computer and telecommunication networks, which implies applications in the areas of information distribution and cooperative work. Multimedia provides the possibility for a spectrum of new applications, many of which are in place today. On the other hand, one also has to keep in mind the problems of global communication, with its social and legal implications. However, these issues will not be discussed in this book.

One of the first and best-known institutes working on different aspects of multimedia is the MIT Media Lab in Boston. Research is going on there on a variety of future applications, such as personal newspaper and holography. In the meantime, many research institutes, universities and computer and telecommunication companies work in the multimedia area.

Branch-overlapping Aspects of Multimedia

In addition to the strong interest in multimedia systems from the applications and technology viewpoints, one should not underestimate the ongoing migration process, and the evolution of the different industrial branches, as follows:
Telecommunication began primarily with the telephone. Telephone networks changed gradually into digital networks, which are similar to computer networks. In the early days, intermediate switching systems had mechanical switching dials. Today, computers do the job. Now the telephone is becoming more and more of a computer, or is part of a computer (e.g., using an ISDN card).
Consumer electronics contributed massively to the reduction of the price of video support in computers. Similarly, optical disc technology in computing is dependent on the success of the CD player. Consequently, the same companies very often produce CD drives for computer and stereo-systems, or both television and computer monitors.
Recording studios and TV producers are pioneers in dealing with professional audio and video equipment. Today, professional systems are available for the digital cutting of TV movies. Some of these systems are standard computers extended through additional special cards. These information providers convey data through cable, satellite and plain old antennas, which will further allow them to serve as information providers via computer networks in the future.
Many of the large publishing houses already offer their publications in electronic form. Further, there are many close relations between publishing houses and movie companies. These branches offer gradually more and more multimedia information.

This short overview shows that the different branches grow together because of coming multimedia technology and applications. However, to allow multimedia applications, many software and hardware components have to be adapted, extended or replaced.

>From the technical perspective, besides handling the huge amount of data, the timing requirements among all components of the data computation is the major challenge. Traditional data computation tries to finish its task as soon as possible. Real-time systems must work internally within given time bounds, mostly as error-tolerant systems. The fault tolerance in multimedia is generally not the most important aspect.

Another challenge is the integration requirement of different types of media in a multimedia application. In such applications, the traditional media (e.g., text, image) as well as the continuous media (e.g., video, audio), must be processed. Moreover, if a timing requirement is set by a multimedia application, it should hold for both classes (traditional and continuous media) to achieve the timing specification of the application. These media are not independent of each other and therefore the integration requires concepts, which are more complex than just the integration of current concepts. In an integrated system, different components have to process both kinds of data, and moreover, different relations can occur in the form of synchronization among the media.

The notion of multimedia is often defined very differently in the literature in comparison to our (above) description. There is some need for clarification. The technology connections and binding different components, were considered only partially and in isolation from each other. Based on [Steinmetz93], we wrote this book to provide an integrated, consistent and total view.

Content

This book has the character of a reference book , covering numerous areas and allowing the reader to learn about a topic of interest without having previously studied extensively or having read areas previously covered in this book. Strong connections are provided among the different areas of this book through its global structure, which is shown in Figure 1. The results presented in this book serve as basis for the development of individual components of a multimedia system and suggest some general parameters one must keep in mind.

Global Structure

This book aims to achieve a complete and balanced view. Figure 1 shows the global view of this book with the main topics covered in it. Figure 1 was developed after many iterations of topic structuring. It shows schematically the main fields of multimedia systems. The basic idea of the figure is to express the interactions among the components through spatial proximity.

The following areas can be distinguished:

Device Domain:

Basic concepts for the processing of digital audio and video data are based on digital signal processing. Hence, these concepts are described and some possible practical implementations are presented. Different methods for the processing of image , graphics and animation are described. The audio techniques section includes music (MIDI) and speech processing. The understanding of video techniques is built mainly on TV development, including digital representation and HDTV. The originated data rates of these media demand, because of the current quality requirements and available technology, corresponding compression methods. The corresponding hardware and some software are briefly described.

The diminishing cost of optical storage space has contributed significantly to the current development of computer technology. Almost all developments are based on CD-DA (Compact Disc-Digital Audio), known from home electronics.

On the other hand, networks, with their higher bandwidth and their capacity for transmitting all media types, have led to networked multimedia systems . Not such a long time ago, local and distributed multimedia systems consisted of a set of external analog devices controlled by a computer. Today, development tends toward full digital working systems.

System Domain:

The interface between the device domain and the system domain is specified by the computer technology . To utilize the device domain, several system services are needed. Basically, three services exist. These services are mostly implemented in software:

  • The operating system serves as an interface between computer hardware/system software and all other software components. It provides the user with a programming and computational environment, which should be easy to operate. In its function as an interface, the operating system provides different services that relate to the computer resources, such as: processor, main memory, secondary storage, input and output devices and network.
  • The database system allows a structured access to data and a management of large databases.
  • The communication system is responsible for data transmission according to the timing and reliability requirements of the networked multimedia application.

    Application Domain

    The services of the system domain are offered to the application domain through proper programming abstractions . Moreover, such abstractions can be, for example, part of a multimedia operating system, programming language or object-oriented class hierarchy.

    Another topic embedded in the application domain is document handling . A document consists of a set of structured information, represented in different media, and generated or recorded at the time of presentation. Many functions of document handling and other applications are accessible and presented to the user through a user interface .

    Cross Domain

    It turns out that some aspects, such as synchronization aspects, are difficult to locate in one or two components or domains. The reason is that synchronization , being the temporal relationship among various media, relates to many components across all domains.

    References

  • Ralf Steinmetz, ``Multimedia Technology: Introduction and Fundamentals (in German), Springer Verlag, 1993