1 Introduction 1
1．1 Branch-overlapping Aspects of Multimedia 2
1．2 Content 3
1．3 Global Structure 4
1．4 Multimedia Literature 6
2 Multimedia: Media and Data Streams 9
2．1 Medium 10
2．1．1 The Perception Medium 10
2．1．2 The Representation Medium 10
2．1．4 The Storage Medium 11
2．1．5 The Transmission Medium 11
2．1．3 The Presentation Medium 11
2．1．6 The Information Exchange Medium 12
2．1．7 Representation Values and Representation Spaces 12
2．1．8 Representation Dimensions 13
2．2 Main Properties of a Multimedia System 14
2．2．1 Multimedia System Definition 14
2．2．2 Combination of Media 15
2．2．3 Independence 15
2．2．4 Computer-supported Integration 16
2．2．5 Communication Systems 16
2．3 Multimedia 17
2．4．1 Asynchronous Transmission Mode 18
2．4 Traditional Data Streams Characteristics 18
2．4．2 Synchronous Transmission Mode 19
2．4．3 Isochronous Transmission Mode 19
2．5 Data Stream Characteristics for Continuous Media 20
2．5．1 The Time Interval Between a Complete Transmission of Consecutive Packets 20
2．5．2 Variation of Consecutive Packet Amount 21
2．5．3 Contiguous Packets 23
2．6 Information Units 25
3 Sound/Audio 27
3．1 Basic Sound Concepts 28
3．1．1 Computer Representation of Sound 29
3．1．2 Audio Formats 31
3．2 Music 32
3．2．1 MIDI Basic Concepts 32
3．2．2 MIDI Devices 34
3．2．3 MIDI Messages 36
3．2．4 MIDI and SMPTE Timing Standards 38
3．2．5 MIDI Software 39
3．3 Speech 41
3．3．1 Speech Generation 42
3．3．2 Speech Analysis 47
3．3．3 Speech Transmission 51
4 Images and Graphics 55
4．1．1 Digital Image Representation 56
4．1 Basic Concepts 56
4．1．2 Image Format 57
4．1．3 Graphics Format 59
4．2 Computer Image Processing 61
4．2．1 Image Synthesis 61
4．2．2 Image Analysis 68
4．2．3 Image Transmission 78
4．3 Comments 79
5 Video and Animation 81
5．1 Basic Concepts 81
5．1．1 Video Signal Representation 81
5．1．2 Computer Video Format 91
5．2．1 Conventional Systems 93
5．2 Television 93
5．2．2 Enhanced Definition Systems 95
5．2．3 High-Definition Systems 98
5．2．4 Transmission 101
5．3 Computer-based Animation 103
5．3．1 Basic Concepts 104
5．3．2 Animation Languages 106
5．3．3 Methods of Controlling Animation 108
5．3．4 Display of Animation 110
5．3．5 Transmission of Animation 111
5．3．6 Comments 112
6．1 Storage Space 113
6 Data Compression 113
6．2 Coding Requirements 114
6．3 Source, Entropy and Hybrid Coding 118
6．4 Some Basic Compression Techniques 121
6．5 JPEG 130
6．5．1 Image Preparation 132
6．5．2 Lossy Sequential DCT-based Mode 137
6．5．3 Expanded Lossy DCT-based Mode 142
6．5．4 Lossless Mode 144
6．5．5 Hierarchical Mode 145
6．6 H.261（px64） 146
6．6．1 Image Preparation 147
6．6．2 Coding Algorithms 148
6．6．3 Data Stream 149
6．7 MPEG 150
6．7．1 Video Encoding 151
6．7．2 Audio Encoding 156
6．7．3 Data Stream 157
6．7．4 MPEG-2 160
6．7．5 MPEG-4 164
6．8 DVI 165
6．8．1 Audio and Still Image Encoding 166
6．8．2 Video Encoding 168
6．8．3 Data Stream 171
6．9 Comments 172
7 Optical Storage Media 175
7．1 History 176
7．2 Basic Technology 177
7．3 Video Disks and Other WORMs 180
7．4 Compact Disk Digital Audio 181
7．4．1 Preliminary Technical Background 181
7．4．2 Eight-to-Fourteen Modulation 183
7．4．3 Error Handling 184
7．4．4 Frames, Tracks, Areas and Blocks of a CD-DA 185
7．5 Compact Disk Read Only Memory 188
7．5．1 Blocks 188
7．4．5 Advantages of Digital CD-DA Technology 188
7．5．2 Modes 189
7．5．3 Logical Data Format 192
7．5．4 Limitations of the CD-ROM Technology 193
7．6 CD-ROM Extended Architecture 194
7．6．1 Form 1 and Form 2 194
7．6．2 Compressed Data of Different Media 196
7．7 Further CD-ROM-based Developments 197
7．7．1 Compact Disk Interactive 197
7．7．2 Compact Disk Interactive Ready Format 201
7．7．3 Compact Disk Bridge Disk 201
7．7．4 Photo Compact Disk 202
7．7．5 Digital Video Interactive 203
7．8 Compact Disk Write Once 204
7．8．1 Principle of the CD-WO 204
7．8．2 Sessions 205
7．9 Compact Disk Magneto Optical 207
7．9．1 Principle of the Magnetic-Optical Method 207
7．9．2 Areas of the CD-MO 207
7．10 The Prospects of CD Technologies 208
8 Computer Technology 211
8．1 Communication Architecture 212
8．1．1 Hybrid Systems 213
8．1．2 Digital Systems 217
8．2 Multimedia Workstation 219
8．3 Comments 223
9 Multimedia Operating Systems 225
9．1 Introduction 225
9．2 Real Time 227
9．2．1 The Notion of Real-Time 227
9．2．2 Real Time and Multimedia 230
9．3 Resource Management 232
9．3．1 Resources 233
9．3．2 Requirements 234
9．3．3 Components and Phases 235
9．3．4 Allocation Scheme 237
9．3．5 Continuous Media Resource Model 238
9．4 Process Management 244
9．4．1 Real Time Process Management in Conventional Operating Systems: An Example 245
9．4．2 Real-time Processing Requirements 248
9．4．3 Traditional Real -time Scheduling 249
9．4．4 Real-time Scheduling: System Model 250
9．4．5 Earliest Deadline First Algorithm 252
9．4．6 Rate Monotonic Algorithm 254
9．4．7 EDF and Rate Monotonic：Context switches 256
9．4．8 EDF and Rate Monotonic: Processor Utilizations 256
9．4．9 Extensions to Rate Monotonic Scheduling 259
9．4．10 Other Approaches for In-Time Scheduling 260
9．4．11 Preemptive versus Non-preemptive Task Scheduling 262
9．4．12 Scheduling of Continuous Media Tasks: Prototype Operating Systems 265
9．5 File Systems 268
9．5．1 Traditional File Systems 269
9．5．2 Multimedia File Systems 276
9．6 Additional Operating System Issues 293
9．6．1 Interprocess Communication and Synchronization 293
9．6．2 Memory Management 294
9．6．3 Device Management 295
9．7 System Architecture 298
9．7．1 UNLX-based Systems 302
9．7．2 QuickTime 302
9．7．3 Windows Multimedia Extensions 306
9．7．4 OS/2 Multimedia Presentation Manager/2 308
9．8 Concluding Remarks 310
10 Networking Systems 313
10．1 Layers, Protocols and Services 313
10．2 Networks 319
10．3 Local Area Networks（LANs） 319
10．3．1 High-speed Ethernet 319
10．3．2 Token Ring 322
10．3．3 FDDI 330
10．3．4 Local ATM Networks 344
10．4 Metropolitan Area Networks（MANs） 355
10．4．1 Distributed Queue Dual Bus（DQDB） 356
10．4．2 Orwell 362
10．4．3 MAN Connectivity to ATM Networks 363
10．5 Wide Area Networks（WANs） 365
10．5．1 Traditional WAN S 366
10．5．2 B-ISDN:ATM 371
10．6 Conclusion 380
11 Multimedia Communication Systems 383
11．1 Application Subsystem 384
11．1．1 Collaborative Computing 384
11．1．2 Session Management 397
11．2 Transport Subsystem 401
11．2．1 Requirements 401
11．2．2 Transport Layer 404
11．2．3 Network Layer 410
11．3 Quality of Service and Resource Management 418
11．3．1 Basic Concepts 418
11．3．2 Establishment and Closing of the Multimedia Call 424
11．3．3 Managing Resources during Multimedia Transmission 440
11．3．4 Architectural Issues 457
11．4 Comments 459
11．4．1 Trends in Collaborative Computing 459
11．4．2 Trends in Transport Systems 462
12 Database Systems 463
12．1 Multimedia Database Management System 464
12．2 Characteristics of an MDBMS 465
12．3 Data Analysis 469
12．4 Data Structure 470
12．4．1 Raw Data 470
12．4．2 Registering Data 471
12．4．3 Descriptive Data 471
12．4．4 Examples of Multimedia Structures 471
12．4．5 Comments on Data Analysis 474
12．5 Operations on Data 474
12．6 Integration in a Database Model 476
12．6．1 Relational Database Model 477
12．6．2 Object-oriented Database Model 478
12．7 Comments 479
13 Documents, Hypertext and MHEG 481
13．1 Documents 481
13．1．1 Document Architecture 482
13．1．2 Manipulation of Multimedia Data 483
13．2 Hypertext and Hypermedia 485
13．2．1 Hypertext, Hypermedia and Multimedia 486
13．2．2 Hypermedia Systems: An Example 491
13．2．3 History 499
13．2．4 Systems: Architecture, Nodes and Pointers 502
13．2．5 Some Final Comments about Hypertext Systems 507
13．3 Document Architecture SGML 510
13．3．1 Some Details 511
13．3．2 SGML and Multimedia 514
13．3．3 Closing Comments about SGML 515
13．4 Document Architecture ODA 516
13．4．1 Some Details on ODA 516
13．4．2 ODA and Multimedia 523
13．5 MHEG 527
13．5．1 Example of an Interactive Multimedia Presentation 528
13．5．2 Derivation of a Class Hierarchy 530
13．5．3 Contents 532
13．5．4 Behavior 534
13．5．5 User Interaction 537
13．5．6 Container 539
13．5．7 Closing Comments 541
14 User Interfaces 543
14．1 General Design Issues 544
14．1．1 Architectural Issues 545
14．1．2 Information Characteristics for Presentation 545
14．1．3 Presentation Function 547
14．1．4 Presentation Design Knowledge 547
14．1．5 Effective Human-Computer Interaction 549
14．2 Current Work 549
14．6．2 Context-sensitive Help Functions 550
14．3 Extension through Video and Audio 551
14．4 Video at the User Interface 552
14．4．2 Example: Remote Camera Control Application 553
14．4．1 Hardware for Visualization of Motion Pictures 553
14．5 Audio at the User Interface 556
14．6 User-friendliness as the Primary Goal 557
14．6．1 Easy to Learn Instructions 559
14．6．3 Easy to Remember Instructions 559
14．6．4 Effective Instructions 560
14．6．5 Aesthetics 560
14．6．6 Effective Implementation Support 561
14．6．7 Entry Elements 561
14．6．9 Presentation 562
14．6．8 Meaningful Location of Functions 562
14．6．10 Dialogue Boxes 563
14．6．11 Additional Design Criteria 563
14．6．12 Design-specific Criteria 564
14．7 Comments 566
15 Synchronization 567
15．1 Introduction 567
15．2 Notion of Synchronization 570
15．2．1 Multimedia Systems 570
15．2．2 Basic Synchronization Issues 572
15．2．3 Intra-and Inter-object Synchronization 574
15．2．4 Live and Synthetic Synchronization 581
15．3 Presentation Requirements 585
15．2．5 Comment 585
15．3．1 Lip Synchronization Requirements 588
15．3．2 Pointer Synchronization Requirements 593
15．3．3 Elementary Media Synchronization 595
15．4 A Reference Model for Multimedia Synchronization 599
15．4．1 Existing Classification Approaches 600
15．4．2 The Synchronization Reference Model 601
15．4．3 Synchronization in a Distributed Environment 608
15．4．4 Aggregate Characteristics of the Synchronization Reference Model 615
15．5 Synchronization Specification 615
15．5．1 Quality of Service 618
15．5．2 Multimedia Synchronization Specification Methods 624
15．5．3 Interval-based Specifications 625
15．5．4 Axes-based Synchronization 628
15．5．5 Control Flow-based Specification 631
15．5．6 Event-based Synchronization 639
15．5．7 Scripts 640
15．5．8 Comment 644
15．6 Case Studies 645
15．6．1 Synchronization in MHEG 645
15．6．2 HyTime 648
15．6．3 Firefly System 652
15．6．4 MODE 656
15．6．5 Multimedia Tele-orchestra 660
15．6．6 Littles Framework 663
15．6．7 ACME 665
15．6．8 Further Synchronization-related Systems 666
15．6．9 Comment 668
15．7 Summary and Outlook 668
15．7．1 Summary 668
15．7．2 Future Topics 670
15．7．3 Conclusion 670
16 Abstractions for Programming 671
16．1 Abstraction Levels 672
16．2 Libraries 674
16．3 System Software 675
16．3．1 Data as Time Capsules 676
16．3．2 Data as Streams 677
16．4 Toolkits 678
16．5 Higher Programming Languages 679
16．5．1 Media as Types 679
16．5．2 Media as Files 683
16．5．3 Media as Processes 684
16．5．4 Programming Language Requirements 686
16．6 Object-oriented Approaches 688
16．6．1 Application-specific Metaphors as Classes 691
16．6．2 Application-generic Metaphors as Classes 691
16．6．3 Devices as Classes 692
16．6．4 Processing Units as Classes 693
16．6．5 Media as Classes 702
16．6．6 Communication -specific Metaphors as Classes 705
16．7 Comments 706
17 Multimedia Applications 709
17．1 Introduction 709
17．1．1 Programs 710
17．1．2 Structure 712
17．2 Media Preparation 713
17．2．1 Means 713
17．2．2 Remarks on the Current Status 717
17．3 Media Composition 717
17．3．1 Text and Graphics Editors 718
17．3．2 Image Editors 720
17．3．3 Animation Editors 722
17．3．4 Sound Editors 723
17．3．5 Video Editors 724
17．4 Media Integration 725
17．4．1 Multimedia Editors 725
17．4．2 Hypermedia/Hypertext Editors 727
17．4．3 Authoring Tools 728
17．5 Media Communication 731
17．5．1 Tele-Services 732
17．5．2 Implementation of Conversational Services 737
17．5．3 Implementation of Messaging Services 740
17．5．4 Implementation of Retrieval Services 741
17．5．5 Implementation of Tele-action Services 745
17．5．6 Implementation of Tele-operation Services 747
17．5．7 Applications of Tele-services 747
17．6 Media Consumption 752
17．6．1 Viewing Multimedia Documents 753
17．6．2 Books, Proceedings and Newspapers 754
17．6．3 Kiosks 755
17．6．4 Tele-shopping 756
17．7 Media Entertainment 756
17．7．1 Virtual Reality 757
17．7．2 Interactive Video 759
17．7．3 Interactive Audio 761
17．7．4 Games 762
17．8 Trends 764
18 Future Directions 767
18．1 Where Are We Today? 767
18．1．1 User Interface 767
18．1．2 Operating Systems 768
18．1．3 Multimedia Documents 768
18．1．4 Synchronization 769
18．1．5 Programming 769
18．2 What Are the Next Steps? 770
18．2．1 Devices 771
18．2．2 Visualization 774
18．2．3 Mobility 774
18．2．4 Interactivity 775
18．2．5 Operating Systems 775
18．2．6 Further Issues in Virtual Environments 776
18．2．7 Multimedia User Interface 776
18．2．8 Hypermedia 777
18．2．9 Multimedia Applications 777
18．3 What Are the Multimedia Research Issues? 779
A Abbreviations 781
Bibliography 791
Index 842