Chapter 1 Multimedia Communications 1
1.1 Introduction 1
1.2 Multimedia Communication Model 4
1.3 Elements of Multimedia Systems 4
1.4 User Requirements 5
1.5 Network Requirements 7
1.6 Packet Transfer Concept 8
1.7 Multimedia Requirements and ATM Networks 9
1.8 Multimedia Terminals 10
1.9 Concluding Remarks 12
Chapter 2 Audio-Visual Integration 13
2.1 Introduction 13
2.2 Media Interaction 14
2.3 Bimodality of Human Speech 16
2.4 Lip Reading 18
2.5 Speech-Driven Talking Heads 21
2.6 Lip Synchronization 23
2.7 Lip Tracking 24
2.8 Audio-to-Visual Mapping 27
2.8.1 Classification-Based Conversion 27
2.8.2 HMM for Audio-to-Visual Conversion 28
2.8.3 Audio and Visual Integration for Lip-Reading Applications 28
2.8.4 Audio-Visual Information Preprocessing 29
2.8.5 Pattern-Recognition Strategies 29
2.8.6 Integration Strategy 29
2.9 Bimodal Person Verification 30
2.10 Joint Audio-Video Coding 31
2.11 Concluding Remarks 32
Chapter 3 Multimedia Processing in Communications 35
3.1 Introduction 37
3.2 Digital Media 37
3.3 Signal-Processing Elements 40
3.4 Challenges of Multimedia Information Processing 42
3.4.1 Pre and Postprocessing 44
3.4.2 Speech,Audio and Acoustic Processing for Multimedia 44
3.4.3 Video Signal Processing 46
3.4.4 Content-Based Image Retrieval 48
Texture-Based Methods 48
Shape-Based Methods 49
Color-Based Methods 49
3.5 Perceptual Coding of Digital Audio Signals 51
3.5.1 General Perceptual Audio-Coding Architecture 51
3.5.2 Review of Psychoacoustic Fundamentals 53
Absolute Threshold of Hearing 53
Critical Band Frequency Analysis 54
Simultaneous Masking and the Spread of Masking 55
Temporal Masking 56
PE 57
3.6 Transform Audio Coders 58
3.6.1 Optimum Coding in the Frequency Domain 59
3.6.2 Perceptual Transform Coder 60
3.6.3 Hybrid Coder 61
3.6.4 Transform Coding Using DFT Interblock Redundancy 62
3.6.5 ASPEC 62
3.6.6 Differential Perceptual Audio Coder 63
3.6.7 DFT Noise Substitution 64
3.6.8 DCT with Vector Quantization 65
3.6.9 MDCT 65
3.6.10 MDCT with VQ 66
3.7 Audio Subband Coders 66
3.7.1 Wavelet Decompositions 67
3.7.2 DWT-based Subband Coders 69
3.8 Speech Coder Attributes 69
3.9 CD Audio Coding for Multimedia Applications 71
3.10 Image Coding 71
3.11 Video Coding 74
3.11.1 TC and Subband Coding(SBC) 74
3.11.2 Predictive Coding 75
3.11.3 Motion-Compensated Video Coding 76
3.12 Watermarking 78
3.12.1 Watermarking Techniques 80
3.12.2 Main Features of Watermarking 81
3.12.3 Application Domains 83
3.13 Organization,Storage and Retrieval Issues 85
3.13.1 Streaming Issues for Speech and Audio 85
3.13.2 Streaming Issues for Video 87
3.14 Signal Processing for Networked Multimedia 88
3.15 NNs for Multimedia Processing 89
3.15.1 NNs for Optimal Visualization 90
3.15.2 Neural Techniques for Motion Estimation 91
3.15.3 NN Application to Face Detection and Recognition 91
3.15.4 Personal Authentication by Fusing Image and Speech 92
3.15.5 Subject-Based Retrieval for Image and Video Databases 93
3.15.6 Face-Based Video Indexing and Browsing 94
3.16 Multimedia Processors 94
3.16.1 Image-Processing Hardware and Software 95
3.16.2 Multimedia Processors Classification 96
3.16.3 General Purpose Microprocessors 98
3.16.4 Microprocessors for Embedded Applications 101
3.17 Concluding Remarks 102
Chapter4 Distributed Multimedia Systems 105
4.1 Introduction 105
4.2 Main Features of a DMS 107
4.3 Resource Management of DMS 107
4.4 Networking 108
4.4.1 Ip Networking 109
IP Multicast 110
Resource Reservation Protocol(RSVP) 111
RTP 112
4.4.2 Integrated Management Architecture for IP-Based Networks 112
Performance Management 114
Fault Management 114
Configuration Management 115
Security Management 115
Accounting and Billing Management 116
4.4.3 ATM 116
4.4.4 Integration of IP and ATM 119
4.4.5 Real-Time Multimedia over ATM(RMOA) 120
4.5 Multimedia Operating Systems 122
CPU Management 123
Memory Management 123
I/O Management 124
File System Management 124
4.6 Distributed Multimedia Servers 124
4.6.1 Multimedia Packing 125
4.7 Distributed Multimedia Applications 126
4.7.1 ITV 127
VoD 128
4.7.2 Telecooperation 131
Telecooperation Infrastructure 132
Telecooperative Applications 133
Telemedicine 135
4.7.3 Hypermedia Applications 135
Basic Features of a Hypermedia System 135
The Web 136
4.8 Concluding Remarks 137
Chapter 5 Multimedia Communication Standards 139
5.1 Intro duction 140
5.2 MPEG Approach to Multimedia Standardization 142
5.3 MPEG-1(Coding of Moving Pictures and Associated Audio) 146
5.3.1 The Basic MPEG-1 Interframe Coding Scheme 149
5.3.2 Conditional Replenishment 152
5.3.3 Specific Storage Media Functionalities 152
5.3.4 Rate Control 153
5.4 MPEG-2(Generic Coding of Moving Pictures and Associated Audio) 154
5.4.1 MPEG-2 Video 157
MPEG-2 Video-The Basics 158
MPEG-2 Video Syntax 163
MPEG-2 Video Scalability 163
MPEG-2 Video:Profiles and Levels 166
5.4.2 MPEG-2 Audio 167
5.4.3 MPEG-2 Systems 171
5.4.4 MPEG-2 DSM-CC 175
5.5 MPEG-4——Coding of Audiovisual Objects 176
5.5.1 Overview of MPEG-4:Motivations,Achievement,Process and Requirements 178
Media Objects 180
MPEG-4 Version 1 182
MPEG-4 Version 2 183
Extensions to MPEG-4 Beyond Version 2 188
Profiles in MPEG-4 189
Verification Testing:Checking MPEG s Performance 192
MPEG-4 Standardization Process 193
Requirements for MPEG-4 194
5.5.2 MPEG-4 Systems 195
MPEG-4 Systems Architecture 196
Elementary Stream Management(ESM) 199
Auxiliary Descriptors und Streams 201
Structuring Content by Grouping of Streams 203
Managing Content Complexity 203
Distributed Content-Handling Considerations 204
System Decoder Model(SDM)for ES Synchronization 204
MPEG-4 Systems BIFS 205
5.5.3 DMIF 210
DMIF Computational Model 213
5.5.4 MPEG-4 Video 214
Shape-Coding Tools for MPEG-4 Natural Video 216
Motion Estimation and Campensation 217
Texture-Coding Tools 218
Multifunctional Coding 220
Sprite Coding 221
Scalability 221
Error Resilience 223
Relationship Between Natural and Synthetic Video Coding 223
Synthetic Images 225
Integration of Face Animation with Natural Video 226
FAPs 227
Face Model 232
Coding of FAPs 233
FIT 235
Integration of Face Animation and Text-to-Speech(TTS) Synthesis 235
BIFS for Facial Animation 236
2D Mesh Coding 237
VO Tracking 238
2D-Mesh Object Encoder/Decoder 239
5.5.5 MPEG-4 Audio 243
MPEG-4 Natural Audio Coding 244
General Audio Coding(Advanced Audio Coding Based) 244
Twin VQ 248
Speech Coding in MPEG-4 Audio 248
Scalability in MPEG-4 Natural Audio 252
Synthetic Audio in MPEG-4 252
Audio BIFS 255
5.5.6 Profiles and Levels in MPEG-4 256
Visual Object Types 256
Visual Profiles 259
Audio Object Types 260
Audio Profiles 261
Graphics 261
Systems Profiles 261
5.6 MPEG-4 Visual Texture Coding(VTC) and JPEG 2000 Image Compression Standards 262
5.6.1 JPEG 2000 Development Process 263
5.6.2 Overview of Still-Image Coding Standards 267
MPEG-4 VTC 267
JPEG 267
PNG 268
5.6.3 Significant Features of JPEG 2000 268
Region of Interest(ROI)Coding 268
Scalability 268
Error Resilience 269
IPRs 269
5.6.4 Architecture of JPEG 2000 269
5.6.5 JPEG 2000 Bit Stream 275
5.6.6 Compression Efficiency Comparisons 277
Error Resilience 281
5.7 MPEG-7 Standardization Process of Multimedia Content Description 282
5.7.1 Objective of the MPEG-7 Standard 282
5.7.2 Status of the MPEG-7 Standard 285
5.7.3 Major Functionalities in MPEG-7 286
MPEG-7 Systems 286
MPEG-7 DDL 289
MPEG-7 Audio 289
MPEG-7 Visual 290
MPEG-7 MMDSs 297
MPEG-7 Reference Software(XM) 299
MPEG-7 Conformance 299
5.7.4 Applications Enabled by MPEG-7 300
5.8 MPEG-21 Multimedia Framework 301
5.8.1 Audiovisual Content Representation Issues 303
5.8.2 Description of a Multimedia Framework Architecture 304
MPEG-21 Digital Item Declaration 304
Content Representation 305
Digital Item Identification and Description 305
Content Management and Usage 305
Intellectual Property Management and Protection 305
Terminals and Networks 306
Event Reporting 306
5.8.3 Requirements for Digital Item Declaration 306
5.9 ITU-T Standardization of Audiovisual Communication Systems 308
5.9.1 ITU-T Standardization Process 308
5.9.2 Audiovisual Systems(H.310,H.320,H.321,H.322,H.323,and H.324) 310
H.320 Standard 312
Standards for Audiovisual Services Across ATM H.310 and H.321 312
Standard H.322——Guaranteed Qos LAN Systems 315
ITU-T H.323 Standard 317
H.324 Standard 319
5.9.3 Video-Coding Standards(H.261,H.263 and H.26L) 319
H.261 Standard 319
H.263 Standard 323
H.263+(H.263 Version 2)Standard 327
H.263++ Standard Development 332
H.26L Standard 333
5.9.4 ITU-T Speech-Coding Standards 336
Bit Rate 338
Delay 338
Complexity 339
Quality 339
5.9.5 Multimedia Multiplex and Synchronization Standards 340
ITU-T Recommendation H.221 341
ITU-T Recommendation H.223 341
ITU-T Recommendation H.225 341
Common Control Protocol H.245 343
5.10 IETF and Internet Standards 344
5.10.1 IETF Standardization Process 344
5.10.2 Internet Network Architecture 347
5.10.3 Internet Protocols 348
Classical IP Stack 350
IP Version 6 351
Priority Field 352
Folw Label 353
IPv6 Addresses 353
Hop-by-Hop Options Header 355
Fragment Header 356
Routing Header 356
IPv6 Security 357
5.10.4 Real-Time Multimedia Transmission Across the Internet 359
Signaling 360
Session Control 360
Transport 361
Network Infrastructure 363
Multimedia Data for Network Use 364
5.10.5 MPEG-4 Video Transport Across the Internet 365
Use of RTP 365
System Architecture 366
MPEG-4 Server 368
MPEG-4 Client 369
5.11 Concluding Remarks 370
Chapter 6 Multimedia Communications Across Networks 373
6.1 Packet Audio/Video in the Network Environment 373
6.1.1 Packet Voice 374
6.1.2 Integrated Packet Networks 377
6.1.3 Packet Video 380
6.2 Video Transport Across Generic Networks 382
6.2.1 Layered Video Coding 386
Layered Compression 386
Layered Transmission 388
6.2.2 Error-Resilient Video Coding Techniques 388
Error-Resilient Encoding 390
Decoder Error Concealment 392
Error-Resilient Entropy Code 393
6.2.3 Scalable Rate Control 394
Rate Control Techniques 396
Theoretical Foundation of the SRC 397
6.2.4 Streaming Video Across the Internet 400
Video Compression 401
Requirements Imposed by Streaming Applications 403
Application Layer Qos Control 404
Continuous Media Distribution Services 407
Streaming Servers 409
Media Synchronization 409
Protocols for Streaming Video 410
6.3 Multimedia Transport Across ATM Networks 411
6.3.1 MUltiplexing in ATM Networks 412
6.3.2 Video Delay in ATM Networks 413
6.3.3 Errors and Losses in ATM 417
6.3.4 MPEG Video Error Concealment 420
6.3.5 Loss Concealment 420
6.3.6 Video Across WATM Networks 421
6.3.7 Heterogeneous Networking 422
6.4 Multimedia Across IP Networks 424
6.4.1 Video Transmission Across IP Networks 424
6.4.2 Traffic Specification for MPEG Video Transmission on the Internet 426
6.4.3 Bandwidth Allocation Mechanism 427
6.4.4 Fine-Grained Scalable Video Coding for Multimedia Across IP 428
6.5 Multimedia Across DSLs 432
6.5.1 VoDSL Architecture 433
6.5.2 Delivering Voice Services Across DSL 438
6.5.3 MUltimedia Across ADSL 439
Serial Transmission:TDM 439
Parallel Transmission Frequency Division Multiplexing 441
6.6 Internet Access Networks 441
6.6.1 DSL Networks 443
6.6.2 Cable Access Networks 445
6.6.3 Fixed Wireless Routed for Internet Access 447
6.7 Multimedia Across Wireless 449
6.7.1 Wireless Broadband Communication System(WBCS)for Multimedia 451
6.7.2 Audiovisual Solutions for Wireless Communications 453
6.7.3 Mobile Networks 458
Speech Transmission in GSM 459
Video Across GSM 460
Mobile ATM 462
Mobile IP 462
Wireless Multimedia Delivery 467
SIP in Mobile Environment 469
Multicast Routing in Cellular Networks 470
Broadband Wireless Mobile 471
6.7.4 Broadcasting Networks 473
Digital Video Broadcasting(DVB) 477
Data Transmission Using MPEG-2 and DVB 478
MPEG Program Stream 480
MPEG Transport Stream 481
Broadband Multimedia Satellite Systems 484
Multimedia Home Platform 486
Multimedia Car Platform 487
6.8 Digital Television Infrastructure for Interactive Multimedia Services 488
Interactive Broadcast Data(IDB) Services 490
Data Carousel Concept 492
6.9 Concluding Remarks 493
References 497
Index 537
About the Authors 545