多媒体通信系统PDF电子书下载

Chapter 1 Multimedia Communications 1

1.1 Introduction 1

1.2 Multimedia Communication Model 4

1.3 Elements of Multimedia Systems 4

1.4 User Requirements 5

1.5 Network Requirements 7

1.6 Packet Transfer Concept 8

1.7 Multimedia Requirements and ATM Networks 9

1.8 Multimedia Terminals 10

1.9 Concluding Remarks 12

Chapter 2 Audio-Visual Integration 13

2.1 Introduction 13

2.2 Media Interaction 14

2.3 Bimodality of Human Speech 16

2.4 Lip Reading 18

2.5 Speech-Driven Talking Heads 21

2.6 Lip Synchronization 23

2.7 Lip Tracking 24

2.8 Audio-to-Visual Mapping 27

2.8.1 Classification-Based Conversion 27

2.8.2 HMM for Audio-to-Visual Conversion 28

2.8.3 Audio and Visual Integration for Lip-Reading Applications 28

2.8.4 Audio-Visual Information Preprocessing 29

2.8.5 Pattern-Recognition Strategies 29

2.8.6 Integration Strategy 29

2.9 Bimodal Person Verification 30

2.10 Joint Audio-Video Coding 31

2.11 Concluding Remarks 32

Chapter 3 Multimedia Processing in Communications 35

3.1 Introduction 37

3.2 Digital Media 37

3.3 Signal-Processing Elements 40

3.4 Challenges of Multimedia Information Processing 42

3.4.1 Pre and Postprocessing 44

3.4.2 Speech,Audio and Acoustic Processing for Multimedia 44

3.4.3 Video Signal Processing 46

3.4.4 Content-Based Image Retrieval 48

Texture-Based Methods 48

Shape-Based Methods 49

Color-Based Methods 49

3.5 Perceptual Coding of Digital Audio Signals 51

3.5.1 General Perceptual Audio-Coding Architecture 51

3.5.2 Review of Psychoacoustic Fundamentals 53

Absolute Threshold of Hearing 53

Critical Band Frequency Analysis 54

Simultaneous Masking and the Spread of Masking 55

Temporal Masking 56

PE 57

3.6 Transform Audio Coders 58

3.6.1 Optimum Coding in the Frequency Domain 59

3.6.2 Perceptual Transform Coder 60

3.6.3 Hybrid Coder 61

3.6.4 Transform Coding Using DFT Interblock Redundancy 62

3.6.5 ASPEC 62

3.6.6 Differential Perceptual Audio Coder 63

3.6.7 DFT Noise Substitution 64

3.6.8 DCT with Vector Quantization 65

3.6.9 MDCT 65

3.6.10 MDCT with VQ 66

3.7 Audio Subband Coders 66

3.7.1 Wavelet Decompositions 67

3.7.2 DWT-based Subband Coders 69

3.8 Speech Coder Attributes 69

3.9 CD Audio Coding for Multimedia Applications 71

3.10 Image Coding 71

3.11 Video Coding 74

3.11.1 TC and Subband Coding(SBC) 74

3.11.2 Predictive Coding 75

3.11.3 Motion-Compensated Video Coding 76

3.12 Watermarking 78

3.12.1 Watermarking Techniques 80

3.12.2 Main Features of Watermarking 81

3.12.3 Application Domains 83

3.13 Organization,Storage and Retrieval Issues 85

3.13.1 Streaming Issues for Speech and Audio 85

3.13.2 Streaming Issues for Video 87

3.14 Signal Processing for Networked Multimedia 88

3.15 NNs for Multimedia Processing 89

3.15.1 NNs for Optimal Visualization 90

3.15.2 Neural Techniques for Motion Estimation 91

3.15.3 NN Application to Face Detection and Recognition 91

3.15.4 Personal Authentication by Fusing Image and Speech 92

3.15.5 Subject-Based Retrieval for Image and Video Databases 93

3.15.6 Face-Based Video Indexing and Browsing 94

3.16 Multimedia Processors 94

3.16.1 Image-Processing Hardware and Software 95

3.16.2 Multimedia Processors Classification 96

3.16.3 General Purpose Microprocessors 98

3.16.4 Microprocessors for Embedded Applications 101

3.17 Concluding Remarks 102

Chapter4 Distributed Multimedia Systems 105

4.1 Introduction 105

4.2 Main Features of a DMS 107

4.3 Resource Management of DMS 107

4.4 Networking 108

4.4.1 Ip Networking 109

IP Multicast 110

Resource Reservation Protocol(RSVP) 111

RTP 112

4.4.2 Integrated Management Architecture for IP-Based Networks 112

Performance Management 114

Fault Management 114

Configuration Management 115

Security Management 115

Accounting and Billing Management 116

4.4.3 ATM 116

4.4.4 Integration of IP and ATM 119

4.4.5 Real-Time Multimedia over ATM(RMOA) 120

4.5 Multimedia Operating Systems 122

CPU Management 123

Memory Management 123

I/O Management 124

File System Management 124

4.6 Distributed Multimedia Servers 124

4.6.1 Multimedia Packing 125

4.7 Distributed Multimedia Applications 126

4.7.1 ITV 127

VoD 128

4.7.2 Telecooperation 131

Telecooperation Infrastructure 132

Telecooperative Applications 133

Telemedicine 135

4.7.3 Hypermedia Applications 135

Basic Features of a Hypermedia System 135

The Web 136

4.8 Concluding Remarks 137

Chapter 5 Multimedia Communication Standards 139

5.1 Intro duction 140

5.2 MPEG Approach to Multimedia Standardization 142

5.3 MPEG-1(Coding of Moving Pictures and Associated Audio) 146

5.3.1 The Basic MPEG-1 Interframe Coding Scheme 149

5.3.2 Conditional Replenishment 152

5.3.3 Specific Storage Media Functionalities 152

5.3.4 Rate Control 153

5.4 MPEG-2(Generic Coding of Moving Pictures and Associated Audio) 154

5.4.1 MPEG-2 Video 157

MPEG-2 Video-The Basics 158

MPEG-2 Video Syntax 163

MPEG-2 Video Scalability 163

MPEG-2 Video:Profiles and Levels 166

5.4.2 MPEG-2 Audio 167

5.4.3 MPEG-2 Systems 171

5.4.4 MPEG-2 DSM-CC 175

5.5 MPEG-4——Coding of Audiovisual Objects 176

5.5.1 Overview of MPEG-4:Motivations,Achievement,Process and Requirements 178

Media Objects 180

MPEG-4 Version 1 182

MPEG-4 Version 2 183

Extensions to MPEG-4 Beyond Version 2 188

Profiles in MPEG-4 189

Verification Testing:Checking MPEG s Performance 192

MPEG-4 Standardization Process 193

Requirements for MPEG-4 194

5.5.2 MPEG-4 Systems 195

MPEG-4 Systems Architecture 196

Elementary Stream Management(ESM) 199

Auxiliary Descriptors und Streams 201

Structuring Content by Grouping of Streams 203

Managing Content Complexity 203

Distributed Content-Handling Considerations 204

System Decoder Model(SDM)for ES Synchronization 204

MPEG-4 Systems BIFS 205

5.5.3 DMIF 210

DMIF Computational Model 213

5.5.4 MPEG-4 Video 214

Shape-Coding Tools for MPEG-4 Natural Video 216

Motion Estimation and Campensation 217

Texture-Coding Tools 218

Multifunctional Coding 220

Sprite Coding 221

Scalability 221

Error Resilience 223

Relationship Between Natural and Synthetic Video Coding 223

Synthetic Images 225

Integration of Face Animation with Natural Video 226

FAPs 227

Face Model 232

Coding of FAPs 233

FIT 235

Integration of Face Animation and Text-to-Speech(TTS) Synthesis 235

BIFS for Facial Animation 236

2D Mesh Coding 237

VO Tracking 238

2D-Mesh Object Encoder/Decoder 239

5.5.5 MPEG-4 Audio 243

MPEG-4 Natural Audio Coding 244

General Audio Coding(Advanced Audio Coding Based) 244

Twin VQ 248

Speech Coding in MPEG-4 Audio 248

Scalability in MPEG-4 Natural Audio 252

Synthetic Audio in MPEG-4 252

Audio BIFS 255

5.5.6 Profiles and Levels in MPEG-4 256

Visual Object Types 256

Visual Profiles 259

Audio Object Types 260

Audio Profiles 261

Graphics 261

Systems Profiles 261

5.6 MPEG-4 Visual Texture Coding(VTC) and JPEG 2000 Image Compression Standards 262

5.6.1 JPEG 2000 Development Process 263

5.6.2 Overview of Still-Image Coding Standards 267

MPEG-4 VTC 267

JPEG 267

PNG 268

5.6.3 Significant Features of JPEG 2000 268

Region of Interest(ROI)Coding 268

Scalability 268

Error Resilience 269

IPRs 269

5.6.4 Architecture of JPEG 2000 269

5.6.5 JPEG 2000 Bit Stream 275

5.6.6 Compression Efficiency Comparisons 277

Error Resilience 281

5.7 MPEG-7 Standardization Process of Multimedia Content Description 282

5.7.1 Objective of the MPEG-7 Standard 282

5.7.2 Status of the MPEG-7 Standard 285

5.7.3 Major Functionalities in MPEG-7 286

MPEG-7 Systems 286

MPEG-7 DDL 289

MPEG-7 Audio 289

MPEG-7 Visual 290

MPEG-7 MMDSs 297

MPEG-7 Reference Software(XM) 299

MPEG-7 Conformance 299

5.7.4 Applications Enabled by MPEG-7 300

5.8 MPEG-21 Multimedia Framework 301

5.8.1 Audiovisual Content Representation Issues 303

5.8.2 Description of a Multimedia Framework Architecture 304

MPEG-21 Digital Item Declaration 304

Content Representation 305

Digital Item Identification and Description 305

Content Management and Usage 305

Intellectual Property Management and Protection 305

Terminals and Networks 306

Event Reporting 306

5.8.3 Requirements for Digital Item Declaration 306

5.9 ITU-T Standardization of Audiovisual Communication Systems 308

5.9.1 ITU-T Standardization Process 308

5.9.2 Audiovisual Systems(H.310,H.320,H.321,H.322,H.323,and H.324) 310

H.320 Standard 312

Standards for Audiovisual Services Across ATM H.310 and H.321 312

Standard H.322——Guaranteed Qos LAN Systems 315

ITU-T H.323 Standard 317

H.324 Standard 319

5.9.3 Video-Coding Standards(H.261,H.263 and H.26L) 319

H.261 Standard 319

H.263 Standard 323

H.263+(H.263 Version 2)Standard 327

H.263++ Standard Development 332

H.26L Standard 333

5.9.4 ITU-T Speech-Coding Standards 336

Bit Rate 338

Delay 338

Complexity 339

Quality 339

5.9.5 Multimedia Multiplex and Synchronization Standards 340

ITU-T Recommendation H.221 341

ITU-T Recommendation H.223 341

ITU-T Recommendation H.225 341

Common Control Protocol H.245 343

5.10 IETF and Internet Standards 344

5.10.1 IETF Standardization Process 344

5.10.2 Internet Network Architecture 347

5.10.3 Internet Protocols 348

Classical IP Stack 350

IP Version 6 351

Priority Field 352

Folw Label 353

IPv6 Addresses 353

Hop-by-Hop Options Header 355

Fragment Header 356

Routing Header 356

IPv6 Security 357

5.10.4 Real-Time Multimedia Transmission Across the Internet 359

Signaling 360

Session Control 360

Transport 361

Network Infrastructure 363

Multimedia Data for Network Use 364

5.10.5 MPEG-4 Video Transport Across the Internet 365

Use of RTP 365

System Architecture 366

MPEG-4 Server 368

MPEG-4 Client 369

5.11 Concluding Remarks 370

Chapter 6 Multimedia Communications Across Networks 373

6.1 Packet Audio/Video in the Network Environment 373

6.1.1 Packet Voice 374

6.1.2 Integrated Packet Networks 377

6.1.3 Packet Video 380

6.2 Video Transport Across Generic Networks 382

6.2.1 Layered Video Coding 386

Layered Compression 386

Layered Transmission 388

6.2.2 Error-Resilient Video Coding Techniques 388

Error-Resilient Encoding 390

Decoder Error Concealment 392

Error-Resilient Entropy Code 393

6.2.3 Scalable Rate Control 394

Rate Control Techniques 396

Theoretical Foundation of the SRC 397

6.2.4 Streaming Video Across the Internet 400

Video Compression 401

Requirements Imposed by Streaming Applications 403

Application Layer Qos Control 404

Continuous Media Distribution Services 407

Streaming Servers 409

Media Synchronization 409

Protocols for Streaming Video 410

6.3 Multimedia Transport Across ATM Networks 411

6.3.1 MUltiplexing in ATM Networks 412

6.3.2 Video Delay in ATM Networks 413

6.3.3 Errors and Losses in ATM 417

6.3.4 MPEG Video Error Concealment 420

6.3.5 Loss Concealment 420

6.3.6 Video Across WATM Networks 421

6.3.7 Heterogeneous Networking 422

6.4 Multimedia Across IP Networks 424

6.4.1 Video Transmission Across IP Networks 424

6.4.2 Traffic Specification for MPEG Video Transmission on the Internet 426

6.4.3 Bandwidth Allocation Mechanism 427

6.4.4 Fine-Grained Scalable Video Coding for Multimedia Across IP 428

6.5 Multimedia Across DSLs 432

6.5.1 VoDSL Architecture 433

6.5.2 Delivering Voice Services Across DSL 438

6.5.3 MUltimedia Across ADSL 439

Serial Transmission:TDM 439

Parallel Transmission Frequency Division Multiplexing 441

6.6 Internet Access Networks 441

6.6.1 DSL Networks 443

6.6.2 Cable Access Networks 445

6.6.3 Fixed Wireless Routed for Internet Access 447

6.7 Multimedia Across Wireless 449

6.7.1 Wireless Broadband Communication System(WBCS)for Multimedia 451

6.7.2 Audiovisual Solutions for Wireless Communications 453

6.7.3 Mobile Networks 458

Speech Transmission in GSM 459

Video Across GSM 460

Mobile ATM 462

Mobile IP 462

Wireless Multimedia Delivery 467

SIP in Mobile Environment 469

Multicast Routing in Cellular Networks 470

Broadband Wireless Mobile 471

6.7.4 Broadcasting Networks 473

Digital Video Broadcasting(DVB) 477

Data Transmission Using MPEG-2 and DVB 478

MPEG Program Stream 480

MPEG Transport Stream 481

Broadband Multimedia Satellite Systems 484

Multimedia Home Platform 486

Multimedia Car Platform 487

6.8 Digital Television Infrastructure for Interactive Multimedia Services 488

Interactive Broadcast Data(IDB) Services 490

Data Carousel Concept 492

6.9 Concluding Remarks 493

References 497

Index 537

About the Authors 545