1 Introduction ............................................... 1
1.1 Concepts and terminology ................................... 1
1.1.1 Signal representation by source coding .............. 3
1.1.2 Optimization of transmission ........................ 6
1.2 Signal sources and acquisition ............................. 7
1.3 Digital representation of multimedia signals .............. 10
1.3.1 Image and video signals ....................... 11
1.3.2 Speech and audio signals ........................... 18
1.3.3 Need for compression technology .................... 18
1.4 Problems .................................................. 19
2 Fundamentals .............................................. 21
2.1 Signals and systems ....................................... 21
2.1.1 Elementary signals ................................. 21
2.1.2 Systems operations ................................. 24
2.2 Signals and Fourier spectra ............................... 27
2.2.1 Spectra over two- and multi-dimensional
coordinates ........................................ 27
2.2.2 Spatio-temporal signals ............................ 28
2.3 Sampling of multimedia signals ............................ 31
2.3.1 Separable two-dimensional sampling ................. 34
2.3.2 Non-separable two-dimensional sampling ............. 35
2.3.3 Sampling of video signals .......................... 42
2.4 Discrete signal processing ................................ 46
2.4.1 LSI systems ........................................ 46
2.4.2 Discrete Fourier transform ......................... 47
2.4.3 z transform ........................................ 48
2.4.4 Multi-dimensional LSI systems ...................... 50
2.5 Statistical analysis ...................................... 54
2.5.1 Sample statistics .................................. 54
2.5.2 Joint statistical properties ....................... 58
2.5.3 Spectral properties of random signals .............. 65
2.5.4 Markov chain models ................................ 65
2.5.5 Statistical foundation of information theory ....... 68
2.6 Linear prediction ......................................... 74
2.6.1 Autoregressive models .............................. 74
2.6.2 Linear prediction .................................. 82
2.7 Linear block transforms ................................... 94
2.7.1 Orthogonal basis functions ......................... 94
2.7.2 Types of orthogonal transforms ..................... 98
2.7.3 Efficiency of transforms .......................... 111
2.7.4 Transforms with block overlap ..................... 112
2.8 Filterbank transforms .................................... 115
2.8.1 Decimation and interpolation ...................... 116
2.8.2 Properties of subband filters ..................... 121
2.8.3 Implementation of filterbank structures ........... 127
2.8.4 Wavelet transform ................................. 133
2.8.5 Two- and multi-dimensional filter banks ........... 140
2.8.6 Pyramid decomposition ............................. 144
2.9 Problems ................................................. 149
3 Perception and Quality ................................... 157
3.1 Properties of vision ..................................... 157
3.1.1 Physiology of the eye ............................. 157
3.1.2 Sensitivity functions ............................. 159
3.1.3 Color vision ...................................... 162
3.1.4 Binocular vision and reproduction ................. 163
3.2 Properties of hearing .................................... 165
3.2.1 Physiology of the ear ............................. 165
3.2.2 Sensitivity functions ............................. 166
3.3 Quality metrics .......................................... 170
3.3.1 Objective signal quality metrics .................. 171
3.3.2 Subjective assessment ............................. 178
3.4 Problems ................................................. 182
4 Quantization and Coding .................................. 185
4.1 Scalar quantization and pulse code modulation ............ 185
4.2 Coding theory ............................................ 192
4.2.1 Source coding theorem and rate-distortion
function .......................................... 192
4.2.2 Rate-distortion function for correlated signals ... 193
4.2.3 Rate-distortion function for multi-dimensional
signals ........................................... 197
4.3 Rate-distortion optimization of quantizers ............... 199
4.4 Entropy coding ........................................... 203
4.4.1 Properties of variable-length codes ............... 203
4.4.2 Huffman code design ............................... 207
4.4.3 Systematic variable-length codes .................. 208
4.4.4 Arithmetic coding ................................. 213
4.4.5 Adaptive and context-dependent entropy coding ..... 218
4.4.6 Entropy coding and transmission errors ............ 222
4.4.7 Lempel-Ziv coding ................................. 224
4.5 Vector quantization ...................................... 225
4.5.1 Basic principles of VQ ............................ 226
4.5.2 VQ with uniform codebooks ......................... 230
4.5.3 VQ with non-uniform codebooks ..................... 232
4.5.4 Structured codebooks .............................. 236
4.5.5 Adaptive VQ ....................................... 240
4.5.6 Rate-constrained VQ ............................... 242
4.6 Trellis-coded quantization ............................... 245
4.7 Problems ................................................. 247
5 Methods of Signal Compression ............................ 251
5.1 Run-length coding ........................................ 251
5.2 Predictive coding ........................................ 254
5.2.1 Open-loop and closed-loop prediction systems ...... 254
5.2.2 Non-linear and shift-variant prediction ........... 257
5.2.3 Effects of transmission losses .................... 258
5.2.4 Vector prediction ................................. 259
5.2.5 Delayed-decision predictive coding ................ 262
5.2.6 Prediction in multi-resolution pyramids ........... 263
5.3 Transform coding ......................................... 268
5.3.1 Gain in discrete transform coding ................. 269
5.3.2 Quantization of transform coefficients ............ 272
5.3.3 Coding of transform coefficients .................. 276
5.3.4 Transform coding under transmission losses ........ 279
5.4 Bit streams with multiple decoding capability ............ 280
5.4.1 Simulcast and transcoding ......................... 280
5.4.2 Scalable coding ................................... 280
5.4.3 Multiple-description coding ....................... 282
5.5 Distributed source coding ................................ 284
5.6 Problems ................................................. 286
6 Intra-picture Coding ..................................... 289
6.1 Compression of binary pictures ........................... 289
6.1.1 Compression of bi-level images .................... 289
6.1.2 Binary shape coding ............................... 294
6.1.3 Contour coding .................................... 296
6.2 Vector quantization of pictures .......................... 298
6.3 Predictive coding of pictures ............................ 300
6.3.1 2D prediction ..................................... 300
6.3.2 2D block prediction ............................... 302
6.3.3 Quantization and encoding of prediction errors .... 304
6.3.4 Error propagation in 2D DPCM ...................... 307
6.3.5 Coding of side information in 2D predictive
coding ............................................ 307
6.4 Transform coding of pictures ............................. 310
6.4.1 Block transform coding ............................ 310
6.4.2 Quantization and coding of transform
coefficients ...................................... 312
6.4.3 Overlapping-block transform coding ................ 324
6.4.4 Subband and wavelet transform coding .............. 324
6.4.5 Local adaptation of transform bases by signal
properties ........................................ 331
6.5 Lossless and near-lossless picture coding ................ 334
6.6 Synthesis based picture coding ........................... 336
6.6.1 Region-based coding ............................... 338
6.6.2 Color and texture synthesis ....................... 340
6.6.3 Synthesis of illumination effects ................. 343
6.7 Fractal image coding ..................................... 344
6.8 3D picture coding ........................................ 347
6.9 Reconstruction filtering ................................. 348
6.10 Still picture coding standards ........................... 352
6.11 Problems ................................................. 356
7 Inter-picture Coding ..................................... 361
7.1 Intra-picture and replenishment coding ................... 361
7.2 Hybrid video coding ...................................... 364
7.2.1 Motion-compensated hybrid coders .................. 364
7.2.2 Characteristics of inter-picture prediction
error signals ..................................... 366
7.2.3 Quantization error feedback and error
propagation ....................................... 369
7.2.4 Reference pictures in motion-compensated
prediction ........................................ 373
7.2.5 Accuracy of motion compensation ................... 387
7.2.6 Hybrid coding of interlaced video signals ......... 394
7.2.7 Optimization of hybrid encoders ................... 397
7.2.8 Hybrid coding using subband/wavelet transforms .... 399
7.3 Spatio-temporal transform coding ......................... 401
7.3.1 Inter-picture transform and subband coding ........ 401
7.3.2 Motion-compensated temporal filtering ............. 405
7.3.3 Quantization and encoding of MCTF-based
representations ................................... 410
7.4 Coding of side information (motion, modes,
partitioning) ............................................ 413
7.5 Scalable video coding .................................... 422
7.5.1 Scalability in hybrid video coding ................ 423
7.5.2 Standardization in scalable video coding .......... 430
7.6 Multi-view video coding .................................. 435
7.7 Synthesis based video coding ............................. 443
7.7.1 Region-based video coding ......................... 444
7.7.2 Super-resolution synthesis ........................ 447
7.7.3 Moving texture synthesis .......................... 447
7.8 Video coding standards ................................... 450
7.9 Problems ................................................. 459
8 Speech and Audio Coding .................................. 463
8.1 Coding of speech signals ................................. 464
8.1.1 Linear predictive coding .......................... 464
8.1.2 Parametric (synthesis) coding ..................... 468
8.1.3 Speech coding standards ........................... 468
8.2 Generic audio, music and sound coding .................... 470
8.2.1 Transform coding of audio signals ................. 470
8.2.2 Synthesis based coding of audio and sound
signals ........................................... 475
8.2.3 Coding of stereo and multi-channel audio signals .. 478
8.2.4 Music and sound coding standards .................. 486
Transmission and Storage of Multimedia Data .............. 491
9.1 Digital multimedia services .............................. 491
9.2 Network interfaces ....................................... 493
9.3 Adaptation for channel characteristics ................... 497
9.3.1 Rate and transmission control ..................... 498
9.3.2 Error control ..................................... 502
9.4 Media transport, storage and reproduction ................ 508
9.4.1 Broadcast applications ............................ 509
9.4.2 Communication services ............................ 513
9.4.3 File storage ...................................... 514
9.4.4 Internet and mobile media streaming ............... 516
A Vector and Matrix Algebra .................................. 521
B Symbols and Variables ...................................... 529
C Glossary and Acronyms ...................................... 533
D References ................................................. 537
E Index ...................................................... 553
|
