Preface ......................................................... v
1. Matrices .................................................... 1
1.1. Basic Terminology ..................................... 1
1.2. Basic Operations ...................................... 2
1.3. Some Basic Types of Matrices .......................... 5
Exercises .................................................. 10
2. Submatrices and Partitioned Matrices ....................... 13
2.1. Some Terminology and Basic Results ................... 13
2.2. Scalar Multiples, Transposes, Sums, and Products
of Partitioned Matrices .............................. 17
2.3. Some Results on the Product of a Matrix and
a Column Vector ...................................... 19
2.4. Expansion of a Matrix in Terms of Its Rows,
Columns, or Elements ................................. 20
Exercises .................................................. 21
3. Linear Dependence and Independence ......................... 23
3.1. Definitions .......................................... 23
3.2. Some Basic Results ................................... 23
Exercises .................................................. 25
4. Linear Spaces: Row and Column Spaces ....................... 27
4.1. Some Definitions, Notation, and Basic
Relationships and Properties ......................... 27
4.2. Subspaces ............................................ 29
4.3. Bases ................................................ 31
4.4. Rank of a Matrix ..................................... 36
4.5. Some Basic Results on Partitioned Matrices and
on Sums of Matrices .................................. 41
Exercises .................................................. 46
5. Trace of a (Square) Matrix ................................. 49
5.1. Definition and Basic Properties ...................... 49
5.2. Trace of a Product ................................... 50
5.3. Some Equivalent Conditions ........................... 52
Exercises .................................................. 52
6. Geometrical Considerations ................................. 55
6.1. Definitions: Norm, Distance, Angle, Inner
Product, and Orthogonality ........................... 55
6.2. Orthogonal and Orthonormal Sets ...................... 61
6.3. Schwarz Inequality ................................... 62
6.4. Orthonormal Bases .................................... 63
Exercises .................................................. 68
7. Linear Systems: Consistency and Compatibility .............. 71
7.1. Some Basic Terminology ............................... 71
7.2. Consistency .......................................... 72
7.3. Compatibility ........................................ 73
7.4. Linear Systems of the Form A'AX = A'B ................ 74
Exercise ................................................... 77
8. Inverse Matrices ........................................... 79
8.1. Some Definitions and Basic Results ................... 79
8.2. Properties of Inverse Matrices ....................... 81
8.3. Premultiplication or Postmultiplication by
a Matrix of Full Column or Row Rank .................. 82
8.4. Orthogonal Matrices .................................. 84
8.5. Some Basic Results on the Ranks and Inverses of
Partitioned Matrices ................................. 88
Exercises ................................................. 103
9. Generalized Inverses ...................................... 107
9.1. Definition, Existence, and a Connection to the
Solution of Linear Systems .......................... 107
9.2. Some Alternative Characterizations .................. 109
9.3. Some Elementary Properties .......................... 117
9.4. Invariance to the Choice of a Generalized Inverse ... 119
9.5. A Necessary and Sufficient Condition for the
Consistency of a Linear System ...................... 120
9.6. Some Results on the Ranks and Generalized
Inverses of Partitioned Matrices .................... 121
9.7. Extension of Some Results on Systems of the
Form AX = В to Systems of the Form AXC = В .......... 126
Exercises ................................................. 126
10. Idempotent Matrices ....................................... 133
10.1. Definition and Some Basic Properties ................ 133
10.2. Some Basic Results .................................. 134
Exercises ................................................. 136
11. Linear Systems: Solutions ................................. 139
11.1. Some Terminology, Notation, and Basic Results ....... 139
11.2. General Form of a Solution .......................... 140
11.3. Number of Solutions ................................. 142
11.4. A Basic Result on Null Spaces ....................... 144
11.5. An Alternative Expression for the General Form
of a Solution ....................................... 144
11.6. Equivalent Linear Systems ........................... 145
11.7. Null and Column Spaces of Idempotent Matrices ....... 146
11.8. Linear Systems With Nonsingular Triangular or
Block-Triangular Coefficient Matrices ............... 146
11.9. A Computational Approach ............................ 149
11.10.Linear Combinations of the Unknowns ................. 150
11.11.Absorption .......................................... 152
11.12.Extensions to Systems of the Form AXC = В ........... 157
Exercises ................................................. 158
12. Projections and Projection Matrices ....................... 161
12.1. Some General Results, Terminology, and Notation ..... 161
12.2. Projection of a Column Vector ....................... 163
12.3. Projection Matrices ................................. 166
12.4. Least Squares Problem ............................... 170
12.5. Orthogonal Complements .............................. 172
Exercises ................................................. 177
13. Determinants .............................................. 179
13.1. Some Definitions, Notation, and Special Cases ....... 179
13.2. Some Basic Properties of Determinants ............... 183
13.3. Partitioned Matrices, Products of Matrices,
and Inverse Matrices ................................ 187
13.4. A Computational Approach ............................ 191
13.5. Cofactors ........................................... 191
13.6. Vandermonde Matrices ................................ 195
13.7. Some Results on the Determinant of the Sum of
Two Matrices ........................................ 197
13.8. Laplace's Theorem and the Binet-Cauchy Formula ...... 200
Exercises ................................................. 205
14. Linear, Bilinear, and Quadratic Forms ..................... 209
14.1. Some Terminology and Basic Results .................. 209
14.2. Nonnegative Definite Quadratic Forms and Matrices ... 212
14.3. Decomposition of Symmetric and Symmetric
Nonnegative Definite Matrices ....................... 218
14.4. Generalized Inverses of Symmetric Nonnegative
Definite Matrices ................................... 222
14.5. LDU, U'DU, and Cholesky Decompositions .............. 223
14.6. Skew-Symmetric Matrices ............................. 239
14.7. Trace of a Nonnegative Definite Matrix .............. 240
14.8. Partitioned Nonnegative Definite Matrices ........... 243
14.9. Some Results on Determinants ........................ 247
14.10.Geometrical Considerations .......................... 255
14.11.Some Results on Ranks and Row and Column Spaces
and on Linear Systems ............................... 259
14.12.Projections, Projection Matrices, and Orthogonal
Complements ......................................... 260
Exercises ........................................... 277
15. Matrix Differentiation .................................... 289
15.1. Definitions, Notation, and Other Preliminaries ...... 290
15.2. Differentiation of (Scalar-Valued) Functions:
Some Elementary Results ............................. 296
15.3. Differentiation of Linear and Quadratic Forms ....... 298
15.4. Differentiation of Matrix Sums, Products, and
Transposes (and of Matrices of Constants) ........... 300
15.5. Differentiation of a Vector or (Unrestricted
or Symmetric) Matrix With Respect to Its Elements ... 303
15.6. Differentiation of a Trace of a Matrix .............. 304
15.7. The Chain Rule ...................................... 306
15.8. First-Order Partial Derivatives of Determinants
and Inverse and Adjoint Matrices .................... 308
15.9. Second-Order Partial Derivatives of Determinants
and Inverse Matrices ................................ 312
15.10.Differentiation of Generalized Inverses ............. 314
15.11.Differentiation of Projection Matrices .............. 319
15.12.Evaluation of Some Multiple Integrals ............... 324
Exercises ................................................. 327
Bibliographic and Supplementary Notes ..................... 335
16. Kronecker Products and the Vec and Vech Operators ......... 337
16.1. The Kronecker Product of Two or More Matrices:
Definition and Some Basic Properties ................ 337
16.2. The Vec Operator: Definition and Some Basic
Properties .......................................... 343
16.3. Vec-Permutation Matrix .............................. 347
16.4. The Vech Operator ................................... 354
16.5. Reformulation of a Linear System .................... 367
16.6. Some Results on Jacobian Matrices ................... 368
Exercises ................................................. 371
Bibliographic and Supplementary Notes ..................... 377
17. Intersections and Sums of Subspaces ....................... 379
17.1. Definitions and Some Basic Properties ............... 379
17.2. Some Results on Row and Column Spaces and on
the Ranks of Partitioned Matrices ................... 385
17.3. Some Results on Linear Systems and on Generalized
Inverses of Partitioned Matrices .................... 392
17.4. Subspaces: Sum of Their Dimensions Versus
Dimension of Their Sum .............................. 396
17.5. Some Results on the Rank of a Product of Matrices ... 398
17.6. Projections Along a Subspace ........................ 402
17.7. Some Further Results on the Essential
Disjointness and Orthogonality of Subspaces
and on Projections and Projection Matrices .......... 409
Exercises ................................................. 411
Bibliographic and Supplementary Notes ..................... 417
18. Sums (and Differences) of Matrices ........................ 419
18.1. Some Results on Determinants ........................ 419
18.2. Some Results on Inverses and Generalized
Inverses and on Linear Systems ...................... 423
18.3. Some Results on Positive (and Nonnegative)
Definiteness ........................................ 437
18.4. Some Results on Idempotency ......................... 439
18.5. Some Results on Ranks ............................... 444
Exercises ................................................. 450
Bibliographic and Supplementary Notes ..................... 458
19. Minimization of a Second-Degree Polynomial
(in n Variables) Subject to Linear Constraints ............ 459
19.1. Unconstrained Minimization .......................... 460
19.2. Constrained Minimization ............................ 463
19.3. Explicit Expressions for Solutions to the
Constrained Minimization Problem .................... 468
19.4. Some Results on Generalized Inverses of
Partitioned Matrices ................................ 476
19.5. Some Additional Results on the Form of
Solutions to the Constrained Minimization Problem ... 483
19.6. Transformation of the Constrained Minimization
Problem to an Unconstrained Minimization Problem .... 489
19.7. The Effect of Constraints on the Generalized
Least Squares Problem ............................... 491
Exercises ................................................. 492
Bibliographic and Supplementary Notes ..................... 495
20 The Moore-Penrose Inverse .................................. 497
20.1. Definition, Existence, and Uniqueness
(of the Moore-Penrose Inverse) ...................... 497
20.2. Some Special Cases .................................. 499
20.3. Special Types of Generalized Inverses ............... 500
20.4. Some Alternative Representations and
Characterizations ................................... 507
20.5. Some Basic Properties and Relationships ............. 508
20.6. Minimum Norm Solution to the Least Squares
Problem ............................................. 512
20.7. Expression of the Moore-Penrose Inverse
as a Limit .......................................... 512
20.8. Differentiation of the Moore-Penrose Inverse ........ 514
Exercises ................................................. 517
Bibliographic and Supplementary Notes ..................... 519
21. Eigenvalues and Eigenvectors .............................. 521
21.1. Definitions, Terminology, and Some Basic Results .... 522
21.2. Eigenvalues of Triangular or Block-Triangular
Matrices and of Diagonal or Block-Diagonal
Matrices ............................................ 528
21.3. Similar Matrices .................................... 530
21.4. Linear Independence of Eigenvectors ................. 534
21.5. Diagonalization ..................................... 537
21.6. Expressions for the Trace and Determinant
of a Matrix ......................................... 545
21.7. Some Results on the Moore-Penrose Inverse
of a Symmetric Matrix ............................... 546
21.8. Eigenvalues of Orthogonal, Idempotent, and
Nonnegative Definite Matrices ....................... 548
21.9. Square Root of a Symmetric Nonnegative
Definite Matrix ..................................... 550
21.10.Some Relationships .................................. 551
21.11.Eigenvalues and Eigenvectors of Kronecker
Products ............................................ 554
21.12.Singular Value Decomposition ........................ 556
21.13.Simultaneous Diagonalization ........................ 566
21.14.Generalized Eigenvalue Problem ...................... 569
21.15.Differentiation of Eigenvalues and Eigenvectors ..... 571
21.16.An Equivalence (Involving Determinants and
Polynomials) ........................................ 574
Appendix: Some Properties of Polynomials .................. 580
Exercises ................................................. 582
Bibliographic and Supplementary Notes ..................... 588
22. Linear Transformations .................................... 589
22.1. Some Definitions, Terminology, and Basic Results .... 589
22.2. Scalar Multiples, Sums, and Products of Linear
Transformations ..................................... 595
22.3. Inverse Transformations and Isomorphic Linear
Spaces .............................................. 598
22.4. Matrix Representation of a Linear Transformation .... 601
22.5. Terminology and Properties Shared by a Linear
Transformation and Its Matrix Representation ........ 609
22.6. Linear Functionals and Dual Transformations ......... 612
Exercises ................................................. 616
References ................................................ 621
Index ......................................................... 625
|
