Part I Fundamentals
1 Introduction ................................................. 3
1.1 Expert Systems .......................................... 3
1.1.1 Representation of Uncertainty .................... 4
1.1.2 Normative Expert Systems ......................... 5
1.2 Rule-Based Systems ...................................... 5
1.2.1 Causality ........................................ 6
1.2.2 Uncertainty in Rule-Based Systems ................ 7
1.2.3 Explaining Away .................................. 8
1.3 Bayesian Networks ....................................... 8
1.3.1 Inference in Bayesian Networks ................... 9
1.3.2 Construction of Bayesian Networks ............... 10
1.3.3 An Example ...................................... 11
1.4 Bayesian Decision Problems ............................. 13
1.5 When to Use Probabilistic Nets ......................... 14
1.6 Concluding Remarks ..................................... 15
2 Networks .................................................... 17
2.1 Graphs ................................................. 18
2.2 Graphical Models ....................................... 20
2.2.1 Variables ....................................... 20
2.2.2 Vertices Vs. Variables .......................... 21
2.2.3 Taxonomy of Vertices/Variables .................. 22
2.2.4 Vertex Symbols .................................. 23
2.2.5 Summary of Notation ............................. 23
2.3 Evidence ............................................... 24
2.4 Causality .............................................. 24
2.5 Flow of Information in Causal Networks ................. 25
2.5.1 Serial Connections .............................. 26
2.5.2 Diverging Connections ........................... 28
2.5.3 Converging Connections .......................... 29
2.5.4 Intercausal Inference (Explaining Away) ......... 30
2.5.5 The Importance of Correct Modeling of
Causality ....................................... 31
2.6 Two Equivalent Irrelevance Criteria .................... 32
2.6.1 d-Separation Criterion .......................... 33
2.6.2 Directed Global Markov Criterion ................ 35
2.7 Summary ................................................ 36
3 Probabilities ............................................... 39
3.1 Basics ................................................. 40
3.1.1 Events .......................................... 40
3.1.2 Axioms .......................................... 40
3.1.3 Conditional Probability ......................... 41
3.2 Probability Distributions for Variables ................ 43
3.2.1 Rule of Total Probability ....................... 44
3.2.2 Graphical Representation ........................ 46
3.3 Probability Potentials ................................. 46
3.3.1 Normalization ................................... 47
3.3.2 Evidence Potentials ............................. 49
3.3.3 Potential Calculus .............................. 50
3.3.4 Barren Variables ................................ 53
3.4 Fundamental Rule and Bayes' Rule ....................... 54
3.4.1 Interpretation of Bayes' Rule ................... 55
3.5 Bayes' Factor .......................................... 58
3.6 Independence ........................................... 59
3.6.1 Independence and DAGs ........................... 60
3.7 Chain Rule ............................................. 62
3.8 Summary ................................................ 64
4 Probabilistic Networks ...................................... 69
4.1 Belief Update .......................................... 70
4.1.1 Discrete Bayesian Networks ...................... 71
4.1.2 Conditional Linear Gaussian Bayesian Networks ... 75
4.2 Decision Making Under Uncertainty ...................... 79
4.2.1 Discrete Influence Diagrams ..................... 80
4.2.2 Conditional LQG Influence Diagrams .............. 89
4.2.3 Limited Memory Influence Diagrams ............... 93
4.3 Object-Oriented Probabilistic Networks ................. 95
4.3.1 Chain Rule ..................................... 100
4.3.2 Unfolded OOPNs ................................. 100
4.3.3 Instance Trees ................................. 100
4.3.4 Inheritance .................................... 101
4.4 Dynamic Models ........................................ 102
4.4.1 Time-Sliced Networks Represented as OOPNs ...... 104
4.5 Summary ............................................... 105
5 Solving Probabilistic Networks ............................. 111
5.1 Probabilistic Inference ............................... 112
5.1.1 Inference in Discrete Bayesian Networks ........ 112
5.1.2 Inference in CLG Bayesian Networks ............. 125
5.2 Solving Decision Models ............................... 128
5.2.1 Solving Discrete Influence Diagrams ............ 128
5.2.2 Solving CLQG Influence Diagrams ................ 132
5.2.3 Relevance Reasoning ............................ 134
5.2.4 Solving LIMIDs ................................. 136
5.3 Solving OOPNs ......................................... 140
5.4 Summary ............................................... 140
Part II Model Construction
6 Eliciting the Model ........................................ 145
6.1 When to Use Probabilistic Networks .................... 146
6.1.1 Characteristics of Probabilistic Networks ...... 147
6.1.2 Some Criteria for Using Probabilistic
Networks ....................................... 148
6.2 Identifying the Variables of a Model .................. 149
6.2.1 Weil-Defined Variables ......................... 149
6.2.2 Types of Variables ............................. 152
6.3 Eliciting the Structure ............................... 154
6.3.1 A Basic Approach ............................... 154
6.3.2 Idioms ......................................... 156
6.3.3 An Example: Extended Chest Clinic Model ........ 163
6.3.4 The Generic Structure of Probabilistic
Networks ....................................... 173
6.4 Model Verification .................................... 174
6.5 Eliciting the Numbers ................................. 176
6.5.1 Eliciting Subjective Conditional
Probabilities .................................. 177
6.5.2 Eliciting Subjective Utilities ................. 179
6.5.3 Specifying CPTs and UTs Through Expressions .... 180
6.6 Concluding Remarks .................................... 183
6.7 Summary ............................................... 185
7 Modeling Techniques ........................................ 191
7.1 Structure-Related Techniques .......................... 191
7.1.1 Parent Divorcing ............................... 192
7.1.2 Temporal Transformation ........................ 196
7.1.3 Structural and Functional Uncertainty .......... 197
7.1.4 Undirected Dependence Relations ................ 201
7.1.5 Bidirectional Relations ........................ 204
7.1.6 Naive Bayes Model .............................. 206
7.2 Probability Distribution-Related Techniques ........... 208
7.2.1 Measurement Uncertainty ........................ 209
7.2.2 Expert Opinions ................................ 211
7.2.3 Node Absorption ................................ 213
7.2.4 Set Value by Intervention ...................... 214
7.2.5 Independence of Causal Influence ............... 216
7.2.6 Mixture of Gaussian Distributions .............. 221
7.3 Decision-Related Techniques ........................... 224
7.3.1 Test Decisions ................................. 224
7.3.2 Missing Informational Links .................... 227
7.3.3 Missing Observations ........................... 229
7.3.4 Hypothesis of Highest Probability .............. 231
7.3.5 Constraints on Decisions ....................... 233
7.4 Summary ............................................... 235
8 Data-Driven Modeling ....................................... 237
8.1 The Task and Basic Assumptions ........................ 238
8.1.1 Basic Assumptions .............................. 240
8.1.2 Equivalent Models .............................. 240
8.2 Constraint-Based Structure Learning ................... 242
8.2.1 Statistical Hypothesis Tests ................... 242
8.2.2 Structure Constraints .......................... 245
8.2.3 PC Algorithm ................................... 246
8.2.4 PC* Algorithm .................................. 251
8.2.5 NPC Algorithm .................................. 251
8.3 Search and Score-Based Structure Learning ............. 256
8.3.1 Space of Structures ............................ 256
8.3.2 Search Procedures .............................. 257
8.3.3 Score Functions ................................ 258
8.3.4 Learning Structure Restricted Models ........... 265
8.4 Worked Example on Structure Learning .................. 271
8.4.1 PC Algorithm ................................... 272
8.4.2 NPC Algorithm .................................. 273
8.4.3 Search and Score-Based Algorithm ............... 275
8.4.4 Chow-Liu Tree .................................. 276
8.4.5 Comparison ..................................... 277
8.5 Batch Parameter Learning .............................. 278
8.5.1 Expectation-Maximization Algorithm ............. 279
8.5.2 Penalized EM Algorithm ......................... 281
8.6 Sequential Parameter Learning ......................... 283
8.7 Summary ............................................... 285
Part III Model Analysis
9 Conflict Analysis .......................................... 291
9.1 Evidence-Driven Conflict Analysis ..................... 292
9.1.1 Conflict Measure ............................... 292
9.1.2 Tracing Conflicts .............................. 294
9.1.3 Conflict Resolution ............................ 295
9.2 Hypothesis-Driven Conflict Analysis ................... 297
9.2.1 Cost-of-Omission Measure ....................... 297
9.2.2 Evidence with Conflict Impact .................. 297
9.3 Summary ............................................... 299
10 Sensitivity Analysis ....................................... 303
10.1 Evidence Sensitivity Analysis ......................... 304
10.1.1 Distance and Cost-of-Omission Measures ......... 305
10.1.2 Identify Minimum and Maximum Beliefs ........... 306
10.1.3 Impact of Evidence Subsets ..................... 307
10.1.4 Discrimination of Competing Hypotheses ......... 308
10.1.5 What-If Analysis ............................... 309
10.1.6 Impact of Findings ............................. 309
10.2 Parameter Sensitivity Analysis ........................ 311
10.2.1 Sensitivity Function ........................... 312
10.2.2 Sensitivity Value .............................. 314
10.2.3 Admissible Deviation ........................... 316
10.3 Two-Way Parameter Sensitivity Analysis ................ 317
10.3.1 Sensitivity Function ........................... 317
10.4 Parameter Tuning ...................................... 320
10.5 Summary ............................................... 323
11 Value of Information Analysis .............................. 327
11.1 VOI Analysis in Bayesian Networks ..................... 328
11.1.1 Entropy and Mutual Information ................. 328
11.1.2 Hypothesis-Driven Value of Information
Analysis ....................................... 329
11.2 VOI Analysis in Influence Diagrams .................... 333
11.3 Summary ............................................... 336
Quick Reference to Model Construction ......................... 341
List of Examples .............................................. 351
List of Figures ............................................... 355
List of Tables ................................................ 365
List of Symbols ............................................... 369
Reference ..................................................... 371
Index ......................................................... 377
| 
