Contents
Preface xv
Prologue: A machine learning sampler 1
1 The ingredients of machine learning 13
1.1 Tasks: the problems that can be solved with machine learning . . . . . . . 14
Looking for structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Evaluating performance on a task . . . . . . . . . . . . . . . . . . . . . . . . 18
1.2 Models: the output of machine learning . . . . . . . . . . . . . . . . . . . . 20
Geometric models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Probabilistic models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Logical models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
Grouping and grading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
1.3 Features: the workhorses of machine learning . . . . . . . . . . . . . . . . 38
Two uses of features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
Feature construction and transformation . . . . . . . . . . . . . . . . . . . 41
Interaction between features . . . . . . . . . . . . . . . . . . . . . . . . . . 44
1.4 Summary and outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
What you’ll find in the rest of the book . . . . . . . . . . . . . . . . . . . . . 48
2 Binary classification and related tasks 49
2.1 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
ixx Contents
Assessing classification performance . . . . . . . . . . . . . . . . . . . . . . 53
Visualising classification performance . . . . . . . . . . . . . . . . . . . . . 58
2.2 Scoring and ranking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Assessing and visualising ranking performance . . . . . . . . . . . . . . . . 63
Turning rankers into classifiers . . . . . . . . . . . . . . . . . . . . . . . . . 69
2.3 Class probability estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Assessing class probability estimates . .
1