Notes

Three numbers for every point and every line of the plane, so that joining two points and crossing two lines each take one cross product.

Four ways a flat shape can change, from a turn to the view of a camera, and how to find and undo one from four pairs of points.

Rotations and shifts in space, perspective projection and the camera model, with interactive figures.

Estimating a camera's projection matrix and its eleven parameters from points of a known object, with an interactive figure.

Where a point seen in one image can appear in a second one, the fundamental matrix that says it, and how to estimate it, with interactive figures.

Finding a point of space from its images in two or more views, and predicting where it appears in a third, with an interactive figure.

How to choose again and again among options whose payoffs can only be learned by trying them, estimating what each is worth and balancing trying the others against using the best one found so far.

Learning how good each state and action is by averaging the returns of complete episodes, and improving the policy from those averages, with no model of the world.

Updating an estimate after every step, toward the reward just received plus the estimate of the next state, instead of waiting for the end of the episode, and the same idea for learning to act.

How an agent learns the values of the best behavior from its own experience, one step at a time, while it keeps trying other actions.

Using n real rewards before trusting an estimate: the range of methods between one-step TD and Monte Carlo.

Averaging every n-step return with weights that fade by λ, and learning from that average as the episode runs, with a fading memory of the pairs just visited.

Learning how the world answers each action, and planning with that model: R-Max goes looking for what it has not tried, and Dyna learns from every real step twice, once from the step itself and again from replays of what it remembers.

Estimating values when there are too many states for a table: a few weights give every state its value, learned from returns or from the next estimate, and the two kinds of learning settle on different weights.

Learning to act when there are too many states for a table: values of actions from a few weights, Sarsa that learns them while acting on them, and, for a task that never ends, values measured against the average reward.

How an agent learns to play video games from the frames alone: a network that turns the last few frames into a value for each move, trained with Q-learning on moments drawn at random from a memory of its own play.

How an agent learns to move a paddle at any speed.

How often words appear and how fast a vocabulary grows, and why a few words fill about a quarter of a long text while new words never stop coming.

What a word is made of, how stems and lemmas gather its forms, how the words of a sentence are tagged, and how WordNet relates their meanings.

How a search engine finds the chapter a query is about, by turning words into weights, chapters into vectors and relevance into an angle, and how BM25 refines the weights.

How a weighted sum of a document's words decides which of two classes it belongs to, how logistic regression turns the sum into a probability, and how gradient descent learns the weights.

How a bag of words loses the order of a text, how n-grams and skip-grams keep some of it, and how word2vec learns a vector for every word by guessing the words around it, a training that negative sampling makes cheap.

What the vectors that word2vec learns from Don Quixote end up capturing, why related words point the same way even when they never stand together, how sums and differences of vectors pick the odd word out of a list and complete analogies, what vectors trained on far more text know and one book does not, and how pooling gives a whole text one vector.

How a word that no table has a vector for gets one from its subwords, runs of three to six characters it shares with other words, how fastText trains them, what the vectors you download were trained with, and when shared letters mean shared meaning.

How a model gives every token of a sentence a label, such as the start of a name or a part of speech: a head that scores each token, a table that scores which label may follow which, the probability of a whole labeling computed without listing them all, and the hidden Markov model, trained by counting and decoded with Viterbi.