# Linear Algebra: The Math of What Happens If I Change This? > Linear algebra is the math of transformation: what happens when you stretch, rotate, or combine things. Every photo filter, 3D game, and machine learning model is built from it. This guide starts from real movement and builds up to matrices without the terror. --- # Linear Algebra: The Math of What Happens If I Change This? If you have ever moved a character in a 2D game, applied a filter to a photo, or watched a recommendation system suggest something you might like, you have used linear algebra. The difference is that the computer knew it was doing linear algebra, and you did not. This guide fixes that. We are not going to drill matrix multiplication until your eyes bleed. We are going to start from something you already understand - walking in a direction - and build up to the notation and the ideas. By the end, a matrix will look like a recipe card: "rotate everything 90 degrees" or "make everything twice as big." That is all it is. This is the sixth guide in the Mathematics track. It assumes nothing beyond the mindset and notation from [Why Math Is Not Your Enemy](/guides/why-math-isnt-your-enemy) and the set idea from [Sets, Relations, and Functions](/guides/sets-relations-and-functions). If you can read a sentence and follow a recipe, you can do this. ## How to read this - **Here for the "what is this even for" answer?** Start with [Phase 1](01-vectors-as-arrows-in-the-real-world.md) - vectors as real movements. - **Want the whole toolkit?** Read in order - matrices build on vectors, and the applications build on both. ## The phases 1. **[Vectors as Arrows in the Real World](01-vectors-as-arrows-in-the-real-world.md)** - direction and magnitude, adding movements, and the code that moves a character. 2. **[Matrices as Recipes for Transformation](02-matrices-as-recipes-for-transformation.md)** - scaling, rotating, and shearing, and what matrix multiplication actually does to a shape. 3. **[Why This Is Everywhere](03-why-this-is-everywhere.md)** - PageRank, recommendation systems, neural networks, and the builder's guide to seeing linear algebra in the wild. > This builds on [Numbers & Number Systems](/guides/numbers-and-number-systems) (coordinates as numbers) and pairs with [Counting & Combinatorics](/guides/counting-and-combinatorics) (dimensions as choices). It sets up the machinery behind much of modern computing. --- # Vectors as Arrows in the Real World ## The walk that teaches you everything Stand up. Face north. Walk three blocks. Turn right. Walk four blocks. Stop. Where are you relative to where you started? You did not need a calculator. You already know: you are five blocks away, in a direction that is slightly east of northeast. Your brain performed vector addition. That is what this phase is about. Not symbols. Not formulas. The simple, physical fact that movements combine, and that the combination has a direction and a distance. ## What a vector is A **vector** is an arrow. That is the whole idea. The arrow has two properties: - **Direction** - which way it points. - **Magnitude** - how long it is. In math we write a vector as a list of numbers inside parentheses, like this: ``` (3, 4) ``` But do not let the parentheses scare you. That is a way of writing "three units in the first direction, four units in the second direction." On a city grid, it might mean "three blocks east, four blocks north." In a game, it might mean "move three pixels right, four pixels down." The numbers are called **components**. The first component is the east-west part. The second is the north-south part. If you had a 3D game, there would be a third component for up-down. ## Adding vectors: combining movements Here is the part your brain already knows. If you walk vector A, then walk vector B, the result is the same as walking a single vector C that goes from your start to your final position. ``` A = (3, 4) # three east, four north B = (1, -2) # one east, two south C = A + B # combine them ``` To add vectors, you add matching components: ``` C = (3 + 1, 4 + (-2)) = (4, 2) ``` Four east, two north. That is where you end up. The math is writing down what your legs already did. ## Scaling a vector: walking faster or slower Sometimes you want the same direction but a different length. That is **scaling**. If you double a vector, you walk twice as far in the same direction: ``` 2 * (3, 4) = (6, 8) ``` If you halve it, you walk half as far: ``` 0.5 * (3, 4) = (1.5, 2) ``` The number you multiply by is called a **scalar**. A scalar is a regular number that stretches or shrinks the arrow without turning it. ## The vector that does nothing Every direction has a special vector: the one with length zero. It points nowhere and goes nowhere. ``` (0, 0) ``` Add it to any vector and nothing changes: ``` (3, 4) + (0, 0) = (3, 4) ``` This is the **zero vector**. It is the "do nothing" button of the vector world. ## See it run Here is a character moving on a 2D grid. The position is a vector. Each move adds a new vector to it. ```python runnable # A character starts at (0, 0) position = [0, 0] # Move east 3, north 4 move1 = [3, 4] position = [position[0] + move1[0], position[1] + move1[1]] print("After move 1:", position) # Move east 1, south 2 move2 = [1, -2] position = [position[0] + move2[0], position[1] + move2[1]] print("After move 2:", position) # Scale a movement: walk the same direction but twice as far scaled_move = [2 * move1[0], 2 * move1[1]] print("Scaled move (double):", scaled_move) ``` *What just happened:* The character started at `[0, 0]`. After the first move it was at `[3, 4]`. After the second move it was at `[4, 2]`. Then we took the first move and doubled it to `[6, 8]` - same direction, twice the distance. Every line is adding or scaling vectors. ## For builders If you write code, you already use vectors. You do not call them that. - **2D and 3D positions** in a game or UI are vectors. A character at `(x, y)` is a vector from the origin. - **Velocities** are vectors. "Move 3 pixels per frame right and 2 pixels per frame down" is a velocity vector `(3, 2)`. - **Colors** in RGB are vectors. `(255, 128, 0)` is a direction and intensity in color space. - **Forces** in a physics simulation are vectors. Gravity pulls down `(0, -9.8)`. Wind pushes right `(2, 0)`. The operations you already write - adding positions to velocities, scaling a force by a time step - are vector addition and scalar multiplication. The name is new. The work is not. ## What we have built - A **vector** is an arrow with direction and magnitude, written as a list of components like `(3, 4)`. - **Adding vectors** combines movements: add matching components. - **Scaling a vector** stretches or shrinks it without turning it: multiply every component by the same scalar. - The **zero vector** `(0, 0)` changes nothing when added. - In code, positions, velocities, forces, and colors are all vectors in disguise. A quick check before you move on: ```quiz [ { "q": "You walk 2 blocks east and 3 blocks north, then 1 block east and 2 blocks south. What is your final position relative to the start?", "choices": ["(3, 5)", "(3, 1)", "(1, 5)", "(1, 1)"], "answer": 1, "explain": "Add the east-west components: 2 + 1 = 3. Add the north-south components: 3 + (-2) = 1. Final position is (3, 1) - three east, one north." }, { "q": "What does scaling a vector by 0.5 do?", "choices": ["It turns the vector 90 degrees", "It halves the length while keeping the same direction", "It makes the vector negative", "It adds a new component"], "answer": 1, "explain": "Multiplying a vector by a scalar stretches or shrinks it. 0.5 cuts the length in half without changing the direction." }, { "q": "Which of these is NOT a vector in the sense we have defined?", "choices": ["A velocity of (5, -3) pixels per frame", "A color of (255, 128, 0) in RGB", "The number 42", "A force of (0, -9.8) meters per second squared"], "answer": 2, "explain": "A single number with no direction is a scalar, not a vector. The others all have multiple components that describe direction and magnitude in some space." } ] ``` --- # Matrices as Recipes for Transformation ## The photo filter that taught me everything Open any photo app. Tap "rotate 90 degrees." The picture turns. Tap "black and white." The colors vanish. Tap "vignette." The edges darken. Each of those taps runs a tiny piece of linear algebra. The app takes every pixel in the photo, treats its position as a vector, and applies a transformation to it. The transformation is stored as a matrix. That is what this phase is about. Not the symbols first. The idea first: a matrix is a recipe. It says "here is how to change every point in a shape." ## From vector to point In Phase 1 we treated vectors as movements. The same math works when we treat vectors as **positions**. A point on a screen is a vector from the top-left corner. The point at the top-left is `(0, 0)`. The point 100 pixels to the right and 50 down is `(100, 50)`. If we apply a transformation to that point, we get a new point. The transformation is the matrix. ## The simplest transformation: scaling Suppose you want to make everything twice as wide but keep the same height. That is a **scaling** transformation. The matrix for "double the width, keep the height" looks like this: ``` [2 0] [0 1] ``` Do not memorize it. Read it as two instructions: - The first row says "for the new x coordinate, take 2 times the old x and 0 times the old y." - The second row says "for the new y coordinate, take 0 times the old x and 1 times the old y." Apply it to the point `(100, 50)`: ``` new_x = 2 * 100 + 0 * 50 = 200 new_y = 0 * 100 + 1 * 50 = 50 ``` The point moved from `(100, 50)` to `(200, 50)`. It got twice as far from the left edge, but stayed at the same height. That is exactly what "double the width" means. ## Rotation: turning the whole world Now you want to rotate everything 90 degrees clockwise around the origin. The matrix for that is: ``` [ 0 1] [-1 0] ``` Read it the same way: - new_x = 0 * old_x + 1 * old_y - new_y = -1 * old_x + 0 * old_y Apply it to `(100, 50)`: ``` new_x = 0 * 100 + 1 * 50 = 50 new_y = -1 * 100 + 0 * 50 = -100 ``` The point `(100, 50)` became `(50, -100)`. It moved from the lower-right quadrant to the lower-left quadrant. That is a 90 degree clockwise turn. ## Combining transformations: matrix multiplication Here is the part that feels like magic. Suppose you want to scale (doubling the width) and *then* rotate 90 degrees. You can do it in two steps, or combine the two matrices into one. The combined matrix is the **product** of the two matrices, and the order matters: scale then rotate is different from rotate then scale. The rule to remember: the transformation you apply **first** goes on the **right**. So "scale, then rotate" is `Rotate * Scale`. ``` Scale: [2 0] Rotate: [ 0 1] [0 1] [-1 0] Scale then rotate = Rotate * Scale: [ 0*2 + 1*0 0*0 + 1*1] [ 0 1] [-1*2 + 0*0 -1*0 + 0*1] = [-2 0] ``` The result is a new matrix that does both operations in one shot. Apply it to `(100, 50)`: ``` new_x = 0 * 100 + 1 * 50 = 50 new_y = -2 * 100 + 0 * 50 = -200 ``` This matches doing the two steps by hand: scaling `(100, 50)` to `(200, 50)`, then rotating that to `(50, -200)`. Matrix multiplication is the notation for "do this transformation, then that one." ## Shearing: the transformation nobody talks about Scaling stretches equally in all directions from an axis. Rotation turns around a point. **Shearing** slides one axis based on the other. The matrix for "slide right based on height" is: ``` [1 1] [0 1] ``` Apply it to `(100, 50)`: ``` new_x = 1 * 100 + 1 * 50 = 150 new_y = 0 * 100 + 1 * 50 = 50 ``` The point moved right by an extra 50 pixels because its y coordinate was 50. A rectangle would lean. That is shear. It is what makes italic text look slanted. ## See it run Here is a tiny transformation engine. It defines three matrices and applies them to a point. ```python runnable def apply_matrix(matrix, point): # matrix is a 2x2 list of lists, point is [x, y] new_x = matrix[0][0] * point[0] + matrix[0][1] * point[1] new_y = matrix[1][0] * point[0] + matrix[1][1] * point[1] return [new_x, new_y] point = [100, 50] scale = [[2, 0], [0, 1]] rotate = [[0, 1], [-1, 0]] shear = [[1, 1], [0, 1]] print("Original:", point) print("Scaled 2x:", apply_matrix(scale, point)) print("Rotated 90 deg clockwise:", apply_matrix(rotate, point)) print("Sheared:", apply_matrix(shear, point)) ``` *What just happened:* The function `apply_matrix` takes a 2x2 matrix and a point, then computes the new point by the rule we described. Scaling doubled the x coordinate. Rotation swapped x and y and negated the new y. Shear added the y value to the x value. Each matrix is a different recipe. The same function runs them all. ## For builders You encounter these transformations constantly, even if you never write a matrix by hand. - **CSS transforms** - `scale(2)`, `rotate(90deg)`, `skewX(20deg)` are exactly the scaling, rotation, and shear matrices we built. The browser does the matrix math for you. - **Image processing** - resizing a photo is scaling. Rotating a thumbnail is rotation. The "lens correction" that straightens tilted photos is a combination of rotation and shear. - **2D and 3D graphics** - every vertex in a game world is a vector. Every model view matrix is a recipe for where that vertex should appear on screen. - **Data visualization** - mapping data coordinates to pixel coordinates is a linear transformation. The axes you see on a chart are the result of scaling and translating the data space. > The key insight: a matrix is not a mysterious grid of numbers. It is a set of instructions. "For each point, compute its new position this way." Once you read it that way, the symbols stop being noise and start being a recipe card. ## What we have built - A **matrix** is a compact way to write a transformation: how to turn every point in a shape into a new point. - **Scaling** stretches or shrinks along an axis. - **Rotation** turns everything around a center point. - **Shear** slides one axis based on the value of another. - **Matrix multiplication** combines transformations: do A, then do B, and the product matrix does both. - The order of multiplication matters: scale then rotate is not the same as rotate then scale. A quick check before you move on: ```quiz [ { "q": "What does a matrix represent in the way we have been using it?", "choices": ["A list of unrelated numbers", "A recipe for transforming every point in a shape", "A way to store data in a database", "A type of vector with more components"], "answer": 1, "explain": "A matrix encodes a transformation: for each input point, it computes a new output point. Scaling, rotation, and shear are all examples of transformations written as matrices." }, { "q": "If you scale by 2 and then rotate 90 degrees, is that the same as rotating 90 degrees and then scaling by 2?", "choices": ["Yes, matrix multiplication is always commutative", "No, the order of transformations matters", "Yes, but only for square matrices", "No, but only for rotation matrices"], "answer": 1, "explain": "Matrix multiplication is not commutative. Scale then rotate produces a different result than rotate then scale, because the operations act on the coordinate system in a different order." }, { "q": "What does the shear matrix [1 1; 0 1] do to a point?", "choices": ["It doubles the x coordinate", "It adds the y coordinate to the x coordinate, slanting the shape", "It rotates the point 45 degrees", "It reflects the point across the y axis"], "answer": 1, "explain": "The shear matrix [1 1; 0 1] computes new_x = 1*x + 1*y and new_y = 0*x + 1*y. The x value gets the y value added to it, which slants or shears the shape." } ] ``` --- # Why This Is Everywhere ## The pattern you already know In Phase 1 you learned that a vector is an arrow. In Phase 2 you learned that a matrix is a recipe for transforming arrows. Now you are going to see those same arrows and recipes running the world. The pattern is always the same: 1. Take a bunch of things (people, web pages, products, pixels). 2. Represent each thing as a vector. 3. Use matrices to transform those vectors in a way that reveals something useful. That is Google search. That is Netflix recommending your next show. That is the filter that makes your photo look like it was taken on a film camera. That is the neural network that recognizes cats in pictures. ## PageRank: how Google started In the late 1990s, the web was a mess of pages linking to other pages. The question was: which pages are important? Larry Page and Sergey Brin realized that a link from one page to another is a vote. But not all votes are equal. A vote from an important page should count more than a vote from an unimportant one. So they built a vector. Each web page got a score - its importance. Then they built a matrix that said "if page A links to page B, pass some of A's importance to B." They applied that matrix over and over, like water flowing through pipes, until the scores settled down. The result was PageRank. The math is linear algebra: a vector of importance scores, and a matrix of links. The transformation says "redistribute importance along the links." Run it enough times and you have a ranked list of the web. You do not need to implement PageRank to use the insight. Every time you search for something and the "right" answer appears near the top, you are seeing linear algebra at work. ## Recommendation systems: what you might like Netflix, Spotify, and Amazon all face the same problem: they have millions of users and millions of items, and they need to guess what you want before you know you want it. One approach: represent each user as a vector of preferences, and each movie or song as a vector of features. Then the "match score" between a user and an item is a simple operation on two vectors. If the user vector is `(5, 2, 0, 4)` meaning "I love action, I like comedy, I hate horror, I love sci-fi" and the movie vector is `(4, 1, 0, 5)` meaning "this is an action-comedy with no horror and lots of sci-fi," then the system can compute how well they align. The matrix enters when you have many users and many items. The whole collection can be organized into a giant matrix, and the task of finding the best matches becomes a transformation problem. The result: "Because you liked Inception, you might enjoy Interstellar." That sentence is linear algebra wearing a product manager's vocabulary. ## Photo filters: matrices in your pocket Open your phone's camera app. Tap a filter. The photo changes. What happened under the hood? The app took every pixel in the image, treated its color as a vector in red-green-blue space, and multiplied it by a matrix. - The "sepia" filter uses a matrix that reduces blue, boosts red, and adds a little green. - The "black and white" filter uses a matrix that averages the three color channels into one. - The "vignette" darkens the edges by scaling down vectors that are far from the center. Each filter is a 3x3 matrix. The operation is the same matrix multiplication you practiced in Phase 2, applied to millions of pixels in parallel. Your phone does this in milliseconds because the hardware is built for it. ## Neural networks: layers of linear algebra A neural network sounds exotic. At its core, it is stacks of linear transformations with a tiny bit of non-linear magic sprinkled between them. Each "layer" in a neural network takes a vector of inputs, multiplies it by a matrix (the layer's weights), adds a bias vector, and passes the result through a simple non-linear function. Then the next layer does the same thing. Then the next. When a neural network recognizes a cat in a photo, it is doing this: 1. The raw pixels become a vector. 2. The first layer transforms that vector into a space where edges and textures are highlighted. 3. The second layer transforms that into a space where shapes are highlighted. 4. The final layer transforms that into a space where "cat" and "not cat" are far apart. Every transformation is a matrix multiplication. The "learning" is the process of finding the right matrices. The inference - actually recognizing the cat - is applying the matrices to the input vector. You do not need to build a neural network to appreciate this. You need to know that the intimidating phrase "deep learning" is mostly linear algebra with a little bit of nonlinearity on top. ## For builders This is the part where you stop seeing linear algebra as a school subject and start seeing it as a tool in your workshop. - **Game development** - Every 2D or 3D game is vectors (positions, velocities) and matrices (camera transforms, model rotations). If you have ever used a game engine, you have used linear algebra. - **Data science** - PCA, the technique that reduces a thousand columns of data to the few that matter most, is linear algebra. So is the regression that finds the line of best fit. - **Computer graphics** - Shaders run matrix math on every pixel, every frame. The GPU is essentially a linear algebra engine. - **Robotics and simulation** - Forward kinematics, inverse kinematics, rigid body transforms: all matrices multiplying vectors. - **Audio processing** - An audio signal is a vector of samples over time. Filters, equalization, and compression are all linear transformations. > The next time you see a 3x3 grid of numbers in a library or a spec sheet, do not skip past it. That is a recipe. Ask yourself: what transformation does it apply? The answer is usually simpler than it looks. ## What we have built - A **vector** is an arrow with direction and magnitude. It represents a position, a movement, a color, or a force. - A **matrix** is a recipe for transforming vectors: scaling, rotating, shearing. - **Matrix multiplication** combines recipes: do A, then do B. - **PageRank** is a vector of importance scores transformed by a matrix of links. - **Recommendation systems** match user vectors to item vectors using matrix operations. - **Photo filters** are 3x3 matrices applied to color vectors. - **Neural networks** are stacks of matrix multiplications with a little nonlinearity. You started this guide with the question "what happens if I change this?" Linear algebra is the most precise answer ever invented to that question. Change the input vector, apply the matrix, read the output vector. That is it. That is the whole thing. A quick check before you go: ```quiz [ { "q": "In PageRank, what do the vector and the matrix represent?", "choices": ["The vector is the list of web pages, and the matrix is the list of search queries", "The vector holds importance scores for each page, and the matrix encodes which pages link to which", "The vector is the browser history, and the matrix is the ad click data", "The vector is the page content, and the matrix is the ranking algorithm"], "answer": 1, "explain": "PageRank uses a vector of importance scores (one per page) and a matrix of links. The matrix transformation redistributes importance along the links, and repeating it converges to a stable ranking." }, { "q": "A photo filter that makes an image black and white is best described as what?", "choices": ["A scalar that reduces the image size", "A matrix that transforms each pixel's color vector", "A vector that adds gray to every pixel", "A sorting algorithm that reorders the pixels"], "answer": 1, "explain": "A black and white filter is a matrix that takes each pixel's RGB vector and transforms it into a single intensity value. It is the same matrix multiplication you practiced, applied to millions of pixels." }, { "q": "Why is a neural network mostly linear algebra?", "choices": ["Because it stores data in matrices", "Because each layer multiplies the input vector by a weight matrix, and the whole network is a stack of such multiplications", "Because it was invented by mathematicians", "Because it only works on linear data"], "answer": 1, "explain": "Each layer in a neural network multiplies the input vector by a weight matrix, adds a bias, and applies a small non-linear function. The heavy lifting is matrix multiplication. The 'learning' is finding the right matrices." } ] ```