The Core Idea of Linear Algebra

“Algebra” roughly translates to “relationships” and linear means “line-like,” thus “Linear Algebra” means “line-like relationships”.

Algebra

So why does algebra from school seem to be just about letters rather than numbers.

In my head algebra is just about arranging and rearranging letters according to specific rules. However here is a way of looking at it.

Consider the equation below. You don’t know what value of the x and the y is.

(x + y)² = x² + 2xy + y²

However we can still workout the relationship between (x + y)² and x² + 2xy + y². We know that both expressions are equal, and we know the relation between the value of x and y and each side of the equation. We could write this as a function:

f(x, y) = (x + y)²

Linear

Linear relations are relation similar to what you can describe on a line. Linear relations are predictable. You double the input and you get double the output.

This is not a linear relation

f(x) = x²

If I double the input, the output is quadrupled. This is an example of a linear relation:

f(x) = 4x

However this is not a linear relation, even if you make think so.

f(x) = 2x + 4

Because f(1) is 6, while f(2) is 8. We doubled the input but did not get double the output.

We can make it linear by adding another parameter.

f(x, y) = 2x + 4y

If we double the inputs, we will now be doubling the output as well.

f(2x, 2y) = 2*2x + 2*4y
2(2x + 4y) = 2f(x, y)

Thus we can make linear functions with some flexibility by just adding a number of arguments.

The Matrix and Vector Connection

A problem with these functions is when they have a lot of arguments it becomes repetitive and a lot to write.

That is why matrices and vectors are used. We can express the previous function f as dot product.

[2, 4] ⋅ [x, y] = 2x + 4y
f = [2, 4]
f ⋅ [x, y] = 2x + 4y

Thus you can think of the f vector as the operation being done on the input vector [x, y].