Linear Algebra

Most people first encounter vectors as arrows in two or three dimensions. That is a perfectly good start. But the deeper idea is more general and more powerful.

A vector space is any set of objects that can be added together and scaled by numbers, as long as those operations behave sensibly — associativity, commutativity, distributivity, and so on. The objects could be arrows in space. But they could also be polynomials, or continuous functions on an interval, or infinite sequences of numbers, or quantum states.

This generality is what makes linear algebra so pervasive. Once you identify the right vector space, powerful machinery kicks in: you can talk about bases, dimensions, transformations, and orthogonality, all in complete abstraction from the specific objects involved.

  Object              Addition                   Scaling
  ──────────────      ──────────────────────     ───────────────────
  Arrows in ℝ³        Tip-to-tail                Stretch or shrink
  Polynomials         (p+q)(x) = p(x)+q(x)       (cp)(x) = c·p(x)
  Functions f:[0,1]→ℝ (f+g)(x) = f(x)+g(x)       (cf)(x) = c·f(x)
  Quantum states |ψ⟩  |ψ⟩+|φ⟩  (superposition)  c|ψ⟩  (amplitude)

A basis is a minimal spanning set — a collection of vectors that (a) span the space, meaning every vector in the space can be written as a linear combination of them, and (b) are linearly independent, meaning none of them is redundant.

The number of vectors in any basis is the dimension of the space. This is a theorem — you can prove that any two bases for the same space have the same number of elements. Dimension is a property of the space itself, not of any particular basis.

Once you choose a basis $\{e_1, \ldots, e_n\}$, every vector gets a unique coordinate representation. This is what makes linear algebra computational: abstract vectors become concrete arrays of numbers. But remember — the numbers depend on the basis. The vector itself does not.

A linear transformation is a function between vector spaces that respects the structure — it preserves addition and scalar multiplication. In coordinates, every linear transformation between finite-dimensional spaces corresponds to multiplication by a matrix.

The kernel of $T$ is the set of vectors that get sent to zero. The image is the set of all output vectors. The rank-nullity theorem is a conservation law for dimensions:

This says something intuitive but powerful: if a transformation collapses a lot of directions to zero (large kernel), it can only reach a small part of the output space (small image). You cannot gain dimensions from a linear map.

An eigenvector of a transformation is a vector that the transformation merely scales — it does not change direction. The scaling factor is the eigenvalue.

This is why eigenvectors are special: they are the natural axes of the transformation. If you write a transformation in the basis of its own eigenvectors, it becomes diagonal — just a list of scaling factors. All the apparent complexity of the transformation is reduced to a few numbers.

Eigenvalues are found by solving the characteristic polynomial$\det(A - \lambda I) = 0$. For an $n \times n$ matrix, this gives a degree-$n$ polynomial, hence up to $n$ eigenvalues.

  A general vector v:          An eigenvector v:
  
        Av                            Av = λv
       ╱                              ▲
      ╱                               │  (only stretched)
     ╱                                │
    v ────▶ Av (rotated + scaled)    v │
                                       │
                                       ▼

An inner product gives a vector space a notion of length and angle. For vectors in$\mathbb{R}^n$, it is the dot product. For functions, it is typically an integral $\langle f, g \rangle = \int f(x) g(x)\, dx$.

Two vectors are orthogonal if their inner product is zero. An orthonormal basis is one where all basis vectors are orthogonal and have unit length. In such a basis, coordinates are found by projection: $c_i = \langle \mathbf{v}, e_i \rangle$. This is why Fourier series work — the trigonometric functions form an orthonormal basis for the space of periodic functions.

The spectral theorem is the crown jewel of linear algebra for physicists. It says that any Hermitian (self-adjoint) operator on a finite-dimensional inner product space can be diagonalised by an orthonormal basis of eigenvectors, with real eigenvalues.

In quantum mechanics, observables are Hermitian operators. The spectral theorem guarantees that measurements yield real values (eigenvalues) and that the eigenstates form a complete basis. The act of measurement is a projection onto an eigenspace. All of this is just the spectral theorem, dressed in physics.

Physicists use Dirac notation as a cleaner way to write inner product space operations. A state vector is a ket $|\psi\rangle$; its dual is a bra$\langle\psi|$; their pairing is the bracket (bra-ket):$\langle\phi|\psi\rangle$.

  Linear algebra           Dirac notation
  ──────────────           ──────────────
  Vector v                 |ψ⟩           (ket)
  Dual vector              ⟨ψ|           (bra)  
  Inner product ⟨u,v⟩      ⟨φ|ψ⟩         (braket)
  Outer product u⊗v*       |ψ⟩⟨φ|        (projection operator)
  Matrix element Aᵢⱼ       ⟨i|A|j⟩       (matrix element)

The completeness relation $\sum_i |e_i\rangle\langle e_i| = \mathbf{1}$is just the statement that the eigenstates form a complete basis — you can expand any state in terms of them. It looks like a trivial identity, but it is actually a powerful computational tool: insert it strategically inside a calculation to switch between bases.