Linear Algebra

UPSC Mathematics Optional · Paper I · Theorems, Formulae & PYQ Insights

1. Vector Spaces 2. Linear Transformations 3. Matrices & Determinants 4. System of Linear Equations 5. Eigenvalues & Eigenvectors 6. Cayley-Hamilton Theorem 7. Diagonalization 8. Inner Product Spaces 9. Quadratic Forms

§1 · Vector Spaces

Vector Space — Axioms

A set \(V\) with addition and scalar multiplication (over field \(F\)) satisfying 8 axioms: commutativity, associativity, identity (\(\mathbf{0}\)), inverse, distributivity (×2), associativity of scalar mult., unit scalar.

Subspace Test quick check

\(W \subseteq V\) is a subspace iff: (i) \(\mathbf{0}\in W\), (ii) closed under addition, (iii) closed under scalar multiplication.

Equivalently: \(\alpha u + \beta v \in W\) for all \(u,v\in W\), \(\alpha,\beta\in F\).

Basis & Dimension

A basis is a linearly independent spanning set.
All bases of a finite-dimensional vector space have the same number of elements = \(\dim V\).
Any \(n\) linearly independent vectors in an \(n\)-dim space form a basis.
Any spanning set of \(n\) vectors in an \(n\)-dim space is a basis.

Dimension Formula for Sum of Subspaces PYQ★★

\[\dim(U+W) = \dim U + \dim W - \dim(U\cap W)\]

Standard Subspaces to Know

Null space (kernel): \(\ker T = \{v : T(v)=0\}\)
Column space (range/image): \(\text{Im}\,T\)
Row space, left null space of a matrix.
\(\dim(\ker T) + \dim(\text{Im}\,T) = \dim V\) (Rank-Nullity)

§2 · Linear Transformations

Linear Transformation (Homomorphism)

\(T:V\to W\) is linear iff \(T(\alpha u+\beta v)=\alpha T(u)+\beta T(v)\).

Completely determined by images of basis vectors.

Rank–Nullity Theorem PYQ★★★

\[\text{rank}(T) + \text{nullity}(T) = \dim V\]

i.e. \(\dim(\text{Im}\,T) + \dim(\ker T) = \dim V\).

Matrix form: rank\((A)\) + nullity\((A)\) = number of columns of \(A\).

Matrix of a Linear Transformation PYQ★★

If \(\mathcal{B}=\{v_1,\ldots,v_n\}\) is a basis of \(V\) and \(\mathcal{C}\) of \(W\), the matrix \([T]_{\mathcal{B}}^{\mathcal{C}}\) has columns = coordinates of \(T(v_j)\) in basis \(\mathcal{C}\).

Change of basis: \([T]_{\mathcal{B}'} = P^{-1}[T]_{\mathcal{B}}P\) where \(P\) = transition matrix.

Isomorphism Theorem

Two finite-dimensional vector spaces over the same field are isomorphic iff they have the same dimension.

\(T:V\to W\) is an isomorphism iff \(T\) is bijective iff \(\ker T = \{0\}\) and \(\text{Im}\,T = W\).

§3 · Matrices & Determinants

Rank of a Matrix PYQ★★★

Rank = number of non-zero rows in row echelon form = size of largest non-zero minor.
\(\text{rank}(A) = \text{rank}(A^T) = \text{rank}(A^TA)\)
\(\text{rank}(AB) \le \min(\text{rank}\,A, \text{rank}\,B)\)
\(\text{rank}(A+B) \le \text{rank}\,A + \text{rank}\,B\)
For \(n\times n\) matrix: \(A\) invertible iff \(\text{rank}(A)=n\) iff \(\det A\ne 0\).

Determinant Properties PYQ★★

\(\det(AB) = \det A \cdot \det B\)
\(\det(A^T) = \det A\)
\(\det(A^{-1}) = (\det A)^{-1}\)
\(\det(kA) = k^n \det A\) for \(n\times n\) matrix
Row interchange: \(\det\) changes sign. Row scalar: \(\det\) scales. Adding multiple of one row to another: \(\det\) unchanged.
Singular matrix \(\Leftrightarrow\) \(\det A = 0\)

Inverse via Adjugate

\[A^{-1} = \frac{1}{\det A}\,\text{adj}(A)\]

where \(\text{adj}(A)_{ij} = (-1)^{i+j}M_{ji}\) (\(M_{ji}\) = minor of \(a_{ji}\)).

\(A\cdot\text{adj}(A) = (\det A)\,I\)

Trace Properties

\(\text{tr}(A) = \sum_i a_{ii} = \sum_i \lambda_i\) (sum of eigenvalues).

\(\text{tr}(AB)=\text{tr}(BA)\). \(\text{tr}(A+B)=\text{tr}A+\text{tr}B\).

§4 · System of Linear Equations

Consistency — Rouché-Capelli Theorem PYQ★★★

The system \(AX=B\) is consistent iff \(\text{rank}(A) = \text{rank}([A|B])\).

Unique solution: \(\text{rank}(A)=\text{rank}([A|B])=n\) (number of unknowns).
Infinite solutions: \(\text{rank}(A)=\text{rank}([A|B])
No solution: \(\text{rank}(A)<\text{rank}([A|B])\).

Homogeneous System \(AX=0\) PYQ★★

Always consistent (trivial solution \(X=0\)).

Non-trivial solution exists iff \(\text{rank}(A) < n\) iff \(\det A = 0\).

Solution space (null space) has dimension \(n - \text{rank}(A)\).

Cramer's Rule (for square non-singular systems)

For \(AX=B\) with \(\det A \ne 0\): \(x_i = \dfrac{\det A_i}{\det A}\), where \(A_i\) = \(A\) with \(i\)-th column replaced by \(B\).

§5 · Eigenvalues & Eigenvectors

Characteristic Equation PYQ★★★

\(\det(A - \lambda I) = 0\) gives the characteristic polynomial \(p(\lambda)\).

Eigenvalues = roots of \(p(\lambda)\). Eigenvectors: solve \((A-\lambda I)X=0\).

Key Eigenvalue Facts

\(\sum \lambda_i = \text{tr}(A)\)
\(\prod \lambda_i = \det(A)\)
If \(\lambda\) is eigenvalue of \(A\), then \(\lambda^n\) is eigenvalue of \(A^n\), \(f(\lambda)\) of \(f(A)\).
If \(A\) invertible, \(1/\lambda\) is eigenvalue of \(A^{-1}\).
\(A\) and \(A^T\) have the same eigenvalues.
Eigenvectors for distinct eigenvalues are linearly independent.
Symmetric real matrix: all eigenvalues real; eigenvectors for distinct eigenvalues are orthogonal.

Special Matrices — Eigenvalues PYQ★★

Orthogonal matrix: eigenvalues are \(\pm 1\) or complex with \(|\lambda|=1\).
Idempotent (\(A^2=A\)): eigenvalues are 0 or 1. \(\text{tr}(A) = \text{rank}(A)\).
Nilpotent (\(A^k=0\)): all eigenvalues 0.
Skew-symmetric real: eigenvalues are 0 or purely imaginary.
Positive definite symmetric: all eigenvalues \(>0\).

§6 · Cayley–Hamilton Theorem

Cayley–Hamilton Theorem PYQ★★★

Every square matrix satisfies its own characteristic equation.

If \(p(\lambda) = \det(\lambda I - A)\), then \(p(A) = 0\).

Use: Find \(A^{-1}\), higher powers of \(A\), minimal polynomial.

Using Cayley-Hamilton to find \(A^{-1}\) PYQ★★★

For \(2\times 2\): \(A^2 - (\text{tr}\,A)\,A + (\det A)\,I = 0\)

Rearranging: \(A^{-1} = \dfrac{1}{\det A}[(\text{tr}\,A)\,I - A]\)

For \(3\times 3\) with char. poly \(\lambda^3-p\lambda^2+q\lambda-r=0\):
\(A^3 - pA^2 + qA - rI = 0 \Rightarrow A^{-1} = \tfrac{1}{r}(A^2-pA+qI)\)

Minimal Polynomial PYQ★★

The monic polynomial of least degree that \(A\) satisfies. It divides the characteristic polynomial. Eigenvalues of \(A\) = roots of minimal polynomial.

If all eigenvalues are distinct, minimal poly = characteristic poly.

§7 · Diagonalization

Diagonalizability Conditions PYQ★★★

\(A\) is diagonalizable iff it has \(n\) linearly independent eigenvectors iff for each eigenvalue \(\lambda_i\), geometric multiplicity = algebraic multiplicity.

If diagonalizable: \(A = PDP^{-1}\) where \(D=\text{diag}(\lambda_1,\ldots,\lambda_n)\), columns of \(P\) = eigenvectors.

Then: \(A^n = PD^nP^{-1}\), \(f(A)=Pf(D)P^{-1}\).

Spectral Theorem (Symmetric Matrices) PYQ★★

Every real symmetric matrix is orthogonally diagonalizable: \(A = Q\Lambda Q^T\) where \(Q\) is orthogonal (columns = orthonormal eigenvectors) and \(\Lambda\) = diagonal matrix of eigenvalues.

When is a matrix NOT diagonalizable?

If some eigenvalue has algebraic multiplicity \(>\) geometric multiplicity. Then we need Jordan Normal Form. Jordan block for eigenvalue \(\lambda\) of size \(k\): \(\lambda\) on diagonal, 1's on superdiagonal.

§8 · Inner Product Spaces

Inner Product — Properties

\(\langle\cdot,\cdot\rangle:V\times V\to\mathbb{R}\) (or \(\mathbb{C}\)) satisfying: linearity in first argument, conjugate symmetry, positive definiteness.

\(\|v\| = \sqrt{\langle v,v\rangle}\). Angle: \(\cos\theta = \dfrac{\langle u,v\rangle}{\|u\|\|v\|}\).

Cauchy-Schwarz Inequality PYQ★★

\[|\langle u,v\rangle| \le \|u\|\,\|v\|\]

Equality iff \(u\) and \(v\) are linearly dependent.

Gram–Schmidt Orthogonalization PYQ★★★

Given basis \(\{v_1,\ldots,v_n\}\), construct orthonormal basis \(\{e_1,\ldots,e_n\}\):

\[u_1 = v_1,\quad u_k = v_k - \sum_{j=1}^{k-1}\frac{\langle v_k,u_j\rangle}{\langle u_j,u_j\rangle}u_j,\quad e_k = \frac{u_k}{\|u_k\|}\]

Orthogonal Complement & Projection

\(W^\perp = \{v\in V : \langle v,w\rangle=0\;\forall w\in W\}\).

\(V = W \oplus W^\perp\). \(\dim W + \dim W^\perp = \dim V\).

Projection of \(v\) onto \(W\): \(\text{proj}_W v = \sum_i \langle v,e_i\rangle e_i\) (orthonormal basis \(\{e_i\}\) of \(W\)).

§9 · Quadratic Forms

Quadratic Form PYQ★★★

\(Q(\mathbf{x}) = \mathbf{x}^T A \mathbf{x}\) where \(A\) is symmetric. \(Q = \sum_{i,j} a_{ij}x_ix_j\).

Canonical form: Under orthogonal transformation, \(Q = \lambda_1 y_1^2 + \cdots + \lambda_n y_n^2\).

Definiteness via Eigenvalues PYQ★★★

Positive definite: all \(\lambda_i > 0\) ↔ all leading principal minors \(> 0\).
Negative definite: all \(\lambda_i < 0\) ↔ leading principal minors alternate in sign \((-,+,-,\ldots)\).
Positive semi-definite: all \(\lambda_i \ge 0\).
Indefinite: mixed signs of \(\lambda_i\).

Sylvester's Law of Inertia PYQ★★

The number of positive, negative, and zero eigenvalues (the signature \((p,q,r)\)) is invariant under congruence transformations.

Index = \(p\) (number of positive eigenvalues). Rank = \(p+q\).

Reduction to Canonical Form — Steps

Write matrix \(A\) of the quadratic form (symmetric).
Find eigenvalues of \(A\).
Apply orthogonal transformation to diagonalize.
Canonical form: \(\lambda_1 y_1^2 + \lambda_2 y_2^2 + \cdots\)