Properties of linear systems of equations

Consider a linear system of \(m\) equations in \(n\) unknowns:

\[ \begin{bmatrix} y_1\\ y_2 \\ \vdots \\ y_m \end{bmatrix}= \begin{bmatrix} a_{11}&a_{12}&\dots&a_{1n}\\a_{21}&a_{22}&\dots&a_{2n} \\ \vdots&\vdots&\vdots&\vdots \\ a_{m1}&a_{m2}&\dots&a_{mn} \end{bmatrix} \begin{bmatrix} x_1\\ x_2 \\ \vdots \\ x_n \end{bmatrix}\]

In compact form this is written as \(y = Ax\).

Such a system:

• - may or may not have a solution, 
• - if a solution exists, this solution may or may not be unique

Consistent systems: at least one solution exists

If a solution exists, the system of equations is called consistent. A solution exists if and only if \(y\) can be written as a linear combination of the column vectors of matrix \(A\):

\[y=\begin{bmatrix} a_{11}\\a_{21}\\ \vdots \\ a_{m1} \end{bmatrix}x_1+\begin{bmatrix} a_{12}\\a_{22}\\ \vdots \\ a_{m2} \end{bmatrix}x_2 + \dots +\begin{bmatrix} a_{1n}\\a_{2n}\\ \vdots \\ a_{mn} \end{bmatrix}x_n\] 

In that case \(y\) is an element of the range space of \(A\): \(y \in \mathcal{R}(A)\).

For a consistent system of equations it is always possible to find a solution \(x\) such that \(y=Ax\). If it is not possible to find a solution, the system is called inconsistent.

Consistency is guaranteed if \(\text{rank}(A) = m\)

Explanation: \( y \in \mathbb{R}^{m}\) and the system is consistent if  \( y \in \mathcal{R}(A) \), hence consistency is guaranteed if \(\mathcal{R}(A)=\mathbb{R}^{m}\). This means that the columns of \(A\) must span the complete space of reals \(\mathbb{R}^{m}\), which is true if \(\text{rank}(A)  = m\).

If \(\text{rank}(A) < m\), the system may or may not be consistent, this depends then on the actual entries of the vector \(y\).

Unique solution

A consistent system has a unique solution if and only if the column vectors of matrix \(A\) are independent, i.e. if \( \text{rank}(A) = n\).

This can be seen as follows: assume that \(x\) and \(x' \neq x\) are two different solutions. Then \(Ax=Ax’\) or \(A(x-x')=0\). But this can only be the case if some of the column vectors of \(A\) are linear dependent, which contradicts the assumption of full column rank.

Determined, overdetermined and underdetermined systems

A system of equations \(y=Ax\) with \(\text{rank}(A) =m=n\) is consistent and has a unique solution: \(\hat{x} = A^{-1}y\). Such a system is called determined.

A system is underdetermined if \(\text{rank}(A)  < n\), i.e. if it does not have a unique solution.

A system is overdetermined if \(\text{rank}(A)  < m\), i.e. the system may or may not be consistent.

Redundancy

The redundancy of a system of equations is equal to \(m - \text{rank}(A)\).

In this course we will restrict ourselves to systems of observation equations that are of full column rank: \(\text{rank}(A) = n \). In that case, the system can either be

• - determined: \(\text{rank}(A) =n =m\), the redundancy is equal to 0
• - overdetermined: \(\text{rank}(A) =n < m\), the redundancy is equal to \(m-n>0\)