Linear Algebra Note#6: Vector Space

Posted on In

This note is based on MIT 18.06 📒

Content

Vector Space

\(R^n\) represents the whole \(n\)-d space, which includes all \(n\)-d vectors.

A vector space:

  • needs to be closed under any linear combination of its components.
  • thus, it must include zero vector.

Sub-space

A sub-space of a vector space \(R\) is included by \(R\), and still satisfies the properties of a vector space.

For example, a sub-space of \(R^2\) can be:

  • \(R^2\) itself
  • Any line in the \(R^2\) that passes the origin
  • Zero vector alone

Note: the lines (sub-spaces of \(R^2\)) are not the same as \(R^1\). Because the vectors inside have two elements. They just form a line-shaped space in \(R^2\).

Extending to \(R^3\), a sub-space can be:

  • \(R^3\) itself
  • Any plane (\(2\)-d space) that passes through the origin in \(R^3\)
  • Any line (\(1\)-d space) that passes through the origin in \(R^3\)
  • Zero vector alone

Union of sub-spaces

The union of two sub-spaces is usually not a sub-space, unless one is a sub-space of the other.

Consider a plane \(P\) and a line \(L\) in \(R^3\). Unless \(L\) is a sub-space of \(P\), the linear combinations of their vectors should fill the whole \(R^3\) space.

Intersection of sub-spaces

The intersection of two sub-spaces forms a sub-space.

Say any two vectors \(v\) and \(w\) from the intersection \(S\cap T\). Since they are both in \(S\) and \(T\), any linear combination of them should be in the intersection as well.

Column Space

A column space is a vector space defined by a specific matrix. Say \(A\):

\[ A= \begin{bmatrix} 1 & 1 & 2 \\\\ 2 & 1 & 3 \\\\ 3 & 1 & 4 \\\\ 4 & 1 & 5 \end{bmatrix} \]

The column space \(C(A)\) is the vector space that contains all possible linear combinations of the column vectors in \(A\).

The size of the column space

For \(Ax=b\), the \(b\) that makes the equation to be solved forms the column space.

In a representational capacity perspective: How many dimensions can be represented by the column vectors of \(A\)?

  • Are the columns independent (each contribute something new to a different dimension)?
  • If a column can be represented by other columns, it is already in the column space represented by other columns. In other words, it does not contribute anything new.

In the \(A\) given above, the third column can be thrown away, so the column space is a \(2\)-d sub-space of \(R^4\).

All the left independent columns are called pivot columns.

Null Space

The null space \(N(A)\) is the vector space that contains all solutions \(x\) that make \(Ax=0\).

For the \(A\) above, it's a sub-space of \(R^3\) (the number of columns).

Why is null space a vector space?

For \(v\) and \(w\) in \(N(A)\), \(Av=0\) and \(Aw=0\), then:

  • \(A(v+w) = Av + Aw = 0+0 = 0\)
  • \(A(cv) = cAv = c \times 0 = 0\)

The null space is closed and contains the zero vector.

Thought (size of null space): the dimension of null space should be the number of columns that are not pivot columns. The coefficient in \(x\) of these columns can be set freely, since the pivot columns can always cancel them out. Thus, the number of these coefficients are the "freedom degree" of the null space. In other words, its dimension.

Dimensions of Rectangular Matrix

For a matrix of size \(m\times n\).

When \(m>n\)

  • Linear equation view: more equations than unknowns, which means more restrictions. For some \(b\) there might not be solution.
  • Column view: columns are too few to provide information to each dim in \(R^m\).

When \(m<n\)

  • Linear equation view: equations give less restrictions so there are always more choice to solve each \(b\), unless there are too many dependent ones.
  • Column view: the columns are more likely to provide enough dimension information in \(R^m\), but sometimes they are too dependent and still not enough.