3.4. Vector Spaces

3.4.1. Vector Spaces

Definition

A vector space (or linear space) consists of the following:

• a field \(F\) of scalars;

• a set \(V\) of objects, called vectors;

• a rule (or operation), called vector addition, which associates with each pair of vectors \(\alpha, \beta \in V\) a vector \(\alpha + \beta \in V\), called the sum of \(\alpha\) and \(\beta\), in such a way that

  • addition is commutative, \(\alpha + \beta = \beta + \alpha\);

  • addition is associative, \(\alpha + (\beta + \gamma) = (\alpha + \beta) + \gamma\);

  • there is a unique vector \(0 \in V\), called the zero vector, such that \(\alpha + 0 = \alpha \quad \forall \alpha \in V\);

  • for each vector \(\alpha \in V\), there is a unique vector \(-\alpha \in V\) such that \(\alpha + (-\alpha) = 0\);

• a rule (or operation), called scalar multiplication, which associates with each scalar \(c \in F\) and vector \(\alpha \in V\) a vector \(c\alpha \in V\) in such a way that

  • \(1\alpha = \alpha \quad \forall \alpha \in V\);

  • \((c_1 c_2)\alpha = c_1 (c_2 \alpha)\);

  • \(c (\alpha + \beta) = c\alpha + c\beta\);

  • \((c_1 + c_2) \alpha = c_1 \alpha + c_2 \alpha\).

Example 1

The n-tuple space, \(F^n\)

Example 2

The space of \(m\times n\) matrices \(F^{m \times n}\)

Example 3

The space of functions from a set \(S\) to a field \(F\).

Example 4

The space of polynomial functions over field \(F\).

\[f(x) = c_0 + c_1 x + \dots + c_n x^n\]

Example 5

Field of complex numbers \(C\) as a vector space over the field \(R\) of real numbers.

Remarks

• \(0\alpha = 0\).

• If \(c \neq 0\) and \(c \alpha = 0\), then \(\alpha = 0\).

• \((-1)\alpha = -\alpha\).

Definition

A vector \(\beta \in V\) is said to be a linear combination of the vectors \(\alpha_1, \dots, \alpha_n \in V\) provided there exist scalars \(c_1, \dots c_n \in F\) such that

\[\begin{split}\beta &= c_1 \alpha_1 + \dots + c_n \alpha_n\\ &= \sum_{i=1}^{n}c_i \alpha_i\end{split}\]

3.4.2. Subspaces

Definition

Let \(V\) be a vector space over the field \(F\). A subspace of \(V\) is a subset \(W \subset V\) which is itself a vector space over \(F\) with the operations of vector addition and scalar multiplication on \(V\).

Theorem 1

A non-empty subset \(W\) of \(V\) is a subspace of \(V\) if and only if for each pair of vectors \(\alpha, \beta\) in \(W\), and each scalar \(c\) in \(F\), the vector \(c\alpha + \beta\) is again in \(W\).

Example 6

• If \(V\) is a vector space, \(V\) is a subspace of \(V\).

• The subset \(\{0\}\) is a subspace of \(V\) called the zero subspace of \(V\).

• In \(F^n\), the set of n-tuples \((x_1, \dots, x_n)\) with \(x_1 = 0\) is a subspace of \(F^n\). The set of n-tuples with \(x_1 = 1 + x_2\) is not a subspace.

• The space of polynomial functions over the field \(F\) is a subspace of the space of all functions from \(F\) into \(F\).

• An \(n \times n\) (square) matrix \(A\) over the field \(F\) is symmetric if \(A_{i j} = A_{j i} \forall i,j\). The symmetric matrices form a subspace of the space of all \(n \times n\) matrices over \(F\).

• An \(n \times n\) matrix \(A\) over the field \(C\) of complex numbers is Hermitian if

  \[A_{j k} = \overline{A_{k j}} \quad \forall j,k.\]

The set of Hermitian matrices is not a subspace of the space of all \(n \times n\) matrices over \(C\).

Example 7

The solution space of a system of homogeneous linear equations.

• Let \(A\) be an \(m\times n\) matrix over field \(F\).

• The set of all \(n \times 1\) (column) matrices over \(F\) such that \(AX = 0\) is a subspace of of the space of all \(n \times 1\) matrices over \(F\).

Lemma

If \(A\) is an \(m \times n\) matrix over \(F\) and \(B,C\) are \(n \times p\) matrices over \(F\), then

\[A (dB + C) = d(AB) + AC \quad \forall d \in F.\]

Theorem 2

Let \(V\) be a vector space over the field \(F\). The intersection of any collection of subspaces of \(V\) is a subspace of \(V\).

Definition

Let \(S\) be a set of vectors in a vector space \(V\). The subspace spanned by \(S\) is defined to be the intersection \(W\) of all subspaces of \(V\) which contain \(S\). When \(S\) is a finite set of vectors, \(S = \{\alpha_1, \dots, \alpha_n\}\), we shall simply call \(W\) the subspace spanned by the vectors \(\alpha_1, \dots \alpha_n\).

Theorem 3

The subspace spanned by a non-empty subset \(S\) of a vector space \(V\) is a set of all linear combinations of vectors in \(S\).

Definition

If \(S_1, S_2, \dots, S_k\) are subsets of a vector space \(V\), the set of all sums

\[\alpha_1 + \alpha_2 + \dots + \alpha_k\]

of vectors \(\alpha_i \in S_i\) is called the sum of the subsets \(S_1, S_2, \dots, S_k\) and is denoted by

\[S_1 + S_2 + \dots + S_k\]

or by

\[\sum_{i=1}^{k} S_i\]

If \(W_1, W_2, \dots, W_k\) are subspaces of \(V\), then the sum

\[W = W_1 + W_2 + \dots + W_k\]

is easily seen to be a subspace of \(V\) which contains each of the subspaces \(W_i\). \(W\) is the subspace spanned by the union of \(W_1, W_2, \dots, W_k\).

Example 10

Let \(A\) be an \(m \times n\) matrix over a field \(F\). The row vectors of \(A\) are the vectors in \(F^n\) given by \(\alpha_i = (A_{i1}, \dots , A_{in}), i = 1,\dots,m\). The subspace of \(F^n\) spanned by the row vectors of \(A\) is called the row space of \(A\).

Example 11

Let \(V\) be the space of all polynomial functions over \(F\). Let \(S\) be the subset of \(V\) consisting of the polynomial functions \(f_0, f_1, f_2,\dots\) defined by:

\[f_n(x) = x^n, \quad n = 0,1,2,\dots\]

Then \(V\) is the subspace spanned by the set \(S\).

3.4.3. Bases and dimensions

Definition

Let \(V\) be a vector space over \(F\). A subset \(S\) of \(V\) is said to be linearly dependent (or simply dependent) if there exist distinct vectors \(\alpha_1, \alpha_2, \dots, \alpha_n\) in \(S\) and scalars \(c_1, c_2, \dots, c_n\) in \(F\), not all of which are 0, such that:

\[c_1 \alpha_1 + c_2 \alpha_2 + \dots + c_n \alpha_n = 0\]

A set which is not linearly dependent is called linearly independent. If the set S contains only finitely many vectors \(\alpha_1, \alpha_2, \dots, \alpha_n\), we sometimes say that \(\alpha_1, \alpha_2, \dots, \alpha_n\) are dependent (or independent) instead of saying \(S\) is dependent (or independent).

• Any set which contains a linearly dependent set is linearly dependent.

• Any subset of a linearly independent set is linearly independent.

• Any set which contains the 0 vector is linearly dependent.

• A set \(S\) of vectors is linearly independent if and only if each finite subset of \(S\) is linearly independent, i.e. if and only if for any distinct vectors \(\alpha_1, \alpha_2, \dots, \alpha_n \in S\), \(c_1 \alpha_1 + c_2 \alpha_2 + \dots + c_n \alpha_n = 0\) implies that each \(c_i = 0\).

Definition

Let \(V\) be a vector space. A basis for \(V\) is a linearly independent set of vectors in \(V\) which spans the space \(V\). The space \(V\) is finite -dimensional if it has a finite basis.

Example 13

Let \(F\) be a field and in \(F^n\) let \(S\) be the subset consisting of vectors \(\epsilon_1, \epsilon_2, \dots, \epsilon_n\) defined by:

\[\begin{split}\epsilon_1 &= (1,0,0,\dots, 0)\\ \epsilon_2 &= (0,1,0,\dots, 0)\\ &\dots\\ \epsilon_n &= (0,0,0,\dots, 1)\end{split}\]

Lets \(x_1, x_2, \dots, x_n\) be scalars in \(F\) and put

\[\alpha = x_1 \epsilon_1 + x_2 \epsilon_2 + \dots + x_n \epsilon_n\]

Then

\[\alpha = (x_1, x_2, \dots, x_n)\]

This shows that \(\epsilon_1, \epsilon_2, \dots, \epsilon_n\) span \(F^n\). Since \(\alpha = 0\) if and only if \(x_1 = x_2 = \dots = x_n = 0\), the vectors \(\epsilon_1, \epsilon_2, \dots, \epsilon_n\) are linearly independent. The set \(S = \{\epsilon_1, \epsilon_2, \dots, \epsilon_n\}\) is accordingly a basis for \(F^n\). We shall call particular basis the standard basis of \(F^n\).

Example 14

• Let \(P\) be an invertible \(n \times n\) matrix with entries in the field \(F\).

• Then \(P_1, \dots, P_n\), the columns of \(P\), form a basis for the space of column matrices, \(F^{n\times 1}\).

• If \(X\) is a column matrix, then

\[PX= x_1P_1 + \dots + x_n P_n\]

• Since \(PX = 0\) has only the trivial solution \(X = 0\), it follows that \(\{P_1, \dots, P_n\}\) is a linearly independent set.

• Why does it span \(F^{n \times 1}\) ?

• Let \(Y\) be any column matrix.

• If \(X = P^{-1} Y\), then \(Y = PX\), i.e.

  \[Y = x_1 P_1 + \dots + x_n P_n.\]

• So \(\{P_1, \dots, P_n\}\) is a basis for \(F^{n \times 1}\).

Example 15

TBD

Example 16

Polynomial functions infinite basis

• Let \(F\) be a subfield of complex numbers and let \(V\) be the space of polynomial functions over \(F\).

  \[f(x) = c_0 + c_1 x + \dots + c_n x^n.\]

• Let \(f_k(x) = x^k, k = 0,1,2,\dots\)

• The infinite set \(\{f_0, f_1, \dots\}\) is a basis for \(V\).

• The set spans \(V\), because the function \(f\) above can be expressed as:

  \[f = c_0 f_0 + c_1 f_1 + \dots + c_n f_n.\]

• The set \(\{f_0, f_1, \dots\}\) is independent if every finite subset of it is independent.

• Consider set \(\{f_0, f_1, \dots, f_n\}\).

• Suppose that:

  \[c_0f_0 + c_1f_1 + \dots + c_n f_n = 0\]

  i.e.

  \[c_0 + c_1 x + c_n x^n = 0 \quad \forall x \in F.\]

• It means: every \(x \in F\) is a root of the polynomial \(f(x) = c_0 + c_1 x + c_n x^n\).

• A polynomial of degree \(n\) cannot have more than \(n\) distinct roots.

• Hence \(c_0 = c_1 = \dots = c_n = 0\).

• Hence the set \(\{f_0, f_1, \dots, f_n\}\) is linearly independent.

Is the vector space \(V\) infinite dimensional?

• Assume that \(V\) has a finite basis.

• Suppose polynomials \(\{g_1, \dots, g_r\}\) form the basis.

• There will be a largest power of \(x\) which appears in one of the \(g_i\).

• If the power is \(k\), then clearly \(f_{k+1}(x) = x^{k+1}\) is not in the span of \(\{g_1, \dots, g_r\}\).

• Hence \(V\) is not finite dimensional.

Theorem 4

Let \(V\) be a vector space which is spanned by a finite set of vectors \(\beta_1, \beta_2, \dots \beta_m\). Then any independent set of vectors in \(V\) is finite and contains no more than \(m\) elements.

Corollary 1

If \(V\) is a finite-dimensional vector space, then any two bases of \(V\) have the same (finite) number of elements.

Definition

The dimension of a finite-dimensional vector space \(V\) is defined as the number of elements in a basis for \(V\). This is denoted by \(\dim V\).

Corollary 2

Let \(V\) be a finite-dimensional vector space and let \(n = \dim V\). Then

• any subset of \(V\) which contains more than \(n\) vectors is linearly dependent.

• No subset of \(V\) which contains less than \(n\) vectors can span \(V\).

Lemma

Let \(S\) be a linearly independent subset of a vector space \(V\). Suppose \(\beta\) is a vector in \(V\) which is not in the subspace spanned by \(S\). Then the set obtained by adjoining \(\beta\) to \(S\) is linearly independent.

Example 17

TBD

Theorem 5

If \(W\) is a subspace of a finite-dimensional vector space \(V\), every linearly independent subset of \(W\) is finite and is part of a (finite) basis for \(W\).

Corollary 1

If \(W\) is a proper subspace of a finite-dimensional vector space \(V\), then \(W\) is finite dimensional and \(\dim W < \dim V\).

Corollary 2

In a finite dimensional vector space \(V\) every non-empty linearly independent set of vectors is part of a basis.

Corollary 3

Let \(A\) be an \(n \times n\) matrix over a field \(F\), and suppose the row vectors of \(A\) form a linearly independent set of vectors in \(F^n\). Then \(A\) is invertible.

Theorem 6

If \(W_1\) and \(W_2\) are finite-dimensional subspaces of a vector space \(V\), then \(W_1 + W_2\) is finite-dimensional and

\[\dim W_1 + \dim W_2 = dim (W_1 \cap W_2) + dim (W_1 + W_2)\]

3.4.4. Coordinates

A basis \(\mathfrak{B}\) in an n-dimensional space \(V\) enables one to introduce coordinates in \(V\) analogous to the ‘natural coordinates’ \(x_i\) of a vector \(\alpha = (x_1, \dots, x_n)\) in the space \(F^n\).

Definition

IF \(V\) is a finite-dimensional vector space, an ordered basis for \(V\) is a finite sequence of vectors which is linearly independent and spans \(V\).

• If \(\alpha_1, \dots, \alpha_n\) is an ordered basis for \(V\), then the set \(\{\alpha_1, \dots, \alpha_n\}\) is a basis for \(V\).

• Ordered basis is a set together with a specified ordering.

• We shall abuse the notation and say that:

  \[\mathfrak{B} = \{\alpha_1, \dots, \alpha_n\}\]

  is an ordered basis for \(V\).

• Given \(\alpha \in V\), there is a unique n-tuple \(x = (x_1, x_2, \dots, x_n)\) such that:

  \[\alpha = \sum_{i=1}^{n} x_i \alpha_i\]

• We shall call \(x_i\) the ith coordinate of \(\alpha\) relative to the ordered basis \(\mathfrak{B}\).

• If

  \[\beta = \sum_{i=1}^{n} y_i \alpha_i\]

• then

  \[\alpha + \beta = \sum_{i=1}^{n} (x_i + y_i) \alpha_i\]

• The ith coordinate of \((\alpha + \beta)\) in the ordered basis is \((x_i + y_i)\).

• Similarly ith coordinate of \(c\alpha\) is \(c x_i\).

• Every n-tuple \(x = (x_1, x_2, \dots, x_n)\) in \(F^n\) is the n-tuple of coordinates of some vector in \(V\) namely the vector:

  \[\sum_{i=1}^{n} x_i \alpha_i\]

• Each ordered basis for \(V\) determines a one-one correspondence

  \[\alpha \mapsto (x_1, \dots, x_n)\]

  between the set of all vectors in \(V\) and the set of all n-tuples in \(F^n\).

Definition

Coordinate matrix of \(\alpha\) relative to the ordered basis \(\mathfrak{B}\):

\[\begin{split}X = \left[\begin{array} {c} x_1\\ \vdots \\ x_n \end{array}\right]\end{split}\]

To indicate the dependence of this coordinate matrix on the basis, we shall use the symbol

\[[\alpha]_{\mathfrak{B}}\]

Theorem 7

Let \(V\) be an n-dimensional vector space over the field \(F\), and let \(\mathfrak{B}\), and \(\mathfrak{B}'\) be two ordered bases of \(V\). Then there is a unique, necessarily invertible, \(n \times n\) matrix \(P\) with entries in \(F\) such that

• \([\alpha]_{\mathfrak{B}} = P [\alpha]_{\mathfrak{B}'}\)

• \([\alpha]_{\mathfrak{B}'} = P^{-1} [\alpha]_{\mathfrak{B}}\)

for every vector \(\alpha\) in \(V\). The columns of \(P\) are given by

\[P_j = [\alpha'_j]_{\mathfrak{B}}\]

where

\[ \begin{align}\begin{aligned}\mathfrak{B} = \{ \alpha_1, \dots, \alpha_n \}\\\mathfrak{B}' = \{ \alpha'_1, \dots, \alpha'_n \}\end{aligned}\end{align} \]

Thus \(P\) represents the basis vectors in \(\mathfrak{B}'\) in terms of basis vectors in \(\mathfrak{B}\).

Theorem 8

Suppose \(P\) is an \(n \times n\) invertible matrix over \(F\). Let \(V\) be an n-dimensional vector space over \(F\), and let \(\mathfrak{B}\) be an ordered basis of \(V\). Then there is a unique ordered basis \(\mathfrak{B}'\) of \(V\) such that

• \([\alpha]_{\mathfrak{B}} = P [\alpha]_{\mathfrak{B}'}\)

• \([\alpha]_{\mathfrak{B}'} = P^{-1} [\alpha]_{\mathfrak{B}}\)

for every vector \(\alpha\) in \(V\).

Example 18

Let \(F\) be a field and let

\[\alpha = (x_1, x_2, \dots, x_n)\]

be a vector in \(F^n\). If \(\mathfrak{B}\) is the standard ordered basis of \(F^n\),

\[\mathfrak{B} = {\epsilon_1, \epsilon_2, \dots,\epsilon_n}\]

the coordinate matrix of the vector \(\alpha\) in the basis \(\mathfrak{B}\) is given by :

\[\begin{split}[\alpha]_{\mathfrak{B}} = \left[ \begin{array}{c} x_1\\x_2\\\vdots\\x_n \end{array} \right]\end{split}\]

Let \(R\) be the field of real numbers and let \(\theta\) be a fixed real number. The matrix

\[\begin{split}P = \left[ \begin{array}{rr}\cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{array} \right]\end{split}\]

is invertible with inverse:

\[\begin{split}P^{-1} = \left[ \begin{array}{rr}\cos\theta & \sin\theta \\ -\sin\theta & \cos\theta \end{array} \right]\end{split}\]

Thus for each \(\theta\), the set consisting of vectors \((\cos\theta, \sin\theta), (-\sin\theta,\cos\theta)\) is a basis for \(R^2\); intuitively the basis may be described as the one obtained by rotating the standard basis through the angle \(\theta\).

If \(\alpha\) is the vector \((x_1, x_2)\) then,

\[\begin{split}[\alpha]_{\mathfrak{B}'} = P^{-1} [\alpha]_{\mathfrak{B}} = \left[ \begin{array}{rr}\cos\theta & \sin\theta \\ -\sin\theta & \cos\theta \end{array} \right] \left[\begin{array}{c}x_1 \\ x_2 \end{array} \right]\end{split}\]

or

\[\begin{split}x'_1 &= x_1 \cos\theta + x_2 \sin\theta\\ x'_2 &= -x_1 \sin\theta + x_2 \cos\theta.\end{split}\]

3.4.5. Summary of row-equivalence

• If \(A\) is an \(m \times n\) matrix over the field \(F\) the row vectors of \(A\) are the vectors \(\alpha_1, \dots, \alpha_n\) in \(F^n\) defined by:

  \[\alpha_i = (A_{i 1}, \dots, A_{i n})\]

• The row space of \(A\) is the subspace of \(F^n\) spanned by these vectors.

Definition

The row rank of \(A\) is the dimension of the row space of \(A\).

• If \(P\) is a \(k \times m\) matrix over \(F\), then the product \(B = PA\) is a \(k \times n\) matrix whose row vectors \(\beta_1, \dots, \beta_n\) are linear combinations

  \[\beta_i = P_{i 1} \alpha_1 + \dots + P_{i m} \alpha_m\]

of the row vectors of \(A\).

• Visualizing as a matrix multiplication:

  \[\begin{split}\begin{bmatrix} \beta_1 \\ \vdots \\ \beta_k\end{bmatrix} = \begin{bmatrix} P_{1 1} & \ldots & P_{1 m} \\ \vdots & \ddots & \vdots \\ P_{k 1} & \ldots & P_{k m} \end{bmatrix} \begin{bmatrix} \alpha_1 \\ \vdots \\ \alpha_m \end{bmatrix}\end{split}\]

• The row space of \(B\) is a subspace of row space of \(A\).

• If \(P\) is an \(m \times m\) invertible matrix, then \(B\) is row-equivalent to \(A\).

  • \(A = P^{-1} B\)

  • Thus row-space of \(A\) is also a subspace of row space of \(B\).

Theorem 9

Row-equivalent matrices have the same row space.

Theorem 10

Let \(R\) be a non-zero row-reduced echelon matrix of \(A\). Then the non-zero row vectors of \(R\) form a basis for the row space of \(A\).

Theorem 11

Let \(m\) and \(n\) be positive integers and let \(F\) be a field. Suppose \(W\) is a subspace of \(F^n\) and \(\dim W \leq m\). Then there is precisely one \(m \times n\) row -reduced echelon matrix over \(F\) which has \(W\) as its row space.

Long proof here.

Corollary

Each \(m \times n\) matrix \(A\) is row-equivalent to one and only one row-reduced echelon matrix.

Corollary

Let \(A\) and \(B\) be \(m \times n\) matrices over the field \(F\). Then \(A\) and \(B\) are row equivalent if and only if they have the same row space.

If \(A\) and \(B\) are \(m \times n\) matrices over the field \(F\), the following statements are equivalent.

• \(A\) and \(B\) are row-equivalent.

• \(A\) and \(B\) have same row space.

• \(B = PA\), where \(P\) is an invertible \(m \times m\) matrix.

3.4.6. Computations concerning subspaces

Suppose we are given \(m\) vectors \(\alpha_1,\dots,\alpha_m\) in \(F^n\). We consider the following questions

• How does one determine if the vectors \(\alpha_1,\dots,\alpha_m\) are linearly independent?

• How does one find the dimension of the subspace \(W\) spanned by these vectors?

• Given \(\beta \in F^n\), how does one determine whether \(\beta\) is a linear combination of \(\alpha_1,\dots,\alpha_m\), i.e., whether \(\beta \in W\)?

• How can one give an explicit description of the subspace \(W\)?