Applied Linear Algebra - Packet 03

Packet 03

Tutorial: Inventory of Matrices in 2D and 3D

Vectors can be considered from an abstract perspective as items in a space of vectors all of which satisfy the rules of the “game” of vectors, as well as from a concrete perspective as lists of numbers upon which the operations of vectors are defined (addition and scaling).

Similarly, matrices can be considered from an abstract perspective as “linear operations” upon an abstract space of vectors, or from a concrete perspective as arrays of numbers which operate upon the lists of numbers in concrete vectors.

Matrices acting on vectors

An matrix is an array of numbers. Given a matrix called , the numbers making it up are called “coefficients” and written for and . Sometimes we write . The first index always specifies the row, and the second index specifies the column. For example, we could have:

Suppose we have a 2D vector and a 3D vector :

Now the action of on and of on can be written out with these formulas:

Example

Matrix acting on a vector

Let be an matrix and let be an D vector. Using summation notation for large numbers of indices, the matrix times vector operation is given by the general formula for the row of the output vector, where :

Question 03-01

Summation notation

Write out the formula for the operation of a matrix on a D vector. (Convert the summation notation to ordinary sums.)

The formulas for the action of a matrix on a vector should be understood in two different ways.

• For concrete calculations, it is best to think of each row of the output vector as the dot product of the corresponding matrix row with the input vector.
• For conceptual and abstract understanding, it is better to think of the operation as sending each standard basis vector to the corresponding column vector of the matrix.

Let’s consider now the second way in more detail. A matrix can be considered as a list of column vectors. For example:

Then :

Recall the standard basis vectors are and . So:

Similarly, in general:

Diagonal matrices: stretching

The main diagonal of a matrix consists of the entries . A matrix is called diagonal when all entries are zero except on the main diagonal. The action of a diagonal matrix is to stretch each spatial dimension according to the corresponding diagonal number:

This action is very easy to express algebraically in terms of standard basis vectors:

In other words, standard basis vectors are sent to scalar multiples of themselves. This means that standard basis vectors are eigenvectors (“self-vectors”) for a diagonal matrix, and the diagonal numbers (the scalar multiples) are eigenvalues (“self-values”).

When the diagonal entries are all the same, the effect is to scale all of space by equal proportions:

In this case, when the transformation is called a dilation because vectors expand away from the origin, and when it is called a contraction because vectors shrink towards the origin.

Question 03-02

Stretching and area

Consider the unit square . What is the area of the image of this square under the action of the matrix ?

Shearing

A simple shear mapping (also shear transformation or transvection) performs a displacement of one coordinate by a proportion of another. The matrix of a 2D shear in is given by:

Horizontal lines remain horizontal, while vertical lines take on a new slope of . Interestingly, shearing preserves area. (We will see a very simple explanation later.)

There is also shear in :

A shear in 3D might look like this:

The shear takes place in the component corresponding to the row of the matrix with . There is a general definition of shear mapping in D space, but we do not use it.

Rotating

A rotation in 2D by the angle (counterclockwise) can be performed by a matrix:

Exercise 03-01

Rotation matrix

Using geometry and trig sum identities, verify that this is the correct formula for rotation of the point counterclockwise about the origin by the angle .

From the matrix for rotation by we can deduce that counterclockwise rotation by is given in matrix form by , which acts by , and clockwise rotation by is given in matrix form by , which acts by . Rotation by is given by , which acts by , and this is simply reflection through the origin.

Study of 3D rotations has myriad applications in science and engineering, but rotations in 3D are much more complicated than in 2D, and we do not investigate the theory very much here. Some basic rotations are given by simple 2D rotations in the separate pairs of coordinates:

These rotations are called yaw, pitch, and roll, respectively. (These terms apply to an airplane flying in the -plane in the axis direction.) Simple rotations like these in two chosen axes are also called Givens rotations (after J. Wallace Givens). They are easily generalized to D space.

Question 03-03

Positive pitch

Verify from the formulas that a positive pitch means an airplane is ascending. Does a positive roll imply an airplane is tilting to the right or to the left?

Projecting

A projection matrix acts on vectors by sending them to the nearest point in some chosen linear subspace, for example on a line (in 2D space) or on a plane (in 3D space). The chosen subspace has lower dimension than all of space, and the image under projection is like a shadow of the original.

Consider the projection of to the direction of . Let us assume that is a unit vector; then the projection formula simplifies to:

We can express this formula using a matrix. In 2D we have:

and

The matrix actually has a name: it is called the outer product , where is the matrix . In more detail: is a row vector matrix which, when applied to any vector , gives the scalar quantity . This horizontal matrix can be embedded inside each row of the vector by multiplying by the component in that row of and substituting the result into the row, thereby converting the vector into a matrix. The action of this matrix on is first to take the dot product , and then to apply this scalar to each row of , in other words multiplying overall by this scalar.

In conclusion, the operation is a linear mapping, and it can be represented as the action of a matrix:

Now if is not a unit vector, this matrix is incorrect. The matrix can be corrected by multiplying every coefficient by , which is .

Exercise 03-02

Computing a projection matrix

Find the matrix of the projection of in the direction of the vector . (First compute the unit vector.)

Exercise 03-03

Double projections

Consider a unit vector and another vector . Show that . Do this by applying the matrix of the projection of to the vector twice in sequence (meaning you apply the matrix again on the output vector of the first application of the matrix). To simplify the result, remember that because is a unit vector.

The same matrix formula can be extended to 3D and D. In 3D the matrix would be:

For a unit vector, this matrix acts on by giving , the projection to the direction of . We can see this because it clearly gives .

It is good to highlight the matrices of projection to the standard basis vectors:

Question 03-04

Projection to standard basis

Check that sends and sends . What is the matrix of in 3D, for ?

Reflecting

A reflection matrix is a matrix that acts on vectors by reflecting them across a hyperplane that passes through the origin. These operations are called Householder reflections.

(A hyperplane is a plane of one dimension lower than the surrounding space. So, a hyperplane in 2D is a line, and a hyperplane in 3D is a 2D plane. A hyperplane in D would be an D plane.)

The matrix of a reflection can be derived from the matrix of a certain projection by considering a formula for the reflection in terms of dot products.

If is any vector, the reflection of in the hyperplane through the origin, perpendicular to the vector is given by:

To see how the formula works, consider that is the component of perpendicular to , meaning that it lies in the hyperplane of reflection. Adding to recovers by bringing back the parallel part on the same side of the hyperplane as . Subtracting the term a second time does the opposite, it restores the part parallel to but on the other side of the hyperplane.

Reasoning via the hyperplane is not strictly necessary in 2D, but the same reasoning also works in D. In 3D the same formula gives the reflection across the plane through the origin, perpendicular to the vector . Here, gives the component of coming out of this plane in along . Subtracting once gives a vector inside the plane; subtracting twice gives the mirror image vector on the opposite side of the plane.

A 2D reflection in the hyperplane perpendicular to can be represented as the action of a matrix:

The action of this matrix sends a vector to the vector . (Check this fact yourself!)

Exercise 03-04

Computing a reflection matrix

Compute the matrix representing 3D reflection in the plane perpendicular to the vector . (First find the unit vector!)

Permuting

A permutation is a way of reassigning some labels to a collection of items. For example, if some marbles numbered are placed (out of order) in bins numbered , then a permutation corresponds to withdrawing the marbles and placing them in the same bins but in a new order.

A permutation matrix is a matrix in which each column and each row has a single , and all other entries are . For example:

The action of a permutation matrix on a vector is simply to permute the rows of the vector:

Such a matrix acts by permuting rows because each row, with its single entry of , “picks out” the entry of the vector whose row corresponds to the column of this (all others are given times ); and since each column of the matrix has only a single , each row of the vector is “picked out” only one time.

The action of a permutation matrix on standard basis vectors is easy to describe: it permutes them.

Exercise 03-05

Finding a permutation matrix

Find the permutation matrix that acts as follows:

Problems due Monday 5 Feb 2024 by 12:00pm

Problem 03-01

Composing rotations

Write the matrix of rotation by and the matrix of rotation by . Now apply the first matrix to and the second matrix to the output of under the action of the first matrix. Show that the resulting vector is the same as the vector given by applying a single rotation by the angle . (You will need trig identities.)

Problem 03-02

Yaw and pitch do not commute

In this problem you will show that 3D rotations do not commute, meaning that the outcome of two rotations can be affected by the order in which the rotations are performed.

Write an arbitrary vector symbolically as . Rotate this vector by a yaw of using the yaw rotation matrix, and then rotate the output by a pitch of . Now do this in reverse: a pitch rotation of followed by a yaw rotation of . Show that the results do not agree! Can you visualize this by rotating your hand?

Problem 03-03

Finding matrices

• (a) Find a matrix that acts as follows:

• (b) Find a matrix that acts as follows:

• (c) Find a matrix that acts as follows:

• (d) Find the matrix of a horizontal shear that maps to and does not change .

Problem 03-04

Rotations via reflections

Given an angle for a counterclockwise rotation of the -plane, find two lines such that reflecting in these two lines in sequence performs the rotation by .

(This means that any rotation in the -plane can be accomplished by performing two reflections in sequence!)

Hint: try picking a very simple line as your first reflection line, e.g. an axis, and a more interesting line as your second reflection line.

You may find it convenient to consider that the formula

for a reflection matrix is satisfied by the matrix

(Expand the entries using double-angle formulas to see why!) This matrix gives the reflection through the line which meets the -axis at the origin at an angle of .