Applied Linear Algebra - Packet 02

Packet 02

Tutorial: Vectors in 2D and 3D

Vector space: addition and scaling

A vector is an element in a vector space, which is a collection of objects for which someone has defined operations of addition and scaling:

Vector space operations

• $𝐯+𝐰$ is a vector when $𝐯,𝐰=\text{vectors}$
• $\lambda 𝐯$ is a vector when $𝐯=\text{vector},\lambda =\text{scalar}$

Properties of vector operations

• Commutativity: $𝐯+𝐰=𝐰+𝐯$
• Associativity: $𝐮+(𝐯+𝐰)=(𝐮+𝐯)+𝐰$
• Distributivity: $\lambda (𝐯+𝐰)=\lambda 𝐯+\lambda 𝐰$
• Zero vector: $𝐯+\mathrm{𝟎}=𝐯$

The elements $\lambda 𝐯$ also satisfy intuitive properties like: $1𝐯=𝐯$ and $(\lambda +\mu )𝐯=\lambda 𝐯+\mu 𝐯$ and $\lambda (\mu 𝐯)=(\lambda \mu )𝐯$

Scalars Scalars (“scale-ers”) could in theory be taken in any field, for example complex numbers, but in this class we use real number scalers $\lambda \in ℝ$.

Components of vectors

In weeks 1–4 we consider vectors in the plane ${ℝ}^2$, and vectors in space ${ℝ}^3$. Much of applied linear algebra involves generalizing the theory to vectors in ${ℝ}^n$ where $n$ is a very large number, but we don’t start doing that until week 5.

Vectors in ${ℝ}^2$ are ordered pairs $𝐯=(a_1,a_2)$ with $a_1,a_2\in ℝ$. Vectors in ${ℝ}^3$ are ordered triples $𝐯=(a_1,a_2,a_3)$ with $a_1,a_2,a_3\in ℝ$.

Addition is componentwise: $(a_1,a_2)+(b_1,b_2)=(a_1+b_1,a_2+b_2)$ Scalars act simultaneously across all components: $\lambda (a_1,a_2)=(\lambda a_1,\lambda a_2)$

Standard basis vectors These special vectors have their own symbols:

$${𝐞}_1=(\mathrm{1,0,0}),{𝐞}_2=(\mathrm{0,1,0}),{𝐞}_3=(\mathrm{0,0,1}).$$

In ${ℝ}^n$ there will be standard basis vectors as well:

$${𝐞}_i=(0,\dots ,0,1^{\underbrace{i^{th}}},0,\dots ,0).$$

Components Usually the terminology ‘component of a vector’ refers to one of the numbers $a_i$ in the ordered sequence $(a_1,a_2)$ or $(a_1,a_2,a_3)$. Sometimes ‘component’ refers to the basis vector scaled by this number, e.g. $a_1{𝐞}_1$.

Notice that we always have $(a_1,a_2,a_3)=a_1{𝐞}_1+a_2{𝐞}_2+a_3{𝐞}_3$, which explains why ‘component’ is sometimes used in this second way.

Other bases It is possible to select a different set of special vectors, e.g. $𝐁=\{{𝐮}_1,{𝐮}_2,{𝐮}_3\}$, having the property that any vector $𝐯$ can be written uniquely as a ‘linear combination’:

$$𝐯=a_1{𝐮}_1+a_2{𝐮}_2+a_3{𝐮}_3.$$

Any such set is called a basis, and the $a_i$ in this case are called components in the basis $𝐁$.

(The ‘unique’ aspect means that the coefficient numbers $a_1,a_2,a_3$ are determined once the vector $𝐯$ is determined.)

Geometric vectors

It is possible to consider vectors geometrically as quantities with direction and magnitude in space. (Provided one has defined space purely geometrically, which nobody does.)

Dot product

The vector space ${ℝ}^n$ (any $n$) comes with a type of product called inner product or dot product or scalar product. Every inner product satisfies basic rules of products: Algebraic properties of every inner product

• $𝐮\cdot 𝐯=𝐯\cdot 𝐮\in ℝ$
• $(𝐮+𝐯)\cdot 𝐰=𝐮\cdot 𝐰+𝐯\cdot 𝐰$
• $(\lambda 𝐮)\cdot 𝐯=\lambda (𝐮\cdot 𝐯)$
• $\mathrm{𝟎}\cdot 𝐮=0$

Component definition of the standard dot product The dot products for ${ℝ}^2$ and ${ℝ}^3$ are defined using components:

• $(a_1,a_2)\cdot (b_1,b_2)=a_1{b}_1+a_2{b}_2$
• $(a_1,a_2,a_3)\cdot (b_1,b_2,b_3)=a_1{b}_1+a_2{b}_2+a_3{b}_3$

Norm The dot product of a vector with itself gives the square of a number called the norm or length or magnitude:

• $𝐮\cdot 𝐮=|𝐮{|}^2=||𝐮|{|}^2$

The terminology “length” is justified by the Pythagorean Theorem in ${ℝ}^3$:

Exercise 02-01

Parallelogram identity

Let $𝐮=(\mathrm{4,0,3})$ and $𝐯=(-\mathrm{2,1,5})$. Compute $|𝐮-𝐯|$ and $|𝐮+𝐯|$ and $|𝐮|$ and $|𝐯|$. Verify that:

$$|𝐮+𝐯{|}^2+|𝐮-𝐯{|}^2=2|𝐮{|}^2+2|𝐯{|}^2.$$

Geometry

The dot product is a scalar number that provides information about the geometric relationship between vectors:

Cosine rule for dot products

$$𝐮\cdot 𝐯=|𝐮||𝐯|\cos (\theta )$$

where $\theta$ is the angle between $𝐮$ and $𝐯$.

Derivation of cosine rule for dot products

The rules for dot product imply that:

$$|𝐮-𝐯{|}^2=(𝐮-𝐯)\cdot (𝐮-𝐯)=𝐮\cdot 𝐮+𝐯\cdot 𝐯-2𝐮\cdot 𝐯.$$

On the other hand, the Law of Cosines for the triangle with sides $𝐮,𝐯,𝐮-𝐯$ gives:

$$|𝐮-𝐯{|}^2=|𝐮{|}^2+|𝐯{|}^2-2\cos (\theta )|𝐮||𝐯|.$$

It is worth noticing that the inner product can be computed entirely in terms of the norm:

$$𝐮\cdot 𝐯=\frac{1}{2}(|𝐮{|}^2+|𝐯{|}^2-|𝐮-𝐯{|}^2)$$

Orthogonality and dot product

The cosine formula for dot product has qualitative geometrical consequences:

• If $𝐮\cdot 𝐯=0$ then $𝐮$ and $𝐯$ are orthogonal (perpendicular vectors)
• If $𝐮\cdot 𝐯>0$ then $𝐮$ and $𝐯$ form an acute angle
• If $𝐮\cdot 𝐯<0$ then $𝐮$ and $𝐯$ form an obtuse angle
• If $𝐮\cdot 𝐯=|𝐮||𝐯|$ then $𝐮$ and $𝐯$ are parallel

Question 02-01

Orthogonality by hand

Determine which pairs of $𝐮=(2,-\mathrm{1,1})$ and $𝐯=(\mathrm{2,6,1})$ and $𝐰=(-\mathrm{4,1,2})$ are orthogonal.

Question 02-02

Dot product via unit vectors

Compute the dot product of $4{𝐞}_1-3{𝐞}_2$ and ${𝐞}_1+2{𝐞}_2+{𝐞}_3$.

Exercise 02-02

Angle between vectors

Find the angle between $(\mathrm{3,6,2})$ and $(\mathrm{4,2,4})$.

Exercise 02-03

Acute, orthogonal, or obtuse?

Are the angles between $𝐮=(1/\mathrm{2,1}/\mathrm{2,5})$ and $𝐯=(\mathrm{3,1},-2)$ and $𝐰=(4,-\mathrm{3,0})$ obtuse or acute?

Unit vectors A vector $𝐮$ with $|𝐮|=1$ is called a unit vector. So, ${𝐞}_1,{𝐞}_2,{𝐞}_3$ are unit vectors.

Thinking geometrically, a unit vector carries the information of direction. We could call unit vectors ‘direction vectors’ instead.

Any vector can be renormalized to create a unit vector by dividing out its length:

$${𝐞}_{𝐮}=\frac{1}{|𝐮|}𝐮.$$

Some texts, especially in physics, use the notation $\widehat{𝐮}$ instead of ${𝐞}_{𝐮}$ for the unit vector that points in the direction of $𝐮$.

Projection The dot product is a quantity telling how much one vector aligns with another. The dot product of two unit vectors is $\cos (\theta )$ where $\theta$ is the angle between them.

One can use the dot product to find the components of a vector in the standard basis. Suppose $𝐮=(a_1,a_2,a_3)$ is any vector. Then $𝐮\cdot {𝐞}_i=a_i$, so:

$$𝐮=(𝐮\cdot {𝐞}_1){𝐞}_1+(𝐮\cdot {𝐞}_2){𝐞}_2+(𝐮\cdot {𝐞}_3){𝐞}_3.$$

More generally, one defines the projection of $𝐮$ along $𝐯$ (projection along the direction of $𝐯$) using the dot product:

$${𝐮}_{||}={\mathrm{proj}}_{𝐯}(𝐮)=(𝐮\cdot {𝐞}_{𝐯}){𝐞}_{𝐯}=\left(\frac{𝐮\cdot 𝐯}{𝐯\cdot 𝐯}\right)𝐯$$

The scalar coefficient $𝐮\cdot {𝐞}_{𝐯}$ is called the parallel component of $𝐮$ along $𝐯$. (Or, sometimes, ${𝐮}_{||}$ is called the component.)

The projection ${𝐮}_{||}$ points in the direction of $𝐯$, and it says how much of $𝐮$ lies in that direction.

Another vector called the perpendicular component is given by subtracting the parallel part:

$${𝐮}_{⟂}=𝐮-{𝐮}_{||}$$

It is called ‘perpendicular’ because:

$${𝐮}_{⟂}\cdot 𝐯=𝐮\cdot 𝐯-\left(\frac{𝐮\cdot 𝐯}{𝐯\cdot 𝐯}\right)𝐯\cdot 𝐯=0$$

Exercise 02-04

Projection by hand

Let $𝐮=(\mathrm{5,1},-3)$ and $𝐯=(\mathrm{4,4,2})$.

• (a) Find the projection of $𝐮$ to the direction of $𝐯$.
• (b) Write the vector $𝐮$ in terms of parts parallel and perpendicular to $𝐯$.

Cauchy-Schwarz inequality The Cauchy-Schwarz inequality says that for any vectors $𝐮$, $𝐯$, we know:

$$|𝐮\cdot 𝐯|\le |𝐮||𝐯|.$$

Exercise 02-05

Cauchy-Schwarz

Prove the Cauchy-Schwarz inequality using the angle formula.

The Cauchy-Schwarz inequality is very important in math. There is a version of this inequality in many other settings where it is harder to prove.

Triangle inequality The Triangle inequality says that for any vectors $𝐮$, $𝐯$, we know:

$$|𝐮+𝐯|\le |𝐮|+|𝐯|.$$

Exercise 02-06

Cauchy-Schwarz $\Rightarrow$ Triangle

Prove the Triangle inequality using the Cauchy-Schwarz inequality by “foiling” $|𝐮+𝐯{|}^2$ after writing it using the dot product.

Exercise 02-07

Orthogonality identity

Suppose that vectors $𝐮$ and $𝐯$ satisfy $|𝐮+𝐯{|}^2=|𝐮{|}^2+|𝐯{|}^2$. Show that they must be orthogonal. (Hint: use the “foiling” trick in the previous exercise.)

Lines and planes using vectors

Lines Given a point identified with a vector ${𝐫}_{\mathrm{𝟎}}$ and a direction vector $𝐯$, the line through ${𝐫}_{\mathrm{𝟎}}$ in direction $𝐯$ is given parametrically by the formula:

$$𝐫(t)={𝐫}_0+t𝐯$$

Given two points ${𝐫}_{\mathrm{𝟎}}$ and ${𝐫}_{\mathrm{𝟏}}$, the line passing through these two points is given parametrically by:

$$𝐫(t)=(1-t){𝐫}_0+t{𝐫}_1$$

The line segment from ${𝐫}_0$ to ${𝐫}_1$ corresponds to $t\in [\mathrm{0,1}]$. The point $𝐫(t)$ may be considered to be a weighted average of ${𝐫}_0$ and ${𝐫}_1$ with weights $1-t$ and $t$.

Planes The scalar equation of a plane in ${ℝ}^3$ gives a plane as the set of points $(x,y,z)$ satisfying a single linear equation, such as:

$$ax+by+cz+d=0.$$

Other formulations of this equation can display more meaning. A plane is determined by the vector equation of a plane by the data of a single point ${𝐫}_0$ through which it passes, together with a normal vector $𝐧$ that is perpendicular to the plane. Such an equation of a plane can be formed by simply stating that vectors in this plane emanating ${𝐫}_0$ are perpendicular to $𝐧$:

$$(𝐫-{𝐫}_0)\cdot 𝐧=0.$$

If we have components $𝐫=(x,y,z)$ and ${𝐫}_0=(x_0,y_0,z_0)$ and $𝐧=(a,b,c)$, then this formula can be rewritten as:

$$a(x-x_0)+b(y-y_0)+c(z-z_0)=0.$$

This equation implies that the $a$, $b$, $c$ in the linear equation of the plane are just the components of its normal vector $𝐧$.

The normal vector of a plane is also useful for computing angles with respect to the plane. For example, the angle between two planes (the angle formed at their intersection) is equal to the angle between their normal vectors. (There is a Problem about this.)

Finally, a plane can be described parametrically by giving the component functions of a parametric surface, such as $𝐫:{ℝ}^2\to {ℝ}^3$ with components $({𝐫}_1(t,s),{𝐫}_2(t,s),{𝐫}_3(t,s))$. Each component function must be a linear function of $u$ and $v$. The parametric surface can also be described using vectors by giving one vector that identifies a point ${𝐫}_0$ through which the plane passes along with two additional vectors ${𝐫}_1$ and ${𝐫}_2$ lying in the plane:

$$𝐫(t,s)={𝐫}_0+t{𝐫}_1+s{𝐫}_2.$$

Question 02-03

Vector plane from scalar plane

Find the vector equation of a plane with scalar equation $7x-4y+2z+10=0$.

Question 02-04

Parametric plane from vector plane

Find a parametric equation of the plane passing through ${𝐫}_0=(\mathrm{3,1,0})$ having normal vector $𝐧=(\mathrm{3,2},-5)$. (Hint: write the equation for a vector from ${𝐫}_0$ that lies in the plane, and solve it to find two example vectors.)

Exercise 02-08

Line and plane intersection

Find the point at which the plane $3x-9y+2z-7=0$ and the line $𝐫(t)=(\mathrm{1,2,1})+t(-\mathrm{2,0,1})$ intersect.

Spans

Suppose we are given two vectors $𝐮,𝐯\in {ℝ}^3$. The span of these vectors is the 2D plane of vectors given by all possible linear combinations $a𝐮+b𝐯$ for any $a,b\in ℝ$. Sometimes the span is written using angle brackets:

$$\mathrm{span}\{𝐮,𝐯\}=⟨𝐮,𝐯⟩=\{a𝐮+b𝐯|a,b\in ℝ\}.$$

This set of vectors forms a plane passing through the origin $(\mathrm{0,0,0})$.

Question 02-05

Is it a span?

Consider the plane $\{(x,y,z)|z=1+2x-y\}$. Is this plane the span of two vectors?

The span of three vectors covering all of ${ℝ}^3$:

Example

Span of two vectors in ${ℝ}^3$

Problem: Suppose $𝐮=(\mathrm{1,2},-1)$ and $𝐯=(4,-\mathrm{1,3})$. Describe $⟨𝐮,𝐯⟩$ in vector form. Solution: By the definition, we have:

$$⟨𝐮,𝐯⟩=\{(a+4b,2a-b,-a+3b)|a,b\in ℝ\}.$$

To write this in vector form, we can use the cross product to find a vector perpendicular to both $𝐮$ and $𝐯$:

$$\begin{array}{cc}𝐧& =𝐮\times 𝐯\\ & =(2\cdot 3-(-1)(-1),-(1\cdot 3-4(-1)),1(-1)-4\cdot 2)\\ & =(5,-7,-9).\end{array}$$

Then in vector form the equation is $(x,y,z)\cdot (5,-7,-9)=0$, or in scalar form it is $5x-7y-9z=0$.

Cross product

The cross product $𝐮\times 𝐯$ is a vector perpendicular to both $𝐮$ and $𝐯$. If $𝐮=(a_1,a_2,a_3)$ and $𝐯=(b_1,b_2,b_3)$, then $𝐮\times 𝐯$ is the vector given in components by:

$$𝐮\times 𝐯=(a_2{b}_3-a_3{b}_2,-(a_1{b}_3-a_3{b}_1),a_1{b}_2-a_2{b}_1).$$

(The cross product is not very important for this class and will not be tested in exams, but it is very useful in 3D geometry for finding perpendicular vectors, and it will be required in the homework.)

Independence

A vector $𝐮$ is dependent on the vector $𝐯$ if there is an equation for $𝐮$ in terms of $𝐯$ alone:

$$𝐮=a𝐯\text{ for some }a\in ℝ,a\ne 0.$$

We say $𝐮$ is independent of $𝐯$ if there is no such equation.

Because $a\ne 0$ in the condition, any dependency can be rewritten as the opposite dependency: if $𝐮=a𝐯$, then $𝐯=a^{-1}𝐮$. So $𝐮$ is dependent on $𝐯$ if and only if $𝐯$ is dependent on $𝐮$, and we will also just say $\{𝐮,𝐯\}$ is a dependent set of vectors.

The span of a single non-zero vector is a line, while the span of two independent vectors is a plane. However, the span of two dependent vectors is still a line!

A vector $𝐮$ is dependent on a set of two vectors $\{𝐯,𝐰\}$ if there is an equation for $𝐮$ in terms of $𝐯$ and $𝐰$ using linear combinations:

$$𝐮=a𝐯+b𝐰\text{ for some }a,b\in ℝ,a\ne 0\text{ or }b\ne 0.$$

(Of course, if both $a=b=0$, this so-called dependency actually says $𝐮=0$.)

Notice that “$𝐮$ is dependent on $\{𝐯,𝐰\}$” is true exactly when $𝐮\in ⟨𝐯,𝐰⟩$, which is to say, when $𝐮$ lies in the plane determined by $𝐯$ and $𝐰$.

The set of three vectors $\{𝐮,𝐯,𝐰\}$ is said to be a dependent set when at least one of them is dependent on the others. This fact is equivalent to the truth of some equation:

$$a𝐮+b𝐯+c𝐰=0,$$

where the coefficients $a,b,c$ are not all equal to zero. The set is an independent set when there is no such equation; which is to say, any equation would have $a=b=c=0$.

Distances

Point to point The distance between $𝐩$ and $𝐪$ (more accurately, between the point identified by $𝐩$ and that identified by $𝐪$) is $|𝐩-𝐪|$.

Point to line The vector $𝐮$ from a point $𝐩$ to a line $𝐫(t)={𝐫}_0-t𝐯$, namely to the point on the line closest to $𝐩$, is given by taking the vector from $𝐩$ to ${𝐫}_0$ and removing the component parallel to $𝐯$:

$$𝐮=({𝐫}_0-𝐩)-(({𝐫}_0-𝐩)\cdot {𝐞}_{𝐯}){𝐞}_{𝐯}$$

The distance from $𝐩$ to $𝐫$ is then $|𝐮|$.

Projection gives minimal distance

One can verify the formula for $𝐮$ by minimizing the squared distance from $𝐩$ to $𝐫(t)$ using the standard calculus technique of minimizing a function by setting its derivative equal to zero.

Point to plane The vector $𝐮$ from a point $𝐩$ to the plane through ${𝐫}_0$ with normal vector $𝐧$ (that is, to the nearest point on the plane) is given by projecting $𝐩-{𝐫}_0$ onto $𝐧$, that is:

$$𝐮=((𝐩-{𝐫}_0)\cdot {𝐞}_{𝐧}){𝐞}_{𝐧}.$$

So the distance from $𝐩$ to this plane is:

$$|𝐮|=|(𝐩-{𝐫}_0)\cdot {𝐞}_{𝐧}|=(𝐩-{𝐫}_0)\cdot \frac{𝐧}{|𝐧|}.$$

Exercise 02-09

Distance between parallel planes

Find the distance from the plane given by $15x+3y-3z-10=0$ to the parallel plane given by $5x+y-z-3=0$.

Problems due 29 Jan, 12:00pm

Problem 02-01

Line of intersection between planes

Find a parametric equation for the line formed as the intersection of the two planes $6x-3y+z-5=0$ and $-x+y+5z-5=0$. What is the angle between these planes?

Problem 02-02

Span

Find $a,b,c$ for the scalar equation $ax+by+cz=0$ of a plane which is equal to $⟨𝐮,𝐯⟩$, where $𝐮=(\mathrm{2,0,1})$ and $𝐯=(-\mathrm{1,1,3})$.

Problem 02-03

Independence

Is this a dependent set of vectors?:

$$\{\left(\begin{array}{c}1\\ -2\\ 3\end{array}\right),\left(\begin{array}{c}4\\ 0\\ 2\end{array}\right),\left(\begin{array}{c}3\\ -2\\ 4\end{array}\right)\}$$

Problem 02-04

Distance point to line

Find the distance from the point $𝐩=(\mathrm{1,5},-2)$ to the line given by $𝐫(t)=(-\mathrm{2,3,1})+t(\mathrm{4,0},-1)$.