% DEFINE some information that will be populated throughout the course notes. \def \coursename {Linear Algebra} \def \coursecode {MATH 2221} \def \courseterm {Winter 2021} \def \instructorname {Nathan Johnston} % END DEFINITIONS % IMPORT the course note formatting and templates \input{course_notes_template} % END IMPORT %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \setcounter{chapter}{0} % Set to one less than the week number \chapter{Vectors} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% {\large This week we will learn about: \begin{itemize} \item What vectors are, \item How to manipulate vectors, and \item Linear combinations. \end{itemize}\bigskip\bigskip \noindent Extra reading and watching: \begin{itemize} \item Section 1.1 in the textbook \item Lecture videos \href{https://www.youtube.com/watch?v=ea6p2eb7mTQ&list=PLOAf1ViVP13jmawPabxnAa00YFIetVqbd&index=1}{1}, \href{https://www.youtube.com/watch?v=5H8nCvbLhfc&list=PLOAf1ViVP13jmawPabxnAa00YFIetVqbd&index=2}{2}, and \href{https://www.youtube.com/watch?v=7LDLe131WAI&list=PLOAf1ViVP13jmawPabxnAa00YFIetVqbd&index=3}{3} on YouTube \item \href{http://en.wikipedia.org/wiki/Euclidean_vector}{Vector} at Wikipedia \end{itemize}\bigskip\bigskip \noindent Extra textbook problems: \begin{itemize} \item[$\star$] 1.1.1--1.1.3, 1.1.5--1.1.8 \item[$\star \, \star$] 1.1.9--1.1.12 \item[$\star \star \star$] 1.1.13(a), 1.1.14, 1.1.15 \item[$\skull$] 1.1.13(b) \end{itemize}} \newpage % OPENING SCHPIEL: Up until now, calculus. This is what you thought "real math" was. It's not. Real math is linear algebra. It is the central subject of all of mathematics -- one that works so uncommonly and profoundly well that almost every other subject piles their messes on top of its magnificent foundation. % in calculus: completely obvious theorems that are so obnoxiously hard to prove taht they are left to 3rd year courses (extreme value theorem, intermediate value theorem, rolle's theorem, etc). LinAlg is the opposite: completely shocking things are straightforward to prove. Everything "just works". Linear algebra is one of the branches of mathematics where everything ``just works''. Most of the objects that we introduce in this course will seem rather simple and unremarkable at first, but we will be able to do a \emph{lot} with them. Some of the things we will be able to do are motivated very geometrically... \exx[8]{Lengths, angles, and deformations (oh my!)} % can be used to compute lengths and angles (in arbitrary dimensions) % can be used to describe deformations of objects in 2 or more dimensions (e.g., skew-ing or rotating an image): MANY IMAGES REQUIRED (see below) % THE POINT: Linear algebra is used to manipulate images, display 3D graphics, and so on. % Show an image (week1_apple_raw.png) and then show its shear (week1_apple_shear.png) and rotation (can be done right in PDFAnnotator) % Show 3D graphics? How do we implement motions of these objects in (e.g.) video games? Via linear transformations (which we will study). How do we represent 3D objects on a 2D screen? Via projections (which we will study). \noindent ...while other applications will involve sifting through huge amounts of data: \exx[8]{How does (well, \emph{did}) Google work?} % THE POINT: Linear algebra is used to curate and sift through large amounts of data, even though it is too high-dimensional to visualize. % Describe (at a high level for now!) that Google creates a matrix that describes how different pages on the web link to each other. % A link from a high-ranked page is worth more than a link from a low-ranked page. % The matrix is HUGE (about 4.75 billion web pages). % To determine each page's "rank", we compute something called an "eigenvector". We will learn how to do this in this course! \newpage \section*{Vectors} A \textbf{vector} is an ordered list of numbers like $(3,1)$. These lists can be as long as we like, but we'll start by considering 2-dimensional vectors in order to establish some intuition for how they work, since we can interpret them geometrically in this case. \\ \noindent Several different notations are used for vectors: \horlines{5} % $\overrightarrow{AB}$ or $\mathbf{v}$ or $\vec{v}$ (note that we will usually use \vec{v} since writing in bold is hard) % A is tail % B is head % O is the origin % Draw 2D plane to the right with points A, B, and a vector from A to B The \textbf{coordinates} or \textbf{entries} of a vector only tell us how far the vector stretches in the $x$- and $y$-directions; \textbf{not} where it is located in space. \exx[7]{Coordinates of vectors.} % Draw a 2D picture of two vectors that are the same, but one comes from the origin and the other doesn't. % Every vector can be translated so that its tail is at the origin (called "standard position"). % Emphasize that we will almost always consider vectors in standard position. % Coordinates of a vector are head minus tail. \noindent That is, vectors represent... \horlines{1} % motion or displacement, not absolute location. \newpage The order of the coordinates matters: for example, $(2,3) \neq (3,2)$. For this reason, 2D vectors are sometimes called ``ordered pairs''. In another math class, you might be introduced to objects called ``sets'', where order does \emph{not} matter. \smallskip \begin{itemize} \item Two vectors are equal if and only if... \horlines{2}\vspace*{-0.6cm} % all of their components are equal (iff they have the same length and direction). \item The \textbf{zero vector} is... \horlines{1}\vspace*{-0.6cm} % (0,0), simply denoted by \vec{0}. \item Recall that the set of all real numbers is denoted by $\mathbb{R}$. Similarly, \horlines{1}\vspace*{-0.7cm} % The set of all vectors with $2$ coordinates (and both coordinates are real numbers) is denoted by $\mathbb{R}^2$. \end{itemize} Sometimes we want to combine two (or more) vectors to get new ones. For example, we might want to think about what happens if we move along the path of multiple different vectors, one after another. Where do we end up after doing this? The answer is given by \textbf{vector addition}. \exx[6]{Vector addition.}\vspace*{-0.5cm} % Draw an example where we put w at the head of v, etc. Ask what the coordinates of w+v are, which leads to... \noindent Specifically, if $\mathbf{v} = (v_1,v_2)$ and $\mathbf{w} = (w_1,w_2)$, then $\mathbf{v} + \mathbf{w} = (v_1 + w_1, v_2 + w_2)$ is the vector from the tail of $\mathbf{v}$ to the head of $\mathbf{w}$. \\ \newpage Another common way to manipulate vectors is to ``scale'' them (or ``multiply by a scalar''). The idea here is that we want to move in the same direction as a given vector, but we want to change how \emph{far} we move in that direction. \exx[7]{Scalar multiplication.} % (1/2)v divides its length by 2 % 2v multiplies length by 2 % -v reverses direction % Draw these examples \noindent Specifically, if $\mathbf{v} = (v_1,v_2)$ and $c$ is a real number, then $c\mathbf{v} = (cv_1,cv_2)$ is the vector that points in the same direction as $\mathbf{v}$, but is $c$ times as long (and if $c < 0$ then the direction of the vector is reversed). \\ Finally, how do you think vector subtraction might be defined? If $\mathbf{v} = (v_1,v_2)$ and $\mathbf{w} = (w_1,w_2)$, then \begin{center}$\mathbf{v} - \mathbf{w} = \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad$\end{center} % (v_1 - w_1, v_2 - w_2) \noindent is the vector from the $\quad \quad \quad \quad \quad \quad \quad \quad \quad \quad$ to the $\quad \quad \quad \quad \quad \quad \quad \quad \quad \quad$. \\ % Head of w to the head of v (when both drawn in standard position). % Easy way to remember: (v-w)+w = v, so (v-w) must go from w to v. \exx[5]{Vector subtraction.} % Do a simple example like $(3,2) - (-1,3)$ and draw it. % Note that we will see another geometric interpretation of v-w (and v+w) shortly (next page) \newpage \exx[10]{Suppose that a regular hexagon has its center at the point $(0,0)$ and one of its corners at the point $(1,0)$. Find the sum of the $6$ vectors that point from its center to its corners.} % Start by drawing picture. % The sum is zero since we can match up the vectors in +/- pairs. Much easier than actually computing the coordinates of the vectors using trig. % Mention that this example illustrates that sometimes interpreting a problem geometrically can make life easier. This relationship between geometry and algebra is what makes linear algebra so powerful. If $\mathbf{v}$ and $\mathbf{w}$ are the (non-parallel) sides of a parallelogram, then $\mathbf{v} + \mathbf{w}$ and $\mathbf{v} - \mathbf{w}$ appear very naturally in that parallelogram... \exx[7]{The parallelogram rule.} % Draw a parallelogram with v and w as sides, and v+w and v-w as its diagonals. Also make note of this general fact. \newpage \section*{3-Dimensional Vectors} Everything we have learned about vectors so far extends naturally to $3$ dimensions. \horlines{1}\vspace*{-0.4cm} %\begin{center}\textbf{\color{BrickRed}There is nothing special about 2 dimensions!}\end{center} A vector in $3$ dimensions is an \textbf{ordered triple} like $(1,3,2)$, and the set of all ordered triples is denoted by $\mathbb{R}^3$. These are a bit harder to draw than their $2$-dimensional counterparts, but it's still possible... \exx[7]{Drawing 3D vectors.}\vspace*{-0.4cm} % Draw $(1,3,2)$. Emphasize that it's a bit of a mess, and it's not 100% agreed upon how to best place the axes. % When drawing by hand, I prefer x-axis to the right, y-axis into the page, and z-axis up. % Textbooks often to x-axis to the front-left, y-axis to the front-right, and z-axis up. % INSERT week1_draw_3d.png % Mention that you will never ask students to draw in 3D, but they need to know how to read a 3D drawing. \noindent Everything we have seen in 2D carries over exactly how you would expect in 3D: \begin{itemize} \item Adding vectors still has the geometric interpretation of ``following'' both vectors, one after the other. \item Adding vectors has the same formula you might expect:% if $\mathbf{v} = (v_1,v_2,v_3)$ and $\mathbf{w} = (w_1,w_2,w_3)$, then \horlines{1}\vspace*{-0.6cm} %$\mathbf{v} + \mathbf{w} = (v_1 + w_1, v_2 + w_2, v_3 + w_3)$. \item Scalar multiplication still has the geometric interpretation of stretching the vector. \item Scalar multiplication has the same formula you might expect:% if $\mathbf{v} = (v_1,v_2,v_3)$ and $c$ is a real number, then \horlines{1}\vspace*{-0.6cm} %$c\mathbf{v} = (cv_1, cv_2, cv_3)$. \end{itemize} \newpage \section*{High-Dimensional Vectors} Everything we have learned about vectors so far extends naturally to $4$ (and more!) dimensions. \horlines{1}\vspace*{-0.4cm} %\begin{center}\textbf{\color{BrickRed}There is nothing special about 3 dimensions!}\end{center} \noindent We'll get a bit more general now and consider an \emph{arbitrary} number of dimensions. \\ A vector in $n$ dimensions is an \textbf{ordered $\mathbf{n}$-tuple} like $(1,2,3,\ldots,n)$, and the set of all ordered $n$-tuples is denoted by $\mathbb{R}^n$. These are a bit harder to draw than their $2$- and $3$-dimensional counterparts... \exx[7]{Drawing 4D (and 5D, and 6D...) vectors.} % No, we're not actually going to draw these. Make a joke out of this -- just write "NOPE!" or something. % Mention though that a lot of the intuition that we will get for how these work draws from the 2D and 3D cases. But we don't make use of these vectors geometrically. However, everything \emph{algebraic} that we have seen for 2D and 3D vectors carries over exactly how you would expect in higher dimensions: \begin{itemize} \item Adding vectors has the same formula you might expect:% if $\mathbf{v} = (v_1,v_2,\ldots,v_n)$ and $\mathbf{w} = (w_1,w_2,\ldots,w_n)$, then \horlines{1}\vspace*{-0.6cm} % $\mathbf{v} + \mathbf{w} = (v_1 + w_1, v_2 + w_2, \ldots, v_n + w_n)$. \item Scalar multiplication has the same formula you might expect:% if $\mathbf{v} = (v_1,v_2,\ldots,v_n)$ and $c$ is a real number, then \horlines{1}\vspace*{-0.6cm} %$c\mathbf{v} = (cv_1, cv_2, \ldots, cv_n)$. \end{itemize} \newpage Even though we can't draw vectors in $\mathbb{R}^n$, we still want to be able to manipulate them. We have seen that vector addition and scalar multiplication work the ``naive'' way. The following theorem shows some more properties that are similarly ``obvious'': \begin{theorem}[Properties of Vector Operations] Let $\mathbf{v},\mathbf{w},\mathbf{x} \in \mathbb{R}^n$ be vectors and let $c,d \in \mathbb{R}$ be scalars. Then \begin{enumerate} \item $\mathbf{v} + \mathbf{w} = \mathbf{w} + \mathbf{v}$ \hfill {\color{gray}(commutativity)} \item $(\mathbf{v} + \mathbf{w}) + \mathbf{x} = \mathbf{v} + (\mathbf{w} + \mathbf{x})$ \hfill {\color{gray}(associativity)} \item $c(\mathbf{v} + \mathbf{w}) = c\mathbf{v} + c\mathbf{w}$ \hfill {\color{gray}(distributivity)} \item $(c+d)\mathbf{v} = c\mathbf{v} + d\mathbf{v}$ \hfill {\color{gray}(distributivity)} \item $\mathbf{v} + \mathbf{0} = \mathbf{v}$ \item $\mathbf{v} + (-\mathbf{v}) = \mathbf{0}$ \item $c(d\mathbf{v}) = (cd)\mathbf{v}$ \end{enumerate} \end{theorem} \begin{proof} We will prove property~(a) in class; you can try to prove some of the others on your own (the method is quite similar). \horlines{8} % Let $\mathbf{v} = (v_1,v_2,\ldots,v_n)$ and $\mathbf{w} = (w_1,w_2,\ldots,w_n)$. Then $\v + \w = (v_1+w_1,v_2+w_2,\ldots,v_n+w_n) = (w_1+v_1,w_2+v_2,\ldots,w_n+v_n) = \w + \v$ % Note that you can take properties of real numbers for granted -- we know and love them in this course. Now you can take the statements of this theorem for granted too. \noindent which completes the proof. \end{proof} \newpage Why did we even bother with the theorem on the previous page? They're all the type of thing you can just look at and ``see'' are true, right? \\ One reason is that we have to make sure that certain combinations of symbols even make sense when we are in new and unfamiliar settings. For example, associativity (property (b)) says that this expression makes sense: \horlines{2} % v + w + z % we don't have to guess if it means (v+w)+z or v+(w+z) % Also mention that a 2nd reason is to get students used to proofs, as we will do more later in this course that are less trivial We will soon introduce some operations that do not have these basic properties like commutativity, so we will have to start being very careful. \exx[4]{Simplify $\mathbf{v} + 2(\mathbf{w} - \mathbf{v}) - 3(\mathbf{v} + 2\mathbf{w})$.} % -4v - 4w % Mention that the previous theorem just says that we can simiplify this expression and manipulate it ``how we're used to'' with real numbers % However, we can't multiply or divide vectors \section*{Linear Combinations} One common task in linear algebra is to start out with some given collection of vectors $\v_1,\v_2,\ldots,\v_k$ and then use vector addition and scalar multiplication to construct new vectors out of them. The following definition gives a name to this concept. % DEFINITION: Linear Combinations \begin{definition}[Linear Combinations]\label{defn:linear_combinations} A \textbf{linear combination} of $\v_1,\v_2,\ldots,\v_k \in \R^n$ is a vector of the form \begin{align*}c_1\v_1 + c_2\v_2 + \cdots + c_k\v_k,\end{align*}where $c_1,c_2,\ldots,c_k \in \R$. \end{definition} % END DEFINITION \newpage \exx[3]{Show that $(1,2,3)$ is a linear combination of the vectors $(1,1,1)$ and $(-1,0,1)$.} % $(1,2,3) = 2(1,1,1) + (-1,0,1)$ % Note that we just "eyeballed" this: we will see how to find scalars that work in a later week \exx[3]{Show that $(1,2,3)$ is \emph{not} a linear combination of $(1,1,0)$ and $(2,1,0)$.} % Every vector of the form $c_1(1,1,0) + c_2(2,1,0)$ has a $0$ in its 3rd entry, and thus can't equal $(1,2,3)$. When working with linear combinations, some particularly important vectors are the ones with a single $1$ in one of their entries, and all other entries equal to $0$. These are called the \textbf{standard basis vectors}: \horlines{2} % e_i = (0,0,...,1,...,0) (1 in i-th entry) \exx[7]{List and draw all of the standard basis vectors in $\R^2$ and $\R^3$.} % FILES NEEDED: week1_ei_R3.png % R2: \e_1 = (1,0) and \e_2 = (0,1). % R3: we have the three vectors $\e_1 = (1,0,0)$, $\e_2 = (0,1,0)$, and $\e_3 = (0,0,1)$ % Sketch them in R2. % For R3, INSERT week1_ei_R3.png \newpage For now, the reason for our interest in these standard basis vectors is that every vector $\v = (v_1,v_2,\ldots,v_n) \in \R^n$ can be written as a linear combination of them. In particular, \horlines{1} % \v = v_1(1,0,...,0) + v_2(0,1,0,...,0) + ... + v_n(0,0,...,1) = v_1\e_1 + v_2\e_2 + \cdots + v_n\e_n. This idea of writing vectors in terms of the standard basis vectors is one of the most useful tricks that we make use of in linear algebra: in many situations, if we can prove that some property holds for the standard basis vectors, then we can use linear combinations to show that it must hold for \emph{all} vectors. \exx[2]{Compute $3\e_1 - 2\e_2 + \e_3 \in \R^3$.} % $3\e_1 - 2\e_2 + \e_3 = 3(1,0,0) - 2(0,1,0) + (0,0,1) = (3,-2,1)$. In general, when adding multiples of the standard basis vectors, the resulting vector has the coefficient of $\e_i$ in its $i$-th entry. \exx[2]{Write $(3,5,-2,-1)$ as a linear combination of $\e_1,\e_2,\e_3,\e_4\in\R^4$.} % Just like in part~(a), the entries of the vectors are the scalars in the linear combination: $(3,5,-2,-1) = 3\e_1 + 5\e_2 - 2\e_3 - \e_4$. \end{document}