Vectors and Norms

Geometric View of Vectors

In basic linear algebra with MATLAB, a vector represents a quantity that has both magnitude and direction. In MATLAB, most vectors you work with are numeric arrays that have either one row and multiple columns, or one column and multiple rows.

A row vector has size $1 \times n$, for example v = [1 2 3]. A column vector has size $n \times 1$, for example v = [1; 2; 3]. In linear algebra, column vectors are usually the standard form, and MATLAB functions for linear algebra typically expect column vectors unless stated otherwise.

You can think of a vector in $\mathbb{R}^2$ as an arrow in a plane, and a vector in $\mathbb{R}^3$ as an arrow in 3D space. The components of the vector are its coordinates along each axis. For example, the vector

$$
v = \begin{bmatrix} 3 \\ 4 \end{bmatrix}
$$

has an $x$-component equal to 3 and a $y$-component equal to 4. The direction is given by how the arrow points, and the length is given by its norm, which you will see in detail later in this chapter.

In MATLAB, many linear algebra functions are designed to work on vectors in the sense of column vectors. The distinction between row and column will matter for multiplication, but for norms, many functions accept either shape and treat the vector as a single list of numbers.

To check the shape of a vector in MATLAB you can use size:

v = [3; 4];        % column vector
size(v)
w = [3 4];         % row vector
size(w)

Both v and w represent the same mathematical vector usually written as a column, but they have different shapes inside MATLAB.

Creating Column and Row Vectors for Linear Algebra

Although you may already know how to create vectors in MATLAB, for linear algebra computations it is useful to be systematic about row and column forms.

You create row vectors by listing elements inside square brackets separated by spaces or commas. You create column vectors by separating elements with semicolons. For example:

row_v = [1 2 3 4];       % 1-by-4 row vector
col_v = [1; 2; 3; 4];    % 4-by-1 column vector

If you have a row vector and you want a column vector, you can transpose it using the transpose operator '. For complex data there is an important distinction between the conjugate transpose and the nonconjugate transpose, but for real-valued vectors these give the same result in value and only differ conceptually. Here the focus is on the basic operation to change shape:

row_v = 1:4;          % row vector [1 2 3 4]
col_v = row_v.';      % column vector [1; 2; 3; 4]

The operator ' performs the conjugate transpose. The operator .' performs the nonconjugate transpose. For real data, both act as simple transpose.

Many linear algebra routines interpret a matrix as a collection of column vectors. For example, an $m \times n$ matrix can be seen as $n$ column vectors of length $m$. If you form a matrix like this:

A = [1 2 3; 4 5 6; 7 8 9];

then A(:,1) is the first column vector, A(:,2) is the second, and so on. This column-oriented view is central in MATLAB when working with linear algebra concepts such as norms of columns or operations on multiple vectors at once.

MATLAB also allows you to create vectors by specifying regular patterns. For example, 0:0.1:1 creates a row vector from 0 to 1 with step 0.1. If you need a column vector instead, apply transpose:

t_row = 0:0.1:1;
t_col = t_row.';

When you plan to use vectors in linear algebra operations such as matrix multiplication or solving systems, decide early whether you will store them as rows or columns and be consistent throughout your code.

Length, Dimension, and Number of Elements

For vectors, there are several related but not identical notions: dimension, number of elements, and what MATLAB calls length. In linear algebra, a vector in $\mathbb{R}^n$ has dimension $n$. In MATLAB, the number of elements of a vector is returned by numel, and the function length returns the largest dimension of the array.

For a vector, length(v) is equal to numel(v) because only one dimension is larger than 1. For a general matrix this is not necessarily true. When you are working with vectors, you can safely use length to obtain their size as a one-dimensional object.

For example:

v = [3; 4; 0; -1];     % 4-dimensional column vector
n_dim = length(v)      % returns 4
n_elem = numel(v)      % also returns 4

Conceptually, this vector belongs to $\mathbb{R}^4$, so its dimension is 4. In linear algebra, this dimension is the number of coordinates needed to specify the vector.

At this stage, it is helpful to remember that the word "length" in linear algebra can also refer to the magnitude of a vector, which is a norm, not just the number of elements. MATLAB uses length for the number of entries, and uses specific functions for the magnitude, such as norm. Keeping this distinction clear will avoid confusion when you move between theory and code.

The Euclidean Norm (2-Norm)

The most common measure of the size of a vector is the Euclidean norm, also called the 2-norm. For a vector

$$
x = \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix},
$$

the Euclidean norm is defined as

$$
\|x\|_2 = \sqrt{x_1^2 + x_2^2 + \dots + x_n^2}.
$$

This is a direct generalization of the Pythagorean theorem to $n$ dimensions and corresponds to the geometric length of the vector when interpreted as an arrow from the origin to the point with coordinates $(x_1,\dots,x_n)$.

In MATLAB, the Euclidean norm is computed by the function norm. When you call norm(x) with a vector input and omit the second argument, MATLAB uses the Euclidean norm by default. For example:

x = [3; 4];
n2 = norm(x)           % returns 5, since sqrt(3^2 + 4^2) = 5

This result corresponds exactly to the standard notion of length in the plane. For a vector in three dimensions, for example x = [1; -2; 2], the same formula extends to include all coordinates.

Behind the scenes, MATLAB uses a numerically stable algorithm to compute norms. Compared with a naive implementation using sqrt(sum(x.^2)), the built-in norm is typically more robust when elements of x vary widely in magnitude.

If you have a row vector, you can use norm directly without reshaping. MATLAB treats 1-by-n and n-by-1 inputs as vectors for the purpose of computing norms:

r = [3 4];
norm(r)                % also returns 5

When using norms in linear algebra, the Euclidean norm is often the default choice, for example in distance calculations, error measures, and optimization problems.

Other Vector Norms: 1-Norm and Infinity Norm

In addition to the Euclidean norm, linear algebra uses other norms that quantify vector size in different ways. Two important ones are the 1-norm and the infinity norm.

For a vector

$$
x = \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix},
$$

the 1-norm is defined as the sum of absolute values:

$$
\|x\|_1 = |x_1| + |x_2| + \dots + |x_n|.
$$

The infinity norm is defined as the maximum absolute component:

$$
\|x\|_{\infty} = \max\{|x_1|, |x_2|, \dots, |x_n|\}.
$$

These norms reflect different ways of measuring size. The 1-norm can be seen as the total absolute weight of the components. The infinity norm focuses on the single largest magnitude among the components.

In MATLAB you can compute these norms with the same function norm by passing a second argument that specifies the type:

x = [1; -2; 3];
n1   = norm(x, 1);      % 1-norm: |1| + |-2| + |3| = 6
n2   = norm(x, 2);      % 2-norm (Euclidean)
nInf = norm(x, inf);    % infinity norm: max(|1|,| -2|,|3|) = 3

The second argument to norm tells MATLAB which norm to compute. For vectors, 1 means the 1-norm, 2 means the Euclidean norm, and inf means the infinity norm.

You can also express these norms with basic MATLAB operations. For example:

n1   = sum(abs(x));       % 1-norm
nInf = max(abs(x));       % infinity norm

These explicit formulas can be useful if you want to see directly how each norm is computed or if you want to adapt the computation in custom ways.

While all norms measure size and are related to each other, they can behave differently in applications. For example, the 1-norm may be more sensitive to the overall distribution of components, while the infinity norm is controlled only by the largest element. This difference becomes important when you interpret error measures or design numerical methods.

Unit Vectors and Normalization

A unit vector is a vector whose norm is equal to 1. In Euclidean space, a unit vector has length 1. In linear algebra, unit vectors are convenient for describing directions independently of magnitude. Any nonzero vector can be converted into a unit vector by dividing it by its norm. This process is called normalization.

Given a nonzero vector $x$, its corresponding normalized vector $\hat{x}$ with respect to the Euclidean norm is

$$
\hat{x} = \frac{x}{\|x\|_2}.
$$

By construction, $\|\hat{x}\|_2 = 1$. In MATLAB, you can perform this computation directly:

x = [3; 4];
x_norm = norm(x);          % Euclidean norm
u = x / x_norm;            % normalized vector
norm(u)                    % returns 1 (up to numerical roundoff)

You must check or ensure that the vector is not the zero vector before normalizing, because division by zero is not defined. A typical pattern is:

x = [0; 0; 0];
if norm(x) == 0
    error('Cannot normalize the zero vector.');
end
u = x / norm(x);

Normalization can be done with respect to other norms as well. For example, if you normalize a vector with respect to the 1-norm you obtain a vector whose 1-norm is 1:

x = [1; -2; 3];
u1 = x / norm(x, 1);      % now norm(u1, 1) is 1

However, unless stated otherwise, "normalize a vector" usually refers to the Euclidean norm. Using unit vectors is common when you want to represent directions, such as in geometry, orthonormal bases, or when scaling data to compare shapes rather than sizes.

Because normalization involves division by the norm, numerical issues can arise if the vector norm is very small or very large. MATLAB manages typical double precision scaling well, but it is still useful to be aware that extreme magnitudes can lead to loss of precision when normalizing.

Distances Between Vectors

The Euclidean distance between two vectors $x$ and $y$ in $\mathbb{R}^n$ is defined by

$$
\text{dist}(x,y) = \|x - y\|_2.
$$

This is the straight-line distance between the points represented by $x$ and $y$. In MATLAB, you can compute it using norm directly:

x = [1; 2; 3];
y = [4; 6; 3];
d = norm(x - y);           % Euclidean distance

Here, x - y is the difference vector, and norm computes its Euclidean length. This pattern occurs often in data analysis and geometry, where each data point is represented as a vector and distances between points are of interest.

You can also use other norms to define different distance measures. For example, the 1-distance is norm(x - y, 1) and the infinity distance is norm(x - y, inf). These different distances can behave differently, especially in higher dimensions, and some applications prefer one norm over another.

If you have many vectors and you want to compute distances systematically, you can organize them as columns of a matrix and apply vectorized operations. In that case, carefully manage the sizes of arrays and the choice of norm so that each column represents a single vector whose distance you want to compute.

Norms and Scaling Effects

Norms have a very important homogeneity property. If you multiply a vector $x$ by a scalar $\alpha$, every norm satisfies

$$
\|\alpha x\| = |\alpha| \, \|x\|.
$$

This property expresses the idea that scaling a vector by a factor $\alpha$ scales its length by the absolute value of that factor. In MATLAB, you can verify this for any of the norms you compute:

x = [1; -2; 3];
alpha = -5;
left  = norm(alpha * x);          % norm of scaled vector
right = abs(alpha) * norm(x);     % abs(alpha) times norm of x

Ideally, left and right should be equal up to small floating point differences. This homogeneity is one of the defining properties of a norm and holds for the 1-norm, 2-norm, infinity norm, and other standard norms.

Scaling behavior is important when you compare vectors, because it tells you how sensitive a particular norm is to multiplicative changes. For example, if you double all components of a vector, any norm of that vector doubles in magnitude. If you are tracking relative changes, you might consider using normalized vectors so that magnitude is fixed and only direction varies.

Another useful idea is that different norms are equivalent in finite-dimensional spaces. This means that for any vector, its 1-norm, 2-norm, and infinity norm are all finite and are related by inequalities that use only constant factors. Concretely for a vector in $\mathbb{R}^n$, you always have

$$
\|x\|_{\infty} \le \|x\|_2 \le \|x\|_1 \le \sqrt{n} \, \|x\|_2 \le n \, \|x\|_{\infty}.
$$

In practice, this means that if a vector is small with respect to one norm, it is also small with respect to the others, up to a factor that depends on the dimension. When you work in MATLAB with moderate dimensions, the choice of norm affects numerical values but not the qualitative statement that a vector is small or large.

Understanding how norms scale and relate to each other prepares you for later topics such as condition numbers, stability of algorithms, and convergence criteria in iterative methods, where norms are used systematically.

Using `vecnorm` for Multiple Vectors

When your data contains many vectors stored together, such as the columns or rows of a matrix, MATLAB offers the function vecnorm to compute norms of each vector at once. This is particularly useful when you need norms of many columns or rows instead of a single vector.

Suppose you have a matrix

A = [1  2  3;
     4  5  6;
     7  8  9];

Conceptually, this matrix consists of three column vectors, each of length 3. To compute the Euclidean norm of each column, use vecnorm:

col_norms = vecnorm(A);

This returns a row vector containing norm(A(:,1)), norm(A(:,2)), and norm(A(:,3)). By default, vecnorm computes the 2-norm along the first non-singleton dimension, which for a 2D matrix is the columns.

If you instead want the norms of the rows, specify the dimension:

row_norms = vecnorm(A, 2, 2);    % 2-norm of each row

Here, the second argument 2 specifies the Euclidean norm, and the third argument 2 selects dimension 2, which corresponds to rows in a 2D matrix.

You can also compute other norms with vecnorm. For instance, to compute the 1-norm of each column:

col_n1 = vecnorm(A, 1);         % 1-norm of each column

or the infinity norm of each row:

row_nInf = vecnorm(A, inf, 2);  % infinity norm of each row

Using vecnorm is both concise and efficient, especially when you are working with batches of vectors. It also makes your code more readable, because the intent to compute vector norms is explicit, rather than hidden inside a combination of sum, abs, sqrt, or max.

For higher dimensional arrays, vecnorm generalizes in the same way by computing norms along a specified dimension. Choosing the dimension correctly is essential: the dimension you specify is the one that is treated as the list of entries of each vector.