Using Linear Algebra Functions Safely

Understanding “Safe” Use of Linear Algebra in MATLAB

Using MATLAB’s linear algebra functions is very convenient, but it is also very easy to write code that “works” for a simple example and then fails, becomes slow, or gives misleading results on real data. In this chapter you focus on how to call these functions in a way that is numerically sensible, robust, and maintainable, without re‑deriving the linear algebra theory itself.

You will see recurring themes: avoid explicit matrix inverses, prefer higher level solvers, check sizes, and be aware of numerical conditioning. The goal is not to turn you into a numerical analyst, but to give you practical habits that keep your code reliable.

Avoiding Explicit Matrix Inversion

MATLAB makes it very easy to compute inverses with inv(A) or A^(-1). For most tasks, this is not what you want. Inverting a matrix explicitly is slower and can be much less accurate than using the right solver for your problem.

Suppose you want to solve the linear system
$$
Ax = b.
$$
You might be tempted to write

x = inv(A)*b;

This computes the entire inverse of A, then multiplies it by b. The safer and recommended way in MATLAB is

x = A\b;

The backslash operator \ chooses an appropriate algorithm internally, uses factorizations instead of forming an explicit inverse, and typically produces a more accurate result.

Similarly, when you see an expression like $A^{-1}B$, you should think of rewriting it as the solution to a matrix equation
$$
AX = B,
$$
and then using

X = A\B;

The same idea appears on the right side. If you have $BA^{-1}$, solve
$$
XA = B
$$
for $X$ and write

X = B/A;

The operator / is the right division operator and solves systems where the unknown is on the left in the algebraic expression.

In short, use \ and / to solve systems instead of computing inv(A). Reserve explicit inverses for special cases where you really need the inverse as an object and you know what you are doing.

Interpreting Warnings and Singular Matrix Issues

When you solve systems or compute factorizations, MATLAB may warn you that a matrix is singular or close to singular. For example:

Warning: Matrix is close to singular or badly scaled.
Results may be inaccurate. RCOND = 1.234567e-17.

This tells you that your matrix is ill conditioned, so small changes in the data can cause large changes in the solution. In this situation, blindly trusting the numeric answer is unsafe.

The number RCOND is a rough measure related to the reciprocal of the condition number. A value near 1 suggests a well conditioned matrix; a value close to machine precision (around 1e-16 for double precision) indicates severe ill conditioning.

If you see such warnings when calling A\b, inv(A), or related functions, you should at least:

Check whether your problem is inherently ill posed. Sometimes equations are nearly redundant or parameters are not identifiable.

Consider regularization or reformulating the problem. For example, in least squares, a common idea is to add a small positive value to the diagonal to stabilize the problem.

Work with scaled variables if your data differ by many orders of magnitude. Extremely large or small values can cause numerical trouble.

Ignoring these warnings can give you answers that look plausible but are effectively meaningless.

Being Careful with Matrix Size and Orientation

Many linear algebra errors in MATLAB come from shape mismatches rather than deep numerical issues. Since vectors, row vectors, and column vectors behave differently, you should watch the sizes reported by size and length.

A system $Ax = b$ is typically represented with A as an m‑by‑n matrix and b as an m‑by‑1 column vector. If you accidentally create b as a row vector, you may see dimension mismatch errors or get results that do not match formulas from linear algebra.

To check sizes, you can use

size(A)
size(b)

If you want to be explicit, create column vectors as b = [1; 2; 3]; rather than [1 2 3] when the distinction matters. In many MATLAB operations, [1 2 3] is treated as a row vector of size 1‑by‑3.

When multiplying matrices, make sure that inner dimensions match. If A is m‑by‑n and B is n‑by‑p, then A*B is defined and gives an m‑by‑p matrix. MATLAB will give an error if the dimensions do not line up, so checking dimensions explicitly can save time.

Careful management of sizes also matters when you use functions like eig, svd, or qr. For example, if you want eigenvectors in columns of V and eigenvalues on the diagonal of D, you typically write

[V,D] = eig(A);

You should expect V and D to have the same square size as A. If A is not square, then eig(A) is not appropriate and MATLAB will tell you.

Choosing Appropriate Linear Algebra Functions

MATLAB has many functions that can in principle solve a problem. Using a function that matches your problem structure is safer and more robust.

If you solve linear systems, prefer \ and / over inv, as discussed earlier.

If you minimize the residual in an overdetermined system $Ax \approx b$, use the least squares form

x = A\b;

when A has more rows than columns. MATLAB will choose a numerically appropriate method, such as QR factorization.

If your matrix is symmetric positive definite, using specialized solvers or factorization routines like chol can be more stable and efficient. For example, if A is symmetric positive definite and you want to solve $Ax = b$, you can write

R = chol(A);
x = R\(R'\b);

You do not need to memorize this pattern, but you should know that exploiting structure is both faster and more accurate.

Functions like det(A) can be very sensitive for large matrices. Determinants of large matrices quickly underflow or overflow in floating point arithmetic. If you want to check whether a matrix is singular, relying on det(A) is not safe. It is better to use rank(A), check rcond(A), or analyze the output of lu(A), qr(A), or svd(A). This gives more reliable information about numerical rank and singularity.

Handling Rank Deficiency and Least Squares Safely

When you solve equations with more equations than unknowns, MATLAB usually interprets this as a least squares problem. In matrix form, for $A$ of size m‑by‑n with m > n, the equation
$$
\min_x \|Ax - b\|_2
$$
is solved by x = A\b;.

If the columns of A are linearly dependent, the problem is rank deficient. In that case, there are infinitely many solutions that give the same minimal residual. MATLAB can still return a solution, but you should be aware that:

The solution might not be unique.

Small perturbations in data can lead to large changes in the solution.

To handle this situation more safely, you can:

Use rank(A) or rcond(A) to get a sense of whether A is full rank.

Use pinv(A) to compute the pseudoinverse when a minimum norm solution is needed. For example, x = pinv(A)*b; returns the least squares solution with minimum Euclidean norm.

Be aware that pinv is more expensive than \, but it can be appropriate when rank deficiency is expected and you specifically want the minimum norm solution.

Again, you do not need to master the theory, but you should recognize that rank deficiency changes the meaning of “the” solution.

Working with Eigenvalues and Eigenvectors Carefully

Eigenvalue and eigenvector computations are powerful, but they are also sensitive to matrix properties. When you compute

[V,D] = eig(A);

MATLAB returns D with eigenvalues on the diagonal and V with eigenvectors as columns. In exact arithmetic, you would have $AV = V D$. In floating point arithmetic, this is only approximately true.

When using eigenvectors and eigenvalues:

Check that your matrix is of the right type. For non square matrices, eig is not defined. For non symmetric matrices, eigenvalues may be complex even if all entries are real.

Interpret complex results correctly. MATLAB represents complex eigenvalues as complex numbers in D. If you expected real values but see complex parts, this usually reflects the true properties of your matrix, not an error.

If you know your matrix is symmetric, A = A', it is safer to use the 'vector' option to get eigenvalues in a vector, or to rely on the fact that symmetric matrices have real eigenvalues and orthogonal eigenvectors. For example:

[V,D] = eig(A,'vector');

The general eig function handles many cases, but numerical stability is better for symmetric or Hermitian matrices. If your problem is not symmetric, be cautious when interpreting repeated eigenvalues and associated eigenvectors.

Respecting Numerical Precision Limits

All linear algebra functions in MATLAB operate in finite precision arithmetic. This means they are subject to rounding error. For most purposes, double precision is accurate to about $10^{-16}$ in relative terms.

Comparisons of results from linear algebra functions should take this into account. Directly testing equality with == can be unsafe when results involve floating point operations. For example, instead of checking

if A\b == x_exact
    ...
end

use a tolerance:

x_num = A\b;
if norm(x_num - x_exact) < 1e-10
    ...
end

The tolerance should be chosen in light of the problem scale. Large matrices or very ill conditioned problems typically require looser tolerances.

Similarly, functions like rank(A) and null(A) use tolerances internally. These are based on norms and singular values. You can provide your own tolerance when needed to adapt to your data, but for many beginner problems the defaults are adequate.

Verifying Results and Sanity Checks

A safe use of linear algebra functions includes simple verification steps. After solving a system x = A\b;, you can check the residual

r = A*x - b;
norm_r = norm(r);

A small residual norm compared to norm(b) suggests that your solution at least satisfies the equation numerically:

$$
\frac{\|Ax - b\|_2}{\|b\|_2}
$$

If this relative residual is extremely large, something is probably wrong, either in the model, the data, or the way you formed your matrices.

For eigenvalue problems, you can check that

res = norm(A*V - V*D);

is small. This does not fix fundamental issues with ill conditioning, but it helps catch simple mistakes like using the wrong matrix, misinterpreting dimensions, or transposing improperly.

These quick tests are often enough to detect errors without deep theory.

Reproducibility and Deterministic Behavior

Some advanced linear algebra routines can behave differently on different hardware or with different MATLAB versions, especially those that rely on multithreaded BLAS libraries. For beginner work, this usually does not matter, but you should be aware that exact bitwise equality of floating point results is not guaranteed across platforms.

For safe and reproducible workflows:

Avoid relying on == to compare whole matrices coming from linear algebra computations across runs or machines.

Use norms and tolerances when comparing results.

Document MATLAB versions and important options when numerical results are critical to reproduce.

This habit helps when you start sharing code or collaborating with others.