KAHIBARO Bookshelves
Discord Login Register

15.6. Using Linear Algebra Functions Safely

Understanding “Safe” Use of Linear Algebra in MATLAB

Using MATLAB’s linear algebra functions is very convenient, but it is also very easy to write code that “works” for a simple example and then fails, becomes slow, or gives misleading results on real data. In this chapter you focus on how to call these functions in a way that is numerically sensible, robust, and maintainable, without re‑deriving the linear algebra theory itself.

You will see recurring themes: avoid explicit matrix inverses, prefer higher level solvers, check sizes, and be aware of numerical conditioning. The goal is not to turn you into a numerical analyst, but to give you practical habits that keep your code reliable.

Avoiding Explicit Matrix Inversion

MATLAB makes it very easy to compute inverses with inv(A) or A^(-1). For most tasks, this is not what you want. Inverting a matrix explicitly is slower and can be much less accurate than using the right solver for your problem.

Suppose you want to solve the linear system
$$
Ax = b.
$$
You might be tempted to write

x = inv(A)*b;

This computes the entire inverse of A, then multiplies it by b. The safer and recommended way in MATLAB is

x = A\b;

The backslash operator \ chooses an appropriate algorithm internally, uses factorizations instead of forming an explicit inverse, and typically produces a more accurate result.

Similarly, when you see an expression like $A^{-1}B$, you should think of rewriting it as the solution to a matrix equation
$$
AX = B,
$$
and then using

X = A\B;

The same idea appears on the right side. If you have $BA^{-1}$, solve
$$
XA = B
$$
for $X$ and write

X = B/A;

The operator / is the right division operator and solves systems where the unknown is on the left in the algebraic expression.

In short, use \ and / to solve systems instead of computing inv(A). Reserve explicit inverses for special cases where you really need the inverse as an object and you know what you are doing.

Interpreting Warnings and Singular Matrix Issues

When you solve systems or compute factorizations, MATLAB may warn you that a matrix is singular or close to singular. For example:

Warning: Matrix is close to singular or badly scaled.
Results may be inaccurate. RCOND = 1.234567e-17.

This tells you that your matrix is ill conditioned, so small changes in the data can cause large changes in the solution. In this situation, blindly trusting the numeric answer is unsafe.

The number RCOND is a rough measure related to the reciprocal of the condition number. A value near 1 suggests a well conditioned matrix; a value close to machine precision (around 1e-16 for double precision) indicates severe ill conditioning.

If you see such warnings when calling A\b, inv(A), or related functions, you should at least:

Check whether your problem is inherently ill posed. Sometimes equations are nearly redundant or parameters are not identifiable.

Consider regularization or reformulating the problem. For example, in least squares, a common idea is to add a small positive value to the diagonal to stabilize the problem.

Work with scaled variables if your data differ by many orders of magnitude. Extremely large or small values can cause numerical trouble.

Ignoring these warnings can give you answers that look plausible but are effectively meaningless.

Being Careful with Matrix Size and Orientation

Many linear algebra errors in MATLAB come from shape mismatches rather than deep numerical issues. Since vectors, row vectors, and column vectors behave differently, you should watch the sizes reported by size and length.

A system $Ax = b$ is typically represented with A as an m‑by‑n matrix and b as an m‑by‑1 column vector. If you accidentally create b as a row vector, you may see dimension mismatch errors or get results that do not match formulas from linear algebra.

To check sizes, you can use

size(A)
size(b)

If you want to be explicit, create column vectors as b = [1; 2; 3]; rather than [1 2 3] when the distinction matters. In many MATLAB operations, [1 2 3] is treated as a row vector of size 1‑by‑3.

When multiplying matrices, make sure that inner dimensions match. If A is m‑by‑n and B is n‑by‑p, then A*B is defined and gives an m‑by‑p matrix. MATLAB will give an error if the dimensions do not line up, so checking dimensions explicitly can save time.

Careful management of sizes also matters when you use functions like eig, svd, or qr. For example, if you want eigenvectors in columns of V and eigenvalues on the diagonal of D, you typically write

[V,D] = eig(A);

You should expect V and D to have the same square size as A. If A is not square, then eig(A) is not appropriate and MATLAB will tell you.

Choosing Appropriate Linear Algebra Functions

MATLAB has many functions that can in principle solve a problem. Using a function that matches your problem structure is safer and more robust.

If you solve linear systems, prefer \ and / over inv, as discussed earlier.

If you minimize the residual in an overdetermined system $Ax \approx b$, use the least squares form

x = A\b;

when A has more rows than columns. MATLAB will choose a numerically appropriate method, such as QR factorization.

If your matrix is symmetric positive definite, using specialized solvers or factorization routines like chol can be more stable and efficient. For example, if A is symmetric positive definite and you want to solve $Ax = b$, you can write

R = chol(A);
x = R\(R'\b);

You do not need to memorize this pattern, but you should know that exploiting structure is both faster and more accurate.

Functions like det(A) can be very sensitive for large matrices. Determinants of large matrices quickly underflow or overflow in floating point arithmetic. If you want to check whether a matrix is singular, relying on det(A) is not safe. It is better to use rank(A), check rcond(A), or analyze the output of lu(A), qr(A), or svd(A). This gives more reliable information about numerical rank and singularity.

Handling Rank Deficiency and Least Squares Safely

When you solve equations with more equations than unknowns, MATLAB usually interprets this as a least squares problem. In matrix form, for $A$ of size m‑by‑n with m > n, the equation
$$
\min_x \|Ax - b\|_2
$$
is solved by x = A\b;.

If the columns of A are linearly dependent, the problem is rank deficient. In that case, there are infinitely many solutions that give the same minimal residual. MATLAB can still return a solution, but you should be aware that:

The solution might not be unique.

Small perturbations in data can lead to large changes in the solution.

To handle this situation more safely, you can:

Use rank(A) or rcond(A) to get a sense of whether A is full rank.

Use pinv(A) to compute the pseudoinverse when a minimum norm solution is needed. For example, x = pinv(A)*b; returns the least squares solution with minimum Euclidean norm.

Be aware that pinv is more expensive than \, but it can be appropriate when rank deficiency is expected and you specifically want the minimum norm solution.

Again, you do not need to master the theory, but you should recognize that rank deficiency changes the meaning of “the” solution.

Working with Eigenvalues and Eigenvectors Carefully

Eigenvalue and eigenvector computations are powerful, but they are also sensitive to matrix properties. When you compute

[V,D] = eig(A);

MATLAB returns D with eigenvalues on the diagonal and V with eigenvectors as columns. In exact arithmetic, you would have $AV = V D$. In floating point arithmetic, this is only approximately true.

When using eigenvectors and eigenvalues:

Check that your matrix is of the right type. For non square matrices, eig is not defined. For non symmetric matrices, eigenvalues may be complex even if all entries are real.

Interpret complex results correctly. MATLAB represents complex eigenvalues as complex numbers in D. If you expected real values but see complex parts, this usually reflects the true properties of your matrix, not an error.

If you know your matrix is symmetric, A = A', it is safer to use the 'vector' option to get eigenvalues in a vector, or to rely on the fact that symmetric matrices have real eigenvalues and orthogonal eigenvectors. For example:

[V,D] = eig(A,'vector');

The general eig function handles many cases, but numerical stability is better for symmetric or Hermitian matrices. If your problem is not symmetric, be cautious when interpreting repeated eigenvalues and associated eigenvectors.

Respecting Numerical Precision Limits

All linear algebra functions in MATLAB operate in finite precision arithmetic. This means they are subject to rounding error. For most purposes, double precision is accurate to about $10^{-16}$ in relative terms.

Comparisons of results from linear algebra functions should take this into account. Directly testing equality with == can be unsafe when results involve floating point operations. For example, instead of checking

if A\b == x_exact
    ...
end

use a tolerance:

x_num = A\b;
if norm(x_num - x_exact) < 1e-10
    ...
end

The tolerance should be chosen in light of the problem scale. Large matrices or very ill conditioned problems typically require looser tolerances.

Similarly, functions like rank(A) and null(A) use tolerances internally. These are based on norms and singular values. You can provide your own tolerance when needed to adapt to your data, but for many beginner problems the defaults are adequate.

Verifying Results and Sanity Checks

A safe use of linear algebra functions includes simple verification steps. After solving a system x = A\b;, you can check the residual

r = A*x - b;
norm_r = norm(r);

A small residual norm compared to norm(b) suggests that your solution at least satisfies the equation numerically:

$$
\frac{\|Ax - b\|_2}{\|b\|_2}
$$

If this relative residual is extremely large, something is probably wrong, either in the model, the data, or the way you formed your matrices.

For eigenvalue problems, you can check that

res = norm(A*V - V*D);

is small. This does not fix fundamental issues with ill conditioning, but it helps catch simple mistakes like using the wrong matrix, misinterpreting dimensions, or transposing improperly.

These quick tests are often enough to detect errors without deep theory.

Reproducibility and Deterministic Behavior

Some advanced linear algebra routines can behave differently on different hardware or with different MATLAB versions, especially those that rely on multithreaded BLAS libraries. For beginner work, this usually does not matter, but you should be aware that exact bitwise equality of floating point results is not guaranteed across platforms.

For safe and reproducible workflows:

Avoid relying on == to compare whole matrices coming from linear algebra computations across runs or machines.

Use norms and tolerances when comparing results.

Document MATLAB versions and important options when numerical results are critical to reproduce.

This habit helps when you start sharing code or collaborating with others.

Key points to remember:
Avoid inv(A) and use A\b or B/A to solve linear systems.
Pay attention to matrix sizes and use column vectors for systems like $Ax = b$.
Heed warnings about singular or ill conditioned matrices, and check rcond or rank if needed.
Use specialized functions that match your matrix structure instead of generic ones when possible.
Do not rely on det(A) to detect singularity for large or ill conditioned matrices.
Handle rank deficient and least squares problems with care, and consider pinv for minimum norm solutions.
Treat eigenvalues and eigenvectors cautiously, especially for non symmetric matrices.
Always remember that results are approximate and compare floating point numbers with tolerances, not exact equality.

Views: 0

Comments

Please login to add a comment.

Don't have an account? Register now!

Where to Move

Move chapter:

  • ☰ 1. Getting Started with MATLAB
    • ☰ 1.1. What Is MATLAB and What Is It Used For
      • ☰ 1.2. Installing MATLAB and Setting Up the Environment
        • ☰ 1.3. Understanding the MATLAB Desktop Interface
          • ☰ 1.4. Command Window, Workspace, and Current Folder
            • ☰ 1.5. Basic Workflow and Running Simple Commands
              • ☰ 1.6. Getting Help and Documentation
              • ☰ 2. MATLAB Language Basics
                • ☰ 2.1. MATLAB Syntax and Expressions
                  • ☰ 2.2. Variables and Assignment
                    • ☰ 2.3. Data Types and Type Conversion
                      • ☰ 2.4. Vectors and Matrices Fundamentals
                        • ☰ 2.5. Creating and Indexing Arrays
                          • ☰ 2.6. Basic Arithmetic and Elementwise Operations
                            • ☰ 2.7. Using Built-In Constants and Functions
                            • ☰ 3. Working with Variables and the Workspace
                              • ☰ 3.1. Creating, Viewing, and Editing Variables
                                • ☰ 3.2. Workspace Browser and Variable Inspector
                                  • ☰ 3.3. Saving and Loading Data Files
                                    • ☰ 3.4. MAT-Files and Importing External Data
                                      • ☰ 3.5. Clearing Variables and Managing the Workspace
                                      • ☰ 4. Vectors, Matrices, and Arrays
                                        • ☰ 4.1. Row and Column Vectors
                                          • ☰ 4.2. Matrix Creation and Concatenation
                                            • ☰ 4.3. Indexing, Slicing, and Logical Indexing
                                              • ☰ 4.4. Matrix Operations vs Elementwise Operations
                                                • ☰ 4.5. Reshaping and Rearranging Arrays
                                                  • ☰ 4.6. Special Matrices and Functions
                                                  • ☰ 5. Basic Programming Constructs
                                                    • ☰ 5.1. Scripts vs Functions
                                                      • ☰ 5.2. Control Flow with if, elseif, else
                                                        • ☰ 5.3. Loops with for and while
                                                          • ☰ 5.4. break, continue, and return
                                                            • ☰ 5.5. Logical Expressions and Relational Operators
                                                              • ☰ 5.6. Best Practices for Clear and Readable Code
                                                              • ☰ 6. Functions in MATLAB
                                                                • ☰ 6.1. Creating Simple Functions
                                                                  • ☰ 6.2. Function Inputs and Outputs
                                                                    • ☰ 6.3. Function Files and the Path
                                                                      • ☰ 6.4. Local vs Global Variables
                                                                        • ☰ 6.5. Anonymous Functions
                                                                          • ☰ 6.6. Organizing Code into Multiple Functions
                                                                          • ☰ 7. Working with Numbers and Math
                                                                            • ☰ 7.1. Scalars and Vectorized Computations
                                                                              • ☰ 7.2. Basic Mathematical Functions
                                                                                • ☰ 7.3. Trigonometric and Exponential Functions
                                                                                  • ☰ 7.4. Complex Numbers in MATLAB
                                                                                    • ☰ 7.5. Rounding, Modulo, and Special Values
                                                                                      • ☰ 7.6. Numerical Precision and Floating Point Considerations
                                                                                      • ☰ 8. Strings, Characters, and Categorical Data
                                                                                        • ☰ 8.1. Character Arrays and String Arrays
                                                                                          • ☰ 8.2. Creating and Manipulating Strings
                                                                                            • ☰ 8.3. Converting Between Data Types and Strings
                                                                                              • ☰ 8.4. Comparing and Searching Strings
                                                                                                • ☰ 8.5. Categorical Arrays Basics
                                                                                                • ☰ 9. Tables, Structures, and Cell Arrays
                                                                                                  • ☰ 9.1. Introduction to Tables
                                                                                                    • ☰ 9.2. Creating Tables and Accessing Data
                                                                                                      • ☰ 9.3. Modifying and Joining Tables
                                                                                                        • ☰ 9.4. Structures and Nested Structures
                                                                                                          • ☰ 9.5. Cell Arrays and Cell Indexing
                                                                                                            • ☰ 9.6. Choosing the Right Container Type
                                                                                                            • ☰ 10. Importing and Exporting Data
                                                                                                              • ☰ 10.1. Reading Text and CSV Files
                                                                                                                • ☰ 10.2. Importing Excel Spreadsheets
                                                                                                                  • ☰ 10.3. Importing Data with the Import Tool
                                                                                                                    • ☰ 10.4. Exporting Data to Files
                                                                                                                      • ☰ 10.5. Working with Image and Audio Files
                                                                                                                        • ☰ 10.6. Basic File Management from MATLAB
                                                                                                                        • ☰ 11. Plotting and Visualization
                                                                                                                          • ☰ 11.1. Introduction to 2D Plotting
                                                                                                                            • ☰ 11.2. Customizing Plots and Axes
                                                                                                                              • ☰ 11.3. Multiple Plots in One Figure
                                                                                                                                • ☰ 11.4. Labels, Titles, Legends, and Annotations
                                                                                                                                  • ☰ 11.5. Plotting with Different Line and Marker Styles
                                                                                                                                    • ☰ 11.6. Saving and Exporting Figures
                                                                                                                                    • ☰ 12. Advanced Plotting for Beginners
                                                                                                                                      • ☰ 12.1. Subplots and Tiled Layouts
                                                                                                                                        • ☰ 12.2. Bar, Scatter, and Stem Plots
                                                                                                                                          • ☰ 12.3. Histograms and Density Plots
                                                                                                                                            • ☰ 12.4. 3D Plots and Surface Plots
                                                                                                                                              • ☰ 12.5. Using plottools and Interactive Editing
                                                                                                                                                • ☰ 12.6. Basics of Handle Graphics
                                                                                                                                                • ☰ 13. Working with Scripts and Live Scripts
                                                                                                                                                  • ☰ 13.1. Creating and Running Script Files
                                                                                                                                                    • ☰ 13.2. Sectioning Code and Running Sections
                                                                                                                                                      • ☰ 13.3. Live Scripts and Rich Text
                                                                                                                                                        • ☰ 13.4. Adding Equations, Images, and Outputs
                                                                                                                                                          • ☰ 13.5. Sharing and Exporting Live Scripts
                                                                                                                                                          • ☰ 14. Debugging and Code Quality
                                                                                                                                                            • ☰ 14.1. Finding and Fixing Syntax Errors
                                                                                                                                                              • ☰ 14.2. Setting Breakpoints and Stepping Through Code
                                                                                                                                                                • ☰ 14.3. Inspecting Variable Values During Execution
                                                                                                                                                                  • ☰ 14.4. Using the Editor Debugging Tools
                                                                                                                                                                    • ☰ 14.5. Using Code Analyzer and Warnings
                                                                                                                                                                      • ☰ 14.6. Writing Clean and Maintainable MATLAB Code
                                                                                                                                                                      • ☰ 15. Basic Linear Algebra with MATLAB
                                                                                                                                                                        • ☰ 15.1. Vectors and Norms
                                                                                                                                                                          • ☰ 15.2. Matrix Addition and Multiplication
                                                                                                                                                                            • ☰ 15.3. Solving Linear Systems with Backslash
                                                                                                                                                                              • ☰ 15.4. Inverses, Determinants, and Rank
                                                                                                                                                                                • ☰ 15.5. Eigenvalues and Eigenvectors
                                                                                                                                                                                  • ☰ 15.6. Using Linear Algebra Functions Safely
                                                                                                                                                                                  • ☰ 16. Data Analysis and Statistics Fundamentals
                                                                                                                                                                                    • ☰ 16.1. Descriptive Statistics and Summary Measures
                                                                                                                                                                                      • ☰ 16.2. Sorting, Filtering, and Grouping Data
                                                                                                                                                                                        • ☰ 16.3. Basic Correlation and Regression
                                                                                                                                                                                          • ☰ 16.4. Simple Data Cleaning Techniques
                                                                                                                                                                                            • ☰ 16.5. Basic Time Series Handling
                                                                                                                                                                                            • ☰ 17. Working with Time and Dates
                                                                                                                                                                                              • ☰ 17.1. Representing Dates and Times
                                                                                                                                                                                                • ☰ 17.2. Creating and Formatting datetime Arrays
                                                                                                                                                                                                  • ☰ 17.3. Date Calculations and Differences
                                                                                                                                                                                                    • ☰ 17.4. Plotting Data over Time
                                                                                                                                                                                                    • ☰ 18. Introduction to Signal and Image Handling
                                                                                                                                                                                                      • ☰ 18.1. Basic Signal Representation in MATLAB
                                                                                                                                                                                                        • ☰ 18.2. Simple Filtering and Transformations
                                                                                                                                                                                                          • ☰ 18.3. Reading, Displaying, and Writing Images
                                                                                                                                                                                                            • ☰ 18.4. Basic Image Transformations
                                                                                                                                                                                                            • ☰ 19. Automation and Scripting Tasks
                                                                                                                                                                                                              • ☰ 19.1. Batch Processing of Files
                                                                                                                                                                                                                • ☰ 19.2. Writing Reusable Utility Scripts
                                                                                                                                                                                                                  • ☰ 19.3. Using Relative Paths and the MATLAB Path
                                                                                                                                                                                                                    • ☰ 19.4. Simple User Interaction with input and disp
                                                                                                                                                                                                                    • ☰ 20. Introduction to Toolboxes
                                                                                                                                                                                                                      • ☰ 20.1. What Are Toolboxes and How to Use Them
                                                                                                                                                                                                                        • ☰ 20.2. Exploring Installed Toolboxes
                                                                                                                                                                                                                          • ☰ 20.3. Finding Functions in Toolboxes
                                                                                                                                                                                                                            • ☰ 20.4. Add-Ons and MATLAB File Exchange
                                                                                                                                                                                                                            • ☰ 21. Best Practices and Next Steps
                                                                                                                                                                                                                              • ☰ 21.1. Organizing Projects and Folders
                                                                                                                                                                                                                                • ☰ 21.2. Version Control Basics with MATLAB
                                                                                                                                                                                                                                  • ☰ 21.3. Performance Tips for Beginners
                                                                                                                                                                                                                                    • ☰ 21.4. Learning Resources and Documentation Path
                                                                                                                                                                                                                                      • ☰ 21.5. Suggested Practice Projects for Beginners