Basic Linux command line usage

Getting Started with the Linux Shell

On almost all HPC systems you interact with Linux via a text-based interface called the shell (most commonly bash). You type commands, press Enter, and the shell runs them. This section focuses on the minimum practical skills you need to be productive on a cluster command line.

Terminals, prompts, and basic anatomy of a command

A command line usually looks something like:

user@login-node:~$ ls -l /home/user

Typical elements:

• user – your username
• login-node – the machine you’re logged in to
• ~ – your current directory (here, your home)
• $ – prompt symbol (for normal users; # is usually root)
• ls -l /home/user – the command you typed

General command structure:

command  [options]  [arguments]

Examples:

• ls – command only
• ls -l – command with option -l
• ls -l /scratch/user – command with option and argument
• cp input.txt results/output.txt – command with multiple arguments

Options often come in short and long forms:

• Short: -l, -a, -r
• Long: --long, --all, --recursive

You can combine some short options: ls -l -a → ls -la (not always true for every command, but common).

Navigating the filesystem

You will constantly move around directories and manipulate paths.

Current directory, home, and absolute vs relative paths

Key concepts:

• Current directory: where you “are” right now.
• Home directory: usually /home/username on clusters, often shown as ~.
• Absolute path: starts with / and is from the filesystem root:
• /home/user/project
• /scratch/user/run1
• Relative path: interpreted from your current directory:
• data/input.txt
• ../results

Useful shorthand:

• ~ – your home directory
• . – current directory
• .. – parent directory

Viewing where you are

pwd          # print working directory

Example:

[user@login ~]$ pwd
/home/user

Changing directories

cd path

Useful forms:

cd             # go to your home directory
cd ~           # same as above
cd /scratch    # go to absolute path /scratch
cd project     # go to subdirectory 'project' under current directory
cd ..          # go up one level
cd ../..       # go up two levels

Listing files

ls                      # list files in current directory
ls some/other/dir       # list another directory
ls -l                   # long listing (permissions, size, dates)
ls -a                   # show hidden files (starting with .)
ls -lh                  # human-readable sizes
ls -lrt                 # sorted by time, newest at bottom

Example:

ls -lh

might show:

:::text
-rw-r--r-- 1 user hpc 1.2K Dec 10 10:21 job.slurm
drwxr-xr-x 2 user hpc 4.0K Dec 10 09:00 data

• Leading d = directory, - = regular file.
• 1.2K etc are sizes when using -h.

Working with files and directories

You will create, copy, move, and delete files frequently.

Creating directories

mkdir newdir           # create a single directory
mkdir -p a/b/c         # create nested directories as needed

Copying files and directories

cp source.txt dest.txt         # copy file
cp source.txt otherdir/        # copy into directory
cp -r dir1 dir2                # copy directory recursively
cp -r dir1 /scratch/user/      # copy dir1 to /scratch/user

On HPC systems, you often copy data between your home and scratch areas.

Moving and renaming

mv both moves and renames:

mv oldname.txt newname.txt     # rename
mv file.txt dir/               # move into a directory
mv dir1 /scratch/user/         # move directory

Moving within the same filesystem is usually fast; moving across filesystems may take longer.

Removing files and directories

Use rm carefully; there is no “recycle bin” by default.

rm file.txt              # remove file
rm -i file.txt           # interactive; asks before removing
rm -r dir/               # remove directory and contents
rm -rf dir/              # force recursive remove (DANGEROUS)

Safer habits, especially on shared systems:

• Use rm -i while you learn.
• Double-check the path before using rm -r.
• Avoid rm -rf * unless you are absolutely sure.

Viewing and editing text files

Most HPC work involves text files: scripts, job files, configuration.

Quick file inspection

cat file.txt            # print entire file
less file.txt           # view page by page (recommended)
head file.txt           # first 10 lines
head -n 50 file.txt     # first 50 lines
tail file.txt           # last 10 lines
tail -n 50 file.txt     # last 50 lines
tail -f logfile.log     # follow file as it grows (useful for job logs)

Usage tips for less:

• Arrow keys / PageUp / PageDown to navigate.
• q to quit.
• /word to search for word; press n for next match.

Simple text editing options

Every cluster will have at least one terminal editor like nano, vi, or vim. For beginners, nano is commonly easiest:

nano job.slurm

Inside nano:

• Type to edit.
• Ctrl+O – write file (save).
• Ctrl+X – exit.

More advanced editor usage will typically be introduced gradually; for now, you only need the ability to open, edit, and save simple text files like job scripts.

Creating a file quickly

touch empty.txt         # create an empty file (or update its timestamp)

Or with redirection (see below):

echo "hello" > hello.txt

Running programs and using the shell path

To run a program, you simply type its name if it’s in your PATH:

python myscript.py
gcc code.c -o code

If a program is in the current directory and not in PATH, you must prefix with ./:

./my_program

Key points:

• PATH is an environment variable listing directories the shell searches for commands.
• Adding . (current directory) to PATH is often discouraged for security on shared systems; instead, use ./program.

Input, output, and redirection

Command-line tools typically read from standard input (stdin) and write to standard output (stdout) and standard error (stderr). You can redirect these streams to/from files.

Redirecting output

command > out.txt        # stdout to file (overwrite)
command >> out.txt       # stdout to file (append)

Example:

ls -l > listing.txt      # save listing in a file

Redirecting input

command < input.txt

Common with programs that read from stdin by default:

sort < unsorted.txt

Redirecting error output

command 2> errors.txt    # stderr only
command > out.txt 2>&1   # stderr to same place as stdout

This is particularly useful when capturing logs from HPC jobs.

Using pipes to combine commands

A pipe | connects the output of one command to the input of another:

command1 | command2

Examples:

ls -l | less                  # view long listing page by page
grep "ERROR" logfile | wc -l  # count lines containing "ERROR"

Pipes allow you to build small data-processing chains without writing code.

Finding files and searching text

You will often need to locate files or search within them.

Searching for text with `grep`

grep "pattern" file.txt       # search pattern in file
grep -n "pattern" file.txt    # include line numbers
grep -r "pattern" dir/        # search recursively in a directory tree
grep -i "pattern" file.txt    # case-insensitive

Combined with less:

grep "ERROR" logfile | less

Finding files with `find`

Basic usage:

find path -name "pattern"

Examples:

find . -name "*.txt"          # all .txt files under current dir
find /scratch/user -maxdepth 2 -type f -name "job*.out"

Use find cautiously on large directory trees; it can traverse many files.

Command history and editing

Using command history effectively saves significant time.

History and re-running commands

history              # list recent commands
!n                   # run command number n from history
!!                   # run previous command again

Keyboard shortcuts (in most shells):

• Up / Down arrows – browse history.
• Ctrl+A – move to start of line.
• Ctrl+E – move to end of line.
• Ctrl+U – delete from cursor to beginning.
• Ctrl+K – delete from cursor to end.

Tab completion

Press Tab to auto-complete filenames and commands:

• Type cd proj then press Tab → cd project/ (if unique).
• Type module loa then Tab → often expands to module load.

This is extremely useful when working with long directory names and paths.

Getting help on Linux commands

Linux commands usually include built-in help.

The `--help` option

Many commands provide a quick summary:

ls --help
grep --help

Manual pages with `man`

man ls
man grep
man find

Navigation in man is like less:

• Arrow keys / PageUp / PageDown.
• /word to search.
• q to quit.

`whatis` and `which`

whatis ls        # brief description of a command
which python     # path to the python executable that will run

Permissions and ownership (basic view)

On shared HPC systems, file permissions matter for collaboration and security.

Reading permission bits

Use ls -l:

-rw-r--r-- 1 user hpc  123 Dec 11 09:00 file.txt
drwxr-xr-x 2 user hpc 4096 Dec 11 08:55 scripts

First 10 characters: file type and permissions:

• -rw-r--r--
• - = regular file (d = directory)
• rw- = owner permissions (read, write)
• r-- = group permissions (read)
• r-- = others (read)

For directories:

• r – list contents.
• w – create/delete files in the directory.
• x – enter the directory (cd into it).

Basic permission changes with `chmod`

chmod u+x run.sh      # add execute for user
chmod g-r data.txt    # remove read for group
chmod 600 secret.txt  # user read/write; no one else
chmod 644 notes.txt   # user read/write; others read-only

You rarely need to change ownership on a cluster (that is typically an admin task), but you may adjust permissions for collaborators or to make scripts executable.

Useful everyday commands for HPC work

A few additional commands you will encounter frequently:

whoami         # your username
hostname       # machine you are on (useful on clusters)
date           # current date and time
du -sh .       # size of current directory
df -h          # disk usage by filesystem
wc -l file     # line count
sort file      # sort lines
uniq file      # remove consecutive duplicates

These become especially useful when checking disk usage quotas, inspecting output, or quickly summarizing data.

Safe habits on shared systems

On HPC clusters, your command-line actions affect shared resources. Some good practices:

• Always know your current directory (pwd) before running rm -r.
• Avoid recursive commands (rm -r, chmod -R, chown -R) from high-level directories unless you fully understand the impact.
• Be cautious with commands that can generate huge outputs (cat largefile, ls -R /) — they can clutter your terminal and stress shared storage.
• Use less and pipes to limit what you view at once.

These basic Linux command-line skills are the foundation for all subsequent HPC tasks: editing job scripts, managing input/output data, launching jobs, and interacting with software environments on clusters.