3.1.4 System logs (journalctl)

Table of Contents

Understanding system logs with `journalctl`

Systemd keeps most modern Linux system logs in a central component called the journal. The main tool to read and work with this journal is the journalctl command. This chapter focuses on how to use journalctl in practical, everyday administration tasks.

The systemd journal and `journalctl`

When your system uses systemd, many messages that would traditionally go to files in /var/log are collected by the systemd journal. These include kernel messages, service logs, and many system messages.

journalctl is the standard interface to query and display those logs. You can think of it as a log viewer that can filter by time, service, boot, and many other fields.

journalctl reads from the persistent journal stored on disk, usually under /var/log/journal, and from the volatile journal kept in memory. You normally do not edit these files yourself. Instead, you ask journalctl to show you what you need.

Viewing logs in basic ways

The simplest use of journalctl is to print all logs that the journal knows about.

journalctl

This may produce a lot of output. By default, it is shown through a pager such as less, so you can scroll. You can quit the pager with q.

To see only the most recent lines, you can use the -n option with a number.

journalctl -n 50

This shows the last 50 log lines. If you want journalctl to follow new log entries in real time, similar to tail -f, use the -f option.

journalctl -f

This is very useful while you start or restart a service, because you can immediately see what it logs.

You can combine both options, for example start with the last 50 lines and then follow new ones.

journalctl -n 50 -f

Showing logs from the current boot

The journal stores logs from multiple boots, if persistent logging is enabled. To focus only on messages from the current boot, you can use the -b option.

journalctl -b

If you want the logs from the previous boot, use -b -1. For two boots ago, use -b -2, and so on.

journalctl -b -1

This is very useful when you are investigating why the system failed or misbehaved during a specific boot.

Filtering logs by time

It is often easier to debug a problem if you can see only the messages that happened around a particular time. journalctl provides --since and --until options for this.

You can specify absolute times in many human friendly formats.

journalctl --since "2026-01-07 09:00" --until "2026-01-07 10:00"

You can also use relative times like "1 hour ago" or "yesterday".

journalctl --since "1 hour ago"
journalctl --since yesterday --until now

If you omit --until, it defaults to now. If you omit --since, it defaults to the beginning of the journal.

Important rule: Use --since and --until together when you want to tightly limit the time window of logs. This keeps output small and focused on the problem period.

Viewing logs for a specific service

One of the most common uses of journalctl together with systemd is to see the logs of a specific service unit. When you know the unit name, you can use the -u option.

journalctl -u ssh.service

This shows all logs for that unit from all boots that are still in the journal. To see only the current boot, use -u together with -b.

journalctl -u ssh.service -b

To watch logs in real time for that service, you can add -f.

journalctl -u ssh.service -f

You can also specify multiple units in one command.

journalctl -u ssh.service -u nginx.service -b

This is very helpful when several services interact and you want to see their messages together.

Filtering by priority

Each log message in the journal has a priority level, for example emergency, alert, critical, error, warning, notice, info, or debug. journalctl represents these as numbers from 0 to 7.

The mapping is:

$$
\begin{aligned}
0 &\rightarrow \text{emerg} \\
1 &\rightarrow \text{alert} \\
2 &\rightarrow \text{crit} \\
3 &\rightarrow \text{err} \\
4 &\rightarrow \text{warning} \\
5 &\rightarrow \text{notice} \\
6 &\rightarrow \text{info} \\
7 &\rightarrow \text{debug}
\end{aligned}
$$

You can limit output to messages at or above a given priority with -p. For example, to see only errors and more serious messages, use priority 3.

journalctl -p 3

You can also use the name instead of the number.

journalctl -p err

This shows messages with priority error, critical, alert, or emergency. If you are investigating a problem, this is a good way to focus on the most important messages.

Important rule: -p filters include all messages with priority less than or equal to the given number, which in this scheme means messages that are equally or more severe.

You can combine priority with other filters. For example, errors from a single service in the current boot.

journalctl -u ssh.service -b -p err

Filtering by process, PID, or executable

Sometimes you know which process you care about, but not which service unit. The journal stores additional metadata such as the process ID and the executable path.

To filter by the process ID, use the _PID field.

journalctl _PID=1234

To see logs from a specific executable binary, use _EXE.

journalctl _EXE=/usr/sbin/sshd

You can also list the journal fields present in your logs with -o verbose, which prints each entry with many fields. This is more advanced, but useful when you want to discover what you can filter on.

journalctl -n 1 -o verbose

From there, you can pick a field like _UID, _COMM, or _SYSTEMD_UNIT and use it in further queries, for example:

journalctl _COMM=sshd

Using different output formats

By default, journalctl shows output in a familiar human readable format. You can change this with the -o option.

Common formats include short, short-iso, json, and cat.

journalctl -o short-iso

This prints timestamps in an ISO like format which is more precise and easier for scripts.

To get raw message text without metadata or timestamps, use -o cat.

journalctl -u ssh.service -o cat

For use in scripts or external tools, you can use -o json or -o json-pretty.

journalctl -u ssh.service -o json-pretty

This prints each entry as a JSON object, which is convenient for further processing.

Showing only recent entries with limits

When exploring logs, it is often helpful to combine filters so that you do not drown in output.

You already saw -n to limit the number of lines. Another useful option is --no-pager if you want output to go directly to the terminal without an interactive pager.

journalctl -u ssh.service -n 20 --no-pager

You can also use --lines as a long form of -n, and you can combine this with time ranges and priorities.

journalctl -u nginx.service --since "10 min ago" -p warning

This might be exactly what you need when a problem started just a few minutes ago.

Persistent vs volatile logs

Systemd can keep logs in memory only, or also persist them on disk. If persistent storage is disabled, logs vanish after reboot. To check whether persistent logs exist, look for /var/log/journal.

ls /var/log/journal

If this directory exists and is writable for the journal, logs across boots are stored there. If it does not exist, systemd usually writes logs under /run/log/journal which is a temporary filesystem.

Enabling persistent journaling usually involves creating /var/log/journal with the right permissions and may involve editing a configuration file. The exact configuration belongs to the broader topic of systemd administration, so here it is enough to know that the presence of /var/log/journal affects how many past boots you can query with journalctl -b -1, -b -2, and so on.

Important rule: If you need logs that survive reboots, make sure that persistent storage for the journal is enabled, otherwise valuable diagnostic information will be lost.

Size control and rotation concepts

The journal manages its own storage limits. It decides how much disk space to use for logs, how to rotate old entries, and when to remove them. This behavior is controlled by configuration but can also be influenced with journalctl commands.

One common administrative task is to remove very old logs to free space. You can ask the journal to remove entries older than a certain time.

sudo journalctl --vacuum-time=2weeks

This keeps only the last two weeks of logs. Similarly, you can limit size.

sudo journalctl --vacuum-size=500M

This forces the journal to keep its total size at or below 500 megabytes by deleting older entries.

You can also specify a number of individual journal files to keep.

sudo journalctl --vacuum-files=5

Journal files and their detailed management are an advanced topic, but knowing these --vacuum options is often enough to prevent disk usage problems.

Using `journalctl` with `sudo` and user logs

Some journal entries are only visible to root or users with special permissions. If you run journalctl as a normal user, you may see only your own user related logs.

To see everything, you usually need to run journalctl with sudo.

sudo journalctl -b

Systemd can also store logs per user session. To view logs for your user, you can use --user.

journalctl --user

And to see logs for a specific user unit, use --user with -u.

journalctl --user -u my-user-service.service

This is relevant when you work with user services instead of system wide services.

Combining filters for practical troubleshooting

The real strength of journalctl appears when you combine different filters to answer precise questions.

For example, imagine a web server service failed after a configuration change at around 14:00. You might want to see only error messages for that service from that time until now.

sudo journalctl -u nginx.service --since "2026-01-07 14:00" -p err

If a system crashed during a previous boot and you want to see all kernel and service messages that led to it, you might use:

sudo journalctl -b -1

If the crash affected only specific subsystems, such as networking, you could further narrow it down by unit or by priority.

The general pattern is to combine:

-b to select the boot
-u to select the unit
--since and --until to select the time window
-p to select important severity levels
-n and -f to control scope and live updates.

Important rule: In real troubleshooting, always combine filters. A broad journalctl without -b, -u, --since, or -p can produce overwhelming output and hide the messages you actually need.

By mastering these combinations, you turn journalctl into a precise instrument that shows exactly the logs that matter for your current problem.

Comments

Please login to add a comment.

Don't have an account? Register now!