4.2.5 Troubleshooting boot issues

Understanding Boot Failures

Boot problems usually show up in a few repeatable ways. The system might stop with a black screen, drop to a rescue shell, loop back to the firmware menu, or show specific error messages from GRUB, the kernel, or initramfs. Your main job is to identify which stage of the boot process failed and then collect enough information to fix or work around the problem.

The Linux boot process has several distinct stages, from firmware, to bootloader, to kernel, to initramfs, to the real root filesystem and user space. When troubleshooting, always ask: how far did the system get, and what was the last component that clearly worked?

By classifying the failure this way, you avoid random guessing and focus on the right tools and configuration files.

Common Symptoms and Where to Look

Different symptoms point to different stages.

If the system jumps straight back to the BIOS or UEFI menu without any GRUB screen, this usually indicates a firmware or bootloader installation issue, or a bad boot entry in UEFI.

If GRUB appears, but shows errors or cannot find a kernel or root filesystem, the problem is usually in GRUB configuration, the disk layout, or the partition identifiers.

If the kernel starts but you see messages about failing to mount the root filesystem, or you are dropped into an initramfs shell, the issue often involves storage drivers, UUID mismatches, missing modules, or damaged filesystems.

If the system reaches the point where you see the distribution logo or a text login prompt but then hangs or fails to give you a usable login, the problem is typically in the system initialization stage, such as systemd services, graphics drivers, or some misconfigured system component.

Recognizing these patterns makes it easier to choose the right rescue method.

Accessing a Rescue Environment

When a system will not boot normally, you often need an external environment to investigate and repair it. There are three main approaches.

You can use your distribution installer ISO in a rescue mode or its live desktop. Boot from a USB stick that contains your distribution or a similar one. Once in the live environment, you can mount the disk of the broken system, inspect logs, repair filesystems, and even reinstall the bootloader.

You can use GRUB’s own rescue or command line interface. If GRUB appears but fails to load the kernel automatically, you can drop to its command line, inspect disks and partitions, and attempt a manual boot. A successful manual boot shows you what configuration needs to be changed permanently.

If the kernel reaches an emergency shell or initramfs prompt, you are already in a minimal environment with some tools. You can inspect /dev, run utilities like ls, blkid, and sometimes fsck, and check why the root filesystem is not mounting.

A chroot lets you run commands like grub-install and update-grub in the context of the installed system, even though you booted from USB.

GRUB and Bootloader Problems

If the failure happens before the kernel starts, GRUB or the underlying bootloader configuration is the primary suspect. Different distributions may use slightly different tools, but the basic patterns of failure are similar.

When GRUB displays a prompt with a message such as grub> or grub rescue>, it cannot find its configuration file or required modules. This may happen after repartitioning disks, changing disk order, or cloning disks. Common error texts include messages about missing normal.mod or missing partitions.

You can use GRUB’s command line to inspect the available disks. Commands like ls within GRUB will show entries such as (hd0,msdos1) or (hd0,gpt2) that correspond to partitions. You can then search for /boot or /boot/grub directories to find the correct partition.

Once you identify the correct partition, you can set GRUB variables interactively, load the proper modules, and instruct GRUB to load the kernel and initrd. A typical manual boot sequence in GRUB involves setting root, specifying a linux line with the kernel and root parameter, adding an initrd line, and then running boot. The exact paths depend on your distribution layout and the partition where /boot resides.

If you manage to boot manually, you should regenerate the GRUB configuration permanently from the running system. On Debian based systems this is often done with update-grub. On Fedora or similar, you may need to run grub2-mkconfig with the correct output filename. If the bootloader itself is missing from the disk, or the UEFI firmware does not have a proper boot entry, you may also need to run grub-install to reinstall it, specifying the target disk and any required options.

On UEFI systems, boot problems sometimes arise from broken or missing EFI entries. The firmware uses information stored in NVRAM to know which EFI executable to run. You can manage these entries from a running or rescue system with tools like efibootmgr. If necessary, you can recreate or adjust these entries so they correctly point to your bootloader in the EFI System Partition.

Carefully note the working settings you use in the GRUB prompt so you can replicate them in the configuration files.

Kernel and `initramfs` Related Failures

Once the bootloader successfully starts the kernel, problems may still occur before the system mounts the real root filesystem and hands control to user space. Typical messages include errors about the root device not being found or timeouts waiting for a particular UUID.

A very common reason for this class of error is a mismatch between the kernel command line and the actual disk layout. The kernel command line usually includes a parameter such as root=UUID=<value> or root=/dev/sdXY. If the disk was cloned, repartitioned, or the filesystem was recreated, the UUID will change. The kernel will then look forever for a filesystem that no longer exists.

You can see the current kernel command line in GRUB by editing an entry before boot, or later from within a running system using /proc/cmdline. To fix a broken system, either adjust the command line in GRUB to point to the correct root device, or update the GRUB configuration to use the current UUIDs reported by tools like blkid.

Another frequent cause is an incomplete or outdated initramfs. This small image contains the drivers and scripts required to locate and mount the root filesystem. If it does not include the driver for your storage controller, or if custom hooks are incorrect, the kernel may be unable to see the disk. This is common after moving a disk to a new machine with different hardware, or after removing modules that were previously required.

In such cases, booting from a rescue system and regenerating the initramfs using your distribution’s tools is the usual remedy. You must identify the kernel version you want to repair, then run the relevant command inside a chroot of the installed system. This rebuild ensures that the necessary modules and configuration are embedded again.

Sometimes, cryptographic layers are involved, such as full disk encryption. Missing or broken prompts for decryption passphrases can also lead the kernel to believe that no root filesystem exists. In this situation, verify that the initramfs includes the encryption hooks and that the kernel command line contains the correct parameters for the encrypted device and its mapping name.

When you are dropped to an initramfs shell, use the limited tools available to confirm which devices can be seen and which filesystems can be mounted. By comparing the kernel’s view of devices with what the boot configuration expects, you can determine whether the problem is a missing driver, a bad identifier, or a damaged filesystem.

Filesystem and Disk Issues

Once the kernel and initramfs are configured correctly, boot may still fail due to problems with the underlying disks or filesystems. A common indicator is a message about an unclean or inconsistent filesystem, followed by a prompt to run fsck manually. In some configurations, the system will stop completely and refuse to continue until you repair the filesystem.

You should always treat such situations carefully, because using filesystem repair tools incorrectly can result in data loss. The safest method is to boot into a live or rescue environment, ensure that the affected filesystem is not mounted, and then run the appropriate fsck command on the device. This tool will scan the filesystem structures and attempt to correct inconsistencies. Often, you will be asked to approve changes. Reading the messages closely helps you avoid destructive options.

In other cases, the disk itself may be failing. Symptoms include repeated I/O errors, long timeouts when trying to read certain blocks, or an inability to identify the disk at all. If you suspect failing hardware, consider copying critical data to another medium before running invasive repairs. It is sometimes better to image a failing disk using tools that can cope with read errors, then operate on the image rather than the original disk.

During boot, network filesystems or secondary mount points can also cause delays or failures. For example, if /etc/fstab contains entries for devices or network shares that no longer exist, systemd may wait for a long time or drop into emergency mode. This is particularly noticeable when mounting devices listed as auto without appropriate timeout options.

To diagnose such problems, use the emergency shell to inspect /etc/fstab. You can comment out questionable lines temporarily, or adjust mount options to include nofail where appropriate, which allows the system to continue booting even if a particular mount is not available.

By separating hardware failure from mere filesystem inconsistency, you can choose a recovery strategy that preserves as much data as possible.

Systemd Emergency and Rescue Modes

Even after the root filesystem mounts correctly, the system may fail partway through startup. On modern Linux systems that use systemd, this often results in an emergency shell or a rescue mode. Understanding the difference between these modes is key to recovering the system.

Rescue mode is a minimal multiuser environment where basic services run, but graphical interfaces and most nonessential services are disabled. You usually get a single root shell. This mode is useful for fixing configuration issues without interference from background services.

Emergency mode is even more minimal. In this mode, systemd mounts the root filesystem, usually read only, and gives you a root shell with minimal services. systemd enters this mode when essential mounts or services fail, or when explicitly instructed through the kernel command line.

You can manually request these modes from the bootloader by editing the kernel command line. Adding systemd.unit=rescue.target requests rescue mode, while systemd.unit=emergency.target places the system in emergency mode. This is very helpful when you know a configuration change will cause trouble, but you still want to boot just far enough to fix it.

Once you are in one of these modes, you can use familiar tools to inspect logs, edit configuration files, and restart services. If the root filesystem is read only, remounting it as read write is usually necessary before making changes. After repairs, you can exit the shell or run commands to switch to the normal default target so that systemd continues booting.

Emergency and rescue modes are especially useful when a misconfigured service breaks networking, graphics, or another critical subsystem. Instead of attempting a full boot that fails repeatedly, you can deliberately enter a minimal mode, fix the offending service or configuration, and then continue startup.

Using Logs and Diagnostics

Accurate diagnosis of boot issues relies heavily on logs. On systems that reach at least some part of systemd initialization, the journal contains a detailed history of boot messages, including service start failures, mount problems, and dependency issues.

When the system can boot, but behaves strangely early in startup, you can use tools to analyze the logs from the previous boot. These tools let you filter by service, priority, or time. For example, inspecting only the last boot or only error and warning messages can reduce the noise and reveal the failing component quickly.

On systems that cannot boot fully but can reach emergency or rescue mode, the journal may still be available. You can view it from that minimal environment, sometimes after remounting filesystems appropriately. The logs will often show which service failed and why systemd chose to drop into emergency mode.

Even if you cannot load the system’s kernel at all, you can still examine logs later by mounting the root filesystem from a live environment and inspecting persistent journal directories or plain text log files under /var/log. This is particularly useful for recurring or intermittent boot problems that leave clues across multiple boots.

Besides logs, early boot parameters are a powerful diagnostic tool. You can add debug options to the kernel command line temporarily through GRUB. These options increase verbosity or cause the kernel to stop at certain points. For example, you can disable certain subsystems or select different logging verbosity levels. By experimenting in a controlled way, you can narrow down which component or driver triggers the problem.

Using logs systematically avoids random trial and error and accelerates recovery.

Recovering from Misconfiguration

Many boot issues are self inflicted through configuration changes that went wrong. Editing key files such as the bootloader configuration, /etc/fstab, graphics settings, or central service configuration can easily render the system unbootable.

A general pattern for recovery is to temporarily bypass or neutralize the offending configuration, then restore a correct version. If the system fails soon after a change to the bootloader configuration, you can revert the change by booting into a live system, mounting the root partition, and editing the relevant files directly. After that, regenerate the GRUB configuration if your distribution uses generated files.

If you recently modified /etc/fstab and now the system stops in emergency mode complaining about failed mounts, you can comment out the problematic lines and retry booting. Once the system boots successfully, reintroduce the required mounts carefully, testing one change at a time.

For graphical issues, such as a black screen after the boot splash but before the login screen, misconfigured graphics drivers are common. You can typically work around this by booting with additional kernel parameters that disable certain driver features, or by selecting a more basic driver. Once you reach a shell, adjust or remove the problematic graphics configuration. You can also choose a text only target temporarily and re-enable the graphical environment only after it is stable again.

Using version control or backup copies for critical configuration files is extremely helpful. If you keep previous known good versions under distinct names, recovery becomes as simple as restoring the older file. Some systems also keep automatic backups of certain configuration files, especially after major upgrades. Knowing where these backups reside lets you recover quickly.

No matter the specific configuration issue, a careful and minimal approach is best. Apply only one fix at a time, test the next boot, and avoid making multiple unrelated changes when you are already in a broken state. This incremental strategy makes it easier to know which adjustment truly solved the problem.

Minimizing Downtime and Preventing Future Issues

While troubleshooting is sometimes unavoidable, you can significantly reduce the frequency and impact of boot problems by planning ahead. The first preventive measure is to keep reliable backups of both your data and your critical system configuration. With good backups, you can afford to be conservative during repair attempts and, if necessary, reinstall the system while preserving your data and then restore configuration gradually.

Testing changes before applying them to a primary system is another key practice. For example, you can maintain a small virtual machine or secondary installation where you try new kernel versions, bootloader changes, or encryption setups. Once you understand the procedure and verify that it works, you can apply it to the main system with more confidence.

Monitoring disk health helps you catch hardware failures early. Regularly checking SMART data or similar health indicators and watching for increasing error counts allows you to replace or clone failing disks before they cause boot failures.

When performing operations that affect the boot process, such as repartitioning, converting filesystems, or changing encryption, document exactly what you do. This record is invaluable if you need to retrace your steps during a later investigation. It also helps you recognize patterns in past issues.

Finally, maintain at least one current bootable rescue medium, such as a USB stick with your preferred distribution. Verify that it actually boots on your hardware. When a serious boot problem occurs, a ready and tested rescue tool can be the difference between a quick fix and a prolonged outage.

Boot troubleshooting is much easier when you approach it methodically, keep your tools prepared, and respect the central role of the boot process in a Linux system.