Troubleshooting Linux: reading logs, safe mode and getting help

TUREFF• •0

Sooner or later something on a Linux system will misbehave: Wi-Fi stops working after an update, the screen stays black after boot, or an application refuses to open. Troubleshooting Linux is a skill like any other, and the system gives you more information than most to work with. This guide sets out a calm, methodical approach.

Start with the basics

Note exactly what happened and when it started. Did you just install an update, a driver or a new program?
Restart. It sounds obvious, but a pending kernel or driver update only takes effect after a reboot.
Check for updates. The fix may already be available.
Try another user account. If a problem disappears in a fresh account, it is probably a setting in your own home folder rather than the system.

Run the program from a terminal

If an application will not open or crashes, start it from a terminal by typing its command name. Error messages that would otherwise vanish are printed in the window, and they are often the most useful clue you will get.

Read the logs

Most modern distributions use systemd, which collects logs in a journal.

'journalctl -b' shows messages from the current boot. Adding '-p err' limits it to errors.
'journalctl -b -1' shows the previous boot, which is useful after a crash or freeze. On some systems the journal is only kept in memory, so earlier boots may not be available unless persistent logging is enabled.
'journalctl -u' followed by a service name shows logs for one service, such as the network manager.
'dmesg' shows kernel messages, including hardware and driver problems. It may need sudo.
'systemctl --failed' lists services that did not start.
/var/log holds further log files on many systems.

You do not need to understand every line. Look for lines marked error or failed near the time of the problem, and copy the exact wording.

When the system will not boot properly

Use the boot menu. On many distributions holding Shift or pressing Esc during start-up shows the GRUB menu.
Boot an older kernel. Under the advanced options entry there is usually a list of installed kernels. If an older one works, a new kernel or driver is likely the cause.
Recovery mode. Ubuntu and related distributions offer a recovery mode entry with a menu for repairing packages, checking the file system or dropping to a root shell. Other distributions have equivalent rescue targets; check your distribution's documentation.
A live USB lets you start a working system, rescue files from the internal drive and repair the installation.
Graphics problems after installing a driver are common. Booting with basic graphics, sometimes called nomodeset or safe graphics, can get you to a desktop to remove or change the driver.

Free space and hardware

A full disk causes strange failures. Check free space with 'df -h'.
Check hardware details with commands such as 'lspci', 'lsusb' and 'lsblk'. Knowing exactly which Wi-Fi chip or graphics card you have makes searching for a fix much easier.

Searching for answers

Search with the exact error message in quotation marks, plus your distribution and version.
Prefer official sources first: your distribution's documentation, wiki and forums. The Arch Wiki is widely used as a detailed reference even by people on other distributions, though commands may differ.
Check dates. Old answers may refer to tools that have since been replaced.
Never run commands you do not understand, especially ones that delete files, change partitions or download and run scripts with sudo.

Asking for help well

People on forums and communities are generous with their time, and a good question gets better answers.

Say which distribution, version and desktop you are using.
Describe what you expected and what happened, and what you have already tried.
Paste the relevant log lines or error messages as text, not a photograph of the screen.
Remove personal details such as usernames, network names or serial numbers from logs before sharing.
Report back with what fixed it, so the next person can find the answer.

Prevent the next problem

Keep backups and system snapshots so a bad update is easy to undo.
Change one thing at a time when experimenting, so you know what caused a change.

What is the most stubborn Linux problem you have solved, and how? Share your troubleshooting stories in the comments.