7.2.3 Analyzing suspicious processes

Why Suspicious Processes Matter

On a compromised or misconfigured Linux system, almost every attacker action eventually becomes a process. Understanding how to recognize and inspect suspicious processes lets you move from vague “something is wrong” signals to concrete, observable evidence. In an incident response workflow, process analysis often sits between initial detection and containment, and it directly informs what you preserve as evidence and how you scope the compromise.

This chapter focuses on practical techniques to identify and investigate processes that may be hostile or abnormal. It assumes you already know what a process is and how to view them at a basic level, and concentrates instead on patterns, tools, and ways of thinking that are specific to forensics and incident response.

Building a Mental Model of “Normal”

You cannot reliably spot suspicious behavior without some notion of what is normal on the system you are investigating. In practice, you rarely have perfect baselines, but you can approximate normal by combining system role, running services, and common administrator activity.

On a small web server, for example, you expect to see a web server process like nginx or httpd, a database like mysqld or postgres, SSH like sshd, logging and cron processes, and perhaps a process supervisor. On a desktop, you expect a graphical session, a browser, file manager, and background services. Persistent high CPU load from bash on a lightly used mail server is likely abnormal, while mysqld using memory on a busy database host is probably not.

As you investigate, keep asking whether a process makes sense for the system’s purpose, the time of day, the logged-in users, and known scheduled tasks. Suspicion often starts with a mismatch between expected behavior and observed behavior.

Initial Triage of Running Processes

When you first examine a potentially compromised system, a quick overview of current processes helps you identify obvious red flags and prioritize deeper inspection.

Common tools include ps, top, htop, and pstree. For forensics you are less interested in routine system daemons and more interested in anomalies in names, parents, owners, and resource use.

A useful starting command is:

ps auxf

This shows processes with users, CPU and memory consumption, start time, and, via the f option, a tree-like parent relationship. You can also sort by resource usage with tools like top or htop to immediately see who is consuming CPU, memory, or I/O.

Look in particular for:

Processes consuming unexpected resources. Long-running sh, bash, or python processes using high CPU or network I/O can be signs of cryptominers, brute force scripts, or data exfiltration.

Unexpected interactive shells. An unrecognized user with /bin/bash attached to a controlling terminal may indicate a backdoor shell.

Processes started by unexpected parents. A web server that spawns nc, bash, or python might indicate remote command execution via that service.

Processes running as privileged users that usually should not. A python script running as root from /tmp deserves attention.

At this stage, avoid killing processes unless you must contain active harm and are willing to lose some volatile evidence. Observing and documenting first, then acting, is usually preferable in incident response.

Recognizing Suspicious Naming and Paths

Attackers often rely on tricks that exploit how people skim process lists. Process names, executable paths, and command lines are all opportunities to hide in plain sight or to stand out once you know what to look for.

Common suspicious naming patterns include:

Misspelled system commands. A process called crond on a system that uses cron, or sshd_ instead of sshd, may indicate a trojaned or mimic process.

Numeric or random-looking names. Names like 1234, kworkerds, java-11, or strings that resemble hashes may be an attempt to look unremarkable while not corresponding to an installed package.

Very long or very short names. A binary named . or _ running from /tmp, or a very long name designed to wrap in terminal output to hide arguments, should draw attention.

Names that mimic kernel threads. Kernel threads often appear in ps with names in brackets like [kworker/0:1]. A user-space process with a bracketed or similar name is a common stealth trick.

Executable locations and working directories also provide strong clues. Decent hygiene on production systems means arbitrary binaries rarely live in /tmp or world-writable directories. You should be skeptical of executables under locations like /tmp, /var/tmp, /dev/shm, or user home subdirectories such as ~/.cache or ~/.local that are not tied to installed software.

Use ps to show full command lines, including executable paths and arguments:

ps auxww

Short, system-like names that are launched from odd paths are good candidates for deeper analysis. A process named klogd is expected on some systems, but if it is running from /tmp/.klogd instead of /sbin/klogd, this inconsistency is important.

Analyzing Process Trees and Parent Relationships

Every user-space process has a parent. Attackers often insert themselves into existing processes or spawn new ones from compromised services. Tracing ancestry helps you understand how the suspicious activity started and whether it is tied to a particular entry point.

You can inspect process trees in several ways. The pstree command presents a tree structure. Running:

pstree -alp

shows process hierarchies with PIDs and arguments. Within ps, the --forest option also gives a tree-like view grouped by parent.

Key questions to ask as you examine the tree include:

Who is the parent? If a suspicious bash process is a child of sshd, and its argument shows sshd: user@pts/0, then you likely have a compromised or misbehaving user account instead of a daemon exploit.

Does the parent make sense? If apache2 or nginx directly spawns bash, perl, or python that is not part of a known CGI or application, this suggests remote command execution through the web application.

Are there recurring parent patterns? A process supervisor like systemd or supervisord may restart malicious children if configured to do so. If you see the same suspicious child repeatedly respawned, suspect a persistence mechanism.

Is there evidence of process injection or replacement? While injection itself may not be obvious in ps, an unexpectedly privileged process with a parent that is not normally associated with that privilege may be a clue.

Process trees connect the dots between a single suspicious PID and its origin. For incident response, you should record not only the suspicious process, but also its ancestors and siblings, because these relationships help reconstruct the attacker’s path and may reveal additional compromised components.

Examining Process Ownership and Credentials

Ownership, effective user IDs, groups, and capabilities determine what a process can access. Processes that run under unexpected identities or that escalate to root without a clear reason are risky.

Start by reviewing the user and group fields from ps:

ps aux

Note which processes run as root. A root-owned tool started from /usr/bin and managed by systemd is routine. A root-owned script launched from /tmp, or with a command line that includes untrusted network resources, is highly suspicious.

You also want to know whether a process is using setuid or Linux capabilities. While ps does not directly show capabilities, you can query the executable with getcap:

getcap /path/to/executable

If an uncommon binary has capabilities such as cap_net_raw or cap_sys_admin, and this was not set by package installation, it can be a stealthy way to gain privilege.

Reviewing /proc/<pid>/status reveals more details about a process’s credentials. For example:

cat /proc/<pid>/status

This file includes real, effective, and saved user and group IDs, capabilities bitmasks, and security context labels on systems with SELinux or AppArmor. In forensics, changes between real and effective IDs can indicate privilege escalation.

Inspecting Command Lines and Arguments

The exact command that launched a process, with all its arguments, often reveals the process’s role more clearly than the name alone. Attackers use command-line flags and scripts that can expose their tools and intentions.

To view full command lines, use:

ps auxww

This form attempts to avoid truncation of long command lines. For a given PID, you can also read the cmdline file under /proc:

tr '\0' ' ' < /proc/<pid>/cmdline; echo

Null bytes separate arguments there, so the tr command turns them into spaces.

Look for:

Shell one-liners. Constructs like bash -c 'curl ... | sh' show direct execution of remotely fetched code, a frequent tactic in automated exploitation and quick dropper scripts.

Network tools and redirection. Arguments containing nc, socat, bash -i >& /dev/tcp/..., or SSH with unusual flags can point to reverse shells or tunnels.

Encoded or obfuscated payloads. Long strings passed to languages like python, perl, or php using -e, -c, or base64 decoding commands can hide second-stage code.

Suspicious redirections to or from /dev/null, unusual files under /tmp, or log files outside normal directories.

Command lines are often your best immediate insight into what the process was started to do. Be careful when copying them elsewhere so you do not accidentally execute malicious commands on an analyst system.

Using /proc for Deep Process Inspection

The /proc filesystem exposes rich, per-process information that is essential for detailed analysis. Each running process appears as a directory /proc/<pid>. For forensics on a live system, this is one of your primary data sources.

Important files and links include:

/proc/<pid>/exe which is a symbolic link to the executable file in use by the process. You can examine where the program actually resides:

readlink -f /proc/<pid>/exe

If something named sshd is linked to /tmp/ssh, that is very different from the legitimate /usr/sbin/sshd.

/proc/<pid>/cwd which represents the current working directory. This can indicate where the process started from and where it writes relative paths.

/proc/<pid>/fd/ which contains file descriptors opened by the process. This tells you what files, sockets, and pipes are in active use and can reveal access to unexpected files or hidden storage.

/proc/<pid>/maps and /proc/<pid>/smaps which show memory mappings. These are more advanced, but you can sometimes spot injected or unpacked payloads here, such as anonymous executable memory regions.

/proc/<pid>/environ which holds environment variables for the process. These can contain secrets, configuration, or indicators of persistence hooks.

By copying key /proc/<pid> files to a safe location, you preserve volatile evidence for later offline analysis. In an incident response context, you often want to archive at least the executable (exe target), command line, environment, open descriptors, and possibly the full memory image if you have tooling for that.

Identifying Suspicious Network-Active Processes

Many intrusions involve communication with external systems, whether for command and control, data exfiltration, or lateral movement. Processes that open or listen on unusual network ports, or connect to unrecognized external IP addresses, warrant attention.

Basic tools include ss and netstat that can associate sockets with PIDs. For example:

ss -tupan

The -p option shows the process and PID associated with each socket. You can then pivot from a suspicious port to its owning process.

Focus on connections that are:

Outbound to uncommon or foreign IP addresses and ports, especially if they persist and are owned by processes with names like kworker, syslog, or similarly system-looking names running from strange paths.

Inbound on ports that the system is not supposed to expose, for example, a random high port with LISTEN state managed by an unknown binary.

Maintained by interpreters like python or node under unusual account names or directories, especially if the system is not used for development.

Once you have the PID, you can move back into /proc/<pid> and ps to inspect the executable, command line, parent, and open file descriptors. Open sockets under /proc/<pid>/fd often link to entries like socket:[12345] that correspond to lines in ss output.

On heavily used servers, not all unexpected connections are malicious. However, a network-active process with any of the previously noted anomalies in name, path, or ownership rises quickly in priority for investigation.

Looking at Open Files and Hidden Activity

Attackers frequently try to hide data or tools in unconventional locations, or to keep files open even after deleting them from the filesystem. Linux processes leave traces of these actions through their file descriptors.

You can list all files opened by a specific PID using either ls on /proc/<pid>/fd or tools like lsof. For example:

lsof -p <pid>

This command lists all files and sockets the process currently has open. Pay attention to:

Deleted but still open files. lsof marks these with (deleted) in the output. A large deleted file associated with a suspicious process could contain logs, stolen data, or components of a malware toolset that the attacker tried to hide.

Unusual or hidden file paths. Paths under directories like /tmp/.X11-unix, /.config, or unusual hidden directories under /var that are not part of standard packages are notable.

Unexpected devices. Processes that open block devices, raw disks, or uncommon pseudo-devices can indicate an attempt to bypass filesystem-level logging or encryption.

Because Linux unlinks files immediately when deleted, but keeps them accessible to processes that already have them open, an attacker can try to erase traces while the process still uses the file contents in memory. After you identify such a file, you can sometimes recover its content by copying from /proc/<pid>/fd/<n> before stopping the process.

Static Analysis of the Executable

Once you know the location of a suspicious process’s executable via /proc/<pid>/exe, you should treat that file as potential evidence and subject of static analysis. This analysis aims to determine what the binary or script is, how it may have arrived on the system, and whether it shows known malicious traits.

First, make a copy of the executable to a secure analysis location, ideally read-only from the perspective of the compromised host. Then inspect it with basic tools that do not execute code, such as file and strings:

file suspicious_binary
strings -a suspicious_binary | less

The file command tells you whether it is an ELF binary, a script with a shebang line, or something else. The strings output can reveal URLs, IP addresses, hard-coded commands, encryption keys, or clear-text markers of malware families.

For script-based malware, simply viewing the text content may show readable functions, obfuscated code, or embedded payloads. Exercise caution not to run anything you inspect.

You can also check whether the executable belongs to an installed package. On a system that uses APT, for example:

dpkg -S /path/to/executable

If the package manager does not know the file, but it is stored somewhere that normally contains only packaged binaries, such as /usr/bin or /usr/sbin, this strongly suggests tampering.

More advanced static analysis may use dedicated malware analysis tools or sandboxes, but even the simple checks above can distinguish between misconfigured but benign tools and custom malicious code.

Tracing System Calls and Behavior in Real Time

When incident response rules permit interacting with the live system, tracing a process’s system calls with tools like strace or ltrace can reveal what it is actually doing without necessarily understanding its code.

Attaching strace to a running process shows system calls in progress. For example:

strace -p <pid> -f -o /tmp/strace_<pid>.log

The -f flag follows child processes as they are created, and -o writes output to a file for later review. By examining the trace you can see whether the process is reading or writing particular files, opening network connections, spawning other processes, or performing suspicious operations like changing file permissions or injecting into other processes.

For dynamically linked applications, ltrace can capture library calls, which sometimes makes it easier to understand high-level behavior, such as network or cryptographic operations that are obscured at the system call level.

System call tracing is intrusive to some degree. It can slow down a process, reveal your monitoring to an attacker, and in rare cases contribute to instability. In a time-critical response, you must balance the value of additional insight against these risks.

Handling Malicious or Undesired Processes Safely

After you have enough evidence that a process is malicious or part of an unauthorized activity, you may need to stop it. How you do so depends on the goals of your investigation and containment strategy.

If preserving as much evidence as possible is the priority, first ensure that you have collected:

Command-line and environment from /proc/<pid>

Executable link and a copy of the binary or script

Open files, including any deleted but still open files

Network connections associated with the process

Relevant logs, including system logs and application logs

Only then should you issue a termination signal. Using kill -TERM <pid> requests a graceful shutdown, while kill -KILL <pid> forces immediate termination. Note that some malware may trap or ignore softer signals and may have watchdog processes that restart it. Process trees help you identify these supervisors so you can disable or remove them as well.

In environments where the compromise is severe or you suspect kernel-level tampering, a complete system isolation from the network or even immediate power off for offline forensics may be preferred over selectively killing processes. The decision belongs to your incident response plan, but an understanding of process behavior informs how urgent and broad your containment must be.

Documenting Findings for Incident Response

Process analysis is valuable not only because it lets you remove malicious activity, but also because it provides a narrative of what happened. Each suspicious process you analyze contributes to a timeline and set of hypotheses about the attacker’s goals and methods.

For each process of interest, record at least:

PID, name, and executable path

Parent process, children, and tree position

User, groups, and privileges

Command line and environment highlights

Open files and network connections

Key behaviors observed through logs or tracing

Link these details back to other incident artefacts, such as logs, file integrity alerts, or unusual user activity. When combined with broader evidence collection and a structured incident response workflow, careful analysis of suspicious processes helps you determine initial access, lateral movement, data access, and persistence, and guides your remediation steps.

By treating every suspicious process as both a threat and a source of information, you make your response more effective and your post-incident understanding more complete.