Linux OS - Finding Evil Process
Inspired from the Sans Poster: Find Evil – Know Normal - i.e Knowing what’s normal running process on a Windows OS host helps cut through the noise to quickly locate potential malware.
Refer: https://www.sans.org/security-resources/posters/hunt-evil/35/download
So I thought of implementing the same in the Linux OS side.
Though the windows OS is more targeted by APT other threat groups( as it's mostly used one above 80%).
But in recent years, there was many APT etc groups are targeting the Linux OS platforms as well.
Reference: Intezer's APT chart
https://www.intezer.com/blog/cloud-security/looking-back-on-the-last-decade-of-linux-apt-attacks/
Challenges
Linux has different OS distributionsSuse, OpenSUSE, Debian, Ubuntu, CentOS, Arch, Fedora, RHEL, AWS Linux all are common Linux distribution names.
Some will be have same commands, only kernel variations etc.
Linux OS distribution used in this blog: AWS Linux.
Some will be have same commands, only kernel variations etc.
Linux OS distribution used in this blog: AWS Linux.
Process chart: (For easy understanding)
Notes:
1) [kthreadd] It will have many child process like [migration/0] ..... [crypto] etc with random PID 3-16,30,174-184 etc.
2) Image Path: these kernel thread process will not have any path/elf file associated with them, if there is any path associated then it's a suspicious one, could be a process masquerading attempt.
3) Systemd or init is the parent process for those circular process structure (above screenshot)
Process details & description:
1) Init or Systemd
Image Path: /usr/lib/systemd/systemd --switched-root --system --deserialize 21 (systemd)
Parent Process: None
Number of Instances: 1
Process ID: 1
Parent process ID: 0
Process ID: 1
Parent process ID: 0
Session ID: 1
User Account: root
Start Time: Within few min of system boot
sha256sum: 8dcedc65c32ff5e149343015798c7613254ff1659e133e8a6f51725bdf1afd2e (aws linux)
sha256sum: 8dcedc65c32ff5e149343015798c7613254ff1659e133e8a6f51725bdf1afd2e (aws linux)
Description:
As per wiki: Thus, the kernel initializes devices, mounts the root filesystem specified by the boot loader as read only, and runs Init (/sbin/init) which is designated as the first process run by the system (PID = 1)init is a abbreviation for Initialization. The init is a daemon process which starts as soon as the computer starts and continue running till, it is shutdown. In-fact init is the first process that starts when a computer boots, making it the parent of all other running processes directly or indirectly and hence typically it is assigned “pid=1“.
A systemd is a System Management Daemon named with UNIX convention to add ‘d‘ at the end of daemon. So, that they can be easily recognized. Initially it was released under GNU General Public License, but now the releases are made under GNU Lesser General Public License. Similar to init, systemd is the parent of all other processes directly or indirectly and is the first process that starts at boot hence typically assigned a “pid=1“.
Systemd has many advantages from the init (faster boot, parallel process etc)
A systemd, may refer to all the packages, utilities and libraries around daemon. It was designed to overcome the shortcomings of init. It itself is a background processes which is designed to start processes in parallel, thus reducing the boot time and computational overhead. It has a lot other features as compared to init.
Some legacy features were kept intact (such as init.d to some degree)
2) Kernel thread process - which will have a brackers ["process name"] ex: [kthreadd]
Kthreadd is a worker thread in kernel address space started by the kernel.
A kernel thread is a kernel task running only in kernel mode; it usually has not been created by fork() or clone() system calls.
Image Path: these kernel thread process will not have any path associated with them, if there is any path associated then it's a suspicious one, could be a process masquerading attempt.
One of the way to find the kernel threads is running this command "strings /proc/$pid/cmdline" is empty for kernel threads.
To augment the need for running background operations, the kernel spawns threads (similar to processes). These kernel threads are similar to regular processes, in that they are represented by a task structure and assigned a PID. Unlike user processes, they do not have any address space mapped, and run exclusively in kernel mode, which makes them non-interactive. Various kernel subsystems use kthreads to run periodic and asynchronous operations.
The kthreadd process is created by idle through kernel_thread, and always runs in the kernel space, responsible for the scheduling and management of all kernel threads
[kthreadd] is the parent process for all the kernel threads.
Image Path: N/A
Kthreadd is a worker thread in kernel address space started by the kernel.
A kernel thread is a kernel task running only in kernel mode; it usually has not been created by fork() or clone() system calls.
Image Path: these kernel thread process will not have any path associated with them, if there is any path associated then it's a suspicious one, could be a process masquerading attempt.
One of the way to find the kernel threads is running this command "strings /proc/$pid/cmdline" is empty for kernel threads.
To augment the need for running background operations, the kernel spawns threads (similar to processes). These kernel threads are similar to regular processes, in that they are represented by a task structure and assigned a PID. Unlike user processes, they do not have any address space mapped, and run exclusively in kernel mode, which makes them non-interactive. Various kernel subsystems use kthreads to run periodic and asynchronous operations.
The kthreadd process is created by idle through kernel_thread, and always runs in the kernel space, responsible for the scheduling and management of all kernel threads
[kthreadd] is the parent process for all the kernel threads.
Image Path: N/A
Parent Process: None
Number of Instances: 1
Process ID: 2
Parent process ID: 0
Process ID: 2
Parent process ID: 0
Session ID: 0
User Account: root
Start Time: Within few min of system boot
sha256sum: N/A (because no path/elf associated with this)
Description:
[kthreadd] It will have many child process like [migration/0] etc with random PID 3-16,30,174-184 etc
Ex: [migration/0] is part of the kernel, and is a thread that is responsible for moving processes to another CPU to balance out the workload. [kthreadd] It will have many child process like [migration/0] etc with random PID 3-16,30,174-184 etc
[crypto] - It provides access to the kernel crypto API designed to handle “transformations” of data between “states” (encrypted and unencrypted).It is where cryptographic functions live that are used by kernel modules.Kernel modules are kernel processes that manage your hardware, or filesystems. Most filesystems that use cryptographic functions, use the kernel crypto functions.This is for some security software that lets the Linux kernel manage keys for encryption and decryption, and it will get increasingly relevant as trusted computing evolves and computer hardware can fulfil more and more security functions.
[kworker/0:0] - is a placeholder process for kernel worker threads, which perform most of the actual processing for the kernel, especially in cases where there are interrupts, timers, I/O, etc.
Kernel threads are not children of init or systemd because they can be started before all the userspace processes.
They are typically used to manage hardware that's why they are directly handled by the kernel and have high priority.
For a process to be child of init it needs to be cloned from init and Kthreads aren't that's why their parent PID is 0 meaning "no-one".
Credits:
https://stackoverflow.com/questions/17988526/what-is-kthreadd-process-and-children-and-how-it-is-different-from-init-and-chil
A kernel thread is a kernel entity, like processes and interrupt handlers; it is the entity handled by the system scheduler. A kernel thread runs within a process, but can be referenced by any other thread in the system. The programmer has no direct control over these threads, unless you are writing kernel extensions or device drivers.
3) systemd-journald
Image Path: /usr/lib/systemd/systemd-journald
Parent Process: Systemd
Parent Process: Systemd
Number of Instances: 1
Process ID: subject to change
Parent process ID: 1
Process ID: subject to change
Parent process ID: 1
User Account: root
Start Time: Within few min of system boot
sha256sum: 6040be31c23b5d45ab5ba168499abfb699330b4b3643785578d3e1e23bbcace0 (aws linux)
Description: systemd-journald is a system service that collects and stores logging data
sha256sum: 6040be31c23b5d45ab5ba168499abfb699330b4b3643785578d3e1e23bbcace0 (aws linux)
Description: systemd-journald is a system service that collects and stores logging data
4)lvmetad -f
Image Path: /usr/sbin/lvmetad -f
Parent Process: Systemd
Parent Process: Systemd
Number of Instances: 1
Process ID: subject to change
Parent process ID: 1
Process ID: subject to change
Parent process ID: 1
User Account: root
Start Time: Within few min of system boot
sha256sum: c9cc0b4648bb5ae7b18225a76d20e814ed951080b0b8eefacaab6f8ba3f03735
sha256sum: c9cc0b4648bb5ae7b18225a76d20e814ed951080b0b8eefacaab6f8ba3f03735
Description:
lvmetad is a metadata caching daemon for LVM. The daemon receives notifications from udev rules (which must be installed for LVM to work correctly when lvmetad is in use). Through these notifications, lvmetad has an up-to-date and consistent image of the volume groups available in the system. By default, lvmetad, even if running, is not used by LVM. See lvm.conf(5).5) systemd-udevd
Image Path: /usr/lib/systemd/systemd-udevd
Parent Process: Systemd
Parent Process: Systemd
Number of Instances: 1
Process ID: subject to change
Parent process ID: 1
Process ID: subject to change
Parent process ID: 1
User Account: root
Start Time: Within few min of system boot
sha256sum: 30c86e74ca507f246f532a66a72edf693621ac9b9ad5bca6fc2442d38f90b38a
sha256sum: 30c86e74ca507f246f532a66a72edf693621ac9b9ad5bca6fc2442d38f90b38a
Description:
systemd-udevd listens to kernel uevents. For every event, systemd-udevd executes matching instructions specified in udev rules. The behavior of the daemon can be configured using udev. conf6) auditd
Image Path: /sbin/auditd
Parent Process: Systemd
Parent Process: Systemd
Number of Instances: 1
Process ID: subject to change
Parent process ID: 1
Process ID: subject to change
Parent process ID: 1
User Account: root
Start Time: Within few min of system boot
sha256sum: c8f2e9de73129a5a6ccbfaf1782884e8aac0943dd40443afc73dca5183b3d849
sha256sum: c8f2e9de73129a5a6ccbfaf1782884e8aac0943dd40443afc73dca5183b3d849
Description:
auditd is the userspace component to the Linux Auditing System.
It's responsible for writing audit records to the disk. Viewing
the logs is done with the ausearch or aureport utilities.
Configuring the audit system or loading rules is done with the
auditctl utility. During startup, the rules in
/etc/audit/audit.rules
The Linux Auditing System provides kernel-resident logging of system calls and user space tools to collect and view the logs. The auditd daemon writes the logging records to disk. auditd is configurable to allow control over what information is written to the logs.
7) lsmd -d
Image Path: /usr/bin/lsmd -d
Parent Process: Systemd
Parent Process: Systemd
Number of Instances: 1
Process ID: subject to change
Parent process ID: 1
Process ID: subject to change
Parent process ID: 1
User Account: libstor+
Start Time: Within few min of system boot
sha256sum: 58a1eb48c1cdd5c31d448d5933eaf088642b05d7e605cdcf3c8b771a68c8be75
sha256sum: 58a1eb48c1cdd5c31d448d5933eaf088642b05d7e605cdcf3c8b771a68c8be75
Description:
The daemon (lsmd) of libstoragemgmt require to check existence of
'libstoragemgmt' user and group before droping privilege to it.
The libStorageMgmt plugin daemon (lsmd) will read the lsmd.conf file in the folder defined via --confdir (default is /etc/lsm/).A lightweight service management daemon for linuxi based on System V Init.Features:
Managing services by CLI.
Monitoring and do define action by service status.
Grouping service by type (system, applications, network, user, database).
Possibility of communication between servers (basic service HA feature) - sending status of service and making scenario.
Creating instances of services eg. you can create a several instances of httpd for several users.
Allow user to create services and instances of application (branch in /applications) with user privileges (running services as user).
8) dbus
Image Path: /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation
Parent Process: Systemd
Number of Instances: 1
Process ID: subject to change
Parent process ID: 1
Process ID: subject to change
Parent process ID: 1
User Account: dbus
Start Time: Within few min of system boot
sha256sum: 2928d0edf4abcc73cec7468932e7a3647a6153e4842fa2e7a136d00616920e6b
sha256sum: 2928d0edf4abcc73cec7468932e7a3647a6153e4842fa2e7a136d00616920e6b
Description:
DBus system usually has two buses: a system bus and a session bus.
Session bus is started per user (in your case for root and xralf), lines 3 to 6. Line 2 is a dbus service that was requested by your window manager.
A system bus is needed for system-wide message exchange.
dbus-daemon is the D-Bus message bus daemon. See http://www.freedesktop.org/software/dbus/ for more information about the big picture. D-Bus is first a library that provides one-to-one communication between any two applications; dbus-daemon is an application that uses this library to implement a message bus daemon. Multiple programs connect to the message bus daemon and can exchange messages with one another.
There are two standard message bus instances: the systemwide message bus (installed on many systems as the "messagebus" init service) and the per-user-login-session message bus (started each time a user logs in). dbus-daemon is used for both of these instances, but with a different configuration file.
9) systemd-logind
Image Path: /usr/lib/systemd/systemd-logind
Parent Process: Systemd
Number of Instances: 1
Process ID: subject to change
Parent process ID: 1
Process ID: subject to change
Parent process ID: 1
User Account: root
Start Time: Within few min of system boot
sha256sum: f479984b2276bff417692e059228a39a0d9ac50b345916f6195b487d117f2504
sha256sum: f479984b2276bff417692e059228a39a0d9ac50b345916f6195b487d117f2504
Description:
systemd-logind is a system service that manages user logins. It is responsible for:
• Keeping track of users and sessions, their processes and their idle state
• Providing PolicyKit-based access for users to operations such as system shutdown or sleep
• Implementing a shutdown/sleep inhibition logic for applications
• Handling of power/sleep hardware keys
• Multi-seat management
• Session switch management
• Device access management for users
• Automatic spawning of text logins (gettys) on virtual console activation and user runtime directory management
10) rngd
Image Path: /sbin/rngd -f --fill-watermark=0 --exclude=jitter
Parent Process: systemd
Number of Instances: 1
Process ID: subject to change
Parent process ID: 1
Process ID: subject to change
Parent process ID: 1
User Account: rngd
Start Time: Within few min of system boot
sha256sum: 6b3226f920a4059d8aa1d83ca4fa41b22374ea9f0949f3395984f8d0f5334b28
sha256sum: 6b3226f920a4059d8aa1d83ca4fa41b22374ea9f0949f3395984f8d0f5334b28
Description:
This daemon feeds data from a random number generator to the kernel's random number entropy pool, after first checking the data to ensure that it is properly random.
11) gssproxy
11) gssproxy
Image Path: /usr/sbin/gssproxy -D
Parent Process: systemd
Number of Instances: 1
Process ID: subject to change
Parent process ID: 1
Process ID: subject to change
Parent process ID: 1
User Account: root
Start Time: Within few min of system boot
sha256sum: e2db3d1bf22fdfbe63f2d78e31321710871af11edc685ede33bdcb35904acf8a
sha256sum: e2db3d1bf22fdfbe63f2d78e31321710871af11edc685ede33bdcb35904acf8a
Description:
12) chronyd
Image Path: /usr/sbin/chronyd
Parent Process: systemd
Number of Instances: 1
Process ID:subject to change
Parent process ID: 1
Process ID:subject to change
Parent process ID: 1
User Account: chrony
Start Time: Within few min of system boot
sha256sum: 9ee5d51bd7b1ef35159ddb35947e9244e3682a79c16a0231d96b0d03af0e4565
sha256sum: 9ee5d51bd7b1ef35159ddb35947e9244e3682a79c16a0231d96b0d03af0e4565
Description:
Chrony is an implementation of the Network Time Protocol (NTP). You can use Chrony:
to synchronize the system clock with NTP servers,
to synchronize the system clock with a reference clock, for example a GPS receiver,
to synchronize the system clock with a manual time input,
as an NTPv4(RFC 5905) server or peer to provide a time service to other computers in the network.
13) dhclient
Image Path:/sbin/dhclient -q -lf /var/lib/dhclient/dhclient--eth0.lease -pf /var/run/dhclient-eth0.pid eth0
& sbin/dhclient -6 -nw -lf /var/lib/dhclient/dhclient6--eth0.lease -pf /var/run/dhclient6-eth0.pid eth0
Parent Process: systemd
Parent Process: systemd
Number of Instances: 1
Process ID: subject to change
Parent process ID: 1
Process ID: subject to change
Parent process ID: 1
User Account: root
Start Time: Within few min of system boot
sha256sum: a73d6a147a85b0de2100863523f1c2af73a0b4fc834cdf0eea24c8e0a21bf0b9
sha256sum: a73d6a147a85b0de2100863523f1c2af73a0b4fc834cdf0eea24c8e0a21bf0b9
Description:
14) postfix master
Image Path: /usr/libexec/postfix/master -w
Parent Process: systemd
Number of Instances: 1
Process ID: subject to change
Parent process ID: 1
Process ID: subject to change
Parent process ID: 1
User Account:postfix
Start Time: Within few min of system boot
sha256sum: 8211b07ffcecf987151fcfe8fee1c7d94683197e1d07c9b4170a199442c02e6b
sha256sum: 8211b07ffcecf987151fcfe8fee1c7d94683197e1d07c9b4170a199442c02e6b
Description:
15) postfix pickup
Image Path: pickup -l -t unix -u
Parent Process: systemd
Number of Instances: 1
Process ID: subject to change
Parent process ID: postfix master
Process ID: subject to change
Parent process ID: postfix master
User Account: postfix
Start Time: Within few min of system boot
sha256sum: a0095760f143f767787b5052c21e8924ed8651b0e0416202427e6847956336b6
sha256sum: a0095760f143f767787b5052c21e8924ed8651b0e0416202427e6847956336b6
Description:
16) postfix qmgr
Image Path: qmgr -l -t unix -u
Parent Process: systemd
Number of Instances: 1
Process ID: subject to change
Parent process ID: postfix master
Process ID: subject to change
Parent process ID: postfix master
User Account: postfix
Start Time: Within few min of system boot
sha256sum: adf673f2077e6c850b9f0d922a331048625f21868ebab8d17cb6ce4727271342
sha256sum: adf673f2077e6c850b9f0d922a331048625f21868ebab8d17cb6ce4727271342
Description:
17) rsyslogd
Image Path: /usr/sbin/rsyslogd -n
Parent Process: systemd
Number of Instances: 1
Process ID: subject to change
Parent process ID: 1
Process ID: subject to change
Parent process ID: 1
User Account: root
Start Time: Within few min of system boot
sha256sum: 27cee57613fa7b3a9e6260e3f793c7a0141289c72ffa4acff2c57898155df2f8
sha256sum: 27cee57613fa7b3a9e6260e3f793c7a0141289c72ffa4acff2c57898155df2f8
Description:
Rsyslogd is a system utility providing support for message logging. Support of both internet and unix domain sockets enables this utility to support both local and remote logging.
The main rsyslog configuration file is located at /etc/rsyslog. conf, which loads modules, defines the global directives, contains rules for processing log messages and it also includes all config files in /etc/rsyslog.
18)AWS SSM Agent
Image Path: /usr/bin/amazon-ssm-agent
Parent Process: systemd
Number of Instances: 1
Process ID: subject to change
Parent process ID: 1
Process ID: subject to change
Parent process ID: 1
User Account: root
Start Time: Within few min of system boot
sha256sum: 975084132b5e5bb8a7a033be585a7f65d1ac04ddbf9c274c9b57b1c5d3bbf55c
sha256sum: 975084132b5e5bb8a7a033be585a7f65d1ac04ddbf9c274c9b57b1c5d3bbf55c
Description:
19) atd
Image Path:/usr/sbin/atd -f
Parent Process: systemd
Number of Instances: 1
Process ID: subject to change
Parent process ID: 1
Process ID: subject to change
Parent process ID: 1
User Account: root
Start Time: Within few min of system boot
sha256sum: d065a34ef6b931de8c77a086fcd713136a36a88bd1705e0567a905aca25132ab
sha256sum: d065a34ef6b931de8c77a086fcd713136a36a88bd1705e0567a905aca25132ab
Description:
20)crond -n
Image Path: /usr/sbin/crond -n
Parent Process: systemd
Number of Instances: 1
Process ID: subject to change
Parent process ID: 1
Process ID: subject to change
Parent process ID: 1
User Account: root
Start Time: Within few min of system boot
sha256sum: e9aa913ca46ce5ea058f618ef420422af2862ca81d53d8c88407a26dce4667b0
sha256sum: e9aa913ca46ce5ea058f618ef420422af2862ca81d53d8c88407a26dce4667b0
Description:
21) agetty
Image Path: /sbin/agetty --noclear tty1 linux & /sbin/agetty --keep-baud 115200,38400,9600 ttyS0 vt220
Parent Process: systemd
Number of Instances: 1
Process ID: subject to change
Process ID: subject to change
TTY: tty1 & ttys0
Parent process ID: 1
Parent process ID: 1
User Account: root
Start Time: Within few min of system boot
sha256sum: b74d9ba6a01c3dd2c81960c523f040fc83eaf31605d9f41144a87da72ef37daf
sha256sum: b74d9ba6a01c3dd2c81960c523f040fc83eaf31605d9f41144a87da72ef37daf
Description:
22) AWS ssm-agent-worker
Image Path: /usr/bin/ssm-agent-worker
Parent Process: amazon-ssm-agent
Number of Instances: 1
Process ID: subject to change
Parent process ID: amazon-ssm-agent
Process ID: subject to change
Parent process ID: amazon-ssm-agent
User Account: root
Start Time: Within few min of system boot
sha256sum: 0d490a6b580d86615c5ea7a126c41a396efa34e8b18762a6fcf820c4cf6f4a7b
sha256sum: 0d490a6b580d86615c5ea7a126c41a396efa34e8b18762a6fcf820c4cf6f4a7b
Description:
23) /usr/bin/ssm-session-worker
Image Path: /usr/bin/ssm-session-worker
Parent Process: /usr/bin/ssm-agent-worker
Number of Instances: multiple based on the logged ssm session
Process ID: subject to change
Parent process ID: /usr/bin/ssm-session-worker
Process ID: subject to change
Parent process ID: /usr/bin/ssm-session-worker
User Account: root
Start Time: Within few min of system boot
sha256sum: 391a14827a342cf4b991294b21c80b1d7cc71dc825ba9daf5bfb58949aa7a510
sha256sum: 391a14827a342cf4b991294b21c80b1d7cc71dc825ba9daf5bfb58949aa7a510
Description:
Each /usr/bin/ssm-session-worker will have a child process called sh, TTY: pts/0
24) sshd -D
Image Path:/usr/sbin/sshd -D
Parent Process: systemd
Number of Instances: A new process is created for each new SSH session.
Process ID: Subject to change
Parent process ID: 1
Process ID: Subject to change
Parent process ID: 1
User Account: root
Start Time: Within few min of system boot
sha256sum: d76838f8e55510c49761612618e89b0cecbeeec54863450afff15aa440bb234
sha256sum: d76838f8e55510c49761612618e89b0cecbeeec54863450afff15aa440bb234
Description:
sshd is the OpenSSH server process. It listens to incoming connections using the SSH protocol and acts as the server for the protocol. It handles user authentication, encryption, terminal connections, file transfers, and tunneling
It is also the parent process of all the other sshd processes
The other sshd processes are child processes that serve a single connection. A new process is created for each new SSH session.
If the SSH server is upgraded or restarted, only the master server is generally restarte
On most systems, syslog is configured to log SSH-related messages by default into files under /var/log/. On Debian-derived systems, the default log file is usually /var/log/auth.log. On Red Hat derived systems, the default log file is usually /var/log/secure.
On most systems, syslog is configured to log SSH-related messages by default into files under /var/log/. On Debian-derived systems, the default log file is usually /var/log/auth.log. On Red Hat derived systems, the default log file is usually /var/log/secure.
Finding Evil Tips:
1)These kernel thread process ( [" "]) will not have any path, binaries associated with them and PPID will be zero, if there is any path,binaries associated then it's a suspicious one, could be a process masquerading attempt, ex:
threat actor named Rocke has used this technique.
Refer: Craig H Rowland's blog on detecting process masquerading
https://www.sandflysecurity.com/blog/detecting-linux-kernel-process-masquerading-with-command-line-forensics/
Refer: Craig H Rowland's blog on detecting process masquerading
https://www.sandflysecurity.com/blog/detecting-linux-kernel-process-masquerading-with-command-line-forensics/
https://redcanary.com/blog/rocke-cryptominer/
2) Attackers would name the process in a way to camouflage or blend in plain sight.
Always check commandline, running from unusual paths like /tmp etc, arguments make sense, is the parent process expected on, processes started near time of attack, process name spelled incorrectly, whether it was started by the expected user, unknown process consuming high CPU resource(could be a miner), unsual network connection (netstat) and also check process memory map ( This location is normally where processes show libraries they are linking to and where they mapped to in memory)
About /proc directory, why it's useful?
procfs is a virtual file system ( It doesn’t really exist) which provides access to kernel structures. procfs provides information about processes, networking, devices, processors, memory, and other system statistics.
It will be really useful for forensics and incident-response.
Note: When your taking full disk image, the proc directory will not be copied, as mentioned it's a virtual file system.
Some of the useful proc commands for DFIR:
i)Process directory listing
cd /proc/$pid/ and do ls -al
ii)Process cmdline Parameters
cd /proc/$pid/ then do
cat comm - exposes the process's comm value—that is, the command name associated with the process
cat comm - exposes the process's comm value—that is, the command name associated with the process
strings cmdline - holds the complete command line for the process, unless the process is a zombie
iii)Process path:
ls -al /proc/<PID>/exe
iv)Process Maps - This can be used to see which libraries a process is using.
cat /proc/<PID>/maps.
v)Process Environment variables
strings /proc/<PID>/environ
vi)File descriptors(Similar to lsof)- see what files the process has opened/handle
ls -al /proc/<PID>/fd
vii)Status of the process
/proc/PID/stat and /proc/PID/status
viii)I/O statistics for the process,
cat /proc/PID/io
xi)Netstat with PID info
netstat -plant
xiv)Process hash (sha256sum)
sha256sum /proc/$pid/exe
3) Fileless attacks are increasing -, injecting malicious payloads into benign system processes, using GTFOBins (lolbins) legitimate binaries for process execution, hide in memory and hide process using rootkit techniques
- We can use volatility to detect the hidden process, injected process (will create a separate blog).
Refer:
https://gtfobins.github.io/
https://www.sandflysecurity.com/blog/detecting-linux-memfd_create-fileless-malware-with-command-line-forensics/
4) Always check the hash of the malicious/running process with the Virustotal as well. I know that Malwares can evade AV/EDRs/Sandbox as well. So in addition to the VT, Sandbox, It's always better to do a malware analysis of the ELF executables, as well.(will create a separate blog on the ELF, Malware analysis etc).
5) Check if the process is packed or not. Some malware for Linux is packed or encrypted and shows very high entropy.
Refer: https://www.sandflysecurity.com/blog/sandfly-filescan-open-source-file-entropy-scanner-for-linux/
6) Sometimes malware will delete itself after it starts, check for the deleted binaries that are still running.
v)Process Environment variables
strings /proc/<PID>/environ
vi)File descriptors(Similar to lsof)- see what files the process has opened/handle
ls -al /proc/<PID>/fd
vii)Status of the process
/proc/PID/stat and /proc/PID/status
viii)I/O statistics for the process,
cat /proc/PID/io
ix) This is a symbolic link to the process. you can copy these if the file has been deleted.
cp /proc/PID/exe /tmp/test
x)Current working directory:
ls -alR /proc/PID/cwd
ls -alR /proc/PID/cwd
xi)Netstat with PID info
netstat -plant
xii)Top usage of CPU/RAM:
top
xiii)Process tree
ps -auxwf
xiv)Process hash (sha256sum)
sha256sum /proc/$pid/exe
3) Fileless attacks are increasing -, injecting malicious payloads into benign system processes, using GTFOBins (lolbins) legitimate binaries for process execution, hide in memory and hide process using rootkit techniques
- We can use volatility to detect the hidden process, injected process (will create a separate blog).
Refer:
https://gtfobins.github.io/
https://www.sandflysecurity.com/blog/detecting-linux-memfd_create-fileless-malware-with-command-line-forensics/
4) Always check the hash of the malicious/running process with the Virustotal as well. I know that Malwares can evade AV/EDRs/Sandbox as well. So in addition to the VT, Sandbox, It's always better to do a malware analysis of the ELF executables, as well.(will create a separate blog on the ELF, Malware analysis etc).
5) Check if the process is packed or not. Some malware for Linux is packed or encrypted and shows very high entropy.
Refer: https://www.sandflysecurity.com/blog/sandfly-filescan-open-source-file-entropy-scanner-for-linux/
6) Sometimes malware will delete itself after it starts, check for the deleted binaries that are still running.
using : ls -alR /proc/*/exe 2> /dev/null | grep deleted ( watch out for False positives)
Please comment your thoughts or if I missed anything etc.
Special thanks to Craig H Rowland for the support.
