Round 2: Hands-On¶

sudo useradd -r -s /usr/sbin/nologin appsvc      # no login shell
sudo -u appsvc whoami 2>&1 || true               # confirm it cannot log in
echo 'alice ALL=(appsvc) NOPASSWD: ALL' | sudo tee /etc/sudoers.d/alice-appsvc
sudo visudo -cf /etc/sudoers.d/alice-appsvc       # validate before trusting it

A system account (-r) with a nologin shell cannot start a session, and the sudoers drop-in lets alice run commands as it. Always validate a sudoers file with visudo -c, covered in Sudo and su.

sudo groupadd -f devs
sudo install -d -g devs -m 2775 /srv/shared        # 2 = setgid
sudo setfacl -d -m g:devs:rwx /srv/shared          # default ACL for new files
getfacl /srv/shared | grep default

The setgid bit makes new files take the directory's group, and the default ACL sets their permissions, covered in Special Permissions and ACL.

sudo systemctl edit myapp        # add: [Service] \n Restart=on-failure \n RestartSec=5
sudo systemctl daemon-reload
sudo systemctl enable --now myapp
systemctl show myapp -p Restart -p RestartSec

The drop-in adds the restart policy without editing the shipped unit, covered in Unit Files.

sudo fallocate -l 2G /swapfile && sudo chmod 600 /swapfile
sudo mkswap /swapfile && sudo swapon /swapfile
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab
swapon --show

The /etc/fstab line re-enables it on boot, covered in Swap.

sudo firewall-cmd --add-port=8080/tcp --permanent
sudo firewall-cmd --reload
sudo firewall-cmd --list-ports

--permanent writes the rule; --reload applies it to the running firewall, covered in Firewalld and UFW.

podman volume create appdata
podman run -d --name app --memory 256m -v appdata:/data myimage
podman inspect app --format '{{.HostConfig.Memory}}'

The volume outlives the container; --memory sets the cgroup memory.max, covered in Podman and Quadlet.

CPU is fully saturated in user space (100.0 us, 0.0 wa, 0.0 st), and two stress-ng processes are each pinning a core. This is CPU-bound work, not I/O or memory: memory is healthy and there is no iowait. The fix is on the process, not the host, covered in CPU and Load.

The host is out of usable memory: available is 10Mi, and buff/cache has already been squeezed to 43Mi, so there is nothing left to reclaim. The next allocation risks the OOM killer. Judge by available, not free, covered in Memory.

The run queue r is 3 to 4 on a two-CPU host with us at 100 and id at 0, so more threads are runnable than there are cores: CPU saturation. Swap is idle (si/so 0), ruling out memory paging. More cores or less concurrency is the lever.

/mnt/small is 95% full with 1 MB free, so writes there will soon fail while the root filesystem is fine. If writes fail but df shows space, check df -i for inode exhaustion instead, covered in Disk Usage and Disk Full.

Nothing on the disk: despite 23 MB/s of writes, %util is 7.57 and w_await is 2.64 ms, so the device is keeping up with room to spare. A slow application here is not disk-bound; look at CPU, locks or the network. A saturated disk instead shows %util near 100 with a high aqu-sz and await, covered in Disk I/O.

The host is an NFS server: rpc.mountd and rpcbind are listening, and chronyd binds 323 on loopback only. A service reachable locally but not remotely often binds 127.0.0.1 instead of 0.0.0.0; check the Local Address column, covered in Ports and Sockets and Service Unreachable.

PID 1592 is a zombie (Z): it has exited but its parent (1590) has not reaped it with wait. A few zombies are harmless; many mean a buggy parent, and the fix is to signal or restart the parent, not the zombie. Killing a zombie does nothing because it is already dead, covered in Process States.