While Log4J or Log4Shell stole a lot of the headlines, there have been a lot of other attacks on Kubernetes and containerized environments recently. Our threat research team, Unit 42, has been hard at work identifying new vulnerabilities, attack vectors and active malware. When aggregating these various attacks, there are a few common techniques bad actors use to gain access to a system, exploit resources, or abuse resources and spread.
Let’s look at a few common patterns and discuss how you can protect against them.
The most common technique for bad actors to get in through the front door is exploiting vulnerable components. What’s new is recent attacks targeting Kubernetes components like public kubelets. Attackers scan the internet for any public facing vulnerabilities and unsecure components to exploit. From there, it’s trivial to perform an exploit and get into a system.
Vulnerability management tools will identify all known vulnerabilities in base images and packages and provide upgrade recommendations. When vulnerabilities can’t be patched or there is no patch available, providing virtual patching and other runtime protection can be useful compensating controls.
For Kubernetes components, this is another reason to consider managed Kubernetes offerings, rather than rolling your own. All the major cloud providers’ managed Kubernetes offerings lock down the kubelet component by default and are not susceptible to this exploit. For those self-managing Kubernetes clusters, tools like Prisma Cloud can identify unsecure components to secure using our Kubernetes audits. Integrations with Open Policy Agent (OPA) can also prevent spinning up privileged containers and other violations of secure Kubernetes practices.
Many of the attacks spotted by Unit 42 don’t stop once they’re in. Malware like Hildegard and Black-T immediately begin searching for credentials, such as cloud access keys, access tokens, SSH keys, Kubernetes service tokens, etc. TeamTNT currently searches for 16 different credentials using pattern matching techniques. These keys can be used for lateral movement (discussed later), privilege escalation, spinning up a privileged container or spinning up VMs and containers with cryptominers.
The best way to protect against these kinds of exploits is to make it as difficult as possible to gain access to credentials and to lock down credentials as much as possible. Tools like Prisma Cloud and Bridgecrew can search container images, infrastructure as code, and cloud resources for clear text secrets so they can be revoked and replaced as soon as possible. Using our CIEM capabilities allow you to identify overprivileged cloud credentials and move to a least privileged model.
These malware variants are also scraping memory, so it’s not possible to completely protect against them once they are inside. In that case, the best protection is making sure they don’t get access in the first place. Also, using Prisma Cloud in combination with secrets managers like Vault lets you only inject secrets when absolutely necessary, limiting the number of times a secret shows up in memory.
While the popularity of cryptojacking has declined somewhat, Unit 42 still found evidence of cryptojacking at 17% of organizations. Crypojacking is the process of either spinning up a cryptominer in an infected container or virtual machine (VM) or spinning up a dedicated container or VM with cryptomining software. Another form of cryptojacking is a supply chain poisoning attack where public container images, such as ones stored in Docker Hub, contain cryptomining software. These images are disguised as something useful, such as a popular benign image, and often do function as intended, just with a side process that mines cryptocurrency for bad actors.
Cryptomining processes can be stopped in a few places. When reaching out to command-and-control servers, network monitoring can block traffic to known malicious IPs and domains. Workload protection platforms can block known cryptominers from ever being spun up and trace the events before and after the attempt. If a container or VM attempts to download a file or spin up a container image, security tools can identify the signatures of known cryptomining malware or send it to a sandbox to identify unknown malware. Additionally, these tools can lock down deployments to only known, trusted images and block all other images.
Malware is rarely satisfied with the first entry point. Using port scanning tools, such as masscan, pnscan and zgrab, and CLI dumps, malware can find the next target to infect and spread to. From there, they can open up access to other containers and VMs using service control software like SSH or API software such as Kubernetes API commands.
Blocking lateral movement involves blocking each step of the attempt. Start by limiting the attack surface by using microsegmentation to limit the services that can talk to each other. Leverage cloud workload protection tools to block known port scanning tools. Kubernetes allows you to predefine service communication, so there is unlikely a need for port scanning from a container. Then block SSH and RDP access to just known users and processes and lock down your Kubernetes and Docker APIs.
One of the popular obfuscation techniques is to run memory only malware and to clear log history. This makes it hard for volume-based scanning to detect and prevent these attacks. The malware is only included temporarily, sometimes in the
tmp directory or as a script in memory only, and then wiped clean. This makes forensics without active monitoring nearly impossible.
The only protection against these attacks is active monitoring. Agents can detect and block malicious processes and file modifications at the source. For most Kubernetes environments, preventing shell access is a good practice and effective measure against some attacks and clearing history. Commands like
history -c can be blocked using custom rules to keep an immutable history of actions in logs.
While not new, we’ve noticed that the malware we’re monitoring is killing off competing malware. For example, the Black-T malware variant greps for other worms and cryptominers and runs the kill command. Disabling other worms frees up more resources for the bad actor to use.
The first question is, do you want to stop malware from stopping other malware?
All joking aside, the variants we’ve seen typically use the
kill command. You can take the approach of blocking shell commands in containers. Traditionally all commands should come through code updates and the API controllers should update configurations and live code on your behalf, so manual shell access shouldn’t be necessary. Another, more fine-grained approach is to disable services from using
Threat actors are growing in sophistication and Kubernetes is a prime target. However, there are tools out there to block every step of their attacks. This multi-layered approach to blocking attacks is the most effective approach to runtime security in the cloud.
Try Prisma Cloud’s Kubernetes security capabilities, such as runtime security, vulnerability management, compliance, image analysis sandbox, and more, by requesting a free trial here.