David Balaban is a computer security researcher with over 17 years of experience in malware analysis and antivirus software evaluation. David runs MacSecurity.net and Privacy-PC.com projects that present expert opinions on contemporary information security matters, including social engineering, malware, penetration testing, threat intelligence, online privacy, and white hat hacking. David has a strong malware troubleshooting background, with a recent focus on ransomware countermeasures.

Container technology revolutionized the deployment of applications in enterprise ecosystems. It smooths the edges of the runtime environment by bundling code, its configuration files, libraries, and dependencies into a single package that runs seamlessly across different operating systems and the underlying infrastructures. To take the leap, organizations need a solution that helps deploy, scale, and manage containerized applications in the cloud. The open-source system called Kubernetes bridges the gap.

According to a recent survey, a whopping 88% of companies perform container orchestration using Kubernetes. Viewed from the angle of cyber crime, this statistic means a huge potential attack surface. The common exploitation scenarios include data theft, coin mining, and distributed denial-of-service (DDoS) assaults piggybacking on elements of a compromised Kubernetes cluster, such as the control plane, worker nodes, and pods.

In August 2021, the U.S. National Security Agency (NSA) and Cybersecurity and Infrastructure Security Agency (CISA) released a one-stop set of recommendations to help organizations enhance the security of their Kubernetes systems. The guidance outlines the relevant threat model while providing comprehensive tips to safeguard different areas of container orchestration practices.

Key threats to Kubernetes

The risks can occur both outside and within the organization, with the following three abuse vectors dominating this threat landscape. Although network admins cannot forestall all of them, they can step up the defenses of a Kubernetes cluster by identifying and fixing misconfigurations along with known vulnerabilities.

Supply chain attacks. This means undermining the security of a specific component that constitutes a Kubernetes cluster and plays a role in supplying the end product. The common examples of the target elements include third-party software, containers, and the infrastructure hosting these systems.

Malicious actors. Kubernetes architecture leverages a series of application programming interfaces (APIs) that adversaries may try to exploit to gain remote access to containerized environments. The control plane, worker nodes, and applications that run within a cluster are on the receiving end of most such incursions. Therefore, these entities need to have well-thought-out access controls in place.

Insider threats. The privileges of an organization’s IT personnel, regular users, or representatives of an infrastructure-as-a-service (IaaS) provider are a double-edged sword. While being required for proper operation and maintenance of the Kubernetes system, they can be a source for surreptitious foul play on the inside.

Hardening Kubernetes protection

The NSA and CISA emphasize that organizations need to prioritize the protection of five areas of their Kubernetes setups. Here is a summary of these recommendations.

Kubernetes pod security

Consisting of one or more containers, a pod is the smallest deployable building block of any Kubernetes architecture. After compromising a container, a threat actor will try to mishandle the corresponding pod to gain a foothold in the higher-level components of the organization’s digital infrastructure.

To thwart privileged execution, administrators should use non-root containers. This will limit the permissions of applications that do not require such a level of access to do their thing. So-called rootless container engines can streamline this routine. Keep in mind, though, that this technology has yet to mature.

Leveraging containers with immutable file systems is another layer of protection. This will prevent an intruder from creating malicious files, downloading arbitrary scripts, and changing an application’s code. When it comes to protecting container images, it is best to ensure their isolation by means of hypervisors and scan them for security loopholes as well as configuration slip-ups.

Furthermore, the use of Pod Security Policies (PSPs) allows IT professionals to pull the plug on commonly exploited container features (e.g., hostPID and allowedHostPath), avoid privilege redundancy, and block containers that attempt to run as a root user.

Fortifying the network

Containers, pods, and services actively interact with one another inside a cluster. If an attacker gains access to one of them and there are no networking restrictions in place, they can move laterally across the environment and thereby compromise other entities. To steer clear of this situation, organizations should make the most of firewalls and network policies to isolate resources. Another tip is to secure web traffic and business-critical data at rest through encryption.

The Kubernetes namespaces functionality is incredibly effective when it comes to segregating cluster resources among employees, departments, and applications. Essentially, it allows administrators to form sub-clusters with different access requirements within the virtual infrastructure. The three native namespaces – “kube-system,” “kube-public,” and “default” – apply to Kubernetes components, public resources, and user resources, respectively.

Well-thought-out network policies can add an extra layer of security to communication between cluster elements. They allow a company to manage the traffic between pods, namespaces, and IP addresses outside the Kubernetes territory. This technique blocks all connections not specified in the relevant policy. Bear in mind, though, that you need a plugin that supports the NetworkPolicy API to create new policies.

It is also recommended to raise the bar for accessing the “etcd” server that stores cluster Secrets and other sensitive data. Admins need to ascertain that it communicates with API servers via HTTPS protocol.

Scanning container images

Continuous image scanning is one of the main prerequisites for proactive protection of a Kubernetes setup. This is a sure-shot way to identify malware, weak or compromised credentials, and vulnerabilities in software that runs on an image. With proper alerting features in place, security teams can apply patches or take other appropriate action before threat actors get a chance to parasitize these gaps.

Ideally, an image scanner should be able to pinpoint both zero-day loopholes and known vulnerabilities, automatically halting the development pipeline and providing an effective remediation walkthrough whenever a problem is detected. It is worth emphasizing that this routine must be in effect throughout the image life cycle, from the moment it is created all the way to runtime deployment. The use of a static application security testing (SAST) solution can help discover security issues in the source code of your containerized software more accurately.

Proper access controls

If a cluster lacks effective user verification mechanisms, malefactors can run a scan to identify unsecured Kubernetes ports and make API calls or access sensitive data without having to sign in. Setting up reliable authentication can avert this scenario. However, this workflow isn’t equally frictionless for Kubernetes service accounts and regular user accounts.

Authentication for service accounts is implemented automatically via so-called bearer tokens. The caveat is that these tokens may reside in well-known locations within a pod, which means a competent attacker can get hold of them to infiltrate the network. Therefore, it is wise to narrow down the range of individuals who can access pod Secrets.

There is no automatic authentication for admin accounts and regular users. Things like client certificates or special plugins and protocols are required to add it. IT personnel should avoid storing static password files or following similarly weak authentication practices; otherwise, adversaries may log in as legitimate users without much effort.

The use of role-based access control (RBAC) adds flexibility to the mix. It allows administrators to specify the degree of access to cluster resources for individuals based on their previously assigned roles in the organization. The two types of permissions are Roles and ClusterRoles, with the former applying to particular namespaces and the latter working across all contents of a cluster. RBAC is a default feature in Kubernetes v1.6 and later.

Authorization must be aligned with the principle of least privilege. It means that users should only have access to namespaces that host the required Kubernetes resources. This approach minimizes the damage and prevents lateral movement if a threat actor takes over a specific employee’s account.

Log auditing

Not only do logs help you to ascertain that Kubernetes services are working smoothly, but they also provide actionable insights into the security status of the system. This data can speak volumes about unauthorized misconfigurations and other forms of abnormal activity, which is a call to action for the organization’s IT teams.

Whereas Kubernetes can keep track of cluster actions as well as basic maintenance information such as the CPU and memory consumption details, it does not come with advanced monitoring and alerting mechanisms. To fill the void, admins can leverage Security Information and Event Management (SIEM) software. It will additionally monitor the API request history, network traffic, operating system calls, pod scaling, permission changes, and deployments.

Runtime protection

Container deployment and real-time security monitoring should go hand in hand. The objective of this approach is to instantly detect issues that pose risk to your workloads. These perils run the gamut from malware in container images to privilege escalation attempts fueled by access control misconfigurations or software vulnerabilities.

In Kubernetes, runtime security is a fusion of signature-based analysis and the detection of anomalies in workload profiles. The former boils down to capturing all kernel system calls and comparing them against known hallmarks of malicious activity. The latter hinges on an engine that creates a profile of workloads within a protected cluster and then keeps tabs on discrepancies from this baseline profile.

Whereas Kubernetes comes with a decent event logging functionality, it doesn’t analyze the collected data and lacks alerting features of its own. The good news is that the protection logic of any automatic container security tool worth its salt combines the above-mentioned two mechanisms to foil runtime threats.

Upgrading and application security practices

Since security is a process rather than a plug-and-play product, it is imperative to keep all components of the Kubernetes infrastructure up to date. These include plugins, hypervisors, virtualization tools, the operating system that hosts the environment, and applications deployed on servers.

In addition to bringing new features and performance improvements, updates patch the latest vulnerabilities that expose the system to compromise. If any old components are no longer needed, organizations should remove them to reduce the number of entry points for a potential attacker.

Penetration testing is another area of protection that helps pinpoint insecure configurations, zero-days, and other weak links in the Kubernetes setup. This simulation of cyber crooks’ activity can be very informative, allowing companies to remediate the discovered security loopholes proactively.

Summary

With numerous incredibly useful capabilities under its hood, Kubernetes is here to stay. The flip side of its growing popularity among enterprises is that cyber criminals will keep ramping up their efforts to infect these systems with malware, steal data, quietly chew up the processing power to mine crypto currencies, or parasitize networks to execute DDoS attacks.

To be a moving target, any organization that uses Kubernetes must adhere to the above recommendations. An important remark is that corporate IT professionals’ focus should go beyond external risks. The insider threat is an escalating problem, and therefore the above-mentioned event log monitoring is an inalienable part of the protection puzzle.

Admins still need to make sure that their containerized applications are up-to-date and properly protected via a reputable security provider.

Lastly, to learn more about pressing issues in the cyber world, please join us at the premiere cyber security event of the year – CPX 360 2022. Register here.