‍Overview

Apache Spark is an open-source, distributed computing framework designed for big data processing and analytics, offering high-speed performance through in-memory computation and a unified engine for diverse workloads like batch processing, streaming, and machine learning. 

The Apache Spark UI provides an option to activate Access Control Lists (ACLs) through the spark.acls.enable configuration setting. When paired with an authentication filter, this feature verifies if a user has the necessary permissions to view or alter the application. When ACLs are turned on, a specific section of code in HttpSecurityFilter may permit impersonation by allowing an individual to supply any user name. A harmful user could then potentially access a permission verification function that constructs and runs a Unix shell command using their input. This leads to the execution of arbitrary shell commands under the identity of the user running Spark.

Command injection is a type of security vulnerability that occurs when an application allows untrusted user input to be incorporated into a command that is executed by the operating system or a similar execution environment. This happens when the application fails to properly validate, sanitize, or restrict the input before passing it to a function or process that runs system-level commands, such as a shell or script interpreter. The ramifications can be far-reaching and significant. They are:

• Allowing arbitrary code execution
• Complete system compromise
• Data theft or exposure
• Data manipulation or destruction
• Privilege escalation, and
• Denial of service.

Details

Module Info

Product: Apache Spark

Affected packages: Apache Spark

Affected versions:

>=3.2.0 <=3.2.1

>=3.1.1 <=3.1.3

<=3.0.3

GitHub Repo: N/A

Published packages: N/A

Package manager: npm

‍

Vulnerability Info

This high-severity vulnerability is found in the main Apache Spark package.

Steps To Reproduce

1. Set up an Apache Spark environment that is vulnerable to this exploit, such as 3.1.1. Configure Spark to run with the UI enabled. 
2. Enable ACLs in Spark Configuration through the Spark configuration file (e.g., spark-defaults.conf).
3. Launch the Spark UI  and log in.
4. Exploit impersonation in HttpSecurityFilter  by identifying an endpoint in the Spark UI that processes user input and is protected by ACLs (e.g., a REST API endpoint or a UI form submission).
5. Craft an HTTP request to this endpoint that includes a parameter allowing impersonation. For example, supply an arbitrary username in a header or query parameter (exact parameter depends on implementation, but commonly something like doAs or a custom user field).
6. In the same  request, Inject malicious input to trigger shell command execution by including input that reaches a permission check function vulnerable to command injection. It might look like this (of course, whoami is innocuous in this example):

curl -X POST "http://<spark-master-host>:8080/<vulnerable-endpoint>" \
  -H "Authorization: <valid-auth-token>" \
  -d "user=arbitraryUser&action=checkPermission&input=;whoami;"

‍

7. Send the crafted request to the Spark UI endpoint.
8. If successful, the Spark process executes the injected shell command as the user running the Spark application (e.g., the spark user or the system user who started the process).

Addressing the Issue

Users of the affected components should apply one of the following mitigations:

• Disable ACLs if they aren’t required.

• Upgrade to a secure version of the software.

• Sign up for post-EOL security support; HeroDevs customers get immediate access to a patched version of this software.

Credit(s)

• Sven Krewitt, Flashpoint

‍

‍