Deploying MLflow in AWS App Runner

mlflow.org

AWS launched May 2021 the AWS App Runner, a fully managed service that makes it easy for developers to quickly deploy containerized web applications and APIs. [1]

Unlike AWS Fargate, AWS App Runner has a pricing policy based on requests that your application receives. Apps that don’t receive requests all the time can benefit from it.

Our loved MLflow fits perfectly in this context. The interaction with MLflow is usually made in training jobs, data scientists evaluating their experiments, or APIs that expose our models. In all these cases we don’t have frequent access, it occurs eventually.

So, in this post, I’ll describe how I deployed the MLflow with basic auth (username/password) in AWS App Runner for my personal projects and the costs involved in this solution.

Solution Overview

The following diagram illustrates this architecture.

Figure 1: Architecture. Image by author

An MLflow tracking server also has two components to store data: a backend store and an artifact store. [3]

The backend store is where MLflow Tracking Server stores experiment and run metadata as well as params, metrics, and tags for runs. [3]

I used the Amazon RDS as our backend store because it has a Free Tier available for 12 months. Each calendar month, the free tier will allow you to use the Amazon RDS resources listed below for free: [4]

• 750 hrs of Amazon RDS in a Single-AZ “db.t2.micro” Instance.
• 20 GB of General Purpose Storage (SSD).
• 20 GB for automated backup storage and any user-initiated DB Snapshots.

The artifact store is a location suitable for large data and is where data scientists log their artifact output (for example, models). [3] Obviously, the default choice here is an S3 Bucket.

Amazon S3 is an object storage service. It also has a free tier available for 12 months. [5]

• 5GB of standard storage.
• 20,000 Get Requests
• 2,000 Put Requests

Our MLflow Tracking Server will be hosted in App Runner. This new service launched by AWS is very similar to Cloud Run which is available in Google Cloud. With App Runner, you deploy and run your applications in container instances, which consume compute and memory resources.

When your application is idle, you pay per GB of memory for provisioned container instances that keep your application warm and eliminate cold starts. [2]

When requests come in, your application responds in milliseconds, and you pay for the vCPU and memory consumed by your active container instances as your application is processing requests. [2] See more about the pricing here.

Deployment

I created a Terraform stack that will help you to easily create all resources needed to host the MLflow Server with basic authentication.

The Terraform stack will create the following resources:

• An S3 bucket for storing MLflow artifacts.
• An IAM role and policy for the MLflow server connect to the S3 bucket.
• An RDS database instance (MySQL) for storing MLflow metadata.
• A service in App Runner to run MLflow Tracking Server.

Prerequisites

You’ll need to have the following installed:

• AWS CLI (1.18.69)
• Terraform CLI (v1.0.1)

Deploying MLflow

1. Create an AWS account if you don’t already have one.
2. Configure AWS CLI to use your AWS account.
3. Clone the repository: github.com/DougTrajano/mlflow-server
4. Open mlflow-server/terraform folder.

cd mlflow-server/terraform

5. Run the following command to create all the required resources:

terraform apply -var mlflow_username="USERNAME" -var mlflow_password="PASSWORD"

5. Type “yes” when prompted to continue.

mlflow_username and mlflow_password are used to authenticate you in MLflow UI and API. These variables are informed to Nginx proxy as environment variables in docker. If you don’t provide a mlflow_password, this stack will create a strong password for you. :)

Terraform will create the resources in your account, this may take few minutes. In the end, you will receive the MLflow URL (XXXXX.aws-region.awsapprunner.com) and username/password for basic authentication.

Figure 2: Terraform execution. Image by author

Access the service_url provided and see your MLflow \o/

Figure 3: MLflow UI. Image by author

You can use this link to access the MLflow UI and to track experiments in MLflow API, for example:

MLflow sample usage. Code by author

So, that’s it! If you want to see the code and more information about this project, see the repository below.

DougTrajano/mlflow-server

In this solution, Amazon RDS and Amazon S3 are both parts of the free tier provided by AWS. The only cost involved here is App Runner, but based on my experience and AWS estimates, its cost will be close to 10 dollars a month, depending on your usage.

This post does not cover MLflow usage, but if you want to see how to use MLflow like a pro, I strongly recommend the post below. :)

Complete Data Science Project Template with Mlflow for Non-Dummies.

I hope you enjoyed this post. If you have any comments or contributions, leave a comment and make the community stronger :)

References

[1]”AWS App Runner — Fully managed container application service — Amazon Web Services”, Amazon Web Services, Inc., 2021. [Online]. Available: https://aws.amazon.com/apprunner/. [Accessed: 17- Jul- 2021]

[2]”AWS App Runner Pricing — Fully managed container application service — Amazon Web Services”, Amazon Web Services, Inc., 2021. [Online]. Available: https://aws.amazon.com/apprunner/pricing/. [Accessed: 17- Jul- 2021]

[3]”MLflow Tracking — MLflow 1.19.0 documentation”, Mlflow.org, 2021. [Online]. Available: https://www.mlflow.org/docs/latest/tracking.html. [Accessed: 17- Jul- 2021]

[4]”Amazon RDS | Cloud Relational Database | Amazon Web Services”, Amazon Web Services, Inc., 2021. [Online]. Available: https://aws.amazon.com/rds/. [Accessed: 17- Jul- 2021]

[5]”Cloud Object Storage | Store & Retrieve Data Anywhere | Amazon Simple Storage Service (S3)”, Amazon Web Services, Inc., 2021. [Online]. Available: https://aws.amazon.com/s3/. [Accessed: 17- Jul- 2021]