kubernetes - Install Linux package on launch when deploying Airflow with Helm in Azure Kubernets Service

Question

I'm trying to install Airflow on Azure Kubernetes Service (AKS) using Helm. I've found some guides to do so, and with some difficulty I've managed to get it working fine. I was now trying to run some DAGs I made and in one of those DAGs I use the bash operator to run a specific command, and that command needs a Linux package that does not come with the default image Helm uses for Airflow... https://github.com/airflow-helm/charts/tree/main/charts/airflow

Is there a way to include extra Linux packages for the Helm Airflow chart installation? I've looked all over for it but couldn't find anything

Hope someone can help me on this.

Answer 1

Helm itself is a templating language. Most of the helm charts give you a flexibility to change your base image of the applications. As long as you are using the related image, the helm chart will create a correct deployment for you.

In your case, if you want to extend the functionalities or install more packages in the base image, you will need to create your own image and push to an image repository.

For example, you can install your package like this with Dockerfile defined locally.

FROM apache/airflow:1.10.12-python3.6

RUN apt update && 
    apt install vim -y && 
    rm -rf /var/lib/apt/lists/*

Then, run the following command to build and upload your custom image.

docker build -t my-account/my-airflow:latest .
docker push my-account/my-airflow:latest

In your values file, you can specify your image name and tag then.

airflow:
  image:
    repository: my-account
    tag: my-airflow

Once you apply this values file, helm will help you to change the default image to your customized one.

The tutorial in the doc also mentions the custom image but it is for the DAG building. But with the same technique, you can customize your own base image.