Example: run Apache Airflow in App Studio
This worked example deploys Apache Airflow 3.3.2, web interface included, as an App Studio app in about ten minutes.
Who can use it: the Builder role or higher.
Recorded on .
Prerequisites
- Your platform address, the part of the URL before the first
/when you are signed in. This page writes it asyour-platform-address. - The email address you sign in with.
- A compute tier with at least 2 cores and 8 GB of memory. Airflow's standalone mode runs four processes at once.
Work out the app's address first
Airflow must be told its own address before the first deploy. App Studio serves every app at /apps/name-suffix, and the suffix is the first six characters of a SHA-256 hash of your organisation, your sign-in email and the app name, joined by colons. On AI Platform the organisation part is empty, so the text starts with a colon. On AI Cloud it starts with your organisation's short name.
printf "%s" ":you@example.com:airflow" | sha256sum | cut -c1-6
For you@example.com and the app name airflow this prints f9a4df, so the app lives at https://your-platform-address/apps/airflow-f9a4df. The Review step shows the address with a placeholder where the suffix goes, so work the suffix out yourself.
Settings to copy
Start command. Keep the word streamlit out of it. If it appears, the platform appends Streamlit's own flags to the command.
pip install "apache-airflow==3.3.2" --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-3.3.2/constraints-3.11.txt" && exec airflow standalone
Environment variables. Replace airflow-f9a4df with your own name and suffix.
| Key | Value | Why |
|---|---|---|
AIRFLOW__CORE__SIMPLE_AUTH_MANAGER_ALL_ADMINS | True | Turns off Airflow's own login. Read Limits before you rely on this. |
AIRFLOW__API__PORT | 8501 | The port a Streamlit app is expected to listen on. |
AIRFLOW__API__BASE_URL | https://your-platform-address/apps/airflow-f9a4df | Tells Airflow it is served under that path, so its links and assets resolve. |
AIRFLOW__CORE__EXECUTION_API_SERVER_URL | http://localhost:8501/apps/airflow-f9a4df/execution/ | Lets task processes reach Airflow inside the pod. Without it they go out through the sign-in page and every task fails. |
Deploy Airflow
- Open the wizard
Switch to Studio, open App Studio and choose New app.
- Source
Choose GitHub repo. Enter
https://github.com/octocat/Hello-Worldwith the branchmasterand leave Access token empty. App Studio needs a Git source to run a start command; the repository's files are never used. A Docker image source does not work here, because the official Airflow image needs thestandaloneargument and the wizard cannot pass it. - Framework
Detection says it could not identify a framework, which is expected. Pick Streamlit: it is the framework App Studio serves at the full
/apps/path without rewriting it, which is how Airflow expects to sit behind a proxy. Leave Main file asapp.pyand paste the start command into Build / start command. - Runtime
Set App name to
airflow, the name you hashed. Choose a Compute tier with 2 cores and 8 GB or more, leave Runtime image on Auto (Python 3.11, which must match the constraints file in the start command), and set Min replicas and Max replicas to 1. Leave storage and the dataset on none. Set Health check path to/api/v2/monitor/health. Choose + Add variable four times and enter the table above, leaving Secret unticked. - Network
Choose Private (just me). The default is Shared. Leave everything else as it is.
- Review and ship
Check the summary and choose Ship app. The build takes two to three minutes. The first line of the build log names the app it is creating. If that name is not the
airflow-name you worked out, the address in your variables is wrong: delete the app and create it again with the right suffix.
Verify
Choose Open on the app, or go to its address with a trailing slash.
- The home page shows MetaDatabase, Scheduler, Triggerer and Dag Processor as healthy.
- Dags lists Airflow's example DAGs.
- Open
example_bash_operatorand trigger it. The run finishes as a success, with two tasks skipped by design: one exits with code 99, which skips it and the task after it.
Limits
This is a demonstration, not a production Airflow.
- Access. With Airflow's own login turned off, the app's visibility is the only gate. Private hides the app from other users in App Studio, but sign-in on its address is enforced only where your cluster enables it, and the Network step warns you when it does not. If you see that warning, remove the first variable so Airflow keeps its own login. Never choose Public.
- No persistence. Airflow's database lives inside the pod, so a restart loses run history, connections and variables.
- Slow restarts. Every pod start installs Airflow again, which takes about two minutes.
- Your own DAGs need a real Git repository or a mounted Storage dataset. This example sets up neither.
- Cost. The app is billed for its compute tier while it runs; Review shows the estimate. Stop or delete the app in App Studio when you are done.
The Airflow documentation for 3.3.2 explains each setting: see its pages on running behind a reverse proxy and its configuration reference.