Deploy CosmicAC

Deploy the CosmicAC Docker Compose stack on your host machine. After deployment, CosmicAC connects to your GPU Kubernetes cluster. For the services in the stack and how they interact, see Deployment architecture.

Prerequisites

The recommended host is Ubuntu 22.04 or 24.04 on an x86_64 CPU. Before you start, make sure that you have the following.

Docker Engine and Docker Compose v2.
Task.
Node.js.
The jq and kubectl command-line tools.
Access to the private CosmicAC deployment repository, which contains the Compose files and deployment scripts. To get access, contact the CosmicAC team.
GitHub Container Registry (GHCR) credentials for the private CosmicAC images in ghcr.io/tetherto.
- Your GitHub username.
- A classic GitHub personal access token with the read:packages scope. If the tetherto organization enforces single sign-on (SSO), approve the token for the organization. See Managing your personal access tokens.
A GPU Kubernetes cluster that meets the requirements. CosmicAC connects to the cluster but doesn't create it.
A kubeconfig for the cluster.

Steps

Verify the prerequisites

Enable Docker, and check that each required tool is installed.

sudo systemctl enable --now docker
docker compose version
task --version
jq --version
node --version
kubectl version --client

Set up the deployment

Clone the deployment repository and change to its directory.

git clone <deployment-repo-url>
cd <deployment-repo>

Create the .env file from the example file.

cp .env.example .env

In .env, set the variables that are required before the first deployment. For every supported variable, see Deployment configuration.

If GITHUB_PAT and GITHUB_USER aren't set in .env, bootstrap prompts for them before it pulls the private images from GHCR.

Add the kubeconfig

Get the kubeconfig from your cluster administrator. The kubeconfig must meet the kubeconfig requirements.

Create a file on the host, paste the kubeconfig into it, and save the file.

nano ~/kubeconfig

Get the absolute path of the file.

realpath ~/kubeconfig

In .env, set KUBECONFIG_SRC to that path.

KUBECONFIG_SRC=/home/<user>/kubeconfig

Check that the file isn't empty and that the kubeconfig reaches your cluster.

test -s ~/kubeconfig && echo ok
kubectl --kubeconfig ~/kubeconfig config current-context
kubectl --kubeconfig ~/kubeconfig cluster-info

Run the bootstrap

The task bootstrap command deploys the whole stack with the TAG value in .env. For what bootstrap runs, see Task deployment commands.

Get the path of the deployment directory, and copy it.

pwd

Open a root shell.

sudo -i

A root shell starts in the root home directory. Change back to the deployment directory.

cd <deployment-directory>

Run the bootstrap.

task bootstrap

Run later commands as root

Run all later task commands as root. Bootstrap and the running services create the deployment's configuration and state files as root, so commands such as task backup and task update fail for other users.

Verify the deployment

Check the services and the API.

task ps
curl -s4 -o /dev/null -w '%{http_code}\n' http://127.0.0.1:5173/
curl -s4 "http://127.0.0.1:5173/api/auth/servers?overwrite_cache=true"
curl -s4 "http://127.0.0.1:5173/api/auth/jobs?page=1&pageSize=10"

These requests don't need a token, because a default deployment has authentication turned off.

The deployment is ready when task ps shows every service as Up and the web interface returns HTTP 200. The servers request lists your GPUs. On a new deployment, the jobs request returns an empty list.

Open the web interface

In your browser, go to http://<server-ip>:5173. If you changed UI_PORT, use that port instead.

A default deployment has authentication turned off, so anyone who can reach this port has full access without signing in. Restrict network access to the deployment. For the authentication settings, see Deployment configuration.

Help and troubleshooting

Bootstrap fails with rm: cannot remove ...: Permission denied

Bootstrap resets the HyperMQ stores in ./services/*/store before it deploys. The stack's containers create those files as root, so a non-root user can't delete them.

To fix the error, run bootstrap again from a root shell in the deployment directory.

sudo -i
cd <deployment-directory>
task bootstrap

Bootstrap fails with error from registry: denied

The user who runs bootstrap has no working GHCR credentials. Either that user never logged in to GHCR, or an earlier login left credentials that have since expired or been revoked. Bootstrap logs in only when no credentials are stored, so it can't replace credentials that no longer work. See task ensure-login.

To fix the error, run the following commands as root in the deployment directory.

Remove the stored GHCR credentials.
```
docker logout ghcr.io
```
Include the registry name. A bare docker logout signs you out of Docker Hub instead.
Log in to GHCR again.
```
task login
```
The task login command reads your GitHub username and token from GITHUB_USER and GITHUB_PAT in .env. You don't type the token in a command, so it isn't saved in your shell history.
Run bootstrap again.
```
task bootstrap
```

If the pull still fails, check the Docker configuration file, ~/.docker/config.json by default, for a credsStore or credHelpers entry. Either entry makes Docker store credentials in a separate helper program, and a missing or broken helper fails the same way as an expired token.

Next steps

After you deploy CosmicAC, add a model master for each model that you want to serve. Then install the CLI and create your first job.

Deploy CosmicAC

On this page