Below are some notes on steps you need to take before deploying Garden projects to a remote Kubernetes cluster.
Many of the steps are not specific to Garden as such, so you may have already performed some of these steps and/or may need to follow the provided links in each section for details on how to perform the steps you have not yet completed.
Start by making sure you have a kubectl context set up on your development machine to access your cluster. How you set this up will vary by how and where you have deployed your cluster.
Then configure the project and provider in your project
garden.yml, along with the kubectl context you use to connect to your cluster.
project:name: my-projectenvironments:- name: devproviders:- name: kubernetescontext: my-dev-context # the name of the kubectl context for the cluster...defaultEnvironment: dev
The cluster needs to have a configured nginx ingress controller.
You'll also need to point one or more DNS entries to your cluster, and configure a TLS certificate for the hostnames you will expose for ingress. How you configure DNS and prepare the certificates will depend on how you manage DNS and certificates in general, so we won't cover that in detail here.
Once you have the certificates on hand (the
.key files), create a Secret for each cert in the cluster so that they can be referenced when deploying services:
kubectl create secret tls mydomain-tls-secret --key <path-to-key-file> --cert <path-to-crt-file>
Then configure each certificate/secret in your
garden.yml provider configuration:
project:name: my-projectenvironments:- name: devproviders:- name: kubernetescontext: my-dev-contexttlsCertificates:- name: main# Optionally set particular hostnames to use this certificate for# (useful if you have multiple certs for the same hostname).hostnames: [mydomain.com]secretRef:# Change to whatever name you chose for the secret above.name: my-tls-secret# Change this if you store the secret in another namespace.namespace: default- name: wildcardsecretRef:name: wildcard-tls-secretnamespace: default...defaultEnvironment: dev
NOTE: Configuring a container registry is not necessary when using Garden Enterprise, which features integrated remote building.
When you deploy to the environment (via
garden deploy or
garden dev), containers are first built and then pushed to the configured deployment registry, where the K8s cluster will then pull the built images when deploying. This should generally be a private container registry, or at least a private project in a public registry.
Similarly to the above TLS configuration, you may also need to set up auth for the registry using K8s Secrets, in this case via the
kubectl create secret docker-registry helper.
Note that you do not need to configure the authentication and imagePullSecrets when using GKE along with GCR, as long as your deployment registry is in the same project as the GKE cluster.
Once you've created the auth secret in the cluster, you can configure the registry and the secrets in your
garden.yml project config like this:
project:name: my-projectenvironments:- name: devproviders:- name: kubernetescontext: my-dev-context...deploymentRegistry:# The hostname of the registry, e.g. gcr.io for GCR (Google Container Registry)hostname: my.registry.io# Namespace (aka project ID) to use in the registry for this project.# For GKE/GCR, use the project ID where your cluster is.namespace: my-project-idimagePullSecrets:# The name of the secret you stored using `kubectl create secret docker-registry`- name: my-registry-secret# Change this if you store the secret in another namespace.namespace: defaultdefaultEnvironment: dev
You also need to login to the
docker CLI, so that images can be pushed to the registry. Please refer to your registry's documentation on how to do that (for Docker Hub you simply run
Note that you need to have permissions to create namespaces and to create deployments, daemonsets, services and ingresses within the namespaces created.
The plugin will create two or more namespaces per user and project, one to run services, another to manage metadata and configuration (this is so that your environment can be reset without clearing your configuration variables), and potentially more to support specific plugins/providers.