local-kubernetes

Description

The local-kubernetes provider is a specialized version of the kubernetes provider that automates and simplifies working with local Kubernetes clusters.

For general Kubernetes usage information, please refer to the Kubernetes guides. For local clusters a good place to start is the Local Kubernetes guide. The Quickstart Guide guide is also helpful as an introduction.

If you're working with a remote Kubernetes cluster, please refer to the kubernetes provider docs, and the Remote Kubernetes guide guide.

Below is the full schema reference for the provider configuration. For an introduction to configuring a Garden project with providers, please look at our configuration guide.

The reference is divided into two sections. The first section contains the complete YAML schema, and the second section describes each schema key.

Complete YAML Schema

The values in the schema below are the default values.

providers:
  - # List other providers that should be resolved before this one.
    dependencies: []

    # If specified, this provider will only be used in the listed environments. Note that an empty array effectively
    # disables the provider. To use a provider in all environments, omit this field.
    environments:

    # Choose the mechanism for building container images before deploying. By default your local Docker daemon is
    # used, but you can set it to `cluster-buildkit` or `kaniko` to sync files to the cluster, and build container
    # images there. This removes the need to run Docker locally, and allows you to share layer and image caches
    # between multiple developers, as well as between your development and CI workflows.
    #
    # For more details on all the different options and what makes sense to use for your setup, please check out the
    # [in-cluster building
    # guide](https://docs.garden.io/v/acorn-0.12/kubernetes-plugins/advanced/in-cluster-building).
    #
    # **Note:** The `cluster-docker` mode has been deprecated and will be removed in a future release!
    buildMode: local-docker

    # Configuration options for the `cluster-buildkit` build mode.
    clusterBuildkit: {}
      # Use the `cache` configuration to customize the default cluster-buildkit cache behaviour.
      #
      # The default value is:
      # clusterBuildkit:
      #   cache:
      #     - type: registry
      #       mode: auto
      #
      # For every build, this will
      # - import cached layers from a docker image tag named `_buildcache`
      # - when the build is finished, upload cache information to `_buildcache`
      #
      # For registries that support it, `mode: auto` (the default) will enable the buildkit `mode=max`
      # option.
      #
      # See the following table for details on our detection mechanism:
      #
      # | Registry Name                   | Registry Domain         | Assumed `mode=max` support |
      # |---------------------------------|-------------------------|------------------------------|
      # | Google Cloud Artifact Registry  | `pkg.dev`             | Yes                          |
      # | Azure Container Registry        | `azurecr.io`          | Yes                          |
      # | GitHub Container Registry       | `ghcr.io`             | Yes                          |
      # | DockerHub                       | `hub.docker.com`     | Yes                          |
      # | Garden In-Cluster Registry      |                         | Yes                          |
      # | Any other registry              |                         | No                           |
      #
      # In case you need to override the defaults for your registry, you can do it like so:
      #
      # clusterBuildkit:
      #   cache:
      #     - type: registry
      #       mode: max
      #
      # When you add multiple caches, we will make sure to pass the `--import-cache` options to buildkit in the same
      # order as provided in the cache configuration. This is because buildkit will not actually use all imported
      # caches
      # for every build, but it will stick with the first cache that yields a cache hit for all the following layers.
      #
      # An example for this is the following:
      #
      # clusterBuildkit:
      #   cache:
      #     - type: registry
      #       tag: _buildcache-${slice(kebabCase(git.branch), "0", "30")}
      #     - type: registry
      #       tag: _buildcache-main
      #       export: false
      #
      # Using this cache configuration, every build will first look for a cache specific to your feature branch.
      # If it does not exist yet, it will import caches from the main branch builds (`_buildcache-main`).
      # When the build is finished, it will only export caches to your feature branch, and avoid polluting the `main`
      # branch caches.
      # A configuration like that may improve your cache hit rate and thus save time.
      #
      # If you need to disable caches completely you can achieve that with the following configuration:
      #
      # clusterBuildkit:
      #   cache: []
      cache:
        - # Use the Docker registry configured at `deploymentRegistry` to retrieve and store buildkit cache
          # information.
          #
          # See also the [buildkit registry cache
          # documentation](https://github.com/moby/buildkit#registry-push-image-and-cache-separately)
          type:

          # The registry from which the cache should be imported from, or which it should be exported to.
          #
          # If not specified, use the configured `deploymentRegistry` in your kubernetes provider config, or the
          # internal in-cluster registry in case `deploymentRegistry` is not set.
          #
          # Important: You must make sure `imagePullSecrets` includes authentication with the specified cache
          # registry, that has the appropriate write privileges (usually full write access to the configured
          # `namespace`).
          registry:
            # The hostname (and optionally port, if not the default port) of the registry.
            hostname:

            # The port where the registry listens on, if not the default.
            port:

            # The registry namespace. Will be placed between hostname and image name, like so:
            # <hostname>/<namespace>/<image name>
            namespace: _

            # Set to true to allow insecure connections to the registry (without SSL).
            insecure: false

          # This is the buildkit cache mode to be used.
          #
          # The value `inline` ensures that garden is using the buildkit option `--export-cache inline`. Cache
          # information will be inlined and co-located with the Docker image itself.
          #
          # The values `min` and `max` ensure that garden passes the `mode=max` or `mode=min` modifiers to the
          # buildkit `--export-cache` option. Cache manifests will only be
          # stored stored in the configured `tag`.
          #
          # `auto` is the same as `max` for some registries that are known to support it. Garden will fall back to
          # `inline` for all other registries.
          #  See the [clusterBuildkit cache option](#providers-.clusterbuildkit.cache) for a description of the
          # detection mechanism.
          #
          # See also the [buildkit export cache documentation](https://github.com/moby/buildkit#export-cache)
          mode: auto

          # This is the Docker registry tag name buildkit should use for the registry build cache. Default is
          # `_buildcache`
          #
          # **NOTE**: `tag` can only be used together with the `registry` cache type
          tag: _buildcache

          # If this is false, only pass the `--import-cache` option to buildkit, and not the `--export-cache` option.
          # Defaults to true.
          export: true

      # Enable rootless mode for the cluster-buildkit daemon, which runs the daemon with decreased privileges.
      # Please see [the buildkit docs](https://github.com/moby/buildkit/blob/master/docs/rootless.md) for caveats when
      # using this mode.
      rootless: false

      # Exposes the `nodeSelector` field on the PodSpec of the BuildKit deployment. This allows you to constrain the
      # BuildKit daemon to only run on particular nodes.
      #
      # [See here](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/) for the official Kubernetes
      # guide to assigning Pods to nodes.
      nodeSelector: {}

      # Specify tolerations to apply to cluster-buildkit daemon. Useful to control which nodes in a cluster can run
      # builds.
      tolerations:
        - # "Effect" indicates the taint effect to match. Empty means match all taint effects. When specified,
          # allowed values are "NoSchedule", "PreferNoSchedule" and "NoExecute".
          effect:

          # "Key" is the taint key that the toleration applies to. Empty means match all taint keys.
          # If the key is empty, operator must be "Exists"; this combination means to match all values and all keys.
          key:

          # "Operator" represents a key's relationship to the value. Valid operators are "Exists" and "Equal".
          # Defaults to
          # "Equal". "Exists" is equivalent to wildcard for value, so that a pod can tolerate all taints of a
          # particular category.
          operator: Equal

          # "TolerationSeconds" represents the period of time the toleration (which must be of effect "NoExecute",
          # otherwise this field is ignored) tolerates the taint. By default, it is not set, which means tolerate
          # the taint forever (do not evict). Zero and negative values will be treated as 0 (evict immediately)
          # by the system.
          tolerationSeconds:

          # "Value" is the taint value the toleration matches to. If the operator is "Exists", the value should be
          # empty,
          # otherwise just a regular string.
          value:

      # Specify annotations to apply to both the Pod and Deployment resources associated with cluster-buildkit.
      # Annotations may have an effect on the behaviour of certain components, for example autoscalers.
      annotations:

    # Setting related to Jib image builds.
    jib:
      # In some cases you may need to push images built with Jib to the remote registry via Kubernetes cluster, e.g.
      # if you don't have connectivity or access from where Garden is being run. In that case, set this flag to true,
      # but do note that the build will take considerably take longer to complete! Only applies when using in-cluster
      # building.
      pushViaCluster: false

    # Configuration options for the `kaniko` build mode.
    kaniko:
      # Specify extra flags to use when building the container image with kaniko. Flags set on `container` modules
      # take precedence over these.
      extraFlags:

      # Change the kaniko image (repository/image:tag) to use when building in kaniko mode.
      image: 'gcr.io/kaniko-project/executor:v1.11.0-debug'

      # Choose the namespace where the Kaniko pods will be run. Set to `null` to use the project namespace.
      #
      # **IMPORTANT: The default namespace will change to the project namespace instead of the garden-system namespace
      # in an upcoming release!**
      namespace: garden-system

      # Exposes the `nodeSelector` field on the PodSpec of the Kaniko pods. This allows you to constrain the Kaniko
      # pods to only run on particular nodes. The same nodeSelector will be used for each util pod unless they are
      # specifically set under `util.nodeSelector`.
      #
      # [See here](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/) for the official Kubernetes
      # guide to assigning pods to nodes.
      nodeSelector:

      # Specify tolerations to apply to each Kaniko builder pod. Useful to control which nodes in a cluster can run
      # builds. The same tolerations will be used for each util pod unless they are specifically set under
      # `util.tolerations`
      tolerations:
        - # "Effect" indicates the taint effect to match. Empty means match all taint effects. When specified,
          # allowed values are "NoSchedule", "PreferNoSchedule" and "NoExecute".
          effect:

          # "Key" is the taint key that the toleration applies to. Empty means match all taint keys.
          # If the key is empty, operator must be "Exists"; this combination means to match all values and all keys.
          key:

          # "Operator" represents a key's relationship to the value. Valid operators are "Exists" and "Equal".
          # Defaults to
          # "Equal". "Exists" is equivalent to wildcard for value, so that a pod can tolerate all taints of a
          # particular category.
          operator: Equal

          # "TolerationSeconds" represents the period of time the toleration (which must be of effect "NoExecute",
          # otherwise this field is ignored) tolerates the taint. By default, it is not set, which means tolerate
          # the taint forever (do not evict). Zero and negative values will be treated as 0 (evict immediately)
          # by the system.
          tolerationSeconds:

          # "Value" is the taint value the toleration matches to. If the operator is "Exists", the value should be
          # empty,
          # otherwise just a regular string.
          value:

      # Specify annotations to apply to each Kaniko builder pod. Annotations may have an effect on the behaviour of
      # certain components, for example autoscalers. The same annotations will be used for each util pod unless they
      # are specifically set under `util.annotations`
      annotations:

      util:
        # Specify tolerations to apply to each garden-util pod.
        tolerations:
          - # "Effect" indicates the taint effect to match. Empty means match all taint effects. When specified,
            # allowed values are "NoSchedule", "PreferNoSchedule" and "NoExecute".
            effect:

            # "Key" is the taint key that the toleration applies to. Empty means match all taint keys.
            # If the key is empty, operator must be "Exists"; this combination means to match all values and all keys.
            key:

            # "Operator" represents a key's relationship to the value. Valid operators are "Exists" and "Equal".
            # Defaults to
            # "Equal". "Exists" is equivalent to wildcard for value, so that a pod can tolerate all taints of a
            # particular category.
            operator: Equal

            # "TolerationSeconds" represents the period of time the toleration (which must be of effect "NoExecute",
            # otherwise this field is ignored) tolerates the taint. By default, it is not set, which means tolerate
            # the taint forever (do not evict). Zero and negative values will be treated as 0 (evict immediately)
            # by the system.
            tolerationSeconds:

            # "Value" is the taint value the toleration matches to. If the operator is "Exists", the value should be
            # empty,
            # otherwise just a regular string.
            value:

        # Specify annotations to apply to each garden-util pod and deployments.
        annotations:

        # Specify the nodeSelector constraints for each garden-util pod.
        nodeSelector:

    # A default hostname to use when no hostname is explicitly configured for a service.
    defaultHostname:

    # Sets the deployment strategy for `container` services.
    #
    # The default is `"rolling"`, which performs rolling updates. There is also experimental support for blue/green
    # deployments (via the `"blue-green"` strategy).
    #
    # Note that this setting only applies to `container` services (and not, for example,  `kubernetes` or `helm`
    # services).
    deploymentStrategy: rolling

    # Configuration options for dev mode.
    devMode:
      # Specifies default settings for dev mode syncs (e.g. for `container`, `kubernetes` and `helm` services).
      #
      # These are overridden/extended by the settings of any individual dev mode sync specs for a given module or
      # service.
      #
      # Dev mode is enabled when running the `garden dev` command, and by setting the `--dev` flag on the `garden
      # deploy` command.
      #
      # See the [Code Synchronization guide](https://docs.garden.io/v/acorn-0.12/guides/code-synchronization-dev-mode)
      # for more information.
      defaults:
        # Specify a list of POSIX-style paths or glob patterns that should be excluded from the sync.
        #
        # Any exclusion patterns defined in individual dev mode sync specs will be applied in addition to these
        # patterns.
        #
        # `.git` directories and `.garden` directories are always ignored.
        exclude:

        # The default permission bits, specified as an octal, to set on files at the sync target. Defaults to 0600
        # (user read/write). See the [Mutagen
        # docs](https://mutagen.io/documentation/synchronization/permissions#permissions) for more information.
        fileMode:

        # The default permission bits, specified as an octal, to set on directories at the sync target. Defaults to
        # 0700 (user read/write). See the [Mutagen
        # docs](https://mutagen.io/documentation/synchronization/permissions#permissions) for more information.
        directoryMode:

        # Set the default owner of files and directories at the target. Specify either an integer ID or a string name.
        # See the [Mutagen docs](https://mutagen.io/documentation/synchronization/permissions#owners-and-groups) for
        # more information.
        owner:

        # Set the default group on files and directories at the target. Specify either an integer ID or a string name.
        # See the [Mutagen docs](https://mutagen.io/documentation/synchronization/permissions#owners-and-groups) for
        # more information.
        group:

    # Require SSL on all `container` module services. If set to true, an error is raised when no certificate is
    # available for a configured hostname on a `container` module.
    forceSsl: false

    # References to `docker-registry` secrets to use for authenticating with remote registries when pulling
    # images. This is necessary if you reference private images in your module configuration, and is required
    # when configuring a remote Kubernetes environment with buildMode=local.
    imagePullSecrets:
      - # The name of the Kubernetes secret.
        name:

        # The namespace where the secret is stored. If necessary, the secret may be copied to the appropriate
        # namespace before use.
        namespace: default

    # References to secrets you need to have copied into all namespaces deployed to. These secrets will be
    # ensured to exist in the namespace before deploying any service.
    copySecrets:
      - # The name of the Kubernetes secret.
        name:

        # The namespace where the secret is stored. If necessary, the secret may be copied to the appropriate
        # namespace before use.
        namespace: default

    # Resource requests and limits for the in-cluster builder, container registry and code sync service. (which are
    # automatically installed and used when `buildMode` is `cluster-docker` or `kaniko`).
    resources:
      # Resource requests and limits for the in-cluster builder. It's important to consider which build mode you're
      # using when configuring this.
      #
      # When `buildMode` is `kaniko`, this refers to _each Kaniko pod_, i.e. each individual build, so you'll want to
      # consider the requirements for your individual image builds, with your most expensive/heavy images in mind.
      #
      # When `buildMode` is `cluster-buildkit`, this applies to the BuildKit deployment created in _each project
      # namespace_. So think of this as the resource spec for each individual user or project namespace.
      #
      # When `buildMode` is `cluster-docker`, this applies to the single Docker Daemon that is installed and run
      # cluster-wide. This is shared across all users and builds in the cluster, so it should be resourced
      # accordingly, factoring in how many concurrent builds you expect and how heavy your builds tend to be. **Note
      # that the cluster-docker build mode has been deprecated!**
      builder:
        limits:
          # CPU limit in millicpu.
          cpu: 4000

          # Memory limit in megabytes.
          memory: 8192

          # Ephemeral storage limit in megabytes.
          ephemeralStorage:

        requests:
          # CPU request in millicpu.
          cpu: 100

          # Memory request in megabytes.
          memory: 512

          # Ephemeral storage request in megabytes.
          ephemeralStorage:

      # Resource requests and limits for the in-cluster image registry. Built images are pushed to this registry,
      # so that they are available to all the nodes in your cluster.
      #
      # This is shared across all users and builds, so it should be resourced accordingly, factoring
      # in how many concurrent builds you expect and how large your images tend to be.
      registry:
        limits:
          # CPU limit in millicpu.
          cpu: 2000

          # Memory limit in megabytes.
          memory: 4096

          # Ephemeral storage limit in megabytes.
          ephemeralStorage:

        requests:
          # CPU request in millicpu.
          cpu: 200

          # Memory request in megabytes.
          memory: 512

          # Ephemeral storage request in megabytes.
          ephemeralStorage:

      # Resource requests and limits for the util pod for in-cluster builders.
      # This pod is used to get, start, stop and inquire the status of the builds.
      #
      # This pod is created in each garden namespace.
      util:
        limits:
          # CPU limit in millicpu.
          cpu: 256

          # Memory limit in megabytes.
          memory: 512

          # Ephemeral storage limit in megabytes.
          ephemeralStorage:

        requests:
          # CPU request in millicpu.
          cpu: 256

          # Memory request in megabytes.
          memory: 512

          # Ephemeral storage request in megabytes.
          ephemeralStorage:

    # Storage parameters to set for the in-cluster builder, container registry and code sync persistent volumes
    # (which are automatically installed and used when `buildMode` is `cluster-docker` or `kaniko`).
    #
    # These are all shared cluster-wide across all users and builds, so they should be resourced accordingly,
    # factoring in how many concurrent builds you expect and how large your images and build contexts tend to be.
    storage:
      # Storage parameters for the in-cluster Docker registry volume. Built images are stored here, so that they
      # are available to all the nodes in your cluster.
      #
      # Only applies when `buildMode` is set to `cluster-docker` or `kaniko`, ignored otherwise.
      registry:
        # Volume size in megabytes.
        size: 20480

        # Storage class to use for the volume.
        storageClass: null

    # One or more certificates to use for ingress.
    tlsCertificates:
      - # A unique identifier for this certificate.
        name:

        # A list of hostnames that this certificate should be used for. If you don't specify these, they will be
        # automatically read from the certificate.
        hostnames:

        # A reference to the Kubernetes secret that contains the TLS certificate and key for the domain.
        secretRef:
          # The name of the Kubernetes secret.
          name:

          # The namespace where the secret is stored. If necessary, the secret may be copied to the appropriate
          # namespace before use.
          namespace: default

    # Exposes the `nodeSelector` field on the PodSpec of system services. This allows you to constrain the system
    # services to only run on particular nodes.
    #
    # [See here](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/) for the official Kubernetes guide
    # to assigning Pods to nodes.
    systemNodeSelector: {}

    # For setting tolerations on the registry-proxy when using in-cluster building.
    # The registry-proxy is a DaemonSet that proxies connections to the docker registry service on each node.
    #
    # Use this only if you're doing in-cluster building and the nodes in your cluster
    # have [taints](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/).
    registryProxyTolerations:
      - # "Effect" indicates the taint effect to match. Empty means match all taint effects. When specified,
        # allowed values are "NoSchedule", "PreferNoSchedule" and "NoExecute".
        effect:

        # "Key" is the taint key that the toleration applies to. Empty means match all taint keys.
        # If the key is empty, operator must be "Exists"; this combination means to match all values and all keys.
        key:

        # "Operator" represents a key's relationship to the value. Valid operators are "Exists" and "Equal". Defaults
        # to
        # "Equal". "Exists" is equivalent to wildcard for value, so that a pod can tolerate all taints of a
        # particular category.
        operator: Equal

        # "TolerationSeconds" represents the period of time the toleration (which must be of effect "NoExecute",
        # otherwise this field is ignored) tolerates the taint. By default, it is not set, which means tolerate
        # the taint forever (do not evict). Zero and negative values will be treated as 0 (evict immediately)
        # by the system.
        tolerationSeconds:

        # "Value" is the taint value the toleration matches to. If the operator is "Exists", the value should be
        # empty,
        # otherwise just a regular string.
        value:

    # The name of the provider plugin to use.
    name: local-kubernetes

    # The kubectl context to use to connect to the Kubernetes cluster.
    context:

    # Specify which namespace to deploy services to (defaults to the project name). Note that the framework generates
    # other namespaces as well with this name as a prefix.
    namespace:
      # A valid Kubernetes namespace name. Must be a valid RFC1035/RFC1123 (DNS) label (may contain lowercase letters,
      # numbers and dashes, must start with a letter, and cannot end with a dash) and must not be longer than 63
      # characters.
      name:

      # Map of annotations to apply to the namespace when creating it.
      annotations:

      # Map of labels to apply to the namespace when creating it.
      labels:

    # Set this to null or false to skip installing/enabling the `nginx` ingress controller.
    setupIngressController: nginx

Configuration Keys

`providers[]`

Type

Default

Required

array[object]

[]

`providers[].dependencies[]`

providers > dependencies

List other providers that should be resolved before this one.

Type

Default

Required

array[string]

[]

Example:

providers:
  - dependencies:
      - exec

`providers[].environments[]`

providers > environments

If specified, this provider will only be used in the listed environments. Note that an empty array effectively disables the provider. To use a provider in all environments, omit this field.

Type

Required

array[string]

Example:

providers:
  - environments:
      - dev
      - stage

`providers[].buildMode`

providers > buildMode

Choose the mechanism for building container images before deploying. By default your local Docker daemon is used, but you can set it to cluster-buildkit or kaniko to sync files to the cluster, and build container images there. This removes the need to run Docker locally, and allows you to share layer and image caches between multiple developers, as well as between your development and CI workflows.

For more details on all the different options and what makes sense to use for your setup, please check out the in-cluster building guide.

Note: The cluster-docker mode has been deprecated and will be removed in a future release!

Type

Allowed Values

Default

Required

string

"local-docker", "cluster-docker", "kaniko", "cluster-buildkit"

"local-docker"

Yes

`providers[].clusterBuildkit`

providers > clusterBuildkit

Configuration options for the cluster-buildkit build mode.

Type

Default

Required

object

{}

`providers[].clusterBuildkit.cache[]`

providers > clusterBuildkit > cache

Use the cache configuration to customize the default cluster-buildkit cache behaviour.

The default value is:

clusterBuildkit:
  cache:
    - type: registry
      mode: auto

For every build, this will

import cached layers from a docker image tag named _buildcache
when the build is finished, upload cache information to _buildcache

For registries that support it, mode: auto (the default) will enable the buildkit mode=max option.

See the following table for details on our detection mechanism:

Registry Name

Registry Domain

Assumed mode=max support

Google Cloud Artifact Registry

pkg.dev

Yes

Azure Container Registry

azurecr.io

Yes

GitHub Container Registry

ghcr.io

Yes

DockerHub

hub.docker.com

Yes

Garden In-Cluster Registry

Yes

Any other registry

In case you need to override the defaults for your registry, you can do it like so:

clusterBuildkit:
  cache:
    - type: registry
      mode: max

When you add multiple caches, we will make sure to pass the --import-cache options to buildkit in the same order as provided in the cache configuration. This is because buildkit will not actually use all imported caches for every build, but it will stick with the first cache that yields a cache hit for all the following layers.

An example for this is the following:

clusterBuildkit:
  cache:
    - type: registry
      tag: _buildcache-${slice(kebabCase(git.branch), "0", "30")}
    - type: registry
      tag: _buildcache-main
      export: false

Using this cache configuration, every build will first look for a cache specific to your feature branch. If it does not exist yet, it will import caches from the main branch builds (_buildcache-main). When the build is finished, it will only export caches to your feature branch, and avoid polluting the main branch caches. A configuration like that may improve your cache hit rate and thus save time.

If you need to disable caches completely you can achieve that with the following configuration:

clusterBuildkit:
  cache: []

Type

Default

Required

array[object]

[{"type":"registry","mode":"auto","tag":"_buildcache","export":true}]

`providers[].clusterBuildkit.cache[].type`

providers > clusterBuildkit > cache > type

Use the Docker registry configured at deploymentRegistry to retrieve and store buildkit cache information.

Type

Allowed Values

Required

string

"registry"

Yes

`providers[].clusterBuildkit.cache[].registry`

providers > clusterBuildkit > cache > registry

The registry from which the cache should be imported from, or which it should be exported to.

If not specified, use the configured deploymentRegistry in your kubernetes provider config, or the internal in-cluster registry in case deploymentRegistry is not set.

Important: You must make sure imagePullSecrets includes authentication with the specified cache registry, that has the appropriate write privileges (usually full write access to the configured namespace).

Type

Required

object

`providers[].clusterBuildkit.cache[].registry.hostname`

providers > clusterBuildkit > cache > registry > hostname

The hostname (and optionally port, if not the default port) of the registry.

Type

Required

string

Yes

Example:

providers:
  - clusterBuildkit:
      ...
      cache:
        - registry:
            ...
            hostname: "gcr.io"

`providers[].clusterBuildkit.cache[].registry.port`

providers > clusterBuildkit > cache > registry > port

The port where the registry listens on, if not the default.

Type

Required

number

`providers[].clusterBuildkit.cache[].registry.namespace`

providers > clusterBuildkit > cache > registry > namespace

The registry namespace. Will be placed between hostname and image name, like so: //

Type

Default

Required

string

"_"

Example:

providers:
  - clusterBuildkit:
      ...
      cache:
        - registry:
            ...
            namespace: "my-project"

`providers[].clusterBuildkit.cache[].registry.insecure`

providers > clusterBuildkit > cache > registry > insecure

Set to true to allow insecure connections to the registry (without SSL).

Type

Default

Required

boolean

false

`providers[].clusterBuildkit.cache[].mode`

providers > clusterBuildkit > cache > mode

This is the buildkit cache mode to be used.

The value inline ensures that garden is using the buildkit option --export-cache inline. Cache information will be inlined and co-located with the Docker image itself.

The values min and max ensure that garden passes the mode=max or mode=min modifiers to the buildkit --export-cache option. Cache manifests will only be stored stored in the configured tag.

auto is the same as max for some registries that are known to support it. Garden will fall back to inline for all other registries. See the clusterBuildkit cache option for a description of the detection mechanism.

Type

Allowed Values

Default

Required

string

"auto", "min", "max", "inline"

"auto"

Yes

`providers[].clusterBuildkit.cache[].tag`

providers > clusterBuildkit > cache > tag

This is the Docker registry tag name buildkit should use for the registry build cache. Default is _buildcache

NOTE: tag can only be used together with the registry cache type

Type

Default

Required

string

"_buildcache"

`providers[].clusterBuildkit.cache[].export`

providers > clusterBuildkit > cache > export

If this is false, only pass the --import-cache option to buildkit, and not the --export-cache option. Defaults to true.

Type

Default

Required

boolean

true

`providers[].clusterBuildkit.rootless`

providers > clusterBuildkit > rootless

Enable rootless mode for the cluster-buildkit daemon, which runs the daemon with decreased privileges. Please see the buildkit docs for caveats when using this mode.

Type

Default

Required

boolean

false

`providers[].clusterBuildkit.nodeSelector`

providers > clusterBuildkit > nodeSelector

Exposes the nodeSelector field on the PodSpec of the BuildKit deployment. This allows you to constrain the BuildKit daemon to only run on particular nodes.

See here for the official Kubernetes guide to assigning Pods to nodes.

Type

Default

Required

object

{}

Example:

providers:
  - clusterBuildkit:
      ...
      nodeSelector:
          disktype: ssd

`providers[].clusterBuildkit.tolerations[]`

providers > clusterBuildkit > tolerations

Specify tolerations to apply to cluster-buildkit daemon. Useful to control which nodes in a cluster can run builds.

Type

Default

Required

array[object]

[]

`providers[].clusterBuildkit.tolerations[].effect`

providers > clusterBuildkit > tolerations > effect

"Effect" indicates the taint effect to match. Empty means match all taint effects. When specified, allowed values are "NoSchedule", "PreferNoSchedule" and "NoExecute".

Type

Required

string

`providers[].clusterBuildkit.tolerations[].key`

providers > clusterBuildkit > tolerations > key

"Key" is the taint key that the toleration applies to. Empty means match all taint keys. If the key is empty, operator must be "Exists"; this combination means to match all values and all keys.

Type

Required

string

`providers[].clusterBuildkit.tolerations[].operator`

providers > clusterBuildkit > tolerations > operator

"Operator" represents a key's relationship to the value. Valid operators are "Exists" and "Equal". Defaults to "Equal". "Exists" is equivalent to wildcard for value, so that a pod can tolerate all taints of a particular category.

Type

Default

Required

string

"Equal"

`providers[].clusterBuildkit.tolerations[].tolerationSeconds`

providers > clusterBuildkit > tolerations > tolerationSeconds

"TolerationSeconds" represents the period of time the toleration (which must be of effect "NoExecute", otherwise this field is ignored) tolerates the taint. By default, it is not set, which means tolerate the taint forever (do not evict). Zero and negative values will be treated as 0 (evict immediately) by the system.

Type

Required

string

`providers[].clusterBuildkit.tolerations[].value`

providers > clusterBuildkit > tolerations > value

"Value" is the taint value the toleration matches to. If the operator is "Exists", the value should be empty, otherwise just a regular string.

Type

Required

string

`providers[].clusterBuildkit.annotations`

providers > clusterBuildkit > annotations

Specify annotations to apply to both the Pod and Deployment resources associated with cluster-buildkit. Annotations may have an effect on the behaviour of certain components, for example autoscalers.

Type

Required

object

Example:

providers:
  - clusterBuildkit:
      ...
      annotations:
          cluster-autoscaler.kubernetes.io/safe-to-evict: 'false'

`providers[].clusterDocker`

providers > clusterDocker

Deprecated: This field will be removed in a future release.

Configuration options for the cluster-docker build mode.

Type

Default

Required

object

{}

`providers[].clusterDocker.enableBuildKit`

providers > clusterDocker > enableBuildKit

Deprecated: This field will be removed in a future release.

Enable BuildKit support. This should in most cases work well and be more performant, but we're opting to keep it optional until it's enabled by default in Docker.

Type

Default

Required

boolean

false

`providers[].jib`

providers > jib

Setting related to Jib image builds.

Type

Required

object

`providers[].jib.pushViaCluster`

providers > jib > pushViaCluster

In some cases you may need to push images built with Jib to the remote registry via Kubernetes cluster, e.g. if you don't have connectivity or access from where Garden is being run. In that case, set this flag to true, but do note that the build will take considerably take longer to complete! Only applies when using in-cluster building.

Type

Default

Required

boolean

false

`providers[].kaniko`

providers > kaniko

Configuration options for the kaniko build mode.

Type

Required

object

`providers[].kaniko.extraFlags[]`

providers > kaniko > extraFlags

Specify extra flags to use when building the container image with kaniko. Flags set on container modules take precedence over these.

Type

Required

array[string]

`providers[].kaniko.image`

providers > kaniko > image

Change the kaniko image (repository/image:tag) to use when building in kaniko mode.

Type

Default

Required

string

"gcr.io/kaniko-project/executor:v1.11.0-debug"

`providers[].kaniko.namespace`

providers > kaniko > namespace

Choose the namespace where the Kaniko pods will be run. Set to null to use the project namespace.

IMPORTANT: The default namespace will change to the project namespace instead of the garden-system namespace in an upcoming release!

Type

Default

Required

string

"garden-system"

`providers[].kaniko.nodeSelector`

providers > kaniko > nodeSelector

Exposes the nodeSelector field on the PodSpec of the Kaniko pods. This allows you to constrain the Kaniko pods to only run on particular nodes. The same nodeSelector will be used for each util pod unless they are specifically set under util.nodeSelector.

See here for the official Kubernetes guide to assigning pods to nodes.

Type

Required

object

`providers[].kaniko.tolerations[]`

providers > kaniko > tolerations

Specify tolerations to apply to each Kaniko builder pod. Useful to control which nodes in a cluster can run builds. The same tolerations will be used for each util pod unless they are specifically set under util.tolerations

Type

Default

Required

array[object]

[]

`providers[].kaniko.tolerations[].effect`

providers > kaniko > tolerations > effect

"Effect" indicates the taint effect to match. Empty means match all taint effects. When specified, allowed values are "NoSchedule", "PreferNoSchedule" and "NoExecute".

Type

Required

string

`providers[].kaniko.tolerations[].key`

providers > kaniko > tolerations > key

"Key" is the taint key that the toleration applies to. Empty means match all taint keys. If the key is empty, operator must be "Exists"; this combination means to match all values and all keys.

Type

Required

string

`providers[].kaniko.tolerations[].operator`

providers > kaniko > tolerations > operator

Type

Default

Required

string

"Equal"

`providers[].kaniko.tolerations[].tolerationSeconds`

providers > kaniko > tolerations > tolerationSeconds

Type

Required

string

`providers[].kaniko.tolerations[].value`

providers > kaniko > tolerations > value

"Value" is the taint value the toleration matches to. If the operator is "Exists", the value should be empty, otherwise just a regular string.

Type

Required

string

`providers[].kaniko.annotations`

providers > kaniko > annotations

Specify annotations to apply to each Kaniko builder pod. Annotations may have an effect on the behaviour of certain components, for example autoscalers. The same annotations will be used for each util pod unless they are specifically set under util.annotations

Type

Required

object

Example:

providers:
  - kaniko:
      ...
      annotations:
          cluster-autoscaler.kubernetes.io/safe-to-evict: 'false'

`providers[].kaniko.util`

providers > kaniko > util

Type

Required

object

`providers[].kaniko.util.tolerations[]`

providers > kaniko > util > tolerations

Specify tolerations to apply to each garden-util pod.

Type

Default

Required

array[object]

[]

`providers[].kaniko.util.tolerations[].effect`

providers > kaniko > util > tolerations > effect

"Effect" indicates the taint effect to match. Empty means match all taint effects. When specified, allowed values are "NoSchedule", "PreferNoSchedule" and "NoExecute".

Type

Required

string

`providers[].kaniko.util.tolerations[].key`

providers > kaniko > util > tolerations > key

"Key" is the taint key that the toleration applies to. Empty means match all taint keys. If the key is empty, operator must be "Exists"; this combination means to match all values and all keys.

Type

Required

string

`providers[].kaniko.util.tolerations[].operator`

providers > kaniko > util > tolerations > operator

Type

Default

Required

string

"Equal"

`providers[].kaniko.util.tolerations[].tolerationSeconds`

providers > kaniko > util > tolerations > tolerationSeconds

Type

Required

string

`providers[].kaniko.util.tolerations[].value`

providers > kaniko > util > tolerations > value

"Value" is the taint value the toleration matches to. If the operator is "Exists", the value should be empty, otherwise just a regular string.

Type

Required

string

`providers[].kaniko.util.annotations`

providers > kaniko > util > annotations

Specify annotations to apply to each garden-util pod and deployments.

Type

Required

object

Example:

providers:
  - kaniko:
      ...
      util:
        ...
        annotations:
            cluster-autoscaler.kubernetes.io/safe-to-evict: 'false'

`providers[].kaniko.util.nodeSelector`

providers > kaniko > util > nodeSelector

Specify the nodeSelector constraints for each garden-util pod.

Type

Required

object

`providers[].defaultHostname`

providers > defaultHostname

A default hostname to use when no hostname is explicitly configured for a service.

Type

Required

string

Example:

providers:
  - defaultHostname: "api.mydomain.com"

`providers[].deploymentStrategy`

providers > deploymentStrategy

Experimental: this is an experimental feature and the API might change in the future.

Sets the deployment strategy for container services.

The default is "rolling", which performs rolling updates. There is also experimental support for blue/green deployments (via the "blue-green" strategy).

Note that this setting only applies to container services (and not, for example, kubernetes or helm services).

Type

Default

Required

string

"rolling"

`providers[].devMode`

providers > devMode

Configuration options for dev mode.

Type

Required

object

`providers[].devMode.defaults`

providers > devMode > defaults

Specifies default settings for dev mode syncs (e.g. for container, kubernetes and helm services).

These are overridden/extended by the settings of any individual dev mode sync specs for a given module or service.

Dev mode is enabled when running the garden dev command, and by setting the --dev flag on the garden deploy command.

See the Code Synchronization guide for more information.

Type

Required

object

`providers[].devMode.defaults.exclude[]`

providers > devMode > defaults > exclude

Specify a list of POSIX-style paths or glob patterns that should be excluded from the sync.

Any exclusion patterns defined in individual dev mode sync specs will be applied in addition to these patterns.

.git directories and .garden directories are always ignored.

Type

Required

array[posixPath]

Example:

providers:
  - devMode:
      ...
      defaults:
        ...
        exclude:
          - dist/**/*
          - '*.log'

`providers[].devMode.defaults.fileMode`

providers > devMode > defaults > fileMode

The default permission bits, specified as an octal, to set on files at the sync target. Defaults to 0600 (user read/write). See the Mutagen docs for more information.

Type

Required

number

`providers[].devMode.defaults.directoryMode`

providers > devMode > defaults > directoryMode

The default permission bits, specified as an octal, to set on directories at the sync target. Defaults to 0700 (user read/write). See the Mutagen docs for more information.

Type

Required

number

`providers[].devMode.defaults.owner`

providers > devMode > defaults > owner

Set the default owner of files and directories at the target. Specify either an integer ID or a string name. See the Mutagen docs for more information.

Type

Required

number | string

`providers[].devMode.defaults.group`

providers > devMode > defaults > group

Set the default group on files and directories at the target. Specify either an integer ID or a string name. See the Mutagen docs for more information.

Type

Required

number | string

`providers[].forceSsl`

providers > forceSsl

Require SSL on all container module services. If set to true, an error is raised when no certificate is available for a configured hostname on a container module.

Type

Default

Required

boolean

false

`providers[].imagePullSecrets[]`

providers > imagePullSecrets

References to docker-registry secrets to use for authenticating with remote registries when pulling images. This is necessary if you reference private images in your module configuration, and is required when configuring a remote Kubernetes environment with buildMode=local.

Type

Default

Required

array[object]

[]

`providers[].imagePullSecrets[].name`

providers > imagePullSecrets > name

The name of the Kubernetes secret.

Type

Required

string

Yes

Example:

providers:
  - imagePullSecrets:
      - name: "my-secret"

`providers[].imagePullSecrets[].namespace`

providers > imagePullSecrets > namespace

The namespace where the secret is stored. If necessary, the secret may be copied to the appropriate namespace before use.

Type

Default

Required

string

"default"

`providers[].copySecrets[]`

providers > copySecrets

References to secrets you need to have copied into all namespaces deployed to. These secrets will be ensured to exist in the namespace before deploying any service.

Type

Default

Required

array[object]

[]

`providers[].copySecrets[].name`

providers > copySecrets > name

The name of the Kubernetes secret.

Type

Required

string

Yes

Example:

providers:
  - copySecrets:
      - name: "my-secret"

`providers[].copySecrets[].namespace`

providers > copySecrets > namespace

The namespace where the secret is stored. If necessary, the secret may be copied to the appropriate namespace before use.

Type

Default

Required

string

"default"

`providers[].resources`

providers > resources

Resource requests and limits for the in-cluster builder, container registry and code sync service. (which are automatically installed and used when buildMode is cluster-docker or kaniko).

Type

Default

Required

object

{"builder":{"limits":{"cpu":4000,"memory":8192},"requests":{"cpu":100,"memory":512}},"registry":{"limits":{"cpu":2000,"memory":4096},"requests":{"cpu":200,"memory":512}},"sync":{"limits":{"cpu":500,"memory":512},"requests":{"cpu":100,"memory":90}},"util":{"limits":{"cpu":256,"memory":512},"requests":{"cpu":256,"memory":512}}}

`providers[].resources.builder`

providers > resources > builder

Resource requests and limits for the in-cluster builder. It's important to consider which build mode you're using when configuring this.

When buildMode is kaniko, this refers to each Kaniko pod, i.e. each individual build, so you'll want to consider the requirements for your individual image builds, with your most expensive/heavy images in mind.

When buildMode is cluster-buildkit, this applies to the BuildKit deployment created in each project namespace. So think of this as the resource spec for each individual user or project namespace.

When buildMode is cluster-docker, this applies to the single Docker Daemon that is installed and run cluster-wide. This is shared across all users and builds in the cluster, so it should be resourced accordingly, factoring in how many concurrent builds you expect and how heavy your builds tend to be. Note that the cluster-docker build mode has been deprecated!

Type

Default

Required

object

{"limits":{"cpu":4000,"memory":8192},"requests":{"cpu":100,"memory":512}}

`providers[].resources.builder.limits`

providers > resources > builder > limits

Type

Default

Required

object

{"cpu":4000,"memory":8192}

`providers[].resources.builder.limits.cpu`

providers > resources > builder > limits > cpu

CPU limit in millicpu.

Type

Default

Required

number

4000

Example:

providers:
  - resources:
      ...
      builder:
        ...
        limits:
          ...
          cpu: 4000

`providers[].resources.builder.limits.memory`

providers > resources > builder > limits > memory

Memory limit in megabytes.

Type

Default

Required

number

8192

Example:

providers:
  - resources:
      ...
      builder:
        ...
        limits:
          ...
          memory: 8192

`providers[].resources.builder.limits.ephemeralStorage`

providers > resources > builder > limits > ephemeralStorage

Ephemeral storage limit in megabytes.

Type

Required

number

Example:

providers:
  - resources:
      ...
      builder:
        ...
        limits:
          ...
          ephemeralStorage: 8192

`providers[].resources.builder.requests`

providers > resources > builder > requests

Type

Default

Required

object

{"cpu":100,"memory":512}

`providers[].resources.builder.requests.cpu`

providers > resources > builder > requests > cpu

CPU request in millicpu.

Type

Default

Required

number

100

Example:

providers:
  - resources:
      ...
      builder:
        ...
        requests:
          ...
          cpu: 100

`providers[].resources.builder.requests.memory`

providers > resources > builder > requests > memory

Memory request in megabytes.

Type

Default

Required

number

512

Example:

providers:
  - resources:
      ...
      builder:
        ...
        requests:
          ...
          memory: 512

`providers[].resources.builder.requests.ephemeralStorage`

providers > resources > builder > requests > ephemeralStorage

Ephemeral storage request in megabytes.

Type

Required

number

Example:

providers:
  - resources:
      ...
      builder:
        ...
        requests:
          ...
          ephemeralStorage: 8192

`providers[].resources.registry`

providers > resources > registry

Resource requests and limits for the in-cluster image registry. Built images are pushed to this registry, so that they are available to all the nodes in your cluster.

This is shared across all users and builds, so it should be resourced accordingly, factoring in how many concurrent builds you expect and how large your images tend to be.

Type

Default

Required

object

{"limits":{"cpu":2000,"memory":4096},"requests":{"cpu":200,"memory":512}}

`providers[].resources.registry.limits`

providers > resources > registry > limits

Type

Default

Required

object

{"cpu":2000,"memory":4096}

`providers[].resources.registry.limits.cpu`

providers > resources > registry > limits > cpu

CPU limit in millicpu.

Type

Default

Required

number

2000

Example:

providers:
  - resources:
      ...
      registry:
        ...
        limits:
          ...
          cpu: 2000

`providers[].resources.registry.limits.memory`

providers > resources > registry > limits > memory

Memory limit in megabytes.

Type

Default

Required

number

4096

Example:

providers:
  - resources:
      ...
      registry:
        ...
        limits:
          ...
          memory: 4096

`providers[].resources.registry.limits.ephemeralStorage`

providers > resources > registry > limits > ephemeralStorage

Ephemeral storage limit in megabytes.

Type

Required

number

Example:

providers:
  - resources:
      ...
      registry:
        ...
        limits:
          ...
          ephemeralStorage: 8192

`providers[].resources.registry.requests`

providers > resources > registry > requests

Type

Default

Required

object

{"cpu":200,"memory":512}

`providers[].resources.registry.requests.cpu`

providers > resources > registry > requests > cpu

CPU request in millicpu.

Type

Default

Required

number

200

Example:

providers:
  - resources:
      ...
      registry:
        ...
        requests:
          ...
          cpu: 200

`providers[].resources.registry.requests.memory`

providers > resources > registry > requests > memory

Memory request in megabytes.

Type

Default

Required

number

512

Example:

providers:
  - resources:
      ...
      registry:
        ...
        requests:
          ...
          memory: 512

`providers[].resources.registry.requests.ephemeralStorage`

providers > resources > registry > requests > ephemeralStorage

Ephemeral storage request in megabytes.

Type

Required

number

Example:

providers:
  - resources:
      ...
      registry:
        ...
        requests:
          ...
          ephemeralStorage: 8192

`providers[].resources.util`

providers > resources > util

Resource requests and limits for the util pod for in-cluster builders. This pod is used to get, start, stop and inquire the status of the builds.

This pod is created in each garden namespace.

Type

Default

Required

object

{"limits":{"cpu":256,"memory":512},"requests":{"cpu":256,"memory":512}}

`providers[].resources.util.limits`

providers > resources > util > limits

Type

Default

Required

object

{"cpu":256,"memory":512}

`providers[].resources.util.limits.cpu`

providers > resources > util > limits > cpu

CPU limit in millicpu.

Type

Default

Required

number

256

Example:

providers:
  - resources:
      ...
      util:
        ...
        limits:
          ...
          cpu: 256

`providers[].resources.util.limits.memory`

providers > resources > util > limits > memory

Memory limit in megabytes.

Type

Default

Required

number

512

Example:

providers:
  - resources:
      ...
      util:
        ...
        limits:
          ...
          memory: 512

`providers[].resources.util.limits.ephemeralStorage`

providers > resources > util > limits > ephemeralStorage

Ephemeral storage limit in megabytes.

Type

Required

number

Example:

providers:
  - resources:
      ...
      util:
        ...
        limits:
          ...
          ephemeralStorage: 8192

`providers[].resources.util.requests`

providers > resources > util > requests

Type

Default

Required

object

{"cpu":256,"memory":512}

`providers[].resources.util.requests.cpu`

providers > resources > util > requests > cpu

CPU request in millicpu.

Type

Default

Required

number

256

Example:

providers:
  - resources:
      ...
      util:
        ...
        requests:
          ...
          cpu: 256

`providers[].resources.util.requests.memory`

providers > resources > util > requests > memory

Memory request in megabytes.

Type

Default

Required

number

512

Example:

providers:
  - resources:
      ...
      util:
        ...
        requests:
          ...
          memory: 512

`providers[].resources.util.requests.ephemeralStorage`

providers > resources > util > requests > ephemeralStorage

Ephemeral storage request in megabytes.

Type

Required

number

Example:

providers:
  - resources:
      ...
      util:
        ...
        requests:
          ...
          ephemeralStorage: 8192

`providers[].resources.sync`

providers > resources > sync

Deprecated: This field will be removed in a future release.

Resource requests and limits for the code sync service, which we use to sync build contexts to the cluster ahead of building images. This generally is not resource intensive, but you might want to adjust the defaults if you have many concurrent users.

Type

Default

Required

object

{"limits":{"cpu":500,"memory":512},"requests":{"cpu":100,"memory":90}}

`providers[].resources.sync.limits`

providers > resources > sync > limits

Deprecated: This field will be removed in a future release.

Type

Default

Required

object

{"cpu":500,"memory":512}

`providers[].resources.sync.limits.cpu`

providers > resources > sync > limits > cpu

Deprecated: This field will be removed in a future release.

CPU limit in millicpu.

Type

Default

Required

number

500

Example:

providers:
  - resources:
      ...
      sync:
        ...
        limits:
          ...
          cpu: 500

`providers[].resources.sync.limits.memory`

providers > resources > sync > limits > memory

Deprecated: This field will be removed in a future release.

Memory limit in megabytes.

Type

Default

Required

number

512

Example:

providers:
  - resources:
      ...
      sync:
        ...
        limits:
          ...
          memory: 512

`providers[].resources.sync.limits.ephemeralStorage`

providers > resources > sync > limits > ephemeralStorage

Deprecated: This field will be removed in a future release.

Ephemeral storage limit in megabytes.

Type

Required

number

Example:

providers:
  - resources:
      ...
      sync:
        ...
        limits:
          ...
          ephemeralStorage: 8192

`providers[].resources.sync.requests`

providers > resources > sync > requests

Deprecated: This field will be removed in a future release.

Type

Default

Required

object

{"cpu":100,"memory":90}

`providers[].resources.sync.requests.cpu`

providers > resources > sync > requests > cpu

Deprecated: This field will be removed in a future release.

CPU request in millicpu.

Type

Default

Required

number

100

Example:

providers:
  - resources:
      ...
      sync:
        ...
        requests:
          ...
          cpu: 100

`providers[].resources.sync.requests.memory`

providers > resources > sync > requests > memory

Deprecated: This field will be removed in a future release.

Memory request in megabytes.

Type

Default

Required

number

90

Example:

providers:
  - resources:
      ...
      sync:
        ...
        requests:
          ...
          memory: 90

`providers[].resources.sync.requests.ephemeralStorage`

providers > resources > sync > requests > ephemeralStorage

Deprecated: This field will be removed in a future release.

Ephemeral storage request in megabytes.

Type

Required

number

Example:

providers:
  - resources:
      ...
      sync:
        ...
        requests:
          ...
          ephemeralStorage: 8192

`providers[].storage`

providers > storage

Storage parameters to set for the in-cluster builder, container registry and code sync persistent volumes (which are automatically installed and used when buildMode is cluster-docker or kaniko).

These are all shared cluster-wide across all users and builds, so they should be resourced accordingly, factoring in how many concurrent builds you expect and how large your images and build contexts tend to be.

Type

Default

Required

object

{"builder":{"size":20480,"storageClass":null},"nfs":{"storageClass":null},"registry":{"size":20480,"storageClass":null},"sync":{"size":10240,"storageClass":null}}

`providers[].storage.builder`

providers > storage > builder

Deprecated: This field will be removed in a future release.

Storage parameters for the data volume for the in-cluster Docker Daemon.

Only applies when buildMode is set to cluster-docker, ignored otherwise.

Type

Default

Required

object

{"size":20480,"storageClass":null}

`providers[].storage.builder.size`

providers > storage > builder > size

Deprecated: This field will be removed in a future release.

Volume size in megabytes.

Type

Default

Required

number

20480

`providers[].storage.builder.storageClass`

providers > storage > builder > storageClass

Deprecated: This field will be removed in a future release.

Storage class to use for the volume.

Type

Default

Required

string

null

`providers[].storage.nfs`

providers > storage > nfs

Deprecated: This field will be removed in a future release.

Storage parameters for the NFS provisioner, which we automatically create for the sync volume, unless you specify a storageClass for the sync volume. See the below sync parameter for more.

Only applies when buildMode is set to cluster-docker or kaniko, ignored otherwise.

Type

Default

Required

object

{"storageClass":null}

`providers[].storage.nfs.storageClass`

providers > storage > nfs > storageClass

Deprecated: This field will be removed in a future release.

Storage class to use as backing storage for NFS .

Type

Default

Required

string

null

`providers[].storage.registry`

providers > storage > registry

Storage parameters for the in-cluster Docker registry volume. Built images are stored here, so that they are available to all the nodes in your cluster.

Only applies when buildMode is set to cluster-docker or kaniko, ignored otherwise.

Type

Default

Required

object

{"size":20480,"storageClass":null}

`providers[].storage.registry.size`

providers > storage > registry > size

Volume size in megabytes.

Type

Default

Required

number

20480

`providers[].storage.registry.storageClass`

providers > storage > registry > storageClass

Storage class to use for the volume.

Type

Default

Required

string

null

`providers[].storage.sync`

providers > storage > sync

Deprecated: This field will be removed in a future release.

Storage parameters for the code sync volume, which build contexts are synced to ahead of running in-cluster builds.

Important: The storage class configured here has to support ReadWriteMany access. If you don't specify a storage class, Garden creates an NFS provisioner and provisions an NFS volume for the sync data volume.

Only applies when buildMode is set to cluster-docker, ignored otherwise.

Type

Default

Required

object

{"size":10240,"storageClass":null}

`providers[].storage.sync.size`

providers > storage > sync > size

Deprecated: This field will be removed in a future release.

Volume size in megabytes.

Type

Default

Required

number

10240

`providers[].storage.sync.storageClass`

providers > storage > sync > storageClass

Deprecated: This field will be removed in a future release.

Storage class to use for the volume.

Type

Default

Required

string

null

`providers[].tlsCertificates[]`

providers > tlsCertificates

One or more certificates to use for ingress.

Type

Default

Required

array[object]

[]

`providers[].tlsCertificates[].name`

providers > tlsCertificates > name

A unique identifier for this certificate.

Type

Required

string

Yes

Example:

providers:
  - tlsCertificates:
      - name: "www"

`providers[].tlsCertificates[].hostnames[]`

providers > tlsCertificates > hostnames

A list of hostnames that this certificate should be used for. If you don't specify these, they will be automatically read from the certificate.

Type

Required

array[hostname]

Example:

providers:
  - tlsCertificates:
      - hostnames:
          - www.mydomain.com

`providers[].tlsCertificates[].secretRef`

providers > tlsCertificates > secretRef

A reference to the Kubernetes secret that contains the TLS certificate and key for the domain.

Type

Required

object

Example:

providers:
  - tlsCertificates:
      - secretRef:
            name: my-tls-secret
            namespace: default

`providers[].tlsCertificates[].secretRef.name`

providers > tlsCertificates > secretRef > name

The name of the Kubernetes secret.

Type

Required

string

Yes

Example:

providers:
  - tlsCertificates:
      - secretRef:
            name: my-tls-secret
            namespace: default
          ...
          name: "my-secret"

`providers[].tlsCertificates[].secretRef.namespace`

providers > tlsCertificates > secretRef > namespace

The namespace where the secret is stored. If necessary, the secret may be copied to the appropriate namespace before use.

Type

Default

Required

string

"default"

`providers[].tlsCertificates[].managedBy`

providers > tlsCertificates > managedBy

Deprecated: This field will be removed in a future release.

Set to cert-manager to configure cert-manager to manage this certificate. See our cert-manager integration guide for details.

Type

Required

string

Example:

providers:
  - tlsCertificates:
      - managedBy: "cert-manager"

`providers[].certManager`

providers > certManager

Deprecated: This field will be removed in a future release.

cert-manager configuration, for creating and managing TLS certificates. See the cert-manager guide for details.

Type

Required

object

`providers[].certManager.install`

providers > certManager > install

Deprecated: This field will be removed in a future release.

Automatically install cert-manager on initialization. See the cert-manager integration guide for details.

Type

Default

Required

boolean

false

`providers[].certManager.email`

providers > certManager > email

Deprecated: This field will be removed in a future release.

The email to use when requesting Let's Encrypt certificates.

Type

Required

string

Yes

Example:

providers:
  - certManager:
      ...
      email: "[email protected]"

`providers[].certManager.issuer`

providers > certManager > issuer

Deprecated: This field will be removed in a future release.

The type of issuer for the certificate (only ACME is supported for now).

Type

Default

Required

string

"acme"

Example:

providers:
  - certManager:
      ...
      issuer: "acme"

`providers[].certManager.acmeServer`

providers > certManager > acmeServer

Deprecated: This field will be removed in a future release.

Specify which ACME server to request certificates from. Currently Let's Encrypt staging and prod servers are supported.

Type

Default

Required

string

"letsencrypt-staging"

Example:

providers:
  - certManager:
      ...
      acmeServer: "letsencrypt-staging"

`providers[].certManager.acmeChallengeType`

providers > certManager > acmeChallengeType

Deprecated: This field will be removed in a future release.

The type of ACME challenge used to validate hostnames and generate the certificates (only HTTP-01 is supported for now).

Type

Default

Required

string

"HTTP-01"

Example:

providers:
  - certManager:
      ...
      acmeChallengeType: "HTTP-01"

`providers[].systemNodeSelector`

providers > systemNodeSelector

Exposes the nodeSelector field on the PodSpec of system services. This allows you to constrain the system services to only run on particular nodes.

See here for the official Kubernetes guide to assigning Pods to nodes.

Type

Default

Required

object

{}

Example:

providers:
  - systemNodeSelector:
        disktype: ssd

`providers[].registryProxyTolerations[]`

providers > registryProxyTolerations

For setting tolerations on the registry-proxy when using in-cluster building. The registry-proxy is a DaemonSet that proxies connections to the docker registry service on each node.

Use this only if you're doing in-cluster building and the nodes in your cluster have taints.

Type

Default

Required

array[object]

[]

`providers[].registryProxyTolerations[].effect`

providers > registryProxyTolerations > effect

"Effect" indicates the taint effect to match. Empty means match all taint effects. When specified, allowed values are "NoSchedule", "PreferNoSchedule" and "NoExecute".

Type

Required

string

`providers[].registryProxyTolerations[].key`

providers > registryProxyTolerations > key

"Key" is the taint key that the toleration applies to. Empty means match all taint keys. If the key is empty, operator must be "Exists"; this combination means to match all values and all keys.

Type

Required

string

`providers[].registryProxyTolerations[].operator`

providers > registryProxyTolerations > operator

Type

Default

Required

string

"Equal"

`providers[].registryProxyTolerations[].tolerationSeconds`

providers > registryProxyTolerations > tolerationSeconds

Type

Required

string

`providers[].registryProxyTolerations[].value`

providers > registryProxyTolerations > value

"Value" is the taint value the toleration matches to. If the operator is "Exists", the value should be empty, otherwise just a regular string.

Type

Required

string

`providers[].name`

providers > name

The name of the provider plugin to use.

Type

Default

Required

string

"local-kubernetes"

Yes

Example:

providers:
  - name: "local-kubernetes"

`providers[].context`

providers > context

The kubectl context to use to connect to the Kubernetes cluster.

Type

Required

string

Example:

providers:
  - context: "my-dev-context"

`providers[].namespace`

providers > namespace

Specify which namespace to deploy services to (defaults to the project name). Note that the framework generates other namespaces as well with this name as a prefix.

Type

Required

object | string

`providers[].namespace.name`

providers > namespace > name

A valid Kubernetes namespace name. Must be a valid RFC1035/RFC1123 (DNS) label (may contain lowercase letters, numbers and dashes, must start with a letter, and cannot end with a dash) and must not be longer than 63 characters.

Type

Required

string

`providers[].namespace.annotations`

providers > namespace > annotations

Map of annotations to apply to the namespace when creating it.

Type

Required

object

Example:

providers:
  - namespace: ''
      ...
      annotations:
          cluster-autoscaler.kubernetes.io/safe-to-evict: 'false'

`providers[].namespace.labels`

providers > namespace > labels

Map of labels to apply to the namespace when creating it.

Type

Required

object

`providers[].setupIngressController`

providers > setupIngressController

Set this to null or false to skip installing/enabling the nginx ingress controller.

Type

Default

Required

string

"nginx"

Previouskubernetes Nextmaven-container

Last updated 2 years ago

Was this helpful?

hashtagDescription

hashtagComplete YAML Schema

hashtagConfiguration Keys

hashtagproviders[]

hashtagproviders[].dependencies[]

hashtagproviders[].environments[]

hashtagproviders[].buildMode

hashtagproviders[].clusterBuildkit

hashtagproviders[].clusterBuildkit.cache[]

hashtagproviders[].clusterBuildkit.cache[].type

hashtagproviders[].clusterBuildkit.cache[].registry

hashtagproviders[].clusterBuildkit.cache[].registry.hostname

hashtagproviders[].clusterBuildkit.cache[].registry.port

hashtagproviders[].clusterBuildkit.cache[].registry.namespace

hashtagproviders[].clusterBuildkit.cache[].registry.insecure

hashtagproviders[].clusterBuildkit.cache[].mode

hashtagproviders[].clusterBuildkit.cache[].tag

hashtagproviders[].clusterBuildkit.cache[].export

hashtagproviders[].clusterBuildkit.rootless

hashtagproviders[].clusterBuildkit.nodeSelector

hashtagproviders[].clusterBuildkit.tolerations[]

hashtagproviders[].clusterBuildkit.tolerations[].effect

hashtagproviders[].clusterBuildkit.tolerations[].key

hashtagproviders[].clusterBuildkit.tolerations[].operator

hashtagproviders[].clusterBuildkit.tolerations[].tolerationSeconds

hashtagproviders[].clusterBuildkit.tolerations[].value

hashtagproviders[].clusterBuildkit.annotations

hashtagproviders[].clusterDocker

hashtagproviders[].clusterDocker.enableBuildKit

hashtagproviders[].jib

hashtagproviders[].jib.pushViaCluster

hashtagproviders[].kaniko

hashtagproviders[].kaniko.extraFlags[]

hashtagproviders[].kaniko.image

hashtagproviders[].kaniko.namespace

hashtagproviders[].kaniko.nodeSelector

hashtagproviders[].kaniko.tolerations[]

hashtagproviders[].kaniko.tolerations[].effect

hashtagproviders[].kaniko.tolerations[].key

hashtagproviders[].kaniko.tolerations[].operator

hashtagproviders[].kaniko.tolerations[].tolerationSeconds

hashtagproviders[].kaniko.tolerations[].value

hashtagproviders[].kaniko.annotations

hashtagproviders[].kaniko.util

hashtagproviders[].kaniko.util.tolerations[]

hashtagproviders[].kaniko.util.tolerations[].effect

hashtagproviders[].kaniko.util.tolerations[].key

hashtagproviders[].kaniko.util.tolerations[].operator

hashtagproviders[].kaniko.util.tolerations[].tolerationSeconds

hashtagproviders[].kaniko.util.tolerations[].value

hashtagproviders[].kaniko.util.annotations

hashtagproviders[].kaniko.util.nodeSelector

hashtagproviders[].defaultHostname

hashtagproviders[].deploymentStrategy

hashtagproviders[].devMode

hashtagproviders[].devMode.defaults

hashtagproviders[].devMode.defaults.exclude[]

hashtagproviders[].devMode.defaults.fileMode

hashtagproviders[].devMode.defaults.directoryMode

hashtagproviders[].devMode.defaults.owner

hashtagproviders[].devMode.defaults.group

hashtagproviders[].forceSsl

hashtagproviders[].imagePullSecrets[]

hashtagproviders[].imagePullSecrets[].name

hashtagproviders[].imagePullSecrets[].namespace

hashtagproviders[].copySecrets[]

hashtagproviders[].copySecrets[].name

hashtagproviders[].copySecrets[].namespace

hashtagproviders[].resources

hashtagproviders[].resources.builder

hashtagproviders[].resources.builder.limits

hashtagproviders[].resources.builder.limits.cpu

hashtagproviders[].resources.builder.limits.memory

hashtagproviders[].resources.builder.limits.ephemeralStorage

hashtagproviders[].resources.builder.requests

hashtagproviders[].resources.builder.requests.cpu

hashtagproviders[].resources.builder.requests.memory

hashtagproviders[].resources.builder.requests.ephemeralStorage

hashtagproviders[].resources.registry

hashtagproviders[].resources.registry.limits

Description

Complete YAML Schema

Configuration Keys

`providers[]`

`providers[].dependencies[]`

`providers[].environments[]`

`providers[].buildMode`

`providers[].clusterBuildkit`

`providers[].clusterBuildkit.cache[]`

`providers[].clusterBuildkit.cache[].type`

`providers[].clusterBuildkit.cache[].registry`

`providers[].clusterBuildkit.cache[].registry.hostname`

`providers[].clusterBuildkit.cache[].registry.port`

`providers[].clusterBuildkit.cache[].registry.namespace`

`providers[].clusterBuildkit.cache[].registry.insecure`

`providers[].clusterBuildkit.cache[].mode`

`providers[].clusterBuildkit.cache[].tag`

`providers[].clusterBuildkit.cache[].export`

`providers[].clusterBuildkit.rootless`

`providers[].clusterBuildkit.nodeSelector`

`providers[].clusterBuildkit.tolerations[]`

`providers[].clusterBuildkit.tolerations[].effect`

`providers[].clusterBuildkit.tolerations[].key`

`providers[].clusterBuildkit.tolerations[].operator`

`providers[].clusterBuildkit.tolerations[].tolerationSeconds`

`providers[].clusterBuildkit.tolerations[].value`

`providers[].clusterBuildkit.annotations`

`providers[].clusterDocker`

`providers[].clusterDocker.enableBuildKit`

`providers[].jib`

`providers[].jib.pushViaCluster`

`providers[].kaniko`

`providers[].kaniko.extraFlags[]`

`providers[].kaniko.image`

`providers[].kaniko.namespace`

`providers[].kaniko.nodeSelector`

`providers[].kaniko.tolerations[]`

`providers[].kaniko.tolerations[].effect`

`providers[].kaniko.tolerations[].key`

`providers[].kaniko.tolerations[].operator`

`providers[].kaniko.tolerations[].tolerationSeconds`

`providers[].kaniko.tolerations[].value`

`providers[].kaniko.annotations`

`providers[].kaniko.util`

`providers[].kaniko.util.tolerations[]`

`providers[].kaniko.util.tolerations[].effect`

`providers[].kaniko.util.tolerations[].key`

`providers[].kaniko.util.tolerations[].operator`

`providers[].kaniko.util.tolerations[].tolerationSeconds`

`providers[].kaniko.util.tolerations[].value`

`providers[].kaniko.util.annotations`

`providers[].kaniko.util.nodeSelector`

`providers[].defaultHostname`

`providers[].deploymentStrategy`

`providers[].devMode`

`providers[].devMode.defaults`

`providers[].devMode.defaults.exclude[]`

`providers[].devMode.defaults.fileMode`

`providers[].devMode.defaults.directoryMode`

`providers[].devMode.defaults.owner`

`providers[].devMode.defaults.group`

`providers[].forceSsl`

`providers[].imagePullSecrets[]`

`providers[].imagePullSecrets[].name`

`providers[].imagePullSecrets[].namespace`

`providers[].copySecrets[]`

`providers[].copySecrets[].name`

`providers[].copySecrets[].namespace`

`providers[].resources`

`providers[].resources.builder`

`providers[].resources.builder.limits`

`providers[].resources.builder.limits.cpu`

`providers[].resources.builder.limits.memory`

`providers[].resources.builder.limits.ephemeralStorage`

`providers[].resources.builder.requests`

`providers[].resources.builder.requests.cpu`

`providers[].resources.builder.requests.memory`

`providers[].resources.builder.requests.ephemeralStorage`

`providers[].resources.registry`

`providers[].resources.registry.limits`