Welcome to KubeCFG Documentation

These Docs are a Work in progress and not complete by any means - PRs are welcome

Kubecfg is a tool for managing Kubernetes resources as code.

kubecfg allows you to express the patterns across your infrastructure and reuse these powerful “templates” across many services, and then manage those templates as files in version control. The more complex your infrastructure is, the more you will gain from using kubecfg.

Features

• Supports JSON, YAML or jsonnet files (by file suffix).
• Best-effort sorts objects before updating, so that dependencies are pushed to the server before objects that refer to them.
• Additional jsonnet builtin functions. See lib/kubecfg.libsonnet.
• Optional “garbage collection” of objects removed from config (see --gc-tag).
• TODO: ADD More features here

Infrastructure-as-code Philosophy

The idea is to describe as much as possible about your configuration as files in version control (eg: git).

Changes to the configuration follow a regular review, approve, merge, etc code change workflow (github pull-requests, phabricator diffs, etc). At any point, the config in version control captures the entire desired-state, so the system can be easily recreated in a QA cluster or to recover from disaster.

Jsonnet

Kubecfg relies heavily on jsonnet to describe Kubernetes resources, and is really just a thin Kubernetes-specific wrapper around jsonnet evaluation. You should read the jsonnet tutorial, and skim the functions available in the jsonnet std library.

Installation

Pre-compiled executables exist for some platforms on the Github releases page.

On macOS, it can also be installed via Homebrew: brew install kubecfg

To build from source:

% PATH=$PATH:$GOPATH/bin
% go get github.com/kubecfg/kubecfg

TODO: Add docker info

Basic Usage

Quickstart

All commands in this quickstart are run in the examples directory using the guestbook code

WARNING All commands will use the currently defined context similar to kubectl , be sure not to run these commands with the wrong context

# Show generated YAML
% kubecfg show -o yaml examples/guestbook.jsonnet

# Create resources in the kubernetes cluster defined in the current context
% kubecfg update examples/guestbook.jsonnet

# Modify _something_ (downgrade gb-frontend image)
% sed -i.bak '\,gcr.io/google-samples/gb-frontend,s/:v4/:v3/' examples/guestbook.jsonnet

# See differences vs server
% kubecfg diff examples/guestbook.jsonnet

# Update the kubernetes resources to the new configuration
% kubecfg update examples/guestbook.jsonnet

# Clean up after demo
% kubecfg delete examples/guestbook.jsonnet

Show

Show expanded resource definitions

It will traverse the visible elements in the jsonnet output and render any object that looks like a kubernetes resource

• has a kind field
• has an apiVersion field

Help

Show expanded resource definitions                                                                                                                                                                                 
                                                                                                         
Usage:                                                                                                                                                                                                             
  kubecfg show [flags]                                                                                                                                                                                             
                                                                                                         
Flags:                                                                                                   
  -e, --exec string                        Inline code
      --export-dir string                  Split yaml stream into multiple files and write files into a directory. If the directory exists it must be empty.
      --export-filename-extension string   Override the file extension used when creating filenames when using export-filename-format
      --export-filename-format string      Go template expression used to render path names for resources. (default "{{.apiVersion}}.{{.kind}}-{{default \"default\" .metadata.namespace}}.{{.metadata.name}}")
  -o, --format string                      Output format.  Supported values are: json, yaml (default "yaml")
  -h, --help                               help for show
      --overlay-code string                Inline Jsonnet code to compose to each of the input files
      --overlay-code-file string           Jsonnet file to compose to each of the input files
      --reorder string                     --reorder=server: Reorder resources like the 'update' command does. --reorder=client: TODO
      --show-provenance                    Add provenance annotations showing the file and the field path to each rendered k8s object

Jsonnet Code

local kubecfg = import 'kubecfg.libsonnet';

{
  local outer = self,

  container:: {
    name: 'busybox',
    image: 'busybox:latest',
  },

  deployment: {
    local this = self,

    apiVersion: 'extensions/v1beta1',
    kind: 'Deployment',
    metadata: {
      name: 'busybox',
      labels: { name: 'busybox' },
    },
    spec: {
      replicas: 1,
      template: {
        metadata: { labels: this.metadata.labels },
        spec: {
          containers: [outer.container],
        },
      },
    },
  },
}

Basic Rendering

render jsonnet to stdout in YAML format

kubecfg show source.jsonnet

---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    name: busybox
  name: busybox
spec:
  replicas: 1
  template:
    metadata:
      labels:
        name: busybox
    spec:
      containers:
      - image: busybox:latest
        name: busybox

render to stdout in JSON format

kubecfg show -o json source.jsonnet

{
  "apiVersion": "extensions/v1beta1",
  "kind": "Deployment",
  "metadata": {
    "labels": {
      "name": "busybox"
    },
    "name": "busybox"
  },
  "spec": {
    "replicas": 1,
    "template": {
      "metadata": {
        "labels": {
          "name": "busybox"
        }
      },
      "spec": {
        "containers": [
          {
            "image": "busybox:latest",
            "name": "busybox"
          }
        ]
      }
    }
  }
}

Export Manifests to files

It is possible to export the manifests as a set of split files ( 1 per resource ) to a directory. This can be usefl when using other tools , like Flux or ArgoCD, to actually deploy manifests to a cluster

kubecfg show --export-dir output/ --export-filename-format "{{.apiVersion}}.{{.kind}}-{{default \"default\" .metadata.namespace}}.{{.metadata.name}}" show.jsonnet

➜ ls -1 output 
extensions-v1beta1.Deployment-default.busybox.yaml

➜ cat output/extensions-v1beta1.Deployment-default.busybox.yaml 
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    name: busybox
  name: busybox
spec:
  replicas: 1
  template:
    metadata:
      labels:
        name: busybox
    spec:
      containers:
      - image: busybox:latest
        name: busybox

Inline usage

It is possible to exec inline code with kubecfg, this is useful for debugging purposes

kubecfg show -e "(import 'show.jsonnet') + { deployment+: { metadata+: {namespace: 'foo'}}}"

---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    name: busybox
  name: busybox
  namespace: foo
spec:
  replicas: 1
  template:
    metadata:
      labels:
        name: busybox
    spec:
      containers:
      - image: busybox:latest
        name: busybox

Inline code (-e, --overlay-code, --ext-code and --tla-code) can use kubecfg without importing kubecfg.libsonnet first. See Using kubecfg without importing it.

Eval

Alpha Feature: turn it on with --features=eval (see Feature gates).

Eval is very similar to show but it differs in a fundamental way, it does not traverse the code in search of kubernetes manifests, it instead behaves more like jsonnet command and does just evaluate the jsonnet code to output

This is useful in order to troubleshoot some jsonnet code that , because of the use of kubecfg native extension, cannot be rendered using standard jsonnet

Jsonnet Code

local kubecfg = import 'kubecfg.libsonnet';

{
  local outer = self,

  container:: {
    name: 'busybox',
    image: 'busybox:latest',
  },

  deployment: {
    local this = self,

    apiVersion: 'extensions/v1beta1',
    kind: 'Deployment',
    metadata: {
      name: 'busybox',
      labels: { name: 'busybox' },
    },
    spec: {
      replicas: 3,
      template: {
        metadata: { labels: this.metadata.labels },
        spec: {
          containers: [outer.container],
        },
      },
    },
  },
}

Basic Rendering

render jsonnet to stdout in YAML format

kubecfg --features=eval eval source.jsonnet

➜ kubecfg --features=eval eval code.jsonnet
deployment:
  apiVersion: extensions/v1beta1
  kind: Deployment
  metadata:
    labels:
      name: busybox
    name: busybox
  spec:
    replicas: 3
    template:
      metadata:
        labels:
          name: busybox
      spec:
        containers:
        - image: busybox:latest
          name: busybox

render to stdout in JSON format

kubecfg --features=eval eval -o json source.jsonnet

{
   "deployment": {
      "apiVersion": "extensions/v1beta1",
      "kind": "Deployment",
      "metadata": {
         "labels": {
            "name": "busybox"
         },
         "name": "busybox"
      },
      "spec": {
         "replicas": 3,
         "template": {
            "metadata": {
               "labels": {
                  "name": "busybox"
               }
            },
            "spec": {
               "containers": [
                  {
                     "image": "busybox:latest",
                     "name": "busybox"
                  }
               ]
            }
         }
      }
   }
}

Evaluating an expression

-e evaluates an expression instead of the whole file. In the expression, $ is the value of the file:

$ kubecfg --features=eval eval code.jsonnet -e '$.deployment.spec.replicas'
3

Using kubecfg without importing it

In the expression, kubecfg is already defined, so you can call the functions of kubecfg.libsonnet without importing it:

$ kubecfg --features=eval eval code.jsonnet -e 'kubecfg.fold(kubecfg.layouts.gvkName, $, {})'
extensions/v1beta1.Deployment:
  busybox:
    apiVersion: extensions/v1beta1
    ...

It’s always the copy of kubecfg.libsonnet built into kubecfg. A file with that name on the search path (-J) or next to the evaluated file doesn’t change it. A name you define yourself wins, such as a local kubecfg or a top-level argument named kubecfg. So to use another copy, import it: -e 'local kubecfg = import "kubecfg.libsonnet"; ...' finds it the usual way.

The same goes for the code of --exec, --overlay-code, --ext-code and --tla-code, and for the expressions in YAKL files. It doesn’t for Jsonnet files: they still need the import, so editors, linters and other Jsonnet tools can read them. RFC06 explains why.

Seeing the Jsonnet of a YAKL file

kubecfg --features=eval,yakl eval -o jsonnet app.yakl prints the Jsonnet that a YAKL file compiles to, without evaluating it. See Seeing the generated Jsonnet.

Update

TODO

Delete

TODO

Diff

TODO

Validate

TODO

Advanced Usage

Helm Integration

Supported: from version v0.28.0

Kubecfg supports rendering an Helm chart into Jsonnet at runtime using the parseHelmChart native extension.

• the Helm Chart source code must be vendored and imported as a binary blob using importbin
• the values passed to the Helm Chart are just a jsonnet object that can be created using any jsonnet feature
• the standard output of the parseHelmChart function is a flat objects with the helm template path as a key
  • Some functions are provided in kubecfg.libsonnet to convert the output to a more useful structure
    • kubecfg.layout.gvkName - Helper for ‘fold’. This accumulates a two-level collection of objects by ‘apiVersion.kind’ (GroupVersionKind) and then object ‘name’.
    • kubecfg.layout.gvkNsName - Helper for ‘fold’. This accumulates a three-level collection of objects by ‘apiVersion.kind’ (GroupVersionKind), object ‘namespace’, and then object ‘name’. Namespace is ‘_’ for non-namespaced objects.

Vendoring helm chart version

Vendor 2 versions of the helm chart in the local directory

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
helm pull ingress-nginx/ingress-nginx --version 4.7.1
helm pull ingress-nginx/ingress-nginx --version 4.6.1

➜ ls -1
ingress-nginx-4.6.1.tgz
ingress-nginx-4.7.1.tgz

Jsonnet Code

local kubecfg = import 'kubecfg.libsonnet';

// To support multiple versions they need to be indepently vendored and imported 
// the selection will be done by passing the right key into `parseHelmChart`
local data = {
  '4.7.1': importbin './ingress-nginx-4.7.1.tgz',
  '4.6.1': importbin './ingress-nginx-4.6.1.tgz',
};

{
  _config:: {
    version: '4.6.1',
  },

  values:: {
    nameOverride: 'nginx-example',
    fullNameOverride: 'nginx-example',
    commonLabels: { app: 'nginx-example' },
    controller: {
      minReadySeconds: 10,
    },
  },

  valuesByVersion:: {
    '4.7.1': {
      commonLabels+: { extra: 'label' },
    },
    '4.6.1': {},
  },

  renderHelm::
    kubecfg.fold(
      kubecfg.layouts.gvkNsName,  // Render list output of ParseHelmChart into a hierarchy layout
      kubecfg.parseHelmChart(
        data[$._config.version],
        'nginx-example',
        'ingress',
        $.values + $.valuesByVersion[$._config.version]
      ),
      {}
    ),

  // Extend the Helm rendering by using Jsonnet
  // Access the resources based on the GVK NS NAME hirerarchy

  objects: $.renderHelm {
    'apps/v1.Deployment'+: {  //  apiversion.kind
      ingress+: {  // resource namespace
        'nginx-example-controller'+: {  // resource name
          spec+: {
            replicas: 20,
          },
        },
      },
    },
  },
}

Code WalkThrough

• Importing the helm chart bundle

local data = {
  '4.7.1': importbin './ingress-nginx-4.7.1.tgz',
  '4.6.1': importbin './ingress-nginx-4.6.1.tgz',
  };

Jsonnet does not support dynamic imports so all versions that you might want to render must be imported separately

• Configuration of the helm values and version

  _config:: {
    version: '4.6.1',
  },

  values:: {
    nameOverride: 'nginx-example',
    fullNameOverride: 'nginx-example',
    commonLabels: { app: 'nginx-example' },
    controller: {
      minReadySeconds: 10,
    },
  },

  valuesByVersion:: {
    '4.7.1': {
      commonLabels+: { extra: 'label' },
    },
    '4.6.1': {},
  },

We define a common values in jsonnet syntax and a per-version overlay , we also define a version key in the _config object to use to render the specific version and overlay the right configurations

• render the Helm chart into jsonnet using kubecfg

  renderHelm::
    kubecfg.fold(
      kubecfg.layouts.gvkNsName,  // Render list output of ParseHelmChart into a hierarchy layout
      kubecfg.parseHelmChart(
        data[$._config.version],
        'nginx-example',
        'ingress',
        $.values + $.valuesByVersion[$._config.version]
      ),
      {}
    ),

Here we wrap the actual kubecfg.parseHelmChart into a set of helper functions to get an output that is more flexible to be further extended using jsonnet since the output from the parseHelmChart function is a flat list of manifests that jsonnet is not very good at managing and mangling any further.

from the inside to the outside

• we run kubecfg.parseHelmChart passing
  • the binary import of the **data[$._config.version]** helm chart bundle. by setting the right version at runtime we can render different version of the helm chart
  • the revision name, as expected by helm
  • the namespace, as expected by helm
  • the values to pass to helm, here we use jsonnet to overlay the common $.values with the version specific values
• we call the kubecfg.fold function passsing which will iterate over the list ( the output of parseHelmChart ) and for each element will call the layout function and add the output to the {} empty dict
  • the layout we want to use gvkNsName or gvkName
  • the output of parseHelmChart
  • the initial empty dict to add elements to

the output of such a wrap in renderHelm key is a structure like

renderHelm:
  "apps.v1": # GROUP.VERSION
    "deployments": # KIND
      "ingress": # NAMESPACE
        "ingress-nginx": # NAME
  "v1": #GROUP.VERSION
    "Secrets": # KIND
      "ingress": # NAMESPACE
        "some_secret_name": # NAME
    "ConfigMaps": # KIND
      "ingress": # NAMESPACE
        "some_configmap_name": # NAME

This structure is now easy to further work on using jsonnet

{
  "renderHelm": "...",

  "objects": renderHelm + {
    "apps.v1"+: {
      "deployments"+: {
        "ingress"+: {
          "ingress-nginx"+: {
            "metadata"+: { "labels"+: { "a_label": "a_value" } },
          }
        }
      }
    }

  }
}

Render and Export manifests

kubecfg show --export-dir output/ show.jsonnet

➜ ls -1 output 
admissionregistration.k8s.io-v1.ValidatingWebhookConfiguration-ingress.nginx-example-admission.yaml
apps-v1.Deployment-ingress.nginx-example-controller.yaml
batch-v1.Job-ingress.nginx-example-admission-create.yaml
batch-v1.Job-ingress.nginx-example-admission-patch.yaml
networking.k8s.io-v1.IngressClass-ingress.nginx.yaml
rbac.authorization.k8s.io-v1.ClusterRole-ingress.nginx-example-admission.yaml
rbac.authorization.k8s.io-v1.ClusterRole-ingress.nginx-example.yaml
rbac.authorization.k8s.io-v1.ClusterRoleBinding-ingress.nginx-example-admission.yaml
rbac.authorization.k8s.io-v1.ClusterRoleBinding-ingress.nginx-example.yaml
rbac.authorization.k8s.io-v1.Role-ingress.nginx-example-admission.yaml
rbac.authorization.k8s.io-v1.Role-ingress.nginx-example.yaml
rbac.authorization.k8s.io-v1.RoleBinding-ingress.nginx-example-admission.yaml
rbac.authorization.k8s.io-v1.RoleBinding-ingress.nginx-example.yaml
v1.ConfigMap-ingress.nginx-example-controller.yaml
v1.Service-ingress.nginx-example-controller-admission.yaml
v1.Service-ingress.nginx-example-controller.yaml
v1.ServiceAccount-ingress.nginx-example-admission.yaml
v1.ServiceAccount-ingress.nginx-example.yaml

Metacontroller Integration

Supported: from version v0.29.0

Alpha Feature: turn it on with --features=httpd (see Feature gates).

Metacontroller is an add-on for Kubernetes that makes it easy to write and deploy custom controllers.

For a more detailed explanation of how Metacontroller works and the many options of controllers it support please read the Upstream Documentation

Kubecfg integrates with Metacontroller by exposing an httpd endpoint that accept POSTs from the metacontroller and respond with the expected JSON object. By leveraging kubecfg to produce the response you can use all the extra features provided on top of standard jsonnet.

Example usage

The following example will use code from metacontroller-example

Overview

kubecfg supports exposing individual .jsonnet files over https as hooks by rendering the TopLevelFunction in those jsonnet files as expected by the metacontroller

the functionality is exposed by running kubecfg --features=httpd httpd <jsonnet files>

• each jsonnet file must expose a TopLevelFunction with a request parameter that will be called by kubecfg when the hook is called with a POST
• kubecfg --features=httpd httpd sync.jsonnet - will expose on port :8080/sync the TopLevelFunction defined in the sync.jsonnet
• kubecfg --features=httpd httpd sync-pods.jsonnet sync-services.jsonnet - will expose multiple endpoints :8080/sync-pods and :8080/sync-jsonnet calling each individial TLF for each files

See sync.jsonnet in the metacontroller example for a working example

Flow Diagram

sequenceDiagram
  autonumber
  kubectl->>apiserver: Post CR Instance
  loop Validation
      apiserver->>apiserver: Validate CR vs CRD
  end
  loop ETCd
      apiserver->>apiserver: CR Persisted
  end
  metacontroller->>apiserver: Watch for instances of CRD
  Note right of metacontroller: CompositeController defines hooks
  metacontroller->>kubecfg-httpd: POST request to kubecfg-httpd
  Note right of kubecfg-httpd: kubecfg render json response
  kubecfg-httpd-->>metacontroller: response with manifests to create
  metacontroller-->>apiserver: Manifests to Create

Create base cluster

Create a kind cluster using the provided Makefile in the examples directory

➜ cd examples/metacontroller
➜ make kind
...

➜ kubectl rollout status --timeout 180s -n metacontroller statefulset/metacontroller
partitioned roll out complete: 1 new pods have been updated...

Kubecfg Controller

Create the example useless-controller

➜ make controller

➜ kubectl rollout status --timeout 180s -n metacontroller deployment/useless-controller
deployment "useless-controller" successfully rolled out

this Controller uses the CRD defined in v1/crdv1.yaml ( from api/types.go ) and the compositeController defined in the manifests directory

CompositeController

---
apiVersion: metacontroller.k8s.io/v1alpha1
kind: CompositeController
metadata:
  name: useless-controller
spec:
  generateSelector: true
  parentResource:
    apiVersion: example.com/v1
    resource: uselesspods
    revisionHistory:
      fieldPaths:
      - spec.name
  childResources:
  - apiVersion: apps/v1
    resource: deployments
    updateStrategy:
      method: InPlace
  hooks:
    sync:
      webhook:
        url: http://useless-controller.metacontroller/sync

the compositeController defines how the metacontroller will watch for example.com/v1/uselesspods resources and post the request, with the CR specs, to the kubecfg httpd controller which will execute the sync.jsonnet code

sync.jsonnet hook

the code of the hook can be examined in the examples directory sync.jsonnet

the hook must have a Top Level Function which is executed on each POST passing the request body as json

local process = function(request) {
...
...
  deployment:: { ... },

  resyncAfterSeconds: 30.0,
    status: {
        observedGeneration: std.get(request.parent.metadata, 'generation', 0),
        ready: if std.length(std.objectFields(request.children)) > 0 then 'true' else 'false',
    },
    children: [
      $.deployment,
    ],
}


//Top Level Function
function(request)
  local response = process(request);
    std.trace('request: ' + std.manifestJsonEx(request, '  ') + '\n\nresponse: ' + std.manifestJsonEx(response, '  '), response)

every object returned in the children key will be applied to Kubernetes by the metacontroller

Example

When an instance of the UselessPod is created

1. ApiServer will validate the CR against the CRD definition
2. Persist the CR in etcd
3. Metacontroller watches the UselessPods CRs
4. when an event on a watched CR is observed a POST request will be sent by the metacontroller to the useless-controller with the CR as part of the request
5. the useless-controller will run kubecfg on the sync.jsonnet code Top Level Function using the request as parameter
6. the response to the metacontroller will include a deployment in the children field
7. the metacontroller will apply the children resources ( the deployment ) to the apiserver
8. kubernetes will create the generated deployment

➜ cat << EOF | kubectl apply -f -           
apiVersion: "example.com/v1"
kind: UselessPod
metadata:
  name: test1
  namespace: default
spec:
  name: instance1
EOF
uselesspod.example.com/test1 created

➜ kubectl rollout status -n default deployment/useless-instance1 
deployment "useless-instance1" successfully rolled out

Cleanup

➜ make clean

OCI Support

TODO

Provenance and Tracking

When manifests are generated from a big , nested, codebase it can be hard sometimes to identify where in jsonnet a specific manifest was generated or where a specific field was defined.

kubecfg supports 2 functionatilities to help with this

Provenance

when rendering kubernetes manifests kubecfg can add a custom annotations to help identify which file and which key in the file was used to generate the manifest

kubecfg show code.jsonnet --show-provenance

➜ cat code.jsonnet
local kubecfg = import 'kubecfg.libsonnet';

{
  local outer = self,

  container:: {
    name: 'busybox',
    image: 'busybox:latest',
  },

  deployment: {
    local this = self,

    apiVersion: 'extensions/v1beta1',
    kind: 'Deployment',
    metadata: {
      name: 'busybox',
      labels: { name: 'busybox' },
    },
    spec: {
      replicas: 1,
      template: {
        metadata: { labels: this.metadata.labels },
        spec: {
          containers: [outer.container],
        },
      },
    },
  },
}

➜ kubecfg show code.jsonnet --show-provenance              
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  annotations:
    kubecfg.github.com/provenance-file: code.jsonnet
    kubecfg.github.com/provenance-path: $.deployment
  labels:
    name: busybox
  name: busybox
spec:
  replicas: 1
  template:
    metadata:
      labels:
        name: busybox
    spec:
      containers:
      - image: busybox:latest
        name: busybox

the 2 kubecfg.github.com/provenance- annotations will automatically be added to help identify where a manifest came from in the codebase

Traceback

Alpha Feature: turn it on with --features=traceback (see Feature gates).

when the annotations are added to the rendered manifests is possible to traceback the line in the jsonnet code that did add a specific field

for example, to find which line in jsonnet added the replicas: 1 (which is line 12 in the manifest ) we can

kubecfg --features=traceback traceback RENDERED.yaml:LINE_NUMBER

➜ kubecfg --features=traceback traceback output.yaml:12   
INFO  Tracing file="code.jsonnet", path="$.deployment.spec.replicas"
/dev/shm/kubecfg/example/code.jsonnet:21 

and we will know it was line 21 of the code.jsonnet that added the replicas field

➜ sed -n '21p' code.jsonnet
      replicas: 1,

If the replicas value gets updated by setting the replicas to 3 , the traceback will tell us the correct line

➜ tail code.jsonnet 
    },
  },
} + {

  deployment+: {
    spec+: {
      replicas: 3,
    },
  },
}


➜ kubecfg show code.jsonnet --show-provenance > output.yaml

➜ kubecfg --features=traceback traceback output.yaml:12
INFO  Tracing file="code.jsonnet", path="$.deployment.spec.replicas"
/dev/shm/kubecfg/example/code.jsonnet:34 

➜ sed -n '34p' code.jsonnet                             
      replicas: 3,

Json Schema Validation

Supported: from v0.31.0

The validateJSONSchema function allows the validation of Jsonnet to a particular JSONSchema.

This takes the format of validateJSONSchema(object, schema) and will return a runtime error when the provided object does not uphold the schema.

Example

Using this example schema.json file:

{
   "properties": {
      "age": {
         "description": "Age in years which must be equal to or greater than zero.",
         "minimum": 0,
         "type": "integer"
      },
      "firstName": {
         "description": "The person's first name.",
         "type": "string"
      },
      "lastName": {
         "description": "The person's last name.",
         "type": "string"
      }
   },
   "type": "object"
}

We can use this schema to validate objects which are placed within our Jsonnet code.

Take the following example.jsonnet file:

local kubecfg = import 'kubecfg.libsonnet';
{
  assert kubecfg.validateJSONSchema($.person, import 'schema.json'),

  person: {
    age: 26,
  },
}

This can be successfully evaluated, as it is valid against the schema:

kubecfg eval --features=eval example.jsonnet -o json
{
   "person": {
      "age": 26
   }
}

Note that firstName and lastName are not required items, hence this object remains valid when they are omitted.

By modifying the person.age value to -1 we can force this to produce an error, as this becomes invalid against our schema:

kubecfg eval --features=eval example.jsonnet -o json
ERROR RUNTIME ERROR: object is invalid against the schema: jsonschema: '/age' does not validate with file:///properties/age/minimum: must be >= 0 but found -1

YAKL: YAML with Jsonnet expressions

Warning

Prototype. YAKL is an experimental format, turned on with --features=yakl (see Feature gates). The syntax and semantics below may change.

Jsonnet is a great language, but many people would rather write YAML. Tools like kro keep YAML as the outer language and embed ${...} expressions in it. YAKL (yet another konfig language) does the same, but the language inside ${...} is Jsonnet instead of CEL. Files use the .yakl extension.

foo:
  bar: '${{some: "jsonnet"}.some}'

(the quotes are YAML’s, needed because of the : ; they don’t make it a string) means the same as the Jsonnet

{
  foo: {
    bar: {some: "jsonnet"}.some,
  },
}

and evaluates to {"foo": {"bar": "jsonnet"}}.

The model is: a yakl file is YAML in which any node can be written as a Jsonnet expression. YAKL is not a text template language. Expressions produce values that are placed in the document tree, so there is no indentation or quoting to get right in the output (no nindent, no toYaml), and YAML strings are never interpolated.

On top of that, yakl gives YAML the Jsonnet features that make files reusable, each spelled in plain YAML:

Four ideas carry the design:

• &name always declares a local. On a node, the node stays where it is and also gets a name. As a quoted key, '&port': 8080, the value is only a local.
• The key ^^ names a document. ^^: web among the top-level keys of a document makes the document the field web of the file, which expressions read as $.web. Like any key, it takes + and colons: ^^+: service extends service, and ^^:: config hides config.
• A stream is a list until a document gets a name. From then on the file is one object, as a Jsonnet file usually is.
• Meaning lives in data and anchors, never in quoting, key order or tags. Tools like kubectl, kustomize and prettier rewrite quoting, reorder keys and drop tags, so none of those can carry meaning. No editor needs configuring.

YAML compatibility

A yakl file is valid YAML, as read by kubectl (sigs.k8s.io/yaml), go-yaml v3 (under kustomize), PyYAML and yamllint, eemeli/yaml (prettier and the VS Code YAML server), tree-sitter and VS Code’s highlighter, so editors, linters and other YAML tools can work with it. To them, every expression is a single scalar node, anchors are anchors, and local keys, name keys and key modifiers like spec+ are just keys.

Most expressions are valid plain scalars as they are (replicas: ${params.replicas}). Those containing : or #, and those inside flow collections ([...], {...}), are written as quoted expressions. Those spanning lines are best written as block expressions.

The compiler doesn’t enforce all of it: it still accepts unquoted expressions that a YAML parser would reject or split, like a: ${ {b: 1}.b }.

The examples and the snippets in this document are checked with kubectl’s YAML parser.

What round trips keep

kubectl and kustomize parse a file and write new YAML. Everything spelled as data survives that: expressions, $${, key modifiers, <<+, local keys ('&port':) and names (^^: web). Anchors don’t: both tools expand them. After a round trip, a ${name} that used an anchor’s local fails with “Unknown variable”. kustomize also needs every document to be a Kubernetes resource, so a file with a document of local keys, or a hidden settings document, can’t go through kustomize at all. A file meant for kustomize can use local keys instead of anchors. See What tools do with it.

Implementation model

A yakl file is compiled to Jsonnet and then evaluated like any other Jsonnet file:

1. Lex. A YAML-aware lexer finds every ${ that starts a YAML node and reads the Jsonnet expression up to the matching }. It skips comments, quoted scalars and block scalars, except those whose whole value is an expression. YAML blocks nested inside expressions are lexed the same way, recursively. Expressions are replaced by opaque placeholders.
2. Parse. The result, now plain YAML, is parsed as a YAML stream. The parser also checks the lexer: every placeholder must come back as a whole scalar, and the documents must match. Any disagreement is an error, never silently wrong output.
3. Generate. The YAML tree is translated to Jsonnet: mappings become objects, sequences become arrays, scalars become literals, placeholders become the original expressions. Name keys make fields of the file, anchors and '&x': keys become locals, key modifiers become Jsonnet’s +:, :: and :::, and ${for} and ${if} list items become comprehensions. YAML blocks inside expressions are compiled in place.

The generated code keeps YAML nodes and expressions on their original source lines, so Jsonnet errors and kubecfg traceback / eval --trace point at the right line of the .yakl file. The exceptions are listed in Corner cases.

Seeing the generated Jsonnet

kubecfg --features=eval,yakl eval -o jsonnet app.yakl prints the Jsonnet a file compiles to, formatted to be read, with a field per line and YAML comments as // comments. It doesn’t evaluate the file, so it works when evaluating fails, and needs neither a cluster nor the file’s imports. It is the quickest way to see what names, locals and key modifiers mean. For this document:

'&k': ${import 'k.libsonnet'}
---
^^:: config
replicas: 2 # overlays change it
---
^^: deployment
kind: Deployment
metadata:
  labels: &labels {app: web}
spec:
  replicas: ${$.config.replicas}
  selector: {matchLabels: '${labels}'}

it prints:

{
  local k = import 'k.libsonnet',
  config:: {
    replicas: 2,  // overlays change it
  },
  deployment: local yakl__root = {
    kind: 'Deployment',
    metadata: {
      labels: { app: 'web' },
    },
    spec: {
      replicas: $.config.replicas,
      selector: { matchLabels: labels },
    },
  }, labels = yakl__root.metadata.labels; yakl__root,
}

It’s the same code kubecfg evaluates, laid out differently: kubecfg keeps each node on its source line instead. Internal names, like yakl__root for a document whose nodes have anchors and yakl__unnamed3 for a document without a name, and the check that <<+ extends an object, show up as they are. A comment stays next to the field or item it’s written on; one above a document’s name key goes above the document’s field. Copies made by aliases have no comments. Like other commands, eval reads URLs and /dev/stdin too. If the code doesn’t parse, because of an expression’s syntax, it prints the code unformatted, and the error with its line in the yakl file.

Formatting

kubecfg --features=yakl yaklfmt app.yakl formats the Jsonnet code of the expressions, as jsonnetfmt does, and prints the file; -i rewrites it instead. The YAML around the expressions, comments included, stays as it is: a YAML formatter such as prettier keeps a yakl file’s meaning (see What tools do with it).

• A plain expression gets quoted where its formatted code isn’t a plain YAML scalar, as ${ {a:1} } becomes '${ { a: 1 } }' because of the : . A quoted expression keeps its quotes.
• A block expression keeps its lines inside the block.
• What follows --- moves to the next line, like the | in --- |, since kubectl’s reader allows only a comment there.
• Left as they are: ${for} and ${if} clauses, expressions holding YAML blocks, and plain or quoted expressions whose formatted code would take several lines.

Expressions

${ at the start of a YAML node begins an expression, which runs to the matching }. The body is any Jsonnet expression. Braces are matched with awareness of Jsonnet lexical syntax, so braces inside these do not count:

• string literals: "...", '...' (with backslash escapes)
• verbatim strings: @"...", @'...' (quotes escaped by doubling)
• text blocks: ||| … |||
• comments: // ..., # ..., /* ... */

An expression is always a whole node and produces a value. Its YAML spelling doesn’t matter, because tools rewrite spelling:

replicas: ${params.replicas}            # plain
labels: '${ {app: "web"} }'             # quoted, because of the `: `
ports: |-                               # block, for several lines
  ${[{containerPort: p} for p in [80, 443]]}
literal: $${HOME}                       # the string ${HOME}
script: echo ${HOME}                    # literal text: ${ isn't at the start
image: ${'nginx:' + version}            # build strings in Jsonnet

• A string whose whole value is ${...} is an expression, in any YAML style: plain, single-quoted, double-quoted, literal or folded block. White space at the start or end of a string doesn’t count, here and below, so a tool that rewrites a block as "${...}\n" keeps it an expression.
• $${ at the start of a string is a literal ${. $${HOME} is the string ${HOME}.
• ${ after other text is literal text, in any style: echo ${HOME}, nginx:${version}. A tool may unquote a literal string, so it has to stay literal either way.
• A string that starts with ${ must be exactly one expression, in any style. ${version}-alpine and '${a} ${b}' are errors. Start such text with $${, or build the string in Jsonnet.

Quoted expressions

A quoted scalar whose whole value is ${...} is that expression: '${1 + 1}' and "${1 + 1}" mean exactly ${1 + 1}, the number 2. This is not string interpolation; it is YAML quoting around Jsonnet that is not a valid plain scalar:

a: '${ {x: "y # not a YAML comment"}.x }'   # `: ` and ` #` inside the expression
b: ['${1}', '${"a, b"}']                    # inside a flow sequence

YAML unquotes the scalar first, and its value is the Jsonnet code. So inside '...', write a ' as '' (or use " for Jsonnet strings), and inside "..." YAML’s backslash escapes apply. A quoted expression may span lines, but YAML folds its line breaks into spaces, and yakl compiles the folded text: a // or # comment would swallow the lines after it, and ||| text blocks break. Code with comments, text blocks or YAML blocks needs a block expression.

Block expressions

A block scalar (| or >) whose content is a single ${...} is that expression too. Its lines are the Jsonnet code as written, so it can hold comments, text blocks and YAML blocks, and Jsonnet errors point at the right line. This is how to write a multi-line expression:

ports: |-
  ${[
    {containerPort: p}  // one entry per port
    for p in [80, 443]
  ]}

White space around the expression doesn’t matter, and neither does the block’s style: chomping (|-, |+) has nothing to chomp, and > doesn’t fold the code. The code must stay indented as the block’s content, since a line indented less ends the block.

Where an expression can be

An expression can be a mapping value, a mapping key, a sequence item, an item of a flow collection, or the entire document:

replicas: ${params.replicas}
${name}: value
items:
- ${item}
flow: ['${a}', '${b}']

After the closing } of an unquoted expression, only these may follow on the line: a : (the expression is a key), flow punctuation (,, ], }), a comment, or nothing. An expression can carry an anchor (&a ${x}) but no tag.

Strings and comments are never interpolated

Except for a scalar that is a whole expression, ${ in a string or a comment is plain text. So shell variables need no escaping:

command: [sh, -c, 'echo ${HOME}']   # the string "echo ${HOME}"
script: |
  #!/bin/sh
  echo "${GREETING:-hello}"
# ${ in a comment is just a comment

To put computed values into a multi-line string, build it in Jsonnet, e.g. with a text block:

config.ini: |-
  ${|||
    [server]
    port = %d
  ||| % params.port}

A scalar that starts with ${ and ends with } but isn’t a well-formed expression, like '${"x}', reports the Jsonnet error (here an unterminated string), as it was most likely meant as an expression. A literal string that starts with $${ can’t be written directly, because yakl drops one $: use an expression, ${'$${X}'}.

YAML inside expressions

Inside an expression, \{ followed by a line break opens a YAML block. So ${ always means “Jsonnet here” and \{ always means “YAML here”, and the two languages nest in each other to any depth. As with Jsonnet ||| text blocks, the first content line sets the block’s indentation. The block ends at the first line indented less than that, which must start with }. How deep the block sits in the surrounding code doesn’t matter.

env: |-
  ${[
    \{
      name: ${k}
      value: ${params.env[k]}
    }
    for k in std.objectFields(params.env)
  ]}

The block compiles to a Jsonnet value in place, so expressions in it see the surrounding Jsonnet scope, like the comprehension variable k above. For a loop like this one, a ${for} item is shorter and keeps the body YAML to every tool; blocks cover what it doesn’t. A block is one value, not a file of documents: its anchors and its '&x': keys declare locals as in a document, and a ^^ key is an error.

Rules:

• \{ must end its line; only a YAML comment may follow it. A backslash means nothing in Jsonnet outside string literals, so \{ can’t clash with Jsonnet code. Inside strings it is just text.
• The block’s content must be indented more than its closing }. After the }, the Jsonnet code continues on the same line: } for k in ..., } + {...}, },.
• Blank and comment lines don’t count for indentation.
• A block holds a single YAML document. An empty block is null.

Loops and conditions

A list item can be a Jsonnet comprehension clause, ${for ...} or ${if ...}, with a body indented under it. The loop is YAML; only the clause is Jsonnet.

'&params': ${import 'params.libsonnet'}
'&port': 8080
---
^^: deployment
'&envs': ${std.objectKeysValues(params.env)}
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
  labels:
    app: web
    <<+: ${params.extraLabels}
spec:
  template:
    spec:
      containers:
        - name: web
          env:
            - ${for e in envs}:
                name: ${e.key}
                value: ${e.value}
            - ${if params.debug}:
                name: DEBUG
                value: "1"
            - name: PORT
              value: ${std.toString(port)}
          args:
            - --verbose
            - ${for p in params.ports if p != 9090}: ${'--port=' + p}

With params.env = {LOG: 'info', MODE: 'prod'}, debug: true, ports: [8080, 9090, 9100] and two extra labels, env is LOG, MODE, DEBUG, PORT, in that order; args is --verbose, --port=8080, --port=9100; and labels is app plus the two extra labels.

It means:

deployment: {
  local envs = std.objectKeysValues(params.env),
  apiVersion: 'apps/v1',
  kind: 'Deployment',
  metadata: {
    name: 'web',
    labels: params.extraLabels + { app: 'web' },
  },
  spec: { template: { spec: { containers: [{
    name: 'web',
    env: [{ name: e.key, value: e.value } for e in envs]
      + (if params.debug then [{ name: 'DEBUG', value: '1' }] else [])
      + [{ name: 'PORT', value: std.toString(port) }],
    args: ['--verbose'] + ['--port=' + p for p in params.ports if p != 9090],
  }] } } },
},

A loop becomes an array comprehension, a condition becomes if ... then [body] else [], and the pieces of the list are joined with +.

• The clause is Jsonnet’s own comprehension clause: one for, then any number of further fors and if filters, as in ${for p in params.ports if p != 9090}. On its own, ${if cond} is a condition.
• The body goes under the clause. It is any YAML node, with expressions anywhere in it, and local keys, declared once per iteration. The loop variable is in scope in the body, and hides an outer name, as in Jsonnet. self in a body is the item.
• The clause is the item’s only key. A body written at the clause’s own indentation would put its keys next to the clause, so that’s an error, and so is an empty body; both errors say to indent the body.
• Plain items stay where they are, so a loop, a condition and a fixed item sit side by side, in order, with no + between them.
• A mapping key can’t be a clause. A mapping gets fields from somewhere else through <<+ only, as labels does above: <<+: ${params.extraLabels}, where the mapping’s own app: web wins over an app in params.extraLabels. A condition is <<+: ${if params.debug then {debugPort: 9229}}, since null extends nothing, and several sources are a list: <<+: [${a}, ${b}]. A ${for} or ${if} key in a mapping is an error that says so.
• No anchors inside a body, since a body is repeated or left out, so there’s no single node for an anchor to name. An anchor on a plain item next to loops needs a fixed index: the item comes before every loop of its list, or after every loop.
• A long collection goes into a local first, as envs does above, because a YAML key must fit on one line.
• No modifiers on a clause key. ${for ...}+: is an error, since a string that starts with ${ is one whole expression. Literal keys inside the body keep theirs (spec+:).
• One element per iteration. For several, add a for to the clause (${for a in as for x in [a, a + 1]}), or write two loops. Loops nest: a body can hold more loops and conditions.
• ${if debug then 'debugPort'}: 9229 is still a computed key. for ..., and if without then, aren’t Jsonnet expressions on their own, so a key is a clause only when it isn’t one.
• Overlays just work: env+: with loop items appends them like any other items.
• The escape stays. A block expression with \{ blocks still covers anything this form doesn’t.

Changing every item of a list is a loop whose body extends the item. In an overlay:

^^+: redis
statefulset+:
  spec+:
    volumeClaimTemplates:
      - ${for t in super.volumeClaimTemplates}:
          <<+: ${t}
          spec+:
            resources+:
              requests+: {storage: 20Gi}

Extending and hiding fields

Jsonnet writes how a field combines with an inherited one right after its name: spec+:, _config::. YAKL keys are written the same way.

• The meaning comes from the key’s text. kustomize writes _config:: back as '_config:':. That’s the same text, so it means the same.
• + works on any value, as Jsonnet’s +: does. Objects are extended, lists appended (args+: [--verbose]), numbers added (replicas+: 1). With nothing inherited, the value is used as it is.
• One name, at most two keys in a mapping: one without + (x:, x:: or x:::) and one with it. Together they make one field, x + x+, which lets merge keys and extension work together (example below). The + key must then be plain x+:, and the field takes the other key’s visibility: x:: with x+: gives a hidden x + x+. Anything else is an error: x: with x::, x: with x+::, or two + keys.
• A literal key ending in + or : is written as a key expression: ${'name:'}: value. Kubernetes field names, label and annotation keys, and ConfigMap keys can’t end in + or :.

<<+: extend an object

<<+: S makes the mapping S + {its own fields}.

deployment:
  <<+: ${k.deployment('web', image='nginx:1.27')}
  spec+:
    replicas: 3

That’s deployment: k.deployment('web', image='nginx:1.27') + {spec+: {replicas: 3}}.

• S is an expression or a mapping. An alias to a mapping is fine; tools expand it into the same mapping.
• An expression must give an object, or null, which extends nothing. Anything else is an error, including a list, so + can’t silently join strings or arrays.
• Several sources are a YAML sequence, in << order. Each item is an expression or a mapping, and an earlier source wins over a later one, as in a YAML merge. <<+: [A, B] means B + A + {own fields}.
• S is evaluated outside the mapping, as S in S + {...} is. It sees the document’s anchors and the file’s locals, but not the local keys ('&x':) of the mapping it extends. Inside the mapping, self and super work as in Jsonnet, so fields of S that use self see the mapping’s own fields.
• With << in the same mapping, << first copies fields in, and <<+ is the parent of the result.
• To YAML tools, <<+ is an ordinary key, not a merge key, so kubectl and kustomize keep it. (<<: ${...} would not work: kubectl and kustomize reject a merge key whose value isn’t a mapping.)

Merge keys keep their YAML meaning

• << copies fields, as in YAML. Sources are aliases to mappings, inline mappings, or a list of those. Own keys win, and an earlier source wins over a later one. Keys match as YAML reads them, so 0x1F matches 31; expression keys match by their code, laid out as yaklfmt lays it out, so ${'a'+'b'} matches ${'a' + 'b'}, but not ${'ab'}.
• << must be the very first key, and appear once, before any local key too. With an own key before <<, kubectl keeps the merged value and kustomize the own one. Two << keys are accepted by kubectl and rejected by kustomize.
• Anchors can’t be used across documents, as in YAML.

Two containers sharing defaults:

containers:
  - &app
    name: app
    image: registry.example.com/app:1.4
    resources: {limits: {memory: 512Mi}}
  - <<: *app
    name: worker
    args: [--worker]
    resources+:
      limits+:
        cpu: "2"

The merge copies resources into the worker, and resources+ extends that copy. The worker gets limits: {memory: 512Mi, cpu: "2"}, and so does the file kubectl or kustomize writes back. The anchor &app is also a local, so ${app.image} works anywhere in the document.

The file: a list until something has a name

A YAML file can hold several documents separated by ---, and Kubernetes users put a Deployment, a Service and a ConfigMap in one file all the time. A Jsonnet file is usually one object with a field per resource, so resources can use each other ($.deployment.spec) and overlays can change them (file + {deployment+: {...}}). YAKL lets a stream become that object one document at a time.

1. No names: a document, or a list. Documents of local keys are set aside first. What’s left follows std.parseYaml’s rules: one document gives that document; a --- or several documents give a list; an empty file gives null. A --- that only separates documents of local keys from the rest doesn’t count, so a file of imports plus one document still gives that document, and adding imports never changes the shape. A file with only documents of local keys gives an empty list.
2. At least one name, or a parent: an object. Each named document is a field. Documents without a name go, in file order, into a field called unnamed. It compiles as unnamed+:, so an overlay’s unnamed documents are appended rather than swapped in. When there are none, the field is left out. unnamed is reserved as a name.
3. $ follows Jsonnet: it’s the outermost object. In an object file, $ is the file, in every document, named or not, so $.config reads the document named config. In a file without names, each document is its own $.
4. Order doesn’t matter. Names and file locals belong to one object, so a document can use one defined further down.

yakl takes the ^^ key out of the document. Other YAML tools keep it, so to them a named document is an object with one more key. kubecfg finds the same Kubernetes objects in an object as in a list.

Step by step from plain YAML

Plain Kubernetes YAML compiles to a list:

apiVersion: v1
kind: ConfigMap
metadata: {name: a}
---
apiVersion: v1
kind: ConfigMap
metadata: {name: b}

An import and expressions keep it a list:

'&names': ${import 'names.libsonnet'}
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: ${names.prefix + 'a'}
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: ${names.prefix + 'b'}

local names = import 'names.libsonnet';
[{..., metadata: {name: names.prefix + 'a'}},
 {..., metadata: {name: names.prefix + 'b'}}]

Naming one document makes the file an object, and the other document goes to unnamed:

'&names': ${import 'names.libsonnet'}
---
^^: a
apiVersion: v1
kind: ConfigMap
metadata:
  name: ${names.prefix + 'a'}
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: ${$.a.metadata.name + '-copy'}

{
  local names = import 'names.libsonnet',
  a: {..., metadata: {name: names.prefix + 'a'}},
  unnamed+: [{..., metadata: {name: $.a.metadata.name + '-copy'}}],
}

Naming every document, moving settings into a hidden field and adding an overlay is the worked example.

Naming the first document is the only step that changes the file’s shape, and it fails loudly where it matters: kubecfg finds the same objects; importers of the file get an object instead of a list, so code that indexed it ([0]) fails; and $ now means the file, so an expression that used $ for its own document fails with “field does not exist”.

A document that should have had a name but lacks its ^^ key lands in unnamed and is still deployed, which is what a YAML user expects. Anything that uses the missing name fails: $.service gives “field does not exist”, and an overlay aimed at it leaves a fragment without kind, which kubecfg rejects (“Looking for kubernetes object at … but instead found …”).

Names

The key ^^, among the top-level keys of a document, names it. Its value is the name, and the document becomes the field with that name in the file.

^^:: config
name: web
replicas: 2
---
^^: deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ${$.config.name}
spec:
  replicas: ${$.config.replicas}

{
  config:: { name: 'web', replicas: 2 },
  deployment: { /* ... */ spec: { replicas: $.config.replicas } },
}

• Expressions read it through $, which is the file: ${$.config.replicas}, and ${$['web.conf']} for a name that isn’t a Jsonnet variable name. That’s the field itself, so an overlay that changes config changes what ${$.config.replicas} reads, and overlays read the base the same way. That’s the difference from an anchor, whose local is the node as written.
• A name isn’t a local. $ already reaches every document, and a name of any spelling, like web.conf, works the same way.
• Scope: the whole file, in any order.
• The key says how the field combines, with the + and colons a mapping key has after its name: ^^:: is to ^^: what config:: is to config:.

• Hiding a field doesn’t rename it. Add or remove the colon, and config and $.config keep working.
• Overlays keep the base’s visibility. In Jsonnet, : and +: keep the visibility the base gave a field: {config:: {r: 1}} + {config+: {r: 2}} outputs {}. An overlay writes ^^+: config, without colons, and ^^+::: config makes a hidden base field visible.
• The name is text, never an expression, so the file’s shape is known without evaluating it. Any string is a name: web.conf and my svc are names too. A number must be quoted, ^^: '123'. The + and colons go on the key: ^^: web+ is an error that suggests ^^+: web.
• One per document, among its top-level keys. It goes first by convention, but any place among the keys means the same, since tools that sort keys move it. A ^^ key anywhere else is an error.
• Only a mapping can have a name, since the name is one of its keys. An object that an expression gives goes in <<+: ^^:: lib with <<+: ${import 'lib.libsonnet'} makes the library a hidden field. A string or a list goes in a field of a hidden document.
• Every YAML tool keeps it. ^^ isn’t special in YAML, so it needs no quotes, and to kubectl, kustomize, PyYAML, eemeli/yaml, yq and prettier it’s an ordinary key. kubectl and kustomize sort it to the top of the document.

Anchors are locals

An anchor names a node, as in YAML, and that name is also a Jsonnet local: &name declares a local wherever it appears.

^^: deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: frontend
  labels: &labels {app: guestbook, tier: frontend}
spec:
  selector:
    matchLabels: ${labels}
  template:
    metadata:
      labels: ${labels}

deployment:
  local yakl__root = {
    apiVersion: 'apps/v1',
    kind: 'Deployment',
    metadata: { name: 'frontend', labels: { app: 'guestbook', tier: 'frontend' } },
    spec: { selector: { matchLabels: labels }, template: { metadata: { labels: labels } } },
  }, labels = yakl__root.metadata.labels;
  yakl__root,

The node stays where it’s written, and the local reads it there.

• Scope: the whole document, above or below the anchor. Another document can’t use it: Jsonnet reports “Unknown variable”. Something several documents share belongs in a file local or a field.
• Value: the node as the document writes it, after the document’s own <<+ and x+ keys are applied, but before any overlay. Overlays change fields, never locals: an overlay that adds a label to deployment.metadata.labels changes only that field. The selector and the template keep the local’s value, which is what a Deployment’s immutable selector needs.
• Under a + key, that’s the combined value. labels+: &labels {...} holds the inherited labels too. An item of a list under args+: is counted from the end of the combined list, so it’s still the item written there. If a later + key of the document adds to that list as well, as x+: {args+: [...]} after x: {args+: [&a ...]}, the item’s index isn’t known, and the anchor is an error.
• Aliases still copy. *labels gives the same value as ${labels}, unless the node uses self or $, which a copy reads where the alias is.
• Names: a Jsonnet variable name, not a keyword or std. Other anchors, like &default-labels, &if or &std, stay plain YAML anchors. The same name twice in one document is an error: YAML lets a later anchor replace an earlier one, but a local can’t have two values. An anchor may hide a file local with the same name, as Jsonnet locals do.
• Not in a key of a mapping that holds the node, or a ${for} or ${if} of such a list. Jsonnet needs them to build the mapping or list, and the local reads the node from it, so ${name}: 1 next to name: &name web is an error. Declare the value with a local key, '&name': web, and use ${name} in both places.
• Only on nodes that have a place in the output. An anchor on a key, on the value of << or <<+, inside a merge source, or under a key expression is an error. A common slip: &web apiVersion: v1 on one line anchors the key apiVersion; an anchor for the whole document goes on its own line.
• Expressions in the node are compiled in place, so self, $ and the local keys around them mean what they mean anywhere.
• On a document’s root, the local is the whole document as written, for that document only. Other documents reach it by its name.
• Inside a \{ block, anchors declare locals of the block’s value the same way.

Locals that aren’t output

An anchor keeps its node in the output. For a value that only expressions use, write the anchor as a quoted key: '&name': value. It reads as “an anchor named name, holding this value”, and it declares a local of the mapping it’s in.

^^: deployment
'&labels': {app: guestbook, tier: frontend}
apiVersion: apps/v1
kind: Deployment
metadata:
  name: frontend
  labels: ${labels}
spec:
  selector:
    matchLabels: ${labels}
  template:
    metadata:
      '&port': 8080
      labels: ${labels}
      annotations:
        port: ${std.toString(port)}

deployment: {
  local labels = { app: 'guestbook', tier: 'frontend' },
  apiVersion: 'apps/v1',
  kind: 'Deployment',
  metadata: { name: 'frontend', labels: labels },
  spec: {
    selector: { matchLabels: labels },
    template: { metadata: { local port = 8080, labels: labels, annotations: { port: std.toString(port) } } },
  },
},

• Scope: the mapping that holds the key, and everything inside it, in any order, as a Jsonnet object local. port above exists only in the template’s metadata. A local used in both metadata and spec goes at the top of the document.
• The value is YAML data or an expression. It stays an object local where it’s written, so it can use self, $ and other locals.
• Names: a Jsonnet variable name, not a keyword or std, at most once per mapping. A local may hide an outer one with the same name.
• The quotes are required. Unquoted, &labels: {...} is a YAML anchor on an empty key, which kubectl rejects. yakl sees that shape and says to quote it.
• It needs a value. '&x': with nothing after it is an error, since it’s most likely meant as a name. Write null for a local holding null.
• No anchors inside the value, since the value isn’t in the output.
• It’s a key like any other to YAML tools, so it survives round trips: kustomize keeps it and moves it to the top of its mapping. A merge (<<) copies it too, and an own local key wins over a copied one.

File locals: a document of local keys

A document made only of local keys, with no name key, holds locals of the whole file. That’s where imports go:

'&k': ${import 'k.libsonnet'}
'&names': ${import 'names.libsonnet'}
'&image': nginx:1.27
'&webName': ${function(suffix) 'web-' + suffix}
---
^^: deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ${webName('api')}

{
  local k = import 'k.libsonnet',
  local names = import 'names.libsonnet',
  local image = 'nginx:1.27',
  local webName = function(suffix) 'web-' + suffix,
  deployment: { /* ... */ },
}

In a file without names, the same locals become local k = ..., names = ...; in front of the document or list.

• What counts: a document whose root is a non-empty mapping in which every key is a local key. A ^^ key makes it a named document instead. An anchor on it is an error, since the document isn’t output.
• Not output, and not a list element. Adding one never changes the file’s shape.
• Not tied to position. It can go anywhere, and there can be several. The convention is to put it first, like Jsonnet imports, but nothing depends on it.
• Seen by every document, in any order. File locals can use each other in any order too.
• One name space per file. File locals must all differ.
• Private. Importers and overlays can’t see file locals. A hidden field (^^:: config) is the overlay-friendly alternative: Jsonnet’s local x = ... versus x:: ....
• $ inside a file local is the file in an object file. A file without names has no file object: there Jsonnet would read $ as the local’s own outermost object, so yakl reports it instead. self outside an object gives Jsonnet’s own “Can’t use self outside of an object”.

kubecfg without an import

A file can use kubecfg without importing kubecfg.libsonnet:

apiVersion: v1
kind: Pod
metadata:
  name: web
spec:
  containers:
  - name: web
    image: ${kubecfg.resolveImage('nginx:1.27')}

The compiler adds the file local that this local key would declare:

'&kubecfg': ${import 'internal:///kubecfg.libsonnet'}

• The built-in copy. internal:///kubecfg.libsonnet is the copy built into kubecfg. A file with that name on the search path (-J) or next to the .yakl file doesn’t change it. To use another copy, declare kubecfg yourself, with '&kubecfg': ${import 'kubecfg.libsonnet'}.
• Only when it’s used. The compiler adds it when an expression reads kubecfg and the file doesn’t declare it there. A file with its own '&kubecfg':, or that doesn’t use kubecfg, compiles as before. A field named kubecfg, as in x.kubecfg, isn’t a use.
• Your own names win. An anchor, a local key or a Jsonnet local named kubecfg hides it, as an inner local hides an outer one.
• Same lines. The local goes in front of the first line, without a line of its own, so errors keep their line numbers.
• eval -o jsonnet shows it, as local kubecfg = import 'internal:///kubecfg.libsonnet'; at the top. Saved as a .jsonnet file, that output works in kubecfg as it is. For other Jsonnet tools, change the import to 'kubecfg.libsonnet'.
• Not in Jsonnet files. A Jsonnet file still needs the import, so editors, linters and other Jsonnet tools can read it. RFC06 explains why.

Every way to declare a local

Overlays

An overlay is a yakl file written with the same rules, and kubecfg combines it with the base as base + overlay (--overlay-code-file prod.yakl). Because the overlay spells its own +: and ::, it says exactly what it does to each field.

A single-document overlay uses key modifiers:

spec+:
  replicas: 5
  template+:
    metadata+:
      labels+:
        canary: "true"

An overlay for a stream is a stream too. Each document says what it does to one named document:

• The overlay reaches the base through $, as the base does: $.config.name. $ in the overlay is the combined file, so it sees the base’s fields and the overlay’s changes. The base’s file locals aren’t in scope there.
• The base sees the overlay’s changes too, through $: a ${$.config.replicas} in the base reads the overlaid value.
• unnamed+: keeps both lists. With a plain unnamed:, an overlay’s ServiceAccount would replace the base’s NetworkPolicy, which would silently disappear.
• An overlay without names follows the same file rules. One document is merged into the base’s root, which is the single-document overlay above. Several documents make a list, and an object plus a list is an error. To add documents to a named base, the overlay needs at least one name.
• A file can name its base itself, so it needs no flag: see the next section.
• A whole object as an unnamed single-document overlay is a mistake: merged into a named base’s root, its apiVersion and kind would turn the whole file into that one object. kubecfg reports an overlay with apiVersion and kind but no name when the base has no kind of its own. A flag that names the field, --overlay-code-file sa=whole.yakl, adds it as the document sa.

A file that extends another

A file can say what it extends, as a Jsonnet file does with (import 'base.jsonnet') + {...}. Its first document holds only <<+ and, if needed, local keys:

<<+: ${import 'yakl:web.yakl'}
---
^^+: config
replicas: 5
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: ${$.config.name}

(import 'yakl:web.yakl') + {
  config+: { replicas: 5 },
  unnamed+: [{ apiVersion: 'v1', kind: 'ServiceAccount', metadata: { name: $.config.name } }],
}

kubecfg --features=yakl show prod.yakl then shows web.yakl with the changes, and a third file can extend prod.yakl in turn. With the worked example’s prod.yakl, adding that first document gives the same output as --overlay-code-file.

• <<+ means what it means in a mapping, with the file as the mapping and its documents as the fields: the value is an expression, a mapping, or a list of them, and null extends nothing.
• The file is an object, even without names. Documents without a name are appended to unnamed, and a single one is not merged into the root as it is in an overlay without names. To extend one object, give its document <<+ and the fields, as in <<+.
• It must be the first document, so a reader finds the base at the top.
• The base is evaluated outside the file, as S in S + {...} is, so it can’t use the file’s locals. The file’s documents reach the base through $, and the base sees their changes through its names and $, as with an overlay.
• A single document of only <<+ is that document, as before: a file is extended only when more documents follow.

Worked example

A base file and a prod overlay, both yakl streams. They are also checked by the tests, in pkg/yakl/testdata/overlay.

lib.libsonnet:

{
  conf(name):: name + '-conf',
  probes(port):: { readinessProbe: { httpGet: { path: '/healthz', port: port } } },
}

web.yakl:

'&lib': ${import 'lib.libsonnet'}
'&port': 8080
---
^^:: config
name: web
image: nginx:1.27
replicas: 2
---
^^: deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ${$.config.name}
  labels: &labels
    app: ${$.config.name}
spec:
  replicas: ${$.config.replicas}
  selector:
    matchLabels: ${labels}
  template:
    metadata:
      labels: ${labels}
    spec:
      containers:
        - <<+: ${lib.probes(port)}
          name: web
          image: ${$.config.image}
          ports:
            - containerPort: ${port}
          volumeMounts:
            - name: conf
              mountPath: /etc/nginx/conf.d
      volumes:
        - name: conf
          configMap:
            name: ${$['web.conf'].metadata.name}
---
^^: service
apiVersion: v1
kind: Service
metadata:
  name: ${$.config.name}
spec:
  '&target': ${$.deployment.spec.template.spec.containers[0]}
  selector: ${$.deployment.spec.selector.matchLabels}
  ports:
    - name: ${target.name}
      port: 80
      targetPort: ${target.ports[0].containerPort}
---
^^: web.conf
apiVersion: v1
kind: ConfigMap
metadata:
  name: ${lib.conf($.config.name)}
data:
  default.conf: |
    server { listen 8080; }
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: web-ingress
spec:
  podSelector:
    matchLabels: ${$.deployment.spec.selector.matchLabels}
  ingress:
    - {}

It means:

{
  // the document of local keys
  local lib = import 'lib.libsonnet',
  local port = 8080,

  config:: { name: 'web', image: 'nginx:1.27', replicas: 2 },

  deployment:
    local yakl__root = {
      apiVersion: 'apps/v1',
      kind: 'Deployment',
      metadata: { name: $.config.name, labels: { app: $.config.name } },
      spec: {
        replicas: $.config.replicas,
        selector: { matchLabels: labels },
        template: {
          metadata: { labels: labels },
          spec: {
            containers: [lib.probes(port) + {
              name: 'web',
              image: $.config.image,
              ports: [{ containerPort: port }],
              volumeMounts: [{ name: 'conf', mountPath: '/etc/nginx/conf.d' }],
            }],
            volumes: [{ name: 'conf', configMap: { name: $['web.conf'].metadata.name } }],
          },
        },
      },
    }, labels = yakl__root.metadata.labels;
    yakl__root,

  service: {
    apiVersion: 'v1',
    kind: 'Service',
    metadata: { name: $.config.name },
    spec: {
      local target = $.deployment.spec.template.spec.containers[0],
      selector: $.deployment.spec.selector.matchLabels,
      ports: [{ name: target.name, port: 80, targetPort: target.ports[0].containerPort }],
    },
  },

  'web.conf': {
    apiVersion: 'v1',
    kind: 'ConfigMap',
    metadata: { name: lib.conf($.config.name) },
    data: { 'default.conf': 'server { listen 8080; }\n' },
  },

  unnamed+: [{
    apiVersion: 'networking.k8s.io/v1',
    kind: 'NetworkPolicy',
    metadata: { name: 'web-ingress' },
    spec: { podSelector: { matchLabels: $.deployment.spec.selector.matchLabels }, ingress: [{}] },
  }],
}

prod.yakl:

^^+: config
replicas: 5
image: nginx:1.29
---
^^+: service
spec+:
  type: LoadBalancer
---
^^+: web.conf
data:
  default.conf: |
    server { listen 8080; gzip on; }
---
^^: pdb
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: ${$.config.name}
spec:
  minAvailable: 2
  selector: ${$.deployment.spec.selector}
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: ${$.config.name}

It means:

{
  config+: { replicas: 5, image: 'nginx:1.29' },
  service+: { spec+: { type: 'LoadBalancer' } },
  'web.conf'+: { data: { 'default.conf': 'server { listen 8080; gzip on; }\n' } },
  pdb: {
    apiVersion: 'policy/v1',
    kind: 'PodDisruptionBudget',
    metadata: { name: $.config.name },
    spec: { minAvailable: 2, selector: $.deployment.spec.selector },
  },
  unnamed+: [{ apiVersion: 'v1', kind: 'ServiceAccount', metadata: { name: $.config.name } }],
}

kubecfg --features=yakl show web.yakl and kubecfg --features=yakl,overlay show web.yakl --overlay-code-file prod.yakl give:

• The document of local keys holds the import and the port. Neither is output.
• config is a hidden document. The overlay only touches the config field, yet the Deployment gets 5 replicas and the new image through ${$.config.replicas} and ${$.config.image}, because $ reads the field. ^^+: config keeps config hidden without repeating the colons.
• &labels is a nested anchor: the labels stay in metadata, and the same value fills the selector. An overlay that added a label to metadata.labels would leave the selector as it is.
• '&target' is a local key in the Service’s spec. It names the container once and isn’t output.
• ^^: web.conf gives a field with a dot in its name. That isn’t a variable name, so the Deployment reads it as $['web.conf'].
• <<+ starts the container from the library’s probe and adds its own fields.
• The overlay is a separate file, so it reaches the base through $, and its PDB takes the Deployment’s selector.
• ^^+: web.conf with a plain data: replaces all of data; data+: would merge it.

A larger example, a shop built on k8s-libsonnet with dev and prod overlays that change every object, every pod, or one container, is in examples/k8s-libsonnet.

Semantics

Expressions

An expression evaluates to whatever Jsonnet returns: number, bool, null, string, array or object. That value replaces the node.

Keys

A key expression becomes a Jsonnet computed field name. It must evaluate to a string or null, and null drops the field, which is how to include a field conditionally:

${name}: value
${if debug then 'debugPort'}: 9229

${if debug}:, without then, and ${for ...}: are list items, and errors as mapping keys.

Keys must be scalars. Complex keys (? [a, b]) are rejected. Duplicate keys are errors, also when a computed key collides at runtime.

Scope

Expressions are evaluated in the generated Jsonnet, so:

• $ is the outermost object: the file in a file with names, and otherwise the root mapping of the current YAML document. self is the innermost mapping that contains the expression. Both are late-bound as usual in Jsonnet. Inside a YAML block nested in an expression, self is the block’s innermost mapping, while $ stays the same.
• File locals, the anchors of the current document, the local keys of the mappings around the expression and the variables of the ${for} items around it are in scope. An inner one hides an outer one with the same name, as in Jsonnet.
• std is available, and so are std.extVar (kubecfg -V, --ext-code, …) and the kubecfg native functions. So is kubecfg, the kubecfg library: see kubecfg without an import.
• import, importstr and importbin resolve relative to the .yakl file, then the -J search path, just like in a .jsonnet file. URLs and oci:// work too.

Plain YAML is (almost) unchanged

A .yakl file without expressions means what std.parseYaml and kubectl make of it, so you can rename most .yaml manifests to .yakl and get the same objects. The exceptions:

• YAML 1.1 booleans are errors: unquoted y, n, yes, no, on, off, in any case, as keys or values. kubectl reads them as booleans and kustomize as strings, so the file would mean two things. Write true or false, or quote them. A GitHub Actions workflow’s on: key is one.
• Keys ending in + or : are field modifiers. Kubernetes keys never end that way.
• The keys <<+ and ^^ are reserved, and so are keys starting with &. << must be the first key, and appear once.
• Strings starting with $${ lose one $, and strings starting with ${ are expressions.
• Anchors become locals, which is harmless, except that the same name twice in one document, and an anchor on a key, are now errors when the name is a Jsonnet variable name.

Other YAML 1.1 values keep their meaning, as kubectl and kustomize agree on them:

Quote a scalar to keep it a string ('0777').

Tags

!!str keeps its YAML meaning on ordinary scalars: !!str 0777 is the string "0777", as kubectl reads it. It can’t turn an expression into text: !!str on a string that starts with ${ is an error that points to $${. !!map and !!seq are accepted and ignored. Other tags (!!int, !Ref, the non-specific !, …) are rejected.

Encoding

Files must be UTF-8. A leading byte order mark is ignored, and \r\n and \r line breaks are treated as \n. The Unicode line separators U+0085, U+2028 and U+2029, which YAML also treats as line breaks, are rejected.

Errors

Where a file could mean two things, yakl stops with an error and says why, with the source line:

The &a &b error needs a warning of its own: on a document’s first line, kubectl’s decoder silently reads it as a null document and skips the rest. Every other tool reports the error.

Allowed, with a note:

• ^^: web and ^^+: web in one file combine as web: A + B, by the same rule as two keys in a mapping; ^^:: web with ^^+: web gives a hidden web. It allows a vendored document followed by a local patch.
• A document with only a name gives name: {}, and kubecfg skips it.
• Nested anchors whose names aren’t Jsonnet variable names, like &default-labels, stay plain YAML anchors.
• A file with only documents of local keys has no output. A file imported for its helpers should export them as hidden fields (^^:: helpers), which importers can reach.
• <<+ at the top of a document using the document’s own name or root anchor is a loop, since the document is inside what <<+ extends. Jsonnet stops with “max stack frames exceeded”.

Settings in a visible document

A visible document that isn’t a Kubernetes object makes kubecfg fail. ^^: settings with replicas: 3 puts a bare 3 where kubecfg looks for objects, and it stops with “Looking for kubernetes object … but instead found float64”. The same happens to a plain value without a name, which lands in unnamed. Settings belong in a hidden document (^^:: settings), which overlays can change, or in a document of local keys, which they can’t.

Corner cases

• Unterminated or empty expressions (${ with no matching }, ${}) are errors. They are never treated as literal text, because that would silently drop what the author meant.
• Multi-line plain expressions may use any indentation. YAML never sees them, and YAML error line numbers are mapped back to the source.
• Jsonnet comments inside an expression. A # or // comment runs to the end of the line, so ${ x # note } swallows the }. Put the closing } on the next line or use /* */.
• Source lines. Jsonnet needs some code in a fixed place, so a few things are compiled out of source order, and Jsonnet errors inside them point at another line: file locals placed after a document in a file without names, <<+ that isn’t near the top of its mapping, and the clause of a ${for}, which Jsonnet writes after its body. Anchored nodes stay in place; the code that reads them for their locals comes after the end of the document.
• Reserved names. yakl’s own names start with yakl__, like yakl__root. Don’t declare names with that prefix.
• Private-use characters. Placeholders are built from a Unicode private-use character that doesn’t occur in the source, so any source text is safe.

What tools do with it

Every construct here parses in all the tools listed at the end of this section. The losses are in round trips, and in tools that need each document to be a Kubernetes object.

Rewriting. kubectl and kustomize parse a file and write new YAML, and so does prettier:

• Expressions, $${, key modifiers, <<+, local keys and name keys survive as strings and keys. The spelling doesn’t: both tools unquote "nginx:${version}", and write a block expression as a plain or quoted string. That’s why an expression means the same in every style.
• Anchors don’t survive kubectl or kustomize. Both expand them, so locals written as anchors are gone after a round trip.
• Keys are sorted. kubectl and kustomize sort keys, and & and ^ sort before lowercase letters, so local keys and name keys move to the top of their mapping.
• Quote style changes, meaning doesn’t. kubectl and kustomize write "&x" as '&x', and _config:: as '_config:':. prettier writes '&x' as "&x". On the worked example, prettier changes only spacing and quote style.
• Merges are expanded. A mapping with <<: *app and resources+: comes back with the copied resources and the resources+ key, so it means the same.
• ${for} and ${if} items survive, with their bodies, and list items keep their order.

kubectl parses every document, but its commands need each one to be a Kubernetes object:

• A file with a document of local keys, hidden settings or overlay documents fails with “Object ‘Kind’ is missing”. kubecfg, which applies yakl files, has no such limit.
• kubectl reads metadata into its typed form, so metadata.labels: ${...} fails with “cannot unmarshal string into Go struct field ObjectMeta.metadata.labels”. Keep metadata labels and annotations as mappings; expressions elsewhere, like spec.selector, are fine. <<+: ${...} in metadata.labels is kept, since its value is a string, and so are loops in env or args.

kustomize needs more:

• Every document must be a resource, with apiVersion, kind and metadata.name. A file with a document of local keys or a hidden ^^:: config document fails with “missing Resource metadata”.
• Documents are reordered by kind.
• Transformers that write into a field holding an expression stop with an error. commonLabels fails on metadata.labels: ${labels} and on a Service’s spec.selector: ${...} with “expected MappingNode but got ScalarNode”. Keep labels and selectors as mappings when label transformers must work. labels without selectors, namespace and patches that touch plain data work.
• namePrefix and nameSuffix turn ${params.name} into dev-${params.name}, which is literal text, so the name silently stops being an expression. Use literal names, or apply the prefix in Jsonnet.
• Resources are identified by kind and name text, so two ConfigMaps both named ${params.name} collide: “may not add resource with an already registered id”.
• A strategic-merge patch that merges env entries by name won’t match a loop item, because the item has no name until yakl expands it. Patch the expanded output instead, or keep such entries as plain items. Transformers such as namespace still apply.

Editors and linters:

• A loop’s body stays YAML, so editors fold and highlight it, which they can’t do inside a block expression.
• The VS Code YAML server shows an info-level “Unused anchor” hint on a &labels used only through ${labels}, because it can’t see uses inside expressions.
• yamllint’s default indentation rule wants list items indented under their key, so kubectl-style unindented lists get flagged. That’s style, unrelated to yakl.

Mistakes the tools treat differently. YAKL reports all of these, but other tools don’t always:

YAML 1.2 lets an anchor hold . and :, so eemeli/yaml reads &web.conf: as an anchor web.conf:; kubectl, kustomize and PyYAML allow only [0-9A-Za-z_-].

Versions tested: kubectl 1.37.1 (through kubectl annotate --local, which reads and writes files without a cluster), kustomize 5.8.3, go-yaml v3.0.5 and sigs.k8s.io/yaml v1.6.0, PyYAML 6.0.3, yamllint 1.37.0, eemeli/yaml 2.9.1, prettier 3.9.10, the VS Code YAML extension 1.24.0 (its bundled language server), tree-sitter-yaml 0.7.2, and the TextMate grammar of VS Code 1.136.1.

Migrating from jamlet

YAKL used to be called jamlet. It is an alpha feature, so some meanings changed outright. Most old files fail loudly where their meaning changed.

Three of these change meaning without an error:

• Overlays. A .jamlet overlay that wrote spec: {replicas: 5} merged into spec; as a .yakl overlay it replaces it. Existing overlays need their keys checked.
• Keys ending in : or +. a:: 1 used to give a key a:, and now hides a field a. c++: used to give a key c++, and now extends c+. Kubernetes keys never end that way, so this only touches free-form data.
• An anchor’s local on a mapping used to be the mapping itself, late-bound, and inside an alias copy it named the copy. Now it’s the original node as written, everywhere in the document. The two differ when an overlay changes that mapping, or when an expression inside an alias copy used the name.

$ keeps its meaning in every file without names: the document root in a single-document file, and in each document of a list. It becomes the file only in files with names.

Comparison with kro

kubecfg integration

.yakl files work wherever .jsonnet files do: show, diff, update, delete, validate, eval, overlays (--overlay-code-file x.yakl is applied as base + overlay, see Overlays, or a file names its base with <<+; .yaml and .json overlays are still deep-merged), OCI bundles, and pack (which ships the yakl source). eval -o jsonnet prints the Jsonnet of a yakl file, see Seeing the generated Jsonnet, and yaklfmt formats its expressions, see Formatting.

From Jsonnet, or from an expression in another yakl file, import a yakl file with the yakl: scheme:

(import 'yakl:deployment.yakl') + { spec+: { replicas: 5 } }

importstr 'deployment.yakl' and importbin return the file as it is. A plain import 'deployment.yakl' parses the YAML as Jsonnet and fails with a Jsonnet syntax error. The scheme is needed because go-jsonnet hands import, importstr and importbin the same cached file contents, so compiled code needs a location of its own. The scheme works with any location: yakl:https://..., yakl:oci://....

Plain YAML has a scheme of its own, yaml:, which isn’t behind a feature gate: import 'yaml:team.yaml' is std.parseYaml(importstr 'team.yaml'). It reads the file as YAML and nothing more, so a ${...} in it stays text. One document gives that document; a file with --- gives a list.

Open questions

• A clearer kubecfg error when an overlay names a missing document, such as “overlay field service matches no document”. Today it shows as a fragment without kind.
• Files that go through kustomize. Should the docs recommend local keys instead of anchors there, since kustomize drops anchors? Documents of local keys and hidden settings documents can’t go through kustomize at all.
• String interpolation. Allowing ${...} inside strings (dev-${params.name}) would make namePrefix and nameSuffix work, but yakl deliberately has no interpolation.
• Schema validation. Kubernetes JSON schemas flag spec+, hidden keys, ${...} values and & keys. A yakl-aware schema could allow them.
• Looping over an object’s keys and values, like ${for k, v in params.env}? Jsonnet has no such syntax, and std.objectKeysValues covers it, so the lean is to stay with plain Jsonnet.
• Several items per iteration. Should a list body be spliced, so one iteration can add a few items? The lean is no: one item per iteration, as in Jsonnet, with a second for in the clause for the rest.
• Parameters. A declared set of inputs for a template, like kro’s schema.spec, instead of std.extVar or an imported parameters file.
• Seeing the Jsonnet of imported files while running show or update, for example with a flag that writes the compiled code of every yakl file to a directory. Today, eval -o jsonnet takes one file at a time.

Feature gates

Features that aren’t stable yet are off until you turn them on. Each one has a name and a stage:

• alpha: off by default. It can change or go away in any release.
• beta: on by default, and it only changes by adding things. You can still turn it off.
• GA: always on. Naming it still works, but turning it off is an error.
• removed: an alpha feature that didn’t work out. Naming it prints a warning and does nothing.

Turning features on and off

$ kubecfg show --features=yakl app.yakl
$ kubecfg show --features=yakl,overlay app.yakl --overlay-code-file prod.yakl
$ kubecfg show --features=yakl --features=overlay app.yakl     # same as above
$ kubecfg show --features=yakl=false app.yakl                  # turn a feature off

KUBECFG_FEATURES=yakl,overlay does the same as the flag. If both are set, the flag wins.

--alpha (or KUBECFG_ALPHA=true) turns on every alpha feature, including ones that future releases add. A feature named in --features beats --alpha, so --alpha --features=httpd=false turns on every alpha feature except httpd. Prefer naming the features you use: upgrading kubecfg then can’t turn on something new.

An unknown name is an error, so a typo doesn’t leave a feature off without telling you.

Seeing what’s on

kubecfg features lists every feature, whether it’s on, and what turned it on or off:

$ kubecfg features --alpha --features=httpd=false
NAME              STAGE  DEFAULT  ENABLED  SOURCE      SINCE
eval              alpha  off      on       --alpha     v0.40.0
traceback         alpha  off      on       --alpha     v0.40.0
pack              alpha  off      on       --alpha     v0.40.0
httpd             alpha  off      off      --features  v0.40.0
...

kubecfg features -o json gives the same as data.

All features

RFC02: Flags From Files (FFF)

Background

(when we’re speaking about flags here, we’re also talking about flags mapped from env variables)

Kubecfg’s behaviour is affected by the presence of some flags. Flags can be either passed on the CLI or taken from env variables mapped to the flags. Flags can be problematic; not all flags do, let me tell you why.

Some flags are part of the “interactive experience”. They are effectively part of the language through which users asks kubecfg to do something for them.

Some flags are user preferences.

But other flags just must be there or Jsonnet just doesn’t evaluate correctly. Those flags are problematic. When the user needs to use them, they stray from the “kubecfg way”.

They are problematic because if they must be there, the user has two options:

1. Remember to provide them every time they invoke the command
2. Create a kubecfg wrapper script (or an env file) and remember to always use it.

I’ve seen a lot of people doing (2), with a wrapper script that sets up all the import-path flags to their vendored dependencies, passing ext vars etc etc.

Once the user does that, it effectively puts vital config information into a language that not particularly well suited for describing configs, namely, shell scripts or makefiles or whatnot.

Proposal

Right before starting up, search for a .kubecfgrc.jsonnet in the current dir, and recursively in the parent directories.

If the file is found, evaluate it and use it to populate the default values of any flags.

The kubecfgrc.jsonnet will be overlaid against this file:

{
    flags: { },
    commands: {
        update: $.flags,
        validate: $.flags,
        ....
    } 
}

Every time a command reads the value of a flag it, it will look it up with commands[commandName][flagName], and use that value unless the user overrides via a CLI flag.

Both CLI and env win over .kubecfgrc.jsonnet.

“Env” here means kubecfg’s own KUBECFG_* variables, plus KUBECONFIG: while KUBECONFIG is set, a kubeconfig in .kubecfgrc.jsonnet is ignored, so a checked-in rc file cannot point update or delete at a different cluster than the one the user’s environment selects.

Other variables that libraries read on their own are not tracked, for example POD_NAMESPACE and KUBERNETES_SERVICE_HOST (in-cluster) or HTTPS_PROXY. An rc value for the matching flag (namespace, server, proxy-url) acts like a CLI flag and wins over them, as it does in kubectl.

In other words the kubecfgrc flags are the default values for the flags.

Example:

$ cat .kubecfgrc.jsonnet
{
    commands+: {
        show+: {
            format: 'json',
            jpath: ['testdata'],
        }
    }
}
$ kubecfg show -e "import 'configmap.jsonnet'"
{
  "apiVersion": "v1",
  "kind": "ConfigMap",
  "metadata": {
    "name": "test"
  }
}

Paths

Some flags such as --jpath contain path names. When a relative path is currently provided to such flags, it is interpreted relatively to the current directory at the time of executing the kubecfg command. We can find those flags because they are marked with an annotation, see cobra.MarkFlagFilename.

However, in the context of a .kubecfgrc.jsonnet file, it would be very practical to resolve such path names relative to the location of the .kubecfgrc.jsonnet file.

This is necessary to meet our goals: we’d like the end user to just kubecfg show some/file.jsonnet and have that just work. Forcing the user to always run the kubecfg command for a specific directory defeats the purpose.

This is going to be more and more important as we add some more features like caching+vendoring.

Features

alpha and features turn features on and off (RFC05). They apply to the whole invocation, so they can only be set under flags, not under commands. Every command reads them, including commands that don’t evaluate files, such as kubecfg features: those look for .kubecfgrc.jsonnet from the current directory and take only the features from it, not their other flags.

Multiple files

Most kubecfg commands can operate on multiple files. When presented with multiple files, commands like show, update, delete, diff will evaluate each of them independently. Ideally, the location of the .kubecfgrc.jsonnet should be relative to the jsonnet file itself, not the current dir where the user happens to run kubecfg from. However, while each of the evaluation is independent, some flags may only make sense for the whole command invocation.

I don’t really know how often people invoke kubecfg on multiple file given that The Kubecfg Way is about having a single root file that “wires up” an entire deployment. I don’t know which flags would be problematic if we’d use different flag values for each file evaluation during a single command execution.

But I know it would be hard to reason about and hard to implement. Hence I propose a simple rule: raise an error if while evaluating multiple files, different .kubecfgrc.jsonnet files result from searches starting from the input jsonnet files. I.e.

$ testdata/fffbad
├── .kubecfgrc.jsonnet
├── one
│   └── demo.jsonnet
└── two
    ├── .kubecfgrc.jsonnet
    └── demo.jsonnet
$ kubecfg show testdata/fffbad/{one,two}/demo.jsonnet
ERROR all evaluated files should resolve to the same .kubecfgrc.jsonnet, found ["testdata/fffbad/two/.kubecfgrc.jsonnet" "testdata/fffbad/.kubecfgrc.jsonnet"] instead

RFC05: Feature gates

Background

kubecfg has a single switch for everything that isn’t stable yet: --alpha (or KUBECFG_ALPHA=true). Today it gates eight separate things:

One switch for all of them has a few problems:

1. All or nothing. Someone who only wants pack also gets YAKL, httpd and every alpha feature added in future releases. Upgrading kubecfg can then change what --alpha turns on, without anyone asking.
2. No sense of maturity. eval has been around for years and is used throughout the docs. YAKL was renamed and changed meaning last week. Both show up as “alpha”.
3. No way to graduate. A feature can only go from “behind --alpha” to “always on” in one step. There’s no stage where it’s on by default but can still be turned off if it causes trouble.
4. The error messages drift. pack and traceback both say “eval is an alpha feature”. Each check is written by hand.

Goals

• Turn features on and off one at a time.
• Give each feature a stage (alpha, beta, GA) that says how far along it is, and lets it move forward by changing one line in a table, not by editing every place that checks it.
• Keep every existing command line, env var and script working. This includes --alpha.
• Make it easy for a project to say which features it relies on. This is where RFC02 comes in.

Non-goals

• Changing the meaning of files that already work. Feature gates are for things that are new. Changes in the meaning of existing input need editions; see Follow-ups.
• Remote config, percentage rollouts, or anything a feature-flag service would do. These are build-time switches for a CLI.

Concepts

Feature

A feature is an entry in a registry compiled into kubecfg:

Stages

stateDiagram-v2
    direction LR
    [*] --> Alpha
    Alpha --> Beta
    Beta --> GA
    Alpha --> Removed
    GA --> [*]

• Alpha is off unless someone asks for it. Anything can change between releases, as YAKL did when it was renamed from jamlet.
• Beta is on by default, so people use it without having asked for it. From this point the feature gets kubecfg’s full compatibility promise: its behaviour can only change by adding things. The off switch is there so a user who hits a problem can go back while it’s fixed.
• GA is just “the feature”. The if enabled(...) checks are deleted from the code. The name stays in the registry forever, so command lines and rc files that mention it keep working.
• Removed is for alpha features that didn’t work out. The name stays in the registry as a tombstone: setting it prints a warning instead of failing, and the name is never reused.

Only alpha features can be removed. A beta feature has been on for everyone, so removing it would break people who never opted in.

Which features can go to beta

Beta means “on by default”: everyone gets it on upgrade, without asking. That’s only safe if turning the feature on can’t change the result of anything that worked before.

• Additive features can follow alpha → beta → GA. They add something that used to be an error: a new command, a new flag, a new import scheme, a new file extension, a new standard library function. Every gate in the table above is additive.
• Semantic features change what existing, valid input means: how objects merge, the order objects are applied in, how show formats output. They can have an alpha gate, so people can try them, but they must never reach beta through a feature gate. Their way to graduate is an edition (see Follow-ups).

The registry records which kind each feature is. A test fails if a semantic feature is set to beta or GA.

Lifecycle rules

• Alpha → beta: docs are written, no breaking changes are planned, and it has been used for real for at least one release.
• Beta → GA: at least one release in beta with no reports that needed the off switch.
• Moving a feature forward is a one-line change in the registry, plus release notes.

UX

Turning features on and off

A new global flag, --features, takes a comma-separated list. It can be repeated.

$ kubecfg show --features=yakl app.yakl
$ kubecfg show --features=yakl,knit app.yakl
$ kubecfg show --features=yakl --features=knit app.yakl     # same as above
$ kubecfg show --features=knit=false app.jsonnet            # turn off a beta feature

• name and name=true turn a feature on, and name=false turns it off.
• Settings for different features combine; a later setting for the same feature wins.
• The env var follows the existing KUBECFG_ prefix: KUBECFG_FEATURES=yakl,knit. It’s another way to set the flag, so, as with every kubecfg flag, --features on the command line replaces it.

What happens with each kind of name

An unknown name is an error, not a warning, so a typo doesn’t silently leave a feature off. An rc file written for a newer kubecfg fails the same way on an older kubecfg, which is what should happen: the project needs something that release doesn’t have.

--alpha stays

--alpha and KUBECFG_ALPHA keep their meaning: turn on every alpha feature. To keep that exact behaviour, --alpha also turns on alpha features added in future releases.

A setting that names a feature beats one that doesn’t, so --alpha --features=httpd=false turns on everything alpha except httpd.

The docs move from kubecfg --alpha ... to --features=.... --alpha stays supported forever, but the docs stop recommending it, since it’s the thing that changes under you on upgrade.

Precedence

For each feature, the first source that names it wins:

1. --features (or KUBECFG_FEATURES)
2. the rc file (RFC02)
3. --alpha (or KUBECFG_ALPHA; alpha features only)
4. the stage’s default

Note that this works per feature, which is different from how a plain flag works. With a plain flag, a value on the command line replaces the value from the rc file. With --features, --features=pack on the command line adds pack to the rc file’s yakl; it doesn’t replace it.

Seeing what’s on

$ cat .kubecfgrc.jsonnet
{ flags+: { features+: { yakl: true } } }
$ kubecfg features --alpha --features=httpd=false
NAME              STAGE  DEFAULT  ENABLED  SOURCE              SINCE
eval              alpha  off      on       --alpha             v0.40.0
traceback         alpha  off      on       --alpha             v0.40.0
pack              alpha  off      on       --alpha             v0.40.0
httpd             alpha  off      off      --features          v0.40.0
overlay           alpha  off      on       --alpha             v0.40.0
yakl              alpha  off      on       .kubecfgrc.jsonnet  v0.40.0
kustomize-import  alpha  off      on       --alpha             v0.40.0
binary-import     alpha  off      on       --alpha             v0.40.0

SOURCE is the place that decided the setting, from the list under Precedence. This answers “why is this feature off?” without guessing. An env var shows as its flag: KUBECFG_ALPHA shows as --alpha, because it is another way to set that flag.

-o json gives the same table as data. The docs page listing all features is generated from the registry, so it can’t get out of date.

Error messages

Every gate check produces the same kind of error, generated from the registry:

$ kubecfg show app.yakl
ERROR app.yakl: yakl is an alpha feature; turn it on with --features=yakl

Migration

The first release with feature gates registers the eight features above, all at alpha. On that day nothing behaves differently: --alpha turns on the same things, and nothing is on without it.

Moving any of them to beta is a separate decision, made one feature at a time in later releases.

Implementation sketch

A small package, pkg/features:

package features

type Stage int

const (
	Alpha Stage = iota
	Beta
	GA
	Removed
)

type Kind int

const (
	Additive Kind = iota
	Semantic
)

type Feature struct {
	Name  string
	Stage Stage
	Kind  Kind
	Since string
	Doc   string
}

var (
	Eval = register(Feature{Name: "eval", Stage: Alpha, Since: "v0.40.0", Doc: "the eval command"})
	YAKL = register(Feature{Name: "yakl", Stage: Alpha, Since: "v0.40.0", Doc: "YAML with Jsonnet expressions"})
	// ...
)

// Source is one place features are set from: --features, or the rc file.
type Source struct {
	Name  string   // shown in kubecfg features and in errors
	Items []string // "name", "name=true" or "name=false"
}

// Gates is the set of enabled features for one kubecfg invocation, with where each setting came from.
type Gates struct{ /* ... */ }

// New resolves every feature: the first source that names it wins, then alpha, then the default.
func New(alpha bool, sources ...Source) (*Gates, error)

func (g *Gates) Enabled(f *Feature) bool

// Require returns the standard "X is an alpha feature; turn it on with --features=X" error.
func (g *Gates) Require(f *Feature) error

• cmd/root.go builds one Gates from --features (which viper already merges with KUBECFG_FEATURES), the rc file and --alpha, and passes it down. Gates applies the precedence itself, so it knows which source decided each feature; that is the SOURCE column.
• The alpha bool that goes through kubecfg.WithAlpha, utils.MakeUniversalImporter and kustomizeImporter becomes a *features.Gates. These are exported Go functions that other programs may call, so the old signatures stay as wrappers. WithAlpha(true) means “turn on all alpha features”, exactly as --alpha does.
• wasm/cmd/kubecfg-show has its own --alpha flag; it gets --features too, using the same package.
• Tests:
  • Every feature is listed in the docs page.
  • No two features share a name.
  • A GA feature has no Enabled calls left. This is checked by searching the source for its variable.
  • No semantic feature is beta or GA.

Why not k8s.io/component-base/featuregate

Kubernetes has a feature gate package with the same alpha/beta/GA model and a --feature-gates=Name=true flag. It would be a reasonable base, and kubecfg already depends on parts of Kubernetes. Reasons to write our own (about 200 lines):

• its flag syntax is CamelCase names in a map (Foo=true,Bar=false), which doesn’t match kubecfg’s kebab-case flags;
• it has no notion of a bulk switch like --alpha;
• it doesn’t record where each setting came from, which kubecfg features needs;
• it has no tombstones for removed names, and no additive/semantic distinction.

This is an open question; see below.

Relationship with RFC02 (Flags From Files)

RFC02 lets a project check in a .kubecfgrc.jsonnet that sets default flag values. --features is a flag, so it fits in directly, and this is the main way a project says what it relies on:

// .kubecfgrc.jsonnet
{
  flags+: {
    features+: { yakl: true, knit: true },
  },
}

With this file checked in, every contributor and every CI job gets the same features with a plain kubecfg show. No wrapper scripts, and no --alpha that turns on more than the project needs.

What RFC02 needs for this to work:

• An object value for features. The RFC02 prototype turns arrays into repeated flag values and everything else into a string. features should accept both an array (['yakl', 'knit']) and an object ({yakl: true, knit: false}). The object form is better, because overlays can change one feature (features+: {knit: false}).
• Per-feature merging. As described under Precedence, the rc file’s features and the command line’s are combined feature by feature. RFC02’s general rule (a value on the command line replaces the default) doesn’t apply to this one flag. So the rc file’s features can’t just become the flag’s default value, or --features=pack would replace the rc file’s yakl. RFC02 has to hand them to features.New as a source of their own.
• Global, not per command. RFC02 allows settings under commands.show, commands.update and so on. Features should only be set under flags. If show had knit and update didn’t, show would print something different from what update applies. kubecfg rejects features under commands.
• The rc file is evaluated with default gates. The rc file decides the features, so its own evaluation can’t depend on them. It’s a plain .jsonnet file evaluated with every gate at its default.
• RFC02’s rule that all input files must resolve to the same rc file applies as it is. Features are a property of one invocation.

This RFC doesn’t depend on RFC02: --features and KUBECFG_FEATURES work on their own, and the rc file support lands with RFC02.

Follow-ups

These aren’t part of this RFC, but they show where feature gates fit.

Editions

Feature gates are per invocation: one set of features for the whole run. That’s right for additive features, but it can’t handle changes in meaning, for two reasons:

• a semantic change must never become the default for files that didn’t ask for it;
• a library imported from somewhere else must keep the meaning it was written for, whatever the importing project turns on.

Those need editions: the language version is decided for each file or module, not for each run. Go does this with the go line in go.mod. The nearest go.mod above a file decides its language semantics; that’s how Go 1.22 changed loop variables without breaking older modules. kubecfg could find the edition the same way, from the nearest rc-like file above each imported file. For an OCI bundle, the edition would come from the bundle’s metadata, and for a plain https:// import it would default to the oldest edition.

The two fit together: an alpha feature gate is how people try a semantic change early, and an edition is how it ships. The gate never goes to beta.

knit

The keyed-array overlay merge (kubecfg.knit) is the first feature born under this system. It adds a new namespace to the standard library and changes nothing that exists, so it’s additive and can follow alpha → beta → GA.

An implicit kubecfg local

Making the standard library available as kubecfg without an explicit import is additive: a free kubecfg variable is a static error today, so no working file can depend on its absence. It can start as an alpha feature (implicit-stdlib). It needs a small go-jsonnet change, letting the embedding program add globals the way std is added. Adding the text to the start of each file would change what importstr returns for .jsonnet files.

RFC06 replaces this. It predefines kubecfg only in code that isn’t in a Jsonnet file, which needs no go-jsonnet change, and explains why Jsonnet files are left out.

Caching and vendoring (RFC03)

--cache-dir, --ci and the pruning options in RFC03 are ordinary flags, set through RFC02. Vendoring itself could ship as an alpha feature, vendoring.

OCI bundles

A bundle made with kubecfg pack may need features to evaluate, for example a bundle containing YAKL. pack could record the features it was built with in the bundle’s metadata, next to the kubecfg version it already records. When importing, kubecfg would fail with a clear message listing the missing features. It wouldn’t turn them on by itself: remote code shouldn’t be able to turn on alpha features.

Open questions

• The flag’s name. --features is short and matches kubecfg’s style. --feature-gates is what Kubernetes users know. One could be an alias of the other.
• GA with =false. This RFC proposes an error, since the user asked for something kubecfg can no longer do. A warning would be gentler, but it would silently do something other than what was asked.
• How long in beta. Is one release enough before GA?
• Our own package or component-base/featuregate. See above.
• Should Jsonnet code see feature gates? A library could want to say “this needs knit” with a clear error, e.g. kubecfg.requireFeature('knit'). Letting code read the gates (if enabled then ... else ...) would make output depend on flags in a way that’s hard to see. Requiring a feature seems fine; reading one doesn’t.
• Does --alpha ever print a deprecation warning? It can never be removed. A warning that suggests naming the features would push people toward pinning them.

RFC06: kubecfg predefined outside Jsonnet files

Summary

To use kubecfg’s library, code has to import it first:

local kubecfg = import 'kubecfg.libsonnet';

This RFC makes the name kubecfg available without that line, but only in Jsonnet code that isn’t in a Jsonnet file: code given on the command line, and expressions in .yakl files. Jsonnet files (.jsonnet, .libsonnet) still need the import, so editors, linters and other Jsonnet tools can still read them.

The predefined kubecfg is always the copy of the library built into kubecfg. It isn’t behind a feature gate: code that works today is evaluated exactly as before, and only code that fails with Unknown variable: kubecfg changes. This RFC replaces the “implicit kubecfg local” follow-up in RFC05.

Background

Where the import costs something

In a Jsonnet file, the import is one line next to the file’s other imports. Dropping it saves almost nothing.

Other places have no good spot for it:

• eval -e. To list every object by kind, namespace and name:

  $ kubecfg eval app.jsonnet -e 'local kubecfg = import "kubecfg.libsonnet"; kubecfg.fold(kubecfg.layouts.gvkNsName, $, {})'
  

  (Before this RFC, this failed even with the import; see Imports from command-line code.)

• --overlay-code, --exec, --ext-code, --tla-code. The same: the import has to be part of the one-line expression.

• .yakl files. The file needs a document just for the import, and its author has to know about file locals before calling one function:

  '&kubecfg': ${import 'kubecfg.libsonnet'}
  ---
  apiVersion: v1
  kind: Pod
  metadata:
    name: web
  spec:
    containers:
    - name: web
      image: ${kubecfg.resolveImage('nginx:1.27')}
  

Why not everywhere

RFC05 proposed making kubecfg predefined in every file. That has four problems.

1. Editors and linters. go-jsonnet has exactly one predefined name, std. It’s written into its static check (static_analyzer.go), and there’s no API to add another. The C++ Jsonnet is the same. So every tool built on either one (language servers in editors, jsonnet-lint, jsonnet, Tanka) would report Unknown variable: kubecfg in a file that relied on the predefined name. go-jsonnet rejects the whole file at the first unknown name, before evaluating any of it, so a tool that evaluates the file gets nothing past that error. A missing kubecfg.libsonnet can be fixed in the tool’s settings, by adding a search path or vendoring the file. A name the tool doesn’t know can’t.
2. Libraries used by other tools. The plain-Jsonnet parts of kubecfg.libsonnet (fold, deepMap, toOverlay, layouts, getPath) work in any Jsonnet tool if the file is vendored. std.native returns null for a function the tool doesn’t have, so only the native functions fail, and only when called. If kubecfg were predefined in files, a library author could leave out the import and never notice, because the library would work in every kubecfg run. The first person to see the error would be someone using the library from another tool.
3. importstr. Without changing go-jsonnet, the only way to predefine a name in a file is to add text to the start of the file when it’s imported. go-jsonnet asks the importer for a file in the same way for import, importstr and importbin, so the importer can’t tell them apart. importstr 'app.jsonnet' would return the file with the added text, which changes the result of a file that works today. This is why YAKL needs the yakl: scheme. RFC05 suggested a go-jsonnet change that lets the program add names the way std is added. That fixes this problem, but not problems 1 and 2.
4. The entry file only. Predefining it only in the file named on the command line avoids problem 3. But then base.jsonnet works with kubecfg show base.jsonnet and fails when prod.jsonnet imports it. Whether a file works would depend on who loads it.

The rule

kubecfg is predefined in Jsonnet code that isn’t in a Jsonnet file.

Editors and linters read files. Code that isn’t in a file is only ever read by kubecfg, so no other tool has to understand it. Code in a file is read by other tools, so it keeps the import.

--ext-str and --tla-str values are strings, not code, so the rule doesn’t apply to them.

What “predefined” means

kubecfg checks the code first. If the code reads the variable kubecfg without declaring it, kubecfg evaluates it as if it started with this line:

local kubecfg = import 'internal:///kubecfg.libsonnet';

internal:///kubecfg.libsonnet is the copy of kubecfg.libsonnet built into kubecfg. The rest follows from that line:

1. It’s always the built-in copy. Neither the search path (-J, -U, KUBECFG_JPATH) nor a file next to the code changes it, in the same way that they can’t change std. Why:

   • Nothing in the code says which file the name comes from, so no file should be able to change it. An explicit import is different: you wrote it, and it follows Jsonnet’s lookup rules as before.
   • Much of the library wraps native functions compiled into kubecfg. The built-in copy always matches them. A vendored copy can be older or newer than the kubecfg running it.
   • People vendor kubecfg.libsonnet so that other tools can find it when a Jsonnet file imports it. Those files keep their import, and it still finds the vendored copy. The predefined name only exists in code that other tools never read.
   • There’s nothing to look up. A lookup can fetch a -U URL, or a file next to a .yakl file that was loaded over https:// or from an OCI bundle, and it can fail. The built-in copy is the same everywhere.

   So one run can use two copies: the built-in one through the predefined name, and a vendored one through an explicit import. They don’t interfere: Jsonnet code has no side effects and no shared state, so a value from one copy works with functions from the other. They can only differ in what their functions return, if their versions differ. To use another copy, import it yourself: your import hides the predefined name (point 3).

2. It never reaches into files. In Jsonnet, an imported file doesn’t see the locals of the code that imports it. So kubecfg show -e 'import "base.jsonnet"' doesn’t make kubecfg available inside base.jsonnet.

3. Your own names win. A local, a function parameter or an object local named kubecfg hides it, as with any Jsonnet local. In a .yakl file, an anchor (&kubecfg) or a local key ('&kubecfg':) hides it. In eval -e, so does a top-level argument named kubecfg.

4. Code that works today doesn’t change. The line is only added to code that reads kubecfg without declaring it, and today that code fails with Unknown variable: kubecfg. Everything else is evaluated exactly as before, down to the columns in its error messages. kubecfg finds out with Jsonnet’s own check, the one that reports Unknown variable, so the answer is exact: a field named kubecfg, as in x.kubecfg, or the word in a string or a comment, isn’t a use.

5. Line numbers in errors don’t change. The line goes in front of the code, on its first line, so only the columns of that first line move. (For eval -e the columns already don’t match what you typed, because kubecfg wraps the expression in code of its own. Compiled YAKL doesn’t keep columns exactly either.)

In .yakl files

The YAKL compiler checks the compiled file in the same way. If the file reads kubecfg without declaring it, the compiler adds the local. It means the same as this local key in a document of local keys:

'&kubecfg': ${import 'internal:///kubecfg.libsonnet'}

The compiler adds the local itself, not a document. A document in front of the file would break a file whose first document is <<+: S, because that document has to be the first.

A file that doesn’t use kubecfg, or that declares it wherever it uses it, for example with '&kubecfg':, compiles to the same Jsonnet as before.

A local never changes the shape of a YAKL file’s value (a document, a list or an object), so the added one doesn’t either. Line numbers stay as they are: the compiler writes the local in front of the code of line 1 and doesn’t add a line.

kubecfg eval -o jsonnet prints the Jsonnet with the import in it, after the file’s first comments. For the Pod above, without its first document, it prints:

local kubecfg = import 'internal:///kubecfg.libsonnet';
{
  apiVersion: 'v1',
  kind: 'Pod',
  metadata: {
    name: 'web',
  },
  spec: {
    containers: [
      {
        name: 'web',
        image: kubecfg.resolveImage('nginx:1.27'),
      },
    ],
  },
}

It’s the code that kubecfg evaluates, so it names the built-in copy. Saved as a .jsonnet file, it works in kubecfg as it is. For other Jsonnet tools, change the import to 'kubecfg.libsonnet' and vendor the library, as for any Jsonnet file that uses it.

No feature gate

The rule is on for everyone, without a feature gate:

• It’s additive in the strictest sense. By point 4, code that works today is evaluated exactly as before. Only code that fails today with Unknown variable: kubecfg changes, and it starts working. RFC05 calls such features additive.
• A gate would cancel the gain. The point is to type less on the command line. Typing --features=predefined-kubecfg instead of the import saves almost nothing, and in YAKL files it would mean two gates.
• The unstable parts are gated anyway. eval, --overlay-code and YAKL are alpha features, so the rule can still change there while they are alpha. What becomes stable right away is the rule in --exec, --ext-code and --tla-code, and it can’t be taken back from those flags later. That’s accepted: the reason for the rule, that no other tool reads command-line code, holds for them as it does for eval -e.
• OCI bundles. kubecfg pack stores .yakl files as source, and the kubecfg that imports the bundle compiles them. A bundle with a .yakl file that uses the predefined kubecfg needs a kubecfg with this RFC, as it already needs one with yakl turned on.

Imports from command-line code

Before this RFC, imports didn’t work in eval -e, --ext-code and --tla-code, not even explicit ones:

$ kubecfg --features=eval eval app.jsonnet -e "(import 'kubecfg.libsonnet').isK8sObject({})"
RUNTIME ERROR: Get "/kubecfg.libsonnet": unsupported protocol scheme ""

The cause: go-jsonnet tells kubecfg’s importer the directory of the code doing the import, taken from the code’s name, and the importer resolves a relative import against it. For a file, that directory is a file:// URL. For this code it isn’t a URL. eval -e is named after the path of the evaluated file, so its directory is a path like dir/, or "" for a file in the current directory. go-jsonnet names --ext-code and --tla-code values <extvar:NAME> and <top-level-arg:NAME>, so their directory is "". Resolving against a path gives a URL with no scheme. The importer tried to fetch it, failed, and reported that as an error instead of trying the search path. --exec and --overlay-code are evaluated from the current directory as a file:// URL, so they worked.

The fix: a directory without a scheme is a path relative to the current directory. So eval -e imports relative to the evaluated file, and --ext-code and --tla-code import relative to the current directory, as the jsonnet command does. The search path comes after that, as usual.

The predefined name doesn’t need this fix, because its import is an absolute URL. Explicit imports in the same code do, and an explicit import is how you use a copy other than the built-in one.

Error message

Because of the rule, code that works in -e can fail when it’s moved into a file. To make that quick to fix, when kubecfg fails with Unknown variable: kubecfg, it prints a hint after the error. Everywhere else kubecfg is predefined, so the code that failed is in a Jsonnet file:

hint: kubecfg is not predefined in Jsonnet files. Add this line at the top of the file:
  local kubecfg = import 'kubecfg.libsonnet';

The hint suggests 'kubecfg.libsonnet', not internal:///, because a Jsonnet file should keep an import that other tools can find.

Alternatives

• Everywhere. See Why not everywhere.
• Only in kubecfg’s own places (eval -e, --overlay-code, YAKL), leaving out --exec, --ext-code and --tla-code because the jsonnet command has the same flags. Nothing stable would change. But kubecfg eval -e and kubecfg show -e would disagree, and the rule would become a list to remember instead of one sentence.
• Resolve it like an explicit import (import 'kubecfg.libsonnet', through the search path). Then kubecfg would be the same file as an explicit import at the same spot. But a file on the search path or next to the code would change what a name you never imported means, and the lookup can fetch URLs and fail. See point 1.
• Behind a feature gate. See No feature gate.
• A go-jsonnet change to add predefined names (RFC05’s plan). It fixes importstr, but not editors, linters or shared libraries. This RFC doesn’t need it: every place it covers is code that kubecfg already wraps or compiles.

Implementation

• utils/importer.go, expandImportToCandidateURLs: a directory without a scheme is a path relative to the current directory. This is the import fix.
• pkg/yakl: KubecfgLocal is the line. PredefineKubecfg(code) puts it in front of code that reads kubecfg without declaring it, and returns any other code as it is. The check is freeVars, which the compiler already used to find the variables that a key reads. Compile and Desugar do the same for a yakl file, after its first comments.
• yakl.PredefineKubecfg is called wherever kubecfg builds code from a flag:
  • pkg/kubecfg/eval.go: on the whole eval -e snippet, so a top-level argument named kubecfg still wins; and on the expression that --trace gives to ursonnet.
  • cmd/root.go, readObjs: --exec.
  • pkg/kubecfg/vm.go, ReadObjects: the whole --overlay-code expression. This is safe because the input file is imported, and imports don’t see the line (point 2).
  • pkg/kubecfg/vm.go, JsonnetVM: --ext-code and --tla-code values. Not --ext-str or --tla-str, which are strings. For the *-file flags the value is an import, which the line doesn’t reach.
• cmd.Hint makes the hint, and main.go prints it after the error.
• Tests: each row of the table, names that hide the predefined one, a vendored copy on -J that the predefined name ignores and an explicit import finds, .yakl files that use kubecfg, don’t, or declare their own, a .jsonnet file that still fails, with the hint, and imports from eval -e, --ext-code and --tla-code.
• Docs: eval.md, show.md, yakl.md, and the map entry t8y:/lang/implicit-stdlib.

Follow-ups

• A flag that replaces the built-in library, for example to try a changed kubecfg.libsonnet without building kubecfg. It would replace what internal:///kubecfg.libsonnet serves, so it would change the built-in library everywhere: the predefined name, explicit internal:/// imports, and the last place that import 'kubecfg.libsonnet' looks. It wouldn’t depend on -J. It adds something without changing anything that works, so it can come later.