Standard library · Encoding and data

encoding/yaml

Reading YAML by path, building and writing it, and converting it to JSON.

import encoding.yaml

YAML is the usual format for configuration files. This package converts YAML to JSON internally, which is what makes its reading functions simple and what lets it work alongside the json package.

The reading functions take a path: keys separated by dots, with a list index in square brackets, as in app.hosts[1]. A path that is not there gives an empty string or zero; to tell that apart from a real value, use Has.

It reads the YAML that configuration files use: nested maps, lists in dashed and [flow] form, quoted strings, | and > block text, comments and several documents. For flat key = value settings with sections and overrides, config is simpler, and it reads YAML too.

The things people get wrong:

  • Anchors and aliases (&base, *base, <<: *base) are not supported. A value like &a 1 is read as the string &a 1, and an anchor on a map, as in base: &base followed by indented keys, loses the rest of the document without an error. Write such files out in full, or expand them with another tool first.
  • yes, no, on and off are strings, as in YAML 1.2, so GetBool is false for them. Only true and false are booleans.
  • GetInt returns a 32-bit int and wraps around for larger numbers. Read sizes, ids and timestamps through ToJSON into an i64 field (see below).
  • Every path function parses the whole text again. For many reads from a large file, call ToJSON or Decode once and work on the result.

Guide: read a config file with defaults

The path functions are the quickest way to read a config file. Check first that the sections you depend on exist with HasKeys, then read each setting with the typed getter. For optional settings, wrap Get in a small helper that falls back to a default when Has says the key is not there.

encoding-yaml-guide-config.salam
import encoding.yaml

func setting(text: str, path: str, def: str): str:
    if yaml.Has(text, path):
        ret yaml.Get(text, path)
    end
    ret def
end

func main:
    text := `# settings for the shop
server:
  host: 0.0.0.0
  port: 8080
  workers: 4
log:
  level: debug
features:
  search: true
  beta: false
`
    mut required := Vector {} as Vector<str>
    required.push("server")
    required.push("log")
    if not yaml.HasKeys(text, required):
        println "config is missing a section"
        ret 1
    end

    println "listen on", yaml.Get(text, "server.host") + ":" + yaml.Get(text, "server.port")
    println "workers:", yaml.GetInt(text, "server.workers")
    println "log level:", setting(text, "log.level", "info")
    println "log file:", setting(text, "log.file", "stdout")
    println "search on:", yaml.GetBool(text, "features.search"), "beta on:", yaml.GetBool(text, "features.beta")
    println "feature flags:", yaml.Keys(text, "features")
end
Outputlisten on 0.0.0.0:8080 workers: 4 log level: debug log file: stdout search on: true beta on: false feature flags: ["search", "beta"]

Comments starting with # are ignored. GetInt and GetBool return 0 and false for a missing key, which is also a real value, so a default other than zero needs the Has check.

Guide: YAML into a struct

For a config with many fields, a struct is easier to pass around than a text and a list of paths. There is no YAML Unmarshal of its own, but there does not need to be: ToJSON turns the YAML into JSON, and encoding/json fills the struct, with the same @json names and defaults.

encoding-yaml-guide-struct.salam
import encoding.yaml
import encoding.json
import encoding.encerr

struct Service:
    pub name: str = ""
    pub image: str = ""
    pub replicas: int = 1
    pub ports: Vector<int> = Vector {}
end

func main:
    text := `name: api
image: shop/api:2.3
ports:
  - 80
  - 443
`
    mut s := Service {}
    mut e := encerr.NoError()
    if not json.UnmarshalLenient(yaml.ToJSON(text), s, e):
        println "bad service:", e.message
        ret 1
    end
    println s.name, "runs", s.image
    println "replicas:", s.replicas
    println "ports:", s.ports.len(), s.ports.get(0), s.ports.get(1)
end
Outputapi runs shop/api:2.3 replicas: 1 ports: 2 80 443

replicas is not in the file, so it keeps its default of 1 because the lenient form was used. With json.Unmarshal the missing key would be an error.

Guide: walk a list of records

A list of objects, such as the jobs of a CI file, is read with Len and an index in the path. Build the path for item i once and add the field names to it. [lint] is the short flow form of a list, and reads the same way as the dashed form.

encoding-yaml-guide-list.salam
import encoding.yaml
import str

func main:
    text := `jobs:
  - name: lint
    run: salam format --check
  - name: test
    run: sh tests/run.sh
    needs: [lint]
  - name: deploy
    run: sh deploy.sh
    needs: [test]
    branch: main
`
    n := yaml.Len(text, "jobs")
    println n, "jobs"
    repeat n in i:
        at := "jobs[" + str.FromInt(i) + "]"
        line := yaml.Get(text, at + ".name") + ": " + yaml.Get(text, at + ".run")
        if yaml.Has(text, at + ".needs"):
            println line, "(after", yaml.Get(text, at + ".needs[0]") + ")"
        else:
            println line
        end
        if yaml.Has(text, at + ".branch"):
            println "  only on", yaml.Get(text, at + ".branch")
        end
    end
end
Output3 jobs lint: salam format --check test: sh tests/run.sh (after lint) deploy: sh deploy.sh (after test) only on main

Guide: several documents in one file

Kubernetes manifests and some log formats put several YAML documents in one file, separated by a line with ---. SplitDocuments cuts them apart, and each piece is then an ordinary YAML text for the other functions.

encoding-yaml-guide-multi.salam
import encoding.yaml

func main:
    text := `kind: Deployment
metadata:
  name: web
spec:
  replicas: 3
---
kind: Service
metadata:
  name: web
spec:
  port: 80
---
kind: ConfigMap
metadata:
  name: web-config
`
    docs := yaml.SplitDocuments(text)
    println docs.len(), "documents"
    each d in docs:
        println "-", yaml.Get(d, "kind"), yaml.Get(d, "metadata.name"), "valid:", yaml.Valid(d)
    end
    println "deployment as JSON:", yaml.ToJSON(docs.get(0))
end
Output3 documents - Deployment web valid: true - Service web valid: true - ConfigMap web-config valid: true deployment as JSON: {"kind": "Deployment", "metadata": {"name": "web"}, "spec": {"replicas": 3}}

ToJSON is also how you hand YAML to anything that speaks JSON, such as an HTTP API. Going the other way, Dump writes JSON text as YAML.

Guide: what type did the value get?

YAML decides the type of a value from how it looks, and the result is not always what the author meant. TypeOf shows what the reader saw. Here yes stayed a string, 007 kept its zeros as a string, and the five gigabyte limit is too big for GetInt but fine for an i64 field.

encoding-yaml-guide-types.salam
import encoding.yaml
import encoding.json
import encoding.encerr

struct Limits:
    pub max_upload: i64 = 0
    pub public: bool = false
    pub code: str = ""
end

func load(text: str):
    mut l := Limits {}
    mut err := encerr.EncodingError {}
    if json.UnmarshalLenient(yaml.ToJSON(text), l, err):
        println "  loaded:", l.max_upload, l.public, l.code
    else:
        println "  refused:", err.message
    end
end

func main:
    text := "max_upload: 5368709120\npublic: yes\ncode: 007\n"
    println "public is a", yaml.TypeOf(text, "public"), "so GetBool says", yaml.GetBool(text, "public")
    println "code is a", yaml.TypeOf(text, "code"), "with the text", yaml.Get(text, "code")
    println "GetInt overflows:", yaml.GetInt(text, "max_upload")
    println "as JSON:", yaml.ToJSON(text)
    load(text)
    load("max_upload: 5368709120\npublic: true\ncode: \"007\"\n")
end
Outputpublic is a string so GetBool says false code is a string with the text 007 GetInt overflows: 1073741824 as JSON: {"max_upload": 5368709120, "public": "yes", "code": "007"} refused: value does not match the expected type loaded: 5368709120 true 007

The struct route also checks the types for you: public: yes is refused instead of read as false. Writing true and quoting values that must stay text, such as "007", makes the file mean the same to every reader.

Which function for which job

JobUse
Read a few settingsGet, GetInt, GetBool, GetFloat
Tell a missing key from an empty oneHas, HasKeys
Look at the structureTypeOf, Len, Keys
A whole file into a structToJSON and json.UnmarshalLenient
Unknown shapeDecode to a Value, Encode back
Write YAMLObject, Member, or Dump from JSON
Files with ---SplitDocuments

Functions

Get

func Get(text: str, path: str): str
The value at a path as text; strings come back without their quotes, and a missing path or null gives an empty string.
encoding-yaml-Query.salam
import encoding.yaml

func main:
    text := "app:\n  name: salam\n  port: 8080\n  debug: true\n  ratio: 0.5\n  hosts:\n    - a.example\n    - b.example\n"
    println yaml.Valid(text), yaml.Has(text, "app.name"), yaml.Has(text, "app.user")
    println yaml.Get(text, "app.name"), yaml.GetInt(text, "app.port"), yaml.GetBool(text, "app.debug"), yaml.GetFloat(text, "app.ratio")
    println yaml.TypeOf(text, "app.hosts"), yaml.Len(text, "app.hosts"), yaml.Get(text, "app.hosts[1]")
    println yaml.Keys(text, "app")
    mut required := Vector {} as Vector<str>
    required.push("app")
    println yaml.HasKeys(text, required)
    required.push("db")
    println yaml.HasKeys(text, required)
    println yaml.ToJSON(text)
    println yaml.SplitDocuments("a: 1\n---\nb: 2\n").len()
end
Outputtrue true false salam 8080 true 0.5 list 2 b.example ["name", "port", "debug", "ratio", "hosts"] true false {"app": {"name": "salam", "port": 8080, "debug": true, "ratio": 0.5, "hosts": ["a.example", "b.example"]}} 2

Has

func Has(text: str, path: str): bool
True when the path is there.

The example for this item is shown with the ones above: encoding-yaml-Query.salam

GetInt

func GetInt(text: str, path: str): int
The whole number at a path as a 32-bit int; 0 when it is missing. Larger numbers wrap around.

The example for this item is shown with the ones above: encoding-yaml-Query.salam

GetFloat

func GetFloat(text: str, path: str): f64
The decimal value at a path.

The example for this item is shown with the ones above: encoding-yaml-Query.salam

GetBool

func GetBool(text: str, path: str): bool
The boolean at a path: true only for true. yes and on are strings and give false.

The example for this item is shown with the ones above: encoding-yaml-Query.salam

TypeOf

func TypeOf(text: str, path: str): str
The kind of value at a path: map, list, string, int, float, bool, null, or none when it is absent.

The example for this item is shown with the ones above: encoding-yaml-Query.salam

Len

func Len(text: str, path: str): int
The number of list elements, or object keys, at a path.

The example for this item is shown with the ones above: encoding-yaml-Query.salam

Keys

func Keys(text: str, path: str): Vector<str>
The keys of the object at a path; an empty path means the top level.

The example for this item is shown with the ones above: encoding-yaml-Query.salam

HasKeys

func HasKeys(text: str, required: Vector<str>): bool
True when every key you name is at the top level of the document; for checking a configuration file.

The example for this item is shown with the ones above: encoding-yaml-Query.salam

Decode

func Decode(text: str, err &: encerr.EncodingError): value.Value
Turns YAML into a dynamic Value.
encoding-yaml-Build.salam
import encoding.yaml
import encoding.value
import encoding.encerr
import str

func main:
    mut members := Vector {} as Vector<str>
    members.push(yaml.Member("name", "salam"))
    members.push(yaml.MemberInt("port", 8080))
    members.push(yaml.MemberBool("debug", false))
    members.push(yaml.MemberFloat("ratio", 0.25))
    mut hosts := Vector {} as Vector<str>
    hosts.push(yaml.Str("a: b"))
    members.push(yaml.MemberRaw("hosts", yaml.Array(hosts)))
    object := yaml.Object(members)
    println object
    println yaml.Dump(object)
    println str.Contains(yaml.DumpTabs(`{"a": {"b": 1}}`), "\tb: 1"), yaml.DumpWith(`{"a": [1]}`, "    ")
    println yaml.Escape("one: two")
    println yaml.Parse("x: 1")
    mut e := encerr.NoError()
    v := yaml.Decode("city: Tehran\npopulation: 9000000\n", e)
    println value.AsInt(value.ObjectGet(v, "population"))
    println yaml.Encode(v)
end
Output{"name": "salam", "port": 8080, "debug": false, "ratio": 0.25, "hosts": ["a: b"]} name: salam port: 8080 debug: false ratio: 0.25 hosts: - "a: b" true a: - 1 one: two {"x": 1} 9000000 city: Tehran population: 9000000

Encode

func Encode(v: value.Value): str
Turns a Value into YAML.

The example for this item is shown with the ones above: encoding-yaml-Build.salam

Escape

func Escape(s: str): str
Escapes a string so it can go inside YAML quotes.

The example for this item is shown with the ones above: encoding-yaml-Build.salam

Str

func Str(value: str): str
Turns a string into a ready value, with quotes.

The example for this item is shown with the ones above: encoding-yaml-Build.salam

Member

func Member(key: str, value: str): str
One key and string pair, for building an object.

The example for this item is shown with the ones above: encoding-yaml-Build.salam

MemberInt

func MemberInt(key: str, value: i64): str
A key and whole number pair.

The example for this item is shown with the ones above: encoding-yaml-Build.salam

MemberBool

func MemberBool(key: str, value: bool): str
A key and boolean pair.

The example for this item is shown with the ones above: encoding-yaml-Build.salam

MemberFloat

func MemberFloat(key: str, value: f64): str
A key and decimal pair.

The example for this item is shown with the ones above: encoding-yaml-Build.salam

MemberRaw

func MemberRaw(key: str, raw: str): str
A key paired with a ready-made value.

The example for this item is shown with the ones above: encoding-yaml-Build.salam

Object

func Object(members: Vector<str>): str
Puts the pairs together into an object. The result is JSON, which Dump turns into YAML.

The example for this item is shown with the ones above: encoding-yaml-Build.salam

Array

func Array(items: Vector<str>): str
Puts values together into a list.

The example for this item is shown with the ones above: encoding-yaml-Build.salam

DumpWith

func DumpWith(text: str, unit: str): str
Converts JSON to YAML with the indent unit of your choice.

The example for this item is shown with the ones above: encoding-yaml-Build.salam

Dump

func Dump(text: str): str
Converts JSON to YAML indented with two spaces; strings that need them get quotes.

The example for this item is shown with the ones above: encoding-yaml-Build.salam

DumpTabs

func DumpTabs(text: str): str
Like Dump, with tabs.

The example for this item is shown with the ones above: encoding-yaml-Build.salam

Parse

func Parse(text: str): str
Reads YAML and returns the same data as JSON.

The example for this item is shown with the ones above: encoding-yaml-Build.salam

ToJSON

func ToJSON(text: str): str
Converts YAML to JSON.

The example for this item is shown with the ones above: encoding-yaml-Query.salam

Valid

func Valid(text: str): bool
True when the text can be read as YAML. It is lenient: it does not reject tabs used for indenting.

The example for this item is shown with the ones above: encoding-yaml-Query.salam

SplitDocuments

func SplitDocuments(text: str): Vector<str>
Splits a file of several documents, separated by ---, into separate ones.

The example for this item is shown with the ones above: encoding-yaml-Query.salam