encoding/yaml
Reading YAML by path, building and writing it, and converting it to JSON.
import encoding.yaml
YAML is the usual format for configuration files. This package converts YAML to JSON internally, which is what makes its reading functions simple and what lets it work alongside the json package.
The reading functions take a path: keys separated by dots, with a list index in square brackets, as in app.hosts[1]. A path that is not there gives an empty string or zero; to tell that apart from a real value, use Has.
It reads the YAML that configuration files use: nested maps, lists in dashed and [flow] form, quoted strings, | and > block text, comments and several documents. For flat key = value settings with sections and overrides, config is simpler, and it reads YAML too.
The things people get wrong:
- Anchors and aliases (
&base,*base,<<: *base) are not supported. A value like&a 1is read as the string&a 1, and an anchor on a map, as inbase: &basefollowed by indented keys, loses the rest of the document without an error. Write such files out in full, or expand them with another tool first. yes,no,onandoffare strings, as in YAML 1.2, soGetBoolis false for them. Onlytrueandfalseare booleans.GetIntreturns a 32-bitintand wraps around for larger numbers. Read sizes, ids and timestamps throughToJSONinto ani64field (see below).- Every path function parses the whole text again. For many reads from a large file, call
ToJSONorDecodeonce and work on the result.
Guide: read a config file with defaults
The path functions are the quickest way to read a config file. Check first that the sections you depend on exist with HasKeys, then read each setting with the typed getter. For optional settings, wrap Get in a small helper that falls back to a default when Has says the key is not there.
import encoding.yaml
func setting(text: str, path: str, def: str): str:
if yaml.Has(text, path):
ret yaml.Get(text, path)
end
ret def
end
func main:
text := `# settings for the shop
server:
host: 0.0.0.0
port: 8080
workers: 4
log:
level: debug
features:
search: true
beta: false
`
mut required := Vector {} as Vector<str>
required.push("server")
required.push("log")
if not yaml.HasKeys(text, required):
println "config is missing a section"
ret 1
end
println "listen on", yaml.Get(text, "server.host") + ":" + yaml.Get(text, "server.port")
println "workers:", yaml.GetInt(text, "server.workers")
println "log level:", setting(text, "log.level", "info")
println "log file:", setting(text, "log.file", "stdout")
println "search on:", yaml.GetBool(text, "features.search"), "beta on:", yaml.GetBool(text, "features.beta")
println "feature flags:", yaml.Keys(text, "features")
endComments starting with # are ignored. GetInt and GetBool return 0 and false for a missing key, which is also a real value, so a default other than zero needs the Has check.
Guide: YAML into a struct
For a config with many fields, a struct is easier to pass around than a text and a list of paths. There is no YAML Unmarshal of its own, but there does not need to be: ToJSON turns the YAML into JSON, and encoding/json fills the struct, with the same @json names and defaults.
import encoding.yaml
import encoding.json
import encoding.encerr
struct Service:
pub name: str = ""
pub image: str = ""
pub replicas: int = 1
pub ports: Vector<int> = Vector {}
end
func main:
text := `name: api
image: shop/api:2.3
ports:
- 80
- 443
`
mut s := Service {}
mut e := encerr.NoError()
if not json.UnmarshalLenient(yaml.ToJSON(text), s, e):
println "bad service:", e.message
ret 1
end
println s.name, "runs", s.image
println "replicas:", s.replicas
println "ports:", s.ports.len(), s.ports.get(0), s.ports.get(1)
endreplicas is not in the file, so it keeps its default of 1 because the lenient form was used. With json.Unmarshal the missing key would be an error.
Guide: walk a list of records
A list of objects, such as the jobs of a CI file, is read with Len and an index in the path. Build the path for item i once and add the field names to it. [lint] is the short flow form of a list, and reads the same way as the dashed form.
import encoding.yaml
import str
func main:
text := `jobs:
- name: lint
run: salam format --check
- name: test
run: sh tests/run.sh
needs: [lint]
- name: deploy
run: sh deploy.sh
needs: [test]
branch: main
`
n := yaml.Len(text, "jobs")
println n, "jobs"
repeat n in i:
at := "jobs[" + str.FromInt(i) + "]"
line := yaml.Get(text, at + ".name") + ": " + yaml.Get(text, at + ".run")
if yaml.Has(text, at + ".needs"):
println line, "(after", yaml.Get(text, at + ".needs[0]") + ")"
else:
println line
end
if yaml.Has(text, at + ".branch"):
println " only on", yaml.Get(text, at + ".branch")
end
end
endGuide: several documents in one file
Kubernetes manifests and some log formats put several YAML documents in one file, separated by a line with ---. SplitDocuments cuts them apart, and each piece is then an ordinary YAML text for the other functions.
import encoding.yaml
func main:
text := `kind: Deployment
metadata:
name: web
spec:
replicas: 3
---
kind: Service
metadata:
name: web
spec:
port: 80
---
kind: ConfigMap
metadata:
name: web-config
`
docs := yaml.SplitDocuments(text)
println docs.len(), "documents"
each d in docs:
println "-", yaml.Get(d, "kind"), yaml.Get(d, "metadata.name"), "valid:", yaml.Valid(d)
end
println "deployment as JSON:", yaml.ToJSON(docs.get(0))
endToJSON is also how you hand YAML to anything that speaks JSON, such as an HTTP API. Going the other way, Dump writes JSON text as YAML.
Guide: what type did the value get?
YAML decides the type of a value from how it looks, and the result is not always what the author meant. TypeOf shows what the reader saw. Here yes stayed a string, 007 kept its zeros as a string, and the five gigabyte limit is too big for GetInt but fine for an i64 field.
import encoding.yaml
import encoding.json
import encoding.encerr
struct Limits:
pub max_upload: i64 = 0
pub public: bool = false
pub code: str = ""
end
func load(text: str):
mut l := Limits {}
mut err := encerr.EncodingError {}
if json.UnmarshalLenient(yaml.ToJSON(text), l, err):
println " loaded:", l.max_upload, l.public, l.code
else:
println " refused:", err.message
end
end
func main:
text := "max_upload: 5368709120\npublic: yes\ncode: 007\n"
println "public is a", yaml.TypeOf(text, "public"), "so GetBool says", yaml.GetBool(text, "public")
println "code is a", yaml.TypeOf(text, "code"), "with the text", yaml.Get(text, "code")
println "GetInt overflows:", yaml.GetInt(text, "max_upload")
println "as JSON:", yaml.ToJSON(text)
load(text)
load("max_upload: 5368709120\npublic: true\ncode: \"007\"\n")
endThe struct route also checks the types for you: public: yes is refused instead of read as false. Writing true and quoting values that must stay text, such as "007", makes the file mean the same to every reader.
Which function for which job
| Job | Use |
|---|---|
| Read a few settings | Get, GetInt, GetBool, GetFloat |
| Tell a missing key from an empty one | Has, HasKeys |
| Look at the structure | TypeOf, Len, Keys |
| A whole file into a struct | ToJSON and json.UnmarshalLenient |
| Unknown shape | Decode to a Value, Encode back |
| Write YAML | Object, Member, or Dump from JSON |
Files with --- | SplitDocuments |
Functions
Get
func Get(text: str, path: str): strnull gives an empty string.import encoding.yaml
func main:
text := "app:\n name: salam\n port: 8080\n debug: true\n ratio: 0.5\n hosts:\n - a.example\n - b.example\n"
println yaml.Valid(text), yaml.Has(text, "app.name"), yaml.Has(text, "app.user")
println yaml.Get(text, "app.name"), yaml.GetInt(text, "app.port"), yaml.GetBool(text, "app.debug"), yaml.GetFloat(text, "app.ratio")
println yaml.TypeOf(text, "app.hosts"), yaml.Len(text, "app.hosts"), yaml.Get(text, "app.hosts[1]")
println yaml.Keys(text, "app")
mut required := Vector {} as Vector<str>
required.push("app")
println yaml.HasKeys(text, required)
required.push("db")
println yaml.HasKeys(text, required)
println yaml.ToJSON(text)
println yaml.SplitDocuments("a: 1\n---\nb: 2\n").len()
endHas
func Has(text: str, path: str): boolThe example for this item is shown with the ones above: encoding-yaml-Query.salam
GetInt
func GetInt(text: str, path: str): intint; 0 when it is missing. Larger numbers wrap around.The example for this item is shown with the ones above: encoding-yaml-Query.salam
GetFloat
func GetFloat(text: str, path: str): f64The example for this item is shown with the ones above: encoding-yaml-Query.salam
GetBool
func GetBool(text: str, path: str): booltrue. yes and on are strings and give false.The example for this item is shown with the ones above: encoding-yaml-Query.salam
TypeOf
func TypeOf(text: str, path: str): strmap, list, string, int, float, bool, null, or none when it is absent.The example for this item is shown with the ones above: encoding-yaml-Query.salam
Len
func Len(text: str, path: str): intThe example for this item is shown with the ones above: encoding-yaml-Query.salam
Keys
func Keys(text: str, path: str): Vector<str>The example for this item is shown with the ones above: encoding-yaml-Query.salam
HasKeys
func HasKeys(text: str, required: Vector<str>): boolThe example for this item is shown with the ones above: encoding-yaml-Query.salam
Decode
func Decode(text: str, err &: encerr.EncodingError): value.ValueValue.import encoding.yaml
import encoding.value
import encoding.encerr
import str
func main:
mut members := Vector {} as Vector<str>
members.push(yaml.Member("name", "salam"))
members.push(yaml.MemberInt("port", 8080))
members.push(yaml.MemberBool("debug", false))
members.push(yaml.MemberFloat("ratio", 0.25))
mut hosts := Vector {} as Vector<str>
hosts.push(yaml.Str("a: b"))
members.push(yaml.MemberRaw("hosts", yaml.Array(hosts)))
object := yaml.Object(members)
println object
println yaml.Dump(object)
println str.Contains(yaml.DumpTabs(`{"a": {"b": 1}}`), "\tb: 1"), yaml.DumpWith(`{"a": [1]}`, " ")
println yaml.Escape("one: two")
println yaml.Parse("x: 1")
mut e := encerr.NoError()
v := yaml.Decode("city: Tehran\npopulation: 9000000\n", e)
println value.AsInt(value.ObjectGet(v, "population"))
println yaml.Encode(v)
endEncode
func Encode(v: value.Value): strValue into YAML.The example for this item is shown with the ones above: encoding-yaml-Build.salam
Escape
func Escape(s: str): strThe example for this item is shown with the ones above: encoding-yaml-Build.salam
Str
func Str(value: str): strThe example for this item is shown with the ones above: encoding-yaml-Build.salam
Member
func Member(key: str, value: str): strThe example for this item is shown with the ones above: encoding-yaml-Build.salam
MemberInt
func MemberInt(key: str, value: i64): strThe example for this item is shown with the ones above: encoding-yaml-Build.salam
MemberBool
func MemberBool(key: str, value: bool): strThe example for this item is shown with the ones above: encoding-yaml-Build.salam
MemberFloat
func MemberFloat(key: str, value: f64): strThe example for this item is shown with the ones above: encoding-yaml-Build.salam
MemberRaw
func MemberRaw(key: str, raw: str): strThe example for this item is shown with the ones above: encoding-yaml-Build.salam
Object
func Object(members: Vector<str>): strDump turns into YAML.The example for this item is shown with the ones above: encoding-yaml-Build.salam
Array
func Array(items: Vector<str>): strThe example for this item is shown with the ones above: encoding-yaml-Build.salam
DumpWith
func DumpWith(text: str, unit: str): strThe example for this item is shown with the ones above: encoding-yaml-Build.salam
Dump
func Dump(text: str): strThe example for this item is shown with the ones above: encoding-yaml-Build.salam
DumpTabs
func DumpTabs(text: str): strDump, with tabs.The example for this item is shown with the ones above: encoding-yaml-Build.salam
Parse
func Parse(text: str): strThe example for this item is shown with the ones above: encoding-yaml-Build.salam
ToJSON
func ToJSON(text: str): strThe example for this item is shown with the ones above: encoding-yaml-Query.salam
Valid
func Valid(text: str): boolThe example for this item is shown with the ones above: encoding-yaml-Query.salam
SplitDocuments
func SplitDocuments(text: str): Vector<str>---, into separate ones.The example for this item is shown with the ones above: encoding-yaml-Query.salam