Standard library · Encoding and data

encoding/value

A dynamic value for data whose shape you do not know in advance; the shared language of every format in this group.

import encoding.value

Value is the dynamic data type shared by the encoding packages. One Value is a null, a bool, a 64-bit int, a float, a string, an array of values or an object mapping strings to values, and its kind field says which. encoding/json, encoding/yaml, encoding/toml, encoding/msgpack and others decode into it and encode from it, and encoding/serde converts between formats through it.

Reach for a Value when the shape of the data is not known when you write the program: a webhook payload, a config with optional sections, a document you only pass along. When the shape is fixed, decoding straight into a struct (for example with json.Unmarshal) is shorter and checks the types for you.

The functions in four groups

JobFunctions
Make a valueValueNull, ValueBool, ValueInt, ValueFloat, ValueString, NewValueArray, NewValueObject
Ask the kindIsNull, IsBool, IsInt, IsFloat, IsNumber, IsString, IsArray, IsObject
ReadAsBool, AsInt, AsFloat, AsString, ArrayLen, ArrayGet, ObjectGet, ObjectGetOk, ObjectHas, ObjectKeys
ChangeArrayPush, ArraySet, ObjectSet, ObjectDelete, ValueClone, ValueFree

What trips people up

  • Reading never checks the kind. AsString of a number is "" and AsInt of a string is 0. Ask IsString or IsNumber first when the data comes from outside.
  • Missing is null, all the way down. ObjectGet on a missing key, or on something that is not an object, returns null, so a chain like ObjectGet(ObjectGet(doc, "a"), "b") is safe. ArrayGet is not: an index past ArrayLen stops the program.
  • Copies share parts. A copy of an object shares its map, so ObjectSet through the copy changes the original. A copy of an array shares its elements but not its length, so ArrayPush on an array you got from ObjectGet is lost until you ObjectSet it back. Use ValueClone for a real copy.
  • Key order depends on how the object was built. ObjectKeys returns keys in the order they were added with ObjectSet, and a decoded JSON document in the order of its text. An object made from a map by ValueObject follows that map's own order. Sort the keys, or encode with sort_keys, when output must be alphabetical.

Guide: walk a whole document

A recursive function over Value handles any shape: for an object, visit each key; for an array, each index; anything else is a leaf. This one prints the path of every leaf, which is handy for logging a webhook payload or comparing two documents line by line.

encoding-value-guide-walk.salam
import encoding.value
import encoding.json
import encoding.encerr
import sort
import str

func walk(v: value.Value, path: str):
    if value.IsObject(v):
        keys := value.ObjectKeys(v)
        sort.QuickSort(keys)
        each k in keys:
            walk(value.ObjectGet(v, k), path + "." + k)
        end
    else value.IsArray(v):
        repeat value.ArrayLen(v) in i:
            walk(value.ArrayGet(v, i), path + "[" + str.FromInt(i) + "]")
        end
    else:
        println path, "=", value.ToDebugString(v)
    end
end

func main:
    mut e := encerr.NoError()
    doc := json.Decode(`{"order": 1042, "customer": {"name": "Sara", "vip": true}, "lines": [{"sku": "BK-01", "qty": 2}, {"sku": "PN-07", "qty": 1}], "note": null}`, e)
    walk(doc, "$")
end
Output$.customer.name = string(Sara) $.customer.vip = bool(true) $.lines[0].qty = int(2) $.lines[0].sku = string(BK-01) $.lines[1].qty = int(1) $.lines[1].sku = string(PN-07) $.note = null $.order = int(1042)

ObjectKeys gives the keys in the order they were added, so a decoded document keeps the order of its text. This program sorts them with sort.QuickSort to print them alphabetically whatever order the input had. ToDebugString shows each leaf together with its kind.

Guide: build a payload with optional fields

A struct always writes all its fields. When some fields should only appear when they have a value, as many HTTP APIs expect, build a Value object instead and set only what you have. json.Encode with sort_keys prints it in a stable order.

encoding-value-guide-build.salam
import encoding.value
import encoding.json

func payload(name: str, email: str, age: int, tags: Vector<str>): value.Value:
    mut v := value.NewValueObject()
    value.ObjectSet(v, "name", value.ValueString(name))
    if email != "":
        value.ObjectSet(v, "email", value.ValueString(email))
    end
    if age > 0:
        value.ObjectSet(v, "age", value.ValueInt(age))
    end
    if tags.len() > 0:
        mut arr := value.NewValueArray()
        each t in tags:
            value.ArrayPush(arr, value.ValueString(t))
        end
        value.ObjectSet(v, "tags", arr)
    end
    ret v
end

func main:
    opts := json.JSONOptions { sort_keys = true }
    mut tags := Vector {} as Vector<str>
    println json.Encode(payload("Sara", "", 0, tags), opts)
    tags.push("beta")
    tags.push("vip")
    println json.Encode(payload("Reza", "reza@example.com", 41, tags), opts)
end
Output{"name": "Sara"} {"age": 41, "email": "reza@example.com", "name": "Reza", "tags": ["beta", "vip"]}

Guide: change a nested field

To change something deep in a decoded document, take the inner part with ObjectGet, change it, and store it back with ObjectSet. For objects the store is not strictly needed, because the map is shared, but for arrays it is: the program below shows the new order line missing until the array is set back.

encoding-value-guide-edit.salam
import encoding.value
import encoding.json
import encoding.encerr

func line(sku: str, qty: int): value.Value:
    mut l := value.NewValueObject()
    value.ObjectSet(l, "sku", value.ValueString(sku))
    value.ObjectSet(l, "qty", value.ValueInt(qty))
    ret l
end

func main:
    mut e := encerr.NoError()
    mut order := json.Decode(`{"id": 1042, "customer": {"name": "Sara"}, "lines": [{"sku": "BK-01", "qty": 2}]}`, e)

    mut customer := value.ObjectGet(order, "customer")
    value.ObjectSet(customer, "tier", value.ValueString("gold"))

    mut lines := value.ObjectGet(order, "lines")
    value.ArrayPush(lines, line("PN-07", 1))
    println "before ObjectSet:", value.ArrayLen(value.ObjectGet(order, "lines")), "line(s)"
    value.ObjectSet(order, "lines", lines)
    println "after ObjectSet: ", value.ArrayLen(value.ObjectGet(order, "lines")), "line(s)"

    println json.Encode(order, json.JSONOptions { sort_keys = true })

    city := value.ObjectGet(value.ObjectGet(order, "shipping"), "city")
    println "shipping city given:", not value.IsNull(city), "[" + value.AsString(city) + "]"
end
Outputbefore ObjectSet: 1 line(s) after ObjectSet: 2 line(s) {"customer": {"name": "Sara", "tier": "gold"}, "id": 1042, "lines": [{"qty": 2, "sku": "BK-01"}, {"qty": 1, "sku": "PN-07"}]} shipping city given: false []

Always writing the changed part back with ObjectSet makes the code correct for both kinds, and keeps working if the representation changes. The last line reads a field of a shipping object that does not exist: the chain of ObjectGet calls returns null and AsString gives an empty string, with no crash.

Guide: deep merge of settings

Settings often come in layers: built-in defaults, then a file per site. A deep merge keeps every default the site file does not mention and replaces only what it does, one level at a time. Arrays and plain values in the override win as a whole.

encoding-value-guide-merge.salam
import encoding.value
import encoding.json
import encoding.encerr

func merge(base: value.Value, over: value.Value): value.Value:
    if not value.IsObject(base) or not value.IsObject(over):
        ret value.ValueClone(over)
    end
    mut out := value.ValueClone(base)
    each k in value.ObjectKeys(over):
        o := value.ObjectGet(over, k)
        if value.ObjectHas(out, k):
            value.ObjectSet(out, k, merge(value.ObjectGet(out, k), o))
        else:
            value.ObjectSet(out, k, value.ValueClone(o))
        end
    end
    ret out
end

func main:
    mut e := encerr.NoError()
    defaults := json.Decode(`{"server": {"host": "0.0.0.0", "port": 8080}, "log": {"level": "info", "json": false}, "plugins": ["auth"]}`, e)
    site := json.Decode(`{"server": {"port": 9000}, "log": {"level": "debug"}, "plugins": ["auth", "cache"]}`, e)

    cfg := merge(defaults, site)
    println json.Encode(cfg, json.JSONOptions { indent = "  ", sort_keys = true })
    println "defaults untouched:", value.AsInt(value.ObjectGet(value.ObjectGet(defaults, "server"), "port"))
end
Output{ "log": { "json": false, "level": "debug" }, "plugins": [ "auth", "cache" ], "server": { "host": "0.0.0.0", "port": 9000 } } defaults untouched: 8080

The function clones what it keeps, so defaults is the same afterwards. Without the clones, the result would share its objects with the inputs, as the next section shows.

Guide: copies, conversions and missing keys

Assigning a Value to a new variable does not copy its contents: an object's fields live in a shared map, so a change through the copy shows up in the original. Call ValueClone when you need an independent copy, and ValueFree when you are done with it. Free only values nothing else shares, since freeing a plain copy also frees the original's map.

encoding-value-guide-pitfalls.salam
import encoding.value

func main:
    mut a := value.NewValueObject()
    value.ObjectSet(a, "n", value.ValueInt(1))

    mut shared := a
    value.ObjectSet(shared, "n", value.ValueInt(2))
    println "after editing a plain copy:", value.AsInt(value.ObjectGet(a, "n"))

    mut own := value.ValueClone(a)
    value.ObjectSet(own, "n", value.ValueInt(3))
    println "after editing a clone:", value.AsInt(value.ObjectGet(a, "n"))

    println "AsInt of 2.9:", value.AsInt(value.ValueFloat(2.9))
    println "AsFloat of 3:", value.AsFloat(value.ValueInt(3))
    println "AsInt of \"5\":", value.AsInt(value.ValueString("5"))
    println "1 equals 1.0:", value.Equal(value.ValueInt(1), value.ValueFloat(1.0))

    mut found := false
    _ := value.ObjectGetOk(a, "missing", found)
    println "missing key found:", found

    value.ValueFree(own)
end
Outputafter editing a plain copy: 2 after editing a clone: 2 AsInt of 2.9: 2 AsFloat of 3: 3 AsInt of "5": 0 1 equals 1.0: false missing key found: false
  • AsInt cuts a float toward zero and AsFloat widens an int, but neither parses text: a string gives 0. Check the kind first with IsInt, IsNumber or IsString.
  • Equal compares kinds too, so the integer 1 and the float 1.0 are different values.
  • ObjectGet on a missing key gives a null value; ObjectGetOk also tells you whether the key was there.

Types

Value

struct Value
A dynamic value: null, boolean, whole number, decimal, string, array or object.
NameTypeDescription
kindintThe kind of the value; one of the Kind constants.
bbool = falseThe boolean value.
ii64 = 0The whole number value.
nf64 = 0The decimal value.
sstr = ""The string value.
arrVectorThe elements of an array.
objHashMapThe keys and values of an object.
orderi64 = 0An internal handle to the list that records the order in which keys were added to an object with ObjectSet. ObjectDelete keeps it up to date. Do not use it directly; ObjectKeys reads it.
encoding-value-Value.salam
import encoding.value

func main:
    mut user := value.NewValueObject()
    value.ObjectSet(user, "name", value.ValueString("Narges"))
    value.ObjectSet(user, "age", value.ValueInt(34))
    value.ObjectSet(user, "height", value.ValueFloat(1.68))
    value.ObjectSet(user, "admin", value.ValueBool(false))
    value.ObjectSet(user, "phone", value.ValueNull())
    mut languages := value.NewValueArray()
    value.ArrayPush(languages, value.ValueString("fa"))
    value.ArrayPush(languages, value.ValueString("en"))
    value.ArraySet(languages, 1, value.ValueString("ar"))
    value.ObjectSet(user, "langs", languages)
    println value.ToDebugString(user)
    println value.ObjectLen(user), value.ObjectHas(user, "age"), value.ObjectKeys(user)
    println value.AsString(value.ObjectGet(user, "name")), value.AsInt(value.ObjectGet(user, "age")), value.AsFloat(value.ObjectGet(user, "height")), value.AsBool(value.ObjectGet(user, "admin"))
    langs := value.ObjectGet(user, "langs")
    println value.IsArray(langs), value.ArrayLen(langs), value.AsString(value.ArrayGet(langs, 1))
    mut present := false
    _ := value.ObjectGetOk(user, "email", present)
    println present
    phone := value.ObjectGet(user, "phone")
    println value.IsNull(phone), value.IsBool(phone), value.IsInt(value.ValueInt(1)), value.IsFloat(value.ValueFloat(1.0)), value.IsNumber(value.ValueInt(1))
    println value.IsString(value.ValueString("")), value.IsObject(user), phone.kind == value.KindNull, value.KindObject
    mut copy := value.ValueClone(user)
    println value.Equal(copy, user)
    removed := value.ObjectDelete(copy, "phone")
    println removed, value.Equal(copy, user)
    mut numbers := Vector {} as Vector<value.Value>
    numbers.push(value.ValueInt(1))
    mut fields := HashMap {} as HashMap<str, value.Value>
    fields.put("x", value.ValueArray(numbers))
    println value.ToDebugString(value.ValueObject(fields))
    value.ValueFree(copy)
end
Outputobject{name: string(Narges), age: int(34), height: float(1.68), admin: bool(false), phone: null, langs: array[string(fa), string(ar)]} 6 true ["name", "age", "height", "admin", "phone", "langs"] Narges 34 1.68 false true 2 ar false true false true true true true true true 6 true true false object{x: array[int(1)]}

Constants and variables

Functions

ToDebugString

func ToDebugString(v: Value): str
A readable form with the kind of every element, such as object{name: string(Ali)}, for debugging.

The example for this item is shown with the ones above: encoding-value-Value.salam

ValueBool

func ValueBool(b: bool): Value
A boolean value.

The example for this item is shown with the ones above: encoding-value-Value.salam

ValueInt

func ValueInt(i: i64): Value
A whole number value.

The example for this item is shown with the ones above: encoding-value-Value.salam

ValueFloat

func ValueFloat(n: f64): Value
A decimal value.

The example for this item is shown with the ones above: encoding-value-Value.salam

ValueString

func ValueString(s: str): Value
A string value.

The example for this item is shown with the ones above: encoding-value-Value.salam

ValueArray

func ValueArray(items: Vector<Value>): Value
An array from a vector.

The example for this item is shown with the ones above: encoding-value-Value.salam

ValueObject

func ValueObject(fields: HashMap<str, Value>): Value
An object from a map.

The example for this item is shown with the ones above: encoding-value-Value.salam

NewValueArray

func NewValueArray(): Value
An empty array.

The example for this item is shown with the ones above: encoding-value-Value.salam

NewValueObject

func NewValueObject(): Value
An empty object.

The example for this item is shown with the ones above: encoding-value-Value.salam

IsNull

func IsNull(v: Value): bool
True when the value is null.

The example for this item is shown with the ones above: encoding-value-Value.salam

IsBool

func IsBool(v: Value): bool
True when it is a boolean.

The example for this item is shown with the ones above: encoding-value-Value.salam

IsInt

func IsInt(v: Value): bool
True when it is a whole number.

The example for this item is shown with the ones above: encoding-value-Value.salam

IsFloat

func IsFloat(v: Value): bool
True when it is a decimal.

The example for this item is shown with the ones above: encoding-value-Value.salam

IsNumber

func IsNumber(v: Value): bool
True when it is a whole number or a decimal.

The example for this item is shown with the ones above: encoding-value-Value.salam

IsString

func IsString(v: Value): bool
True when it is a string.

The example for this item is shown with the ones above: encoding-value-Value.salam

IsArray

func IsArray(v: Value): bool
True when it is an array.

The example for this item is shown with the ones above: encoding-value-Value.salam

IsObject

func IsObject(v: Value): bool
True when it is an object.

The example for this item is shown with the ones above: encoding-value-Value.salam

AsBool

func AsBool(v: Value): bool
The boolean value; false for any other kind.

The example for this item is shown with the ones above: encoding-value-Value.salam

AsInt

func AsInt(v: Value): i64
The whole number value; a decimal is truncated toward zero, and any other kind gives 0.

The example for this item is shown with the ones above: encoding-value-Value.salam

AsFloat

func AsFloat(v: Value): f64
The decimal value; a whole number is converted, and any other kind gives 0.

The example for this item is shown with the ones above: encoding-value-Value.salam

AsString

func AsString(v: Value): str
The string value; an empty string for any other kind.

The example for this item is shown with the ones above: encoding-value-Value.salam

ArrayLen

func ArrayLen(v: Value): int
The number of elements in an array; 0 for any other kind.

The example for this item is shown with the ones above: encoding-value-Value.salam

ArrayGet

func ArrayGet(v: Value, i: int): Value
Element number i; an index outside 0 to ArrayLen - 1 stops the program.

The example for this item is shown with the ones above: encoding-value-Value.salam

ArraySet

func ArraySet(v &: Value, i: int, item: Value)
Replaces element number i.

The example for this item is shown with the ones above: encoding-value-Value.salam

ArrayPush

func ArrayPush(v &: Value, item: Value)
Adds an element at the end of an array. If the array came from ObjectGet or ArrayGet, store it back for the parent to see the new length.

The example for this item is shown with the ones above: encoding-value-Value.salam

ObjectLen

func ObjectLen(v: Value): int
The number of keys in an object.

The example for this item is shown with the ones above: encoding-value-Value.salam

ObjectHas

func ObjectHas(v: Value, key: str): bool
True when the key is there.

The example for this item is shown with the ones above: encoding-value-Value.salam

ObjectGet

func ObjectGet(v: Value, key: str): Value
The value for a key, or null when it is absent or v is not an object.

The example for this item is shown with the ones above: encoding-value-Value.salam

ObjectGetOk

func ObjectGetOk(v: Value, key: str, ok &: bool): Value
The value for a key, with ok saying whether the key was there; for telling "absent" apart from "null".

The example for this item is shown with the ones above: encoding-value-Value.salam

ObjectSet

func ObjectSet(v &: Value, key: str, item: Value)
Sets a key.

The example for this item is shown with the ones above: encoding-value-Value.salam

ObjectDelete

func ObjectDelete(v &: Value, key: str): bool
Removes a key and says whether it was there.

The example for this item is shown with the ones above: encoding-value-Value.salam

ObjectKeys

func ObjectKeys(v: Value): Vector<str>
The keys of an object. Keys added with ObjectSet come in the order they were added; the keys of an object made from a map by ValueObject come in that map's own order.

The example for this item is shown with the ones above: encoding-value-Value.salam

ValueClone

func ValueClone(v: Value): Value
A complete, independent copy of a value.

The example for this item is shown with the ones above: encoding-value-Value.salam

ValueFree

func ValueFree(v &: Value)
Releases the memory of a value and its elements.

The example for this item is shown with the ones above: encoding-value-Value.salam

Equal

func Equal(a: Value, b: Value): bool
True when two values, with all their nested elements, are equal.

The example for this item is shown with the ones above: encoding-value-Value.salam