encoding/json
Building and reading JSON: from a plain string through automatic struct conversion to streaming.
import encoding.json
There are three ways to work with JSON here. For simple jobs, functions such as Member, Object and Get work directly on JSON text. For real programs, Marshal and Unmarshal convert a Salam struct to JSON and back. For data whose shape you do not know in advance, Decode returns a dynamic Value.
Set the key name of a field with @json "name" above it; @json "-" leaves the field out. Without it, the field's own name becomes the key.
Use this package for APIs, config files and anything another program will read. Its neighbours: encoding/yaml and encoding/toml for files people edit, encoding/msgpack when the data should be small and binary, and encoding/serde to read or write the same struct in several formats.
The things people get wrong:
GetIntreturns anint, 32 bits, and clamps anything larger:3000000000comes back as2147483647. For ids and timestamps, read the field withUnmarshalinto ani64, or parse the text fromGet.Get,GetIntandHaslook only at the top level of an object. For nested values useGetPath, such asGetPath(text, "user.address.city").- Strict or lenient.
Unmarshalfails on a missing field, an unknown key or a type that does not match;UnmarshalLenientkeeps the struct's defaults for missing fields and skips unknown keys, but a wrong type still fails. - Key order.
Marshalwrites fields in struct order. A decodedValueobject keeps the order of the text andEncodewrites its keys in that order; passsort_keystoEncodewhen output must be alphabetical.
Guide: load a config file into a struct
A config file is the most common JSON a program reads. Describe it as structs, give every field the default you want, and let UnmarshalLenient fill in what the file has. Nested objects map to nested structs and arrays map to Vector fields.
import encoding.json
import encoding.encerr
struct Database:
pub host: str = "localhost"
pub port: int = 5432
pub pool: int = 4
end
struct Config:
pub name: str = "app"
pub debug: bool = false
pub db: Database = Database {}
pub admins: Vector<str> = Vector {}
end
func main:
text := `{
"name": "shop",
"db": {"host": "db.internal", "port": 6432},
"admins": ["sara", "reza"]
}`
mut cfg := Config {}
mut e := encerr.NoError()
if not json.UnmarshalLenient(text, cfg, e):
println "bad config:", e.message
ret 1
end
println "name:", cfg.name
println "debug:", cfg.debug
println "database:", cfg.db.host, cfg.db.port, "pool", cfg.db.pool
println "admins:", cfg.admins.len(), cfg.admins.get(0), cfg.admins.get(1)
endThe file left out debug and db.pool, so they keep their defaults. That is the reason for the lenient form here: plain Unmarshal is strict and fails with missing key 'pool' when a field is absent, and with unknown key when the file has a key the struct does not. Strict is right for messages between programs, where a missing field is a bug; lenient is right for files people edit by hand. To read a file from disk, pass the text of file.ReadAll from fs/file.
Guide: build an API response
Going the other way, Marshal turns a struct into compact JSON for the wire and MarshalIndent into readable JSON for logs and files. A Vector of structs becomes an array of objects, quotes inside strings are escaped, and the field order is the order of the struct.
import encoding.json
struct Item:
pub sku: str = ""
pub title: str = ""
@json "price_cents"
pub price: int = 0
pub in_stock: bool = true
end
struct Page:
pub page: int = 1
pub total: int = 0
pub items: Vector<Item> = Vector {}
@json "-"
pub cache_key: str = ""
end
func main:
mut p := Page { page = 2, total = 14, cache_key = "products:2" }
p.items.push(Item { sku = "BK-01", title = "Notebook", price = 1250 })
p.items.push(Item { sku = "PN-07", title = "Pen \"fine\"", price = 300, in_stock = false })
println json.Marshal(p)
println json.MarshalIndent(p, " ")
end@json "price_cents" renames a field in the JSON without renaming it in the code, and @json "-" keeps internal fields such as cache_key out of the output. Use it for passwords and tokens too, so they can never leak into a response by accident.
Guide: explore JSON of unknown shape
When you do not know the shape ahead of time, for example a webhook from another service, Decode gives a tree of Value nodes. Ask each node what it is with IsString, IsInt, IsObject and the rest, then read it with the matching As function.
import encoding.json
import encoding.value
import encoding.encerr
import str
func describe(v: value.Value): str:
if value.IsString(v): ret "string " + value.AsString(v) end
if value.IsInt(v): ret "int " + str.FromInt(value.AsInt(v)) end
if value.IsFloat(v): ret "float " + str.FromFloat(value.AsFloat(v)) end
if value.IsBool(v):
if value.AsBool(v): ret "bool true" end
ret "bool false"
end
if value.IsArray(v): ret "array of " + str.FromInt(value.ArrayLen(v)) end
if value.IsObject(v): ret "object with " + str.FromInt(value.ObjectLen(v)) + " keys" end
ret "null"
end
func main:
mut e := encerr.NoError()
doc := json.Decode(`{"id": 7, "name": "Lamp", "price": 19.5, "sale": true, "tags": ["home", "light"], "size": {"w": 20, "h": 45}, "note": null}`, e)
if encerr.IsError(e):
println "parse error:", e.message
ret 1
end
println "keys:", value.ObjectLen(doc)
keys := ["id", "name", "price", "sale", "tags", "size", "note", "color"]
each k in keys:
if not value.ObjectHas(doc, k):
println k, "is missing"
continue
end
println k, "=", describe(value.ObjectGet(doc, k))
end
size := value.ObjectGet(doc, "size")
println "area:", value.AsInt(value.ObjectGet(size, "w")) * value.AsInt(value.ObjectGet(size, "h"))
endWhole numbers come back as int and numbers with a fraction as float; IsNumber accepts both. A decoded object keeps its keys in the order of the text, but a sender is free to change that order, so walk a list of the keys you expect, as this program does, and use ObjectHas for the optional ones.
Guide: report bad input
Every decoding function takes an EncodingError from encoding/encerr. When it fails, message says what was wrong and position is the byte offset where the reader stopped, which is what you show the user. The text helpers like Get never fail; they return an empty string for a missing key, so check with Has or Valid first when that matters.
import encoding.json
import encoding.encerr
struct Signup:
pub email: str = ""
pub age: int = 0
end
func load(text: str):
mut s := Signup {}
mut e := encerr.NoError()
if json.Unmarshal(text, s, e):
println "ok:", s.email, s.age
else:
println "error at", e.position, "->", e.message
end
end
func load_lenient(text: str):
mut s := Signup {}
mut e := encerr.NoError()
if json.UnmarshalLenient(text, s, e):
println "lenient ok:", s.email, s.age
end
end
func main:
load(`{"email": "a@example.com", "age": 31}`)
load(`{"email": "a@example.com", "age": "31"}`)
load(`{"email": "a@example.com", "age": 31`)
load(`{"email": "a@example.com", "age": 31, "plan": "pro"}`)
load_lenient(`{"email": "a@example.com", "age": 31, "plan": "pro"}`)
println "valid:", json.Valid(`[1, 2, 3]`), json.Valid(`[1, 2,]`)
println "missing key gives [" + json.Get(`{"a": 1}`, "b") + "]"
endGuide: stream newline-delimited JSON
Logs and exports often hold one JSON value per line (NDJSON). A decoder reads them one at a time: More says whether another value is left, and Decode returns the next one. NewBufDecoder reads from a string and NewDecoder from an open file. Encoders write the same format, one value per line.
import encoding.json
import encoding.value
func main:
logs := `{"level": "info", "msg": "started", "ms": 3}
{"level": "warn", "msg": "slow query", "ms": 820}
{"level": "info", "msg": "request", "ms": 41}
{"level": "error", "msg": "timeout", "ms": 5000}
{"level": "info", "msg": "request", "ms": 37}
`
mut counts := HashMap {} as HashMap<str, int>
mut slow := 0
mut d := json.NewBufDecoder(logs)
until d.More():
line := d.Decode()
level := value.AsString(value.ObjectGet(line, "level"))
counts.put(level, counts.get_or(level, 0) + 1)
if value.AsInt(value.ObjectGet(line, "ms")) > 500:
slow += 1
end
end
println "info:", counts.get_or("info", 0), "warn:", counts.get_or("warn", 0), "error:", counts.get_or("error", 0)
println "slower than 500 ms:", slow
mut out := json.NewBufEncoder()
out.Encode(value.ValueString("summary"))
out.Encode(value.ValueInt(slow))
println out.String()
out.Close()
enduntil d.More(): reads naturally as "while there is more", because until in Salam runs while its condition holds.
Which function for which job
| Job | Use |
|---|---|
| Read one field from a small JSON text | Get, GetInt, GetPath |
| Build a small JSON text by hand | Member, MemberInt, Object, Array |
| Struct to JSON | Marshal, MarshalIndent |
| JSON to struct, exact | Unmarshal |
| JSON to struct, with defaults and extra keys | UnmarshalLenient |
| Unknown shape | Decode, Encode |
| One value per line | NewDecoder, NewEncoder |
| Pretty print or shrink existing text | Indent, Minify |
| A JSON Schema for a struct | Schema |
Types
JSONOptions
struct JSONOptionsEncode.| Name | Type | Description |
|---|---|---|
indent | str = "" | The indent unit; empty means one line. |
sort_keys | bool = false | Write the keys in alphabetical order. Without it they are written in the order the object holds them: the order they were added or decoded. |
ignore_null | bool = false | Leave out the keys whose value is null. |
import encoding.json
import encoding.value
import encoding.encerr
import fs.file
func main:
mut e := encerr.NoError()
v := json.Decode(`{"city": "Tehran", "zones": [1, 2, 22]}`, e)
println encerr.IsError(e), value.AsString(value.ObjectGet(v, "city"))
println json.EncodeDefault(v)
opts := json.JSONOptions { indent = " ", sort_keys = true }
println json.Encode(v, opts)
_ := json.Decode("[1, 2", e)
println encerr.IsError(e), e.position
mut enc := json.NewBufEncoder()
enc.Encode(value.ValueInt(1))
enc.Encode(value.ValueString("two"))
println enc.String()
enc.Close()
mut pretty := json.NewBufEncoderIndent(" ")
pretty.Encode(v)
println pretty.String()
pretty.Close()
out := file.Open("lines.ndjson", "wb")
mut lines := json.NewEncoder(out)
lines.Encode(value.ValueInt(10))
lines.Encode(value.ValueInt(20))
lines.Close()
mut tabbed := json.NewEncoderIndent(file.Open("pretty.json", "wb"), "\t")
tabbed.Encode(v)
tabbed.Close()
mut reader := json.NewDecoder(file.Open("lines.ndjson", "rb"))
mut total := 0 as i64
until reader.More():
total += value.AsInt(reader.Decode())
end
println total
mut two := json.NewBufDecoder("{\"a\": 1}\n{\"a\": 2}\n")
until two.More():
println json.EncodeDefault(two.Decode())
end
endJSONEncoder
struct JSONEncoder| Name | Type | Description |
|---|---|---|
dst | File* | The destination file, when it writes to one. |
useFile | bool = false | Whether it writes to a file or into memory. |
buf | str.StringBuilder | The text buffer, when it writes into memory. |
indent | str = "" | The indent unit; empty means one value per line. |
count | int = 0 | How many values have been written. |
The example for this item is shown with the ones above: encoding-json-Value.salam
Encode
func Encode(v: value.Value): boolThe example for this item is shown with the ones above: encoding-json-Value.salam
String
func String(): strThe example for this item is shown with the ones above: encoding-json-Value.salam
Close
func Close()The example for this item is shown with the ones above: encoding-json-Value.salam
JSONDecoder
struct JSONDecoder| Name | Type | Description |
|---|---|---|
src | File* | The source file. |
useFile | bool = false | Whether it reads from a file. |
buf | str = "" | The text that was read. |
pos | int = 0 | The current position in the text. |
loaded | bool = false | Whether the file has been read. |
eof | bool = false | It has reached the end of the text. |
bad | bool = false | An error happened while reading. |
The example for this item is shown with the ones above: encoding-json-Value.salam
More
func More(): boolThe example for this item is shown with the ones above: encoding-json-Value.salam
Decode
func Decode(): value.ValueThe example for this item is shown with the ones above: encoding-json-Value.salam
Functions
Decode
func Decode(text: str, err &: encerr.EncodingError): value.ValueValue; on an error err is filled in.The example for this item is shown with the ones above: encoding-json-Value.salam
Encode
func Encode(v: value.Value, opts: JSONOptions): strValue into JSON with the given options.The example for this item is shown with the ones above: encoding-json-Value.salam
EncodeDefault
func EncodeDefault(v: value.Value): strValue into JSON with the default options.The example for this item is shown with the ones above: encoding-json-Value.salam
SchemaEnvelope
func SchemaEnvelope<T>(v: T): strSchemaMerge.import encoding.json
import encoding.encerr
struct User:
@json "name"
pub name: str = ""
@json "age"
pub age: int = 0
@json "tags"
pub tags: Vector<str> = Vector {}
@json "-"
pub password: str = ""
end
func main:
mut u := User { name = "Maryam", age = 28, password = "secret" }
u.tags.push("admin")
println json.Marshal(u)
println json.MarshalIndent(u, " ")
mut loaded := User {}
mut e := encerr.NoError()
println json.Unmarshal(`{"name": "Reza", "age": 41, "tags": ["a", "b"]}`, loaded, e)
println loaded.name, loaded.age, loaded.tags.len()
println json.Unmarshal(`{"name": "Reza", "age": "forty"}`, loaded, e), e.message
mut lenient := User {}
println json.UnmarshalLenient(`{"name": "Sara", "extra": 1}`, lenient, e), lenient.name
println json.Schema(u)
envelope := json.SchemaEnvelope(u)
println json.SchemaMerge(envelope) == json.Schema(u)
endSchema
func Schema<T>(v: T): strThe example for this item is shown with the ones above: encoding-json-Marshal.salam
SchemaMerge
func SchemaMerge(envelope: str): strThe example for this item is shown with the ones above: encoding-json-Marshal.salam
NewEncoder
func NewEncoder(f: File*): JSONEncoderThe example for this item is shown with the ones above: encoding-json-Value.salam
NewEncoderIndent
func NewEncoderIndent(f: File*, unit: str): JSONEncoderThe example for this item is shown with the ones above: encoding-json-Value.salam
NewBufEncoder
func NewBufEncoder(): JSONEncoderString.The example for this item is shown with the ones above: encoding-json-Value.salam
NewBufEncoderIndent
func NewBufEncoderIndent(unit: str): JSONEncoderThe example for this item is shown with the ones above: encoding-json-Value.salam
NewDecoder
func NewDecoder(f: File*): JSONDecoderThe example for this item is shown with the ones above: encoding-json-Value.salam
NewBufDecoder
func NewBufDecoder(s: str): JSONDecoderThe example for this item is shown with the ones above: encoding-json-Value.salam
Valid
func Valid(text: str): boolimport encoding.json
func main:
mut tags := Vector {} as Vector<str>
tags.push(json.Str("hello"))
tags.push(json.Str(`a "quoted" line`))
mut members := Vector {} as Vector<str>
members.push(json.Member("name", "Ali"))
members.push(json.MemberInt("age", 30))
members.push(json.MemberBool("admin", false))
members.push(json.MemberFloat("score", 18.5))
members.push(json.MemberRaw("tags", json.Array(tags)))
text := json.Object(members)
println text
println json.Valid(text), json.Valid("{broken")
println json.Get(text, "name"), json.GetInt(text, "age"), json.GetFloat(text, "score")
println json.Has(text, "admin"), json.Has(text, "email"), json.Keys(text)
println json.Escape("one\ttwo")
pretty := json.Indent(text)
println pretty
println json.Minify(pretty) == json.Minify(text)
println json.Indent4(`{"a":1}`)
println json.IndentTabs(`{"a":1}`) == "{\n\t\"a\": 1\n}", json.IndentWith(`[1]`, "--")
nested := `{"user":{"tags":["a","b"],"name":"Ali"}}`
println json.GetPath(nested, "user.name"), json.GetPath(nested, "user.tags.1"), json.Items(json.GetPath(nested, "user.tags")).len()
println json.Unquote(`"\u0633\u0644\u0627\u0645"`)
endGet
func Get(text: str, key: str): strThe example for this item is shown with the ones above: encoding-json-Build.salam
Escape
func Escape(s: str): strThe example for this item is shown with the ones above: encoding-json-Build.salam
Minify
func Minify(text: str): strThe example for this item is shown with the ones above: encoding-json-Build.salam
Member
func Member(key: str, value: str): str"name": "Ali", for building an object.The example for this item is shown with the ones above: encoding-json-Build.salam
GetInt
func GetInt(text: str, key: str): intint. Values beyond 32 bits are clamped; use Unmarshal into an i64 for large numbers.The example for this item is shown with the ones above: encoding-json-Build.salam
GetFloat
func GetFloat(text: str, key: str): f64The example for this item is shown with the ones above: encoding-json-Build.salam
Has
func Has(text: str, key: str): boolThe example for this item is shown with the ones above: encoding-json-Build.salam
MemberInt
func MemberInt(key: str, value: i64): strThe example for this item is shown with the ones above: encoding-json-Build.salam
MemberBool
func MemberBool(key: str, value: bool): strThe example for this item is shown with the ones above: encoding-json-Build.salam
MemberFloat
func MemberFloat(key: str, value: f64): strThe example for this item is shown with the ones above: encoding-json-Build.salam
MemberRaw
func MemberRaw(key: str, raw: str): strThe example for this item is shown with the ones above: encoding-json-Build.salam
Str
func Str(value: str): strThe example for this item is shown with the ones above: encoding-json-Build.salam
Object
func Object(members: Vector<str>): str{...}.The example for this item is shown with the ones above: encoding-json-Build.salam
Array
func Array(items: Vector<str>): str[...].The example for this item is shown with the ones above: encoding-json-Build.salam
IndentWith
func IndentWith(text: str, unit: str): strThe example for this item is shown with the ones above: encoding-json-Build.salam
IndentTabs
func IndentTabs(text: str): strThe example for this item is shown with the ones above: encoding-json-Build.salam
Indent
func Indent(text: str): strThe example for this item is shown with the ones above: encoding-json-Build.salam
Indent4
func Indent4(text: str): strThe example for this item is shown with the ones above: encoding-json-Build.salam
Keys
func Keys(text: str): Vector<str>The example for this item is shown with the ones above: encoding-json-Build.salam
Items
func Items(text: str): Vector<str>Get or Items again.The example for this item is shown with the ones above: encoding-json-Build.salam
Unquote
func Unquote(raw: str): strThe example for this item is shown with the ones above: encoding-json-Build.salam
GetPath
func GetPath(text: str, path: str): strhits.hits.0._source; strings come back decoded, anything else as raw JSON, and a missing step gives an empty string.The example for this item is shown with the ones above: encoding-json-Build.salam
Marshal
func Marshal<T>(v: T): strThe example for this item is shown with the ones above: encoding-json-Marshal.salam
MarshalIndent
func MarshalIndent<T>(v: T, unit: str): strMarshal, indented.The example for this item is shown with the ones above: encoding-json-Marshal.salam
Unmarshal
func Unmarshal<T>(text: str, out &: T, err &: encerr.EncodingError): boolout from JSON. Every field has to be present with a matching type and an unknown key is an error too; on failure err is filled in and it returns false.The example for this item is shown with the ones above: encoding-json-Marshal.salam
UnmarshalLenient
func UnmarshalLenient<T>(text: str, out &: T, err &: encerr.EncodingError): boolUnmarshal, but unknown keys are skipped and missing fields keep the value they already had. A value of the wrong type is still an error.The example for this item is shown with the ones above: encoding-json-Marshal.salam