Standard library · Encoding and data

encoding/encerr

The shared error of the format readers: a kind, a message and a position in the text.

import encoding.encerr

EncodingError is the error type the encoding packages share. It is a plain struct with three fields: kind (one of the Kind constants), message (English text) and position (a byte offset into the input, or -1). Functions that can fail take it as an err &: out-parameter: create one with NoError(), pass it in, then test IsError(err).

Among the encoding packages, the ones that fill it are encoding/json (Decode, Unmarshal and the stream reader), encoding/envfile (ParseStrict), encoding/msgpack (Unmarshal) and encoding/yaml (Decode). Other readers, such as encoding/toml and encoding/xml, report a plain ok bool instead. For errors outside encoding, see error.

What trips people up

  • A successful call does not clear the error. Readers only write err when something goes wrong, so an error left over from an earlier call stays. msgpack.Unmarshal goes further and returns null at once when it is handed an error that is already set. Reset with e = encerr.NoError() before each call.
  • Not every kind is used. The standard readers produce KindUnexpectedEOF, KindUnexpectedToken, KindTrailingData and KindCustom. A field of the wrong type in json.Unmarshal arrives as KindUnexpectedToken, and too deep nesting as KindCustom. Branch on the kinds you will actually see, and treat everything else as "bad input".
  • position counts bytes, not characters or lines; the first guide converts it. For yaml.Decode it points into the JSON the YAML was translated to internally, so do not show it to users.
  • message is English and meant for logs. Show users your own text chosen by kind.

Guide: point the user at the mistake

position is a byte offset into the text, which is exact but not what a person looks for in an editor. Count the line breaks before it to turn it into a line and a column, and print it the way compilers do: file, line, column, message.

encoding-encerr-guide-position.salam
import encoding.encerr
import encoding.json
import str

func where(text: str, pos: int): str:
    mut line := 1
    mut col := 1
    repeat pos in i:
        if text.substr(i, 1) == "\n":
            line += 1
            col = 1
        else:
            col += 1
        end
    end
    ret "line " + str.FromInt(line) + ", column " + str.FromInt(col)
end

func main:
    text := `{
  "name": "shop",
  "port": 8080
  "debug": true
}`
    mut e := encerr.NoError()
    _ := json.Decode(text, e)
    if encerr.IsError(e):
        println "settings.json:", where(text, e.position) + ":", e.message
    end
end
Outputsettings.json: line 4, column 3: expected ',' or '}'

The reader stops where it noticed the problem, which is often just after the real mistake: here the comma is missing at the end of line 3, and the reader complains at the start of line 4. Columns count bytes, so a line with Persian text before the mistake shows a larger number than the letters you see.

Guide: one error report for several formats

A program that loads a JSON file, a .env file and a MessagePack cache can handle all three failures with one function, because they share the error type. The report function below turns the kind into a short description and prints the byte position and the message for the log.

encoding-encerr-guide-handler.salam
import encoding.encerr
import encoding.json
import encoding.envfile
import encoding.msgpack
import encoding.value
import compress

struct Limits:
    pub max_users: int = 0
    pub max_storage_gb: int = 0
end

func report(source: str, e: encerr.EncodingError):
    if not encerr.IsError(e):
        println source, "ok"
        ret
    end
    mut what := "bad input"
    if e.kind == encerr.KindUnexpectedEOF:
        what = "cut off"
    else e.kind == encerr.KindTrailingData:
        what = "extra data after the end"
    else e.kind == encerr.KindCustom:
        what = "not supported"
    end
    println source, what, "at byte", e.position, "(" + e.message + ")"
end

func main:
    mut e := encerr.NoError()
    mut lim := Limits {}
    _ := json.Unmarshal(`{"max_users": "fifty", "max_storage_gb": 20}`, lim, e)
    report("limits.json:", e)

    e = encerr.NoError()
    _ := json.Decode(`{"plan": "team"} {"plan": "pro"}`, e)
    report("plan.json:  ", e)

    e = encerr.NoError()
    mut env := envfile.ParseStrict("PLAN=team\nSEATS=\"12\n", e)
    report(".env:       ", e)
    envfile.Free(env)

    mut obj := value.NewValueObject()
    value.ObjectSet(obj, "seats", value.ValueInt(12))
    bytes := msgpack.Marshal(obj)
    e = encerr.NoError()
    _ := msgpack.UnmarshalBytes(bytes.data, compress.BufLen(bytes) - 1, e)
    report("cache.bin:  ", e)

    _ := msgpack.Unmarshal(bytes, e)
    report("stale error:", e)
    e = encerr.NoError()
    back := msgpack.Unmarshal(bytes, e)
    report("after reset:", e)
    println "seats:", value.AsInt(value.ObjectGet(back, "seats"))
end
Outputlimits.json: bad input at byte 14 (value does not match the expected type) plan.json: extra data after the end at byte 17 (trailing data after JSON value) .env: bad input at byte 10 (envfile: unterminated quoted value) cache.bin: cut off at byte 7 (unexpected end of input) stale error: cut off at byte 7 (unexpected end of input) after reset: ok seats: 12

The wrong type in limits.json comes out as an unexpected token, as described above. The MessagePack data is cut one byte short, so the reader runs out of input. The last three lines show why to reset: decoding the complete cache with the old error still set fails at once, and the same call after NoError() works.

Guide: the same errors in your own parser

When you write a small parser of your own, report errors in the same type, so callers can handle your format and the standard ones with the same code. ErrUnexpectedEOF, ErrUnexpectedToken and ErrTypeMismatch fill in the kind and a standard message; NewError with KindCustom takes any message. Callers can then branch on kind.

encoding-encerr-guide-custom.salam
import encoding.encerr

struct Size:
    pub w: int = 0
    pub h: int = 0
end

func parse_size(text: str, out &: Size, err &: encerr.EncodingError): bool:
    x := text.find("x")
    if x < 0:
        err = encerr.ErrUnexpectedEOF(text.len())
        ret false
    end
    w := text.substr(0, x)
    h := text.substr(x + 1, text.len() - x - 1)
    if w == "" or h == "":
        err = encerr.ErrUnexpectedToken("expected a number on both sides of 'x'", x)
        ret false
    end
    out.w = w.to_int()
    out.h = h.to_int()
    if out.w <= 0 or out.h <= 0:
        err = encerr.NewError(encerr.KindCustom, "sizes must be positive", 0)
        ret false
    end
    ret true
end

func main:
    inputs := ["1920x1080", "800", "x600", "0x10"]
    each s in inputs:
        mut size := Size {}
        mut e := encerr.NoError()
        if parse_size(s, size, e):
            println s, "->", size.w, "by", size.h
        else e.kind == encerr.KindUnexpectedEOF:
            println s, "-> incomplete:", e.message
        else:
            println s, "-> kind", e.kind, "at byte", e.position
            println "   ", e.message
        end
    end
end
Output1920x1080 -> 1920 by 1080 800 -> incomplete: unexpected end of input x600 -> kind 2 at byte 0 expected a number on both sides of 'x' 0x10 -> kind 99 at byte 0 sizes must be positive

The error kinds

ConstantValueMeaningProduced by
KindNone0No errorNoError()
KindUnexpectedEOF1The input ended in the middle of a valuejson, msgpack
KindUnexpectedToken2A character or word that does not belong there, or a JSON value of the wrong type for a struct fieldjson, envfile
KindInvalidUTF83Bytes that are not valid UTF-8reserved, not produced yet
KindInvalidNumber4A number that cannot be readreserved, not produced yet
KindDepthExceeded5Nesting deeper than the reader allowsreserved; json and msgpack report depth as KindCustom
KindTypeMismatch6A value of the wrong kindonly ErrTypeMismatch, for your own code
KindKeyNotFound7A required key is missingreserved, not produced yet
KindTrailingData8More input after a complete valuejson
KindCustom99Anything elsejson and msgpack (nesting over 512 levels, unsupported MessagePack types), and your own errors

The reserved kinds exist so that your own parsers, and later versions of the standard readers, have a precise name for these cases. Your code may compare against them, but do not expect the standard readers to return them today.

Types

EncodingError

struct EncodingError
An error from reading or writing a format.
NameTypeDescription
kindintThe kind of error; one of the Kind constants.
messagestr = ""The error message, in English.
positionint = -1Where the error is in the input, as a byte offset from 0, or -1 when it is not known.
encoding-encerr-Error.salam
import encoding.encerr
import encoding.json

func main:
    mut e := encerr.NoError()
    println encerr.IsError(e), e.kind == encerr.KindNone
    _ := json.Decode(`{"a": }`, e)
    println encerr.IsError(e), e.kind, e.position, e.message
    custom := encerr.NewError(encerr.KindCustom, "a negative value is not allowed", 12)
    println custom.kind, custom.message, custom.position
    println encerr.ErrUnexpectedEOF(40).kind == encerr.KindUnexpectedEOF
    println encerr.ErrUnexpectedToken("expected ':'", 3).kind == encerr.KindUnexpectedToken
    println encerr.ErrTypeMismatch("a number", 5).message
    println encerr.KindInvalidUTF8, encerr.KindInvalidNumber, encerr.KindDepthExceeded, encerr.KindTypeMismatch, encerr.KindKeyNotFound, encerr.KindTrailingData
end
Outputfalse true true 2 6 invalid value 99 a negative value is not allowed 12 true true expected a number 3 4 5 6 7 8

Constants and variables

KindUnexpectedEOF

const KindUnexpectedEOF: i32 = 1
The text ended sooner than expected.

The example for this item is shown with the ones above: encoding-encerr-Error.salam

KindUnexpectedToken

const KindUnexpectedToken: i32 = 2
A character or token turned up out of place.

The example for this item is shown with the ones above: encoding-encerr-Error.salam

KindInvalidUTF8

const KindInvalidUTF8: i32 = 3
The text is not valid UTF-8.

The example for this item is shown with the ones above: encoding-encerr-Error.salam

KindInvalidNumber

const KindInvalidNumber: i32 = 4
An invalid number.

The example for this item is shown with the ones above: encoding-encerr-Error.salam

KindDepthExceeded

const KindDepthExceeded: i32 = 5
The nesting went past the allowed depth.

The example for this item is shown with the ones above: encoding-encerr-Error.salam

KindTypeMismatch

const KindTypeMismatch: i32 = 6
The value's type does not match the expected one.

The example for this item is shown with the ones above: encoding-encerr-Error.salam

KindKeyNotFound

const KindKeyNotFound: i32 = 7
A required key was not found.

The example for this item is shown with the ones above: encoding-encerr-Error.salam

KindTrailingData

const KindTrailingData: i32 = 8
Extra data turned up after the end of the value.

The example for this item is shown with the ones above: encoding-encerr-Error.salam

KindCustom

const KindCustom: i32 = 99
An error defined by the program.

The example for this item is shown with the ones above: encoding-encerr-Error.salam

Functions

NoError

func NoError(): EncodingError
The empty error, meaning "no error". Assign it again to reset a variable before reusing it.

The example for this item is shown with the ones above: encoding-encerr-Error.salam

NewError

func NewError(kind: int, message: str, position: int): EncodingError
A new error with a kind, a message and a position.

The example for this item is shown with the ones above: encoding-encerr-Error.salam

ErrUnexpectedEOF

func ErrUnexpectedEOF(position: int): EncodingError
An unexpected end of text error at the given position.

The example for this item is shown with the ones above: encoding-encerr-Error.salam

ErrUnexpectedToken

func ErrUnexpectedToken(message: str, position: int): EncodingError
An out of place token error with a message.

The example for this item is shown with the ones above: encoding-encerr-Error.salam

ErrTypeMismatch

func ErrTypeMismatch(expected: str, position: int): EncodingError
A type mismatch error; its message names the expected type.

The example for this item is shown with the ones above: encoding-encerr-Error.salam

IsError

func IsError(e: EncodingError): bool
True when kind is not KindNone.

The example for this item is shown with the ones above: encoding-encerr-Error.salam