db/postgres
A PostgreSQL client written in Salam: connection strings and URIs, TLS, queries, transactions, pools and migrations.
import db.postgres
- Tutorial: a bookshop in eleven steps
- 1. Connect
- 2. Connection strings, URIs and TLS
- 3. Create the tables
- 4. Insert rows and get the new ids
- 5. Read rows into your own structs
- 6. Values from users, safely
- 7. NULL values
- 8. Transactions
- 9. Errors
- 10. A pool, and one connection per thread
- 11. PostgreSQL features
- More
- Guide: schema migrations
- Guide: settings from the environment
- Guide: nested results as JSON
- Guide: which function for which job
- Types
- Functions
This package talks to PostgreSQL directly: it implements the Postgres wire protocol in Salam itself, with password, MD5 and SCRAM-SHA-256 logins and optional TLS, so it needs no libpq or any other outside library. Every row of a result is fetched from the server at once, so RowCount is known before you read any of them. A Database also fits the shared Connection interface of db, which gives you engine neutral helpers, connection pools and migrations. This page starts with a tutorial; the full reference of every function and type comes after it.
Tutorial: a bookshop in eleven steps
The steps below build the database of a small bookshop: books, stock and orders. Each step is a complete program, and the output under it is what it printed against a real PostgreSQL 17. Run them in order, because each step uses the data the earlier ones left behind.
First start a server. With Docker that is one command; it creates the bookshop database and the postgres user with the password secret:
docker run -d --name bookshop-db -p 5432:5432 -e POSTGRES_PASSWORD=secret -e POSTGRES_DB=bookshop postgres:17-alpine
Without Docker, install PostgreSQL from your system's packages, then run createdb bookshop and give the postgres user that password. The programs connect to 127.0.0.1 on port 5432. A real application logs in with its own user rather than postgres, as step 2 shows.
1. Connect
Open takes a connection string of key=value pairs, the same format psql and libpq use. It always returns a Database, even when the login failed, so check Ok before anything else; Error then holds the server's message. Ping sends a tiny query to make sure the connection still works, which is a good health check for a long running service.
import db.postgres
func main:
d := postgres.Open("host=127.0.0.1 port=5432 user=postgres password=secret dbname=bookshop")
defer postgres.Close(d)
if not postgres.Ok(d):
print "could not connect: " + postgres.Error(d)
ret 1
end
println "connected to", postgres.DatabaseName(d), "as", postgres.UserName(d)
println "server", postgres.HostName(d) + ":" + postgres.Port(d), "version", postgres.ServerVersion(d)
println postgres.QueryText(d, "SELECT split_part(version(), ' on ', 1)")
println "alive:", postgres.Ping(d)
bad := postgres.Open("host=127.0.0.1 user=postgres password=wrong dbname=bookshop")
defer postgres.Close(bad)
println "with a wrong password:", postgres.Ok(bad)
print postgres.Error(bad)
off := postgres.Open("host=127.0.0.1 port=5499 user=postgres password=secret connect_timeout=3")
defer postgres.Close(off)
println "with no server on the port:", postgres.Ok(off)
print postgres.Error(off)
endThe server's own messages keep their PostgreSQL wording and start with the severity: FATAL for a refused login, ERROR for a failed statement. An error message ends with a newline, which is why the examples print it with print rather than println. With nothing listening on the port, Open fails at once. connect_timeout, in seconds, limits how long the login waits for each answer from the server, 30 seconds when you leave it out.
2. Connection strings, URIs and TLS
The same settings can be written three ways: as key=value pairs, as a postgresql://user:password@host:port/dbname?key=value URI, or as the five arguments of OpenFull. This step also creates a login role for the shop, with a password that has a space and an @ in it, to show how such a value is written in each form.
import db.postgres
func main:
admin := postgres.OpenFull("127.0.0.1", 5432, "postgres", "secret", "bookshop")
defer postgres.Close(admin)
println "OpenFull:", postgres.Ok(admin)
_ := postgres.Exec(admin, "DROP ROLE IF EXISTS clerk")
_ := postgres.Exec(admin, "CREATE ROLE clerk LOGIN PASSWORD 'p@ss word'")
a := postgres.Open("host=127.0.0.1 user=clerk password='p@ss word' dbname=bookshop")
defer postgres.Close(a)
println "quoted value:", postgres.Ok(a), postgres.UserName(a)
b := postgres.Open("postgresql://clerk:p%40ss%20word@127.0.0.1:5432/bookshop?connect_timeout=5")
defer postgres.Close(b)
println "uri:", postgres.Ok(b), postgres.UserName(b), postgres.DatabaseName(b)
c := postgres.Open("host=127.0.0.1 user=clerk password='p@ss word' dbname=bookshop sslmode=disable")
defer postgres.Close(c)
println "sslmode=disable:", postgres.Ok(c), postgres.QueryText(c, "SHOW ssl")
t := postgres.Open("host=127.0.0.1 user=clerk password='p@ss word' dbname=bookshop sslmode=require")
defer postgres.Close(t)
println "sslmode=require:", postgres.Ok(t)
print postgres.Error(t)
endIn the key=value form, a value with spaces goes in single quotes. In a URI, characters with a meaning of their own are percent encoded: %40 for @ and %20 for a space. OpenFull joins its arguments without quoting them, so use it only for values without spaces.
The keys this package reads are host (or hostaddr), port, user, password, dbname, sslmode, sslrootcert and connect_timeout; other keys are ignored. sslmode works as in libpq: prefer (the default) uses TLS when the server offers it, require refuses to go on without it, verify-ca and verify-full also check the server's certificate (against the CA file in sslrootcert when you give one), and disable and allow never ask for TLS. Without one of the verify modes the certificate is not checked. The test server here has TLS turned off, so require fails. Unix domain sockets are not supported; connect over TCP, even on the same machine.
3. Create the tables
Exec runs SQL that returns no rows and gives back 0 on success and -1 on failure. It accepts several statements separated by semicolons, which suits a schema. Use the PostgreSQL types that fit the data: bigint GENERATED ALWAYS AS IDENTITY for ids, numeric for money, text[] for a list of tags and jsonb for free form details.
import db.postgres
func main:
d := postgres.Open("host=127.0.0.1 user=postgres password=secret dbname=bookshop")
defer postgres.Close(d)
schema := `
DROP TABLE IF EXISTS orders, books;
CREATE TABLE books (
id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
title text NOT NULL UNIQUE,
author text NOT NULL,
year int CHECK (year > 0),
price numeric(6, 2) NOT NULL CHECK (price >= 0),
stock int NOT NULL DEFAULT 0 CHECK (stock >= 0),
isbn text,
tags text[] NOT NULL DEFAULT '{}',
info jsonb NOT NULL DEFAULT '{}'
);
CREATE TABLE orders (
id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
customer text NOT NULL,
book_id bigint NOT NULL REFERENCES books (id),
qty int NOT NULL CHECK (qty > 0),
total numeric(8, 2) NOT NULL
);`
if postgres.Exec(d, schema) != 0:
print postgres.Error(d)
ret 1
end
r := postgres.Query(d, `SELECT table_name, column_name, data_type, is_nullable
FROM information_schema.columns
WHERE table_name IN ('books', 'orders') ORDER BY table_name, ordinal_position`)
defer r.Close()
until r.Next():
println r.Text(0) + "." + r.Text(1), r.Text(2), "null:", r.Text(3)
end
endConstraints such as UNIQUE, CHECK (stock >= 0) and REFERENCES let the database refuse bad data even when a bug in a program tries to write it. Step 9 shows what such a refusal looks like.
4. Insert rows and get the new ids
An INSERT ... RETURNING id is a query that returns the generated id, so run it with Query and read the one row. Text that comes from outside the program goes through Quote, which returns it as a quoted, escaped SQL literal: the apostrophe in the second title is doubled and cannot end the string early. ExecCount runs one statement and returns how many rows it inserted, updated or deleted.
import db.postgres
import str
func addBook(d: postgres.Database, title: str, author: str, year: int, price: str, stock: int): i64:
sql := "INSERT INTO books (title, author, year, price, stock) VALUES (" + postgres.Quote(d, title) + ", " + postgres.Quote(d, author) + ", " + str.FromInt(year) + ", " + price + ", " + str.FromInt(stock) + ") RETURNING id"
r := postgres.Query(d, sql)
defer r.Close()
if not r.Next():
print postgres.Error(d)
ret 0
end
ret r.Int64(0)
end
func main:
d := postgres.Open("host=127.0.0.1 user=postgres password=secret dbname=bookshop")
defer postgres.Close(d)
println postgres.Quote(d, "The Hitchhiker's Guide to the Galaxy")
println "Dune got id", addBook(d, "Dune", "Frank Herbert", 1965, "12.50", 4)
println "Hitchhiker got id", addBook(d, "The Hitchhiker's Guide to the Galaxy", "Douglas Adams", 1979, "9.99", 2)
n := postgres.ExecCount(d, `INSERT INTO books (title, author, year, price, stock, isbn, tags) VALUES
('Persuasion', 'Jane Austen', 1817, 7.25, 3, '978-0141439686', '{novel,classic}'),
('The Little Prince', 'Antoine de Saint-Exupéry', 1943, 6.00, 10, '978-0156012195', '{fable,classic}'),
('Neuromancer', 'William Gibson', 1984, 11.00, 1, NULL, '{scifi}')`)
println "inserted", n, "more, last id", d.LastInsertId()
r := postgres.Query(d, "UPDATE books SET tags = '{scifi,classic}' WHERE title = 'Dune' RETURNING id, title, tags")
until r.Next():
println "updated", r.Int(0), r.Text(1), r.Text(2)
end
r.Close()
println "price rise on", postgres.ExecCount(d, "UPDATE books SET price = price + 1 WHERE year < 1900"), "old book"
endLastInsertId asks the server for lastval(), the last value any sequence or identity column produced in this session. After the three row insert that is the id of the last row; RETURNING is the clearer choice when you need every id. RETURNING also works on UPDATE and DELETE, as the tag change shows.
d.Changes() is the count of rows the last d.Exec(...) changed. It is only kept when you call the method on the value; the package function postgres.Exec(d, ...) works on a copy, so d.Changes() stays 0 after it. ExecCount avoids the question.
5. Read rows into your own structs
Query returns a Rows. Next moves to the next row and returns false when there is none, so until r.Next(): visits every row; inside the loop Text, Int, Int64 and Float read one column by its position, starting at 0. Wrap the loop in a function that fills a Vector of your own struct and the rest of the program never sees SQL rows.
import db
import db.postgres
import str
struct Book:
pub id: i64 = 0
pub title: str = ""
pub author: str = ""
pub year: int = 0
pub cents: int = 0
pub stock: int = 0
end
func money(cents: int): str:
c := cents % 100
if c < 10:
ret str.FromInt(cents / 100) + ".0" + str.FromInt(c)
end
ret str.FromInt(cents / 100) + "." + str.FromInt(c)
end
func loadBooks(d: postgres.Database, where: str): Vector<Book>:
mut out := Vector {} as Vector<Book>
r := postgres.Query(d, "SELECT id, title, author, year, (price * 100)::int AS cents, stock FROM books " + where)
defer r.Close()
until r.Next():
out.push(Book { id = r.Int64(0), title = r.Text(1), author = r.Text(2), year = r.Int(3), cents = r.Int(4), stock = r.Int(5) })
end
ret out
end
func main:
d := postgres.Open("host=127.0.0.1 user=postgres password=secret dbname=bookshop")
defer postgres.Close(d)
books := loadBooks(d, "ORDER BY year")
println books.len(), "books"
each b in books:
println b.id, b.title, "by", b.author, b.year, money(b.cents), "stock", b.stock
end
cheap := loadBooks(d, "WHERE price < 10 ORDER BY price")
println "under 10:", cheap.len()
r := postgres.Query(d, "SELECT title, price, stock, isbn FROM books WHERE id = 3")
defer r.Close()
r.Next()
repeat r.ColumnCount() in i:
kind := r.ColumnType(i)
mut name := "text"
if kind == db.TYPE_INT: name = "int" end
if kind == db.TYPE_FLOAT: name = "float" end
println r.ColumnName(i), "=", r.Text(i), "as", name
end
endThe prices are read as whole cents ((price * 100)::int in the SQL) rather than with Float, because a binary float cannot hold most decimal amounts exactly. Text gives the exact numeric value as the server wrote it, such as 8.25, which is the other safe choice. ColumnName and ColumnType describe the columns: integers and booleans report db.TYPE_INT, real, double precision and numeric report db.TYPE_FLOAT, bytea reports db.TYPE_BLOB, and everything else, dates, arrays and jsonb included, is db.TYPE_TEXT. Always close a Rows when you are done; defer r.Close() right after the query is the easy way.
6. Values from users, safely
This package sends SQL as text and has no placeholders, so every value that comes from a user must go through Quote before it is joined into a statement. A quoted value can contain anything, quotes and semicolons included, and it stays one string. Numbers do not need quoting, but they must really be numbers: convert the input to an integer and check it.
import db.postgres
import conv
import str
func likePattern(s: str): str:
ret "%" + str.Replace(str.Replace(str.Replace(s, "\\", "\\\\"), "%", "\\%"), "_", "\\_") + "%"
end
func search(d: postgres.Database, words: str): int:
sql := "SELECT title FROM books WHERE title ILIKE " + postgres.Quote(d, likePattern(words)) + " ORDER BY title"
r := postgres.Query(d, sql)
defer r.Close()
println "search", postgres.Quote(d, words), "found", r.RowCount()
until r.Next():
println " " + r.Text(0)
end
ret r.RowCount()
end
func main:
d := postgres.Open("host=127.0.0.1 user=postgres password=secret dbname=bookshop")
defer postgres.Close(d)
_ := search(d, "the")
_ := search(d, "x' OR '1'='1")
_ := search(d, "'; DROP TABLE books; --")
_ := search(d, "100%")
println "books still there:", postgres.QueryInt(d, "SELECT count(*) FROM books")
typed := "3 OR 1=1"
id := conv.ParseInt(typed)
if conv.FormatInt(id) != typed:
println "not a book id:", typed
end
println postgres.Quote(d, `C:\books\new`)
endBoth attacks only searched for odd titles, and the table is still there. Inside a LIKE or ILIKE pattern, % and _ are wildcards, so the search escapes them with a backslash first; otherwise a search for 100% would match every title that starts with 100. When the text has a backslash, Quote writes it in PostgreSQL's E'...' form with a leading space, which is still a single literal.
Never join raw input into a table or column name. If the user picks a sort column, compare their choice with a fixed list of names in your program and use your own copy of the name.
7. NULL values
A column can hold no value at all: SQL's NULL. Text returns an empty string for it and Int returns 0, which cannot be told apart from a real empty string or a real 0, so call IsNull whenever the difference matters. To write a NULL, put the word NULL into the SQL instead of a quoted value; a small helper keeps that tidy.
import db.postgres
func orNull(d: postgres.Database, s: str): str:
if s == "":
ret "NULL"
end
ret postgres.Quote(d, s)
end
func main:
d := postgres.Open("host=127.0.0.1 user=postgres password=secret dbname=bookshop")
defer postgres.Close(d)
_ := postgres.Exec(d, "UPDATE books SET isbn = " + orNull(d, "978-0441172719") + " WHERE title = 'Dune'")
_ := postgres.Exec(d, "UPDATE books SET isbn = " + orNull(d, "") + " WHERE title LIKE 'The Hitch%'")
r := postgres.Query(d, "SELECT title, isbn FROM books ORDER BY id")
defer r.Close()
until r.Next():
if r.IsNull(1):
println r.Text(0), "has no ISBN, Text gives", "[" + r.Text(1) + "]"
else:
println r.Text(0), r.Text(1)
end
end
println "missing:", postgres.QueryInt(d, "SELECT count(*) FROM books WHERE isbn IS NULL")
println "= NULL matches", postgres.QueryInt(d, "SELECT count(*) FROM books WHERE isbn = NULL")
newest := postgres.Query(d, "SELECT max(year) FROM books WHERE year > 2100")
defer newest.Close()
newest.Next()
println "max over no rows:", newest.Int(0), newest.IsNull(0)
println postgres.QueryText(d, "SELECT string_agg(coalesce(isbn, 'none'), ', ' ORDER BY id) FROM books")
endCompare with IS NULL, never = NULL, which is never true. Aggregates such as max and sum over no rows return NULL, not 0; coalesce(x, fallback) replaces a NULL in the SQL itself.
8. Transactions
An order changes two tables: the stock of the book goes down and a row is added to orders. Either both happen or neither. Begin starts a transaction and returns a Tx whose Exec and Query run inside it; Commit keeps the changes and Rollback throws them away.
import db.postgres
import str
func placeOrder(d: postgres.Database, customer: str, book: int, qty: int): bool:
tx := d.Begin()
if not tx.Ok():
ret false
end
r := tx.Query("UPDATE books SET stock = stock - " + str.FromInt(qty) + " WHERE id = " + str.FromInt(book) + " AND stock >= " + str.FromInt(qty) + " RETURNING price * " + str.FromInt(qty))
defer r.Close()
if not r.Next():
tx.Rollback()
println customer, "wants", qty, "of book " + str.FromInt(book) + ": not enough stock, rolled back"
ret false
end
total := r.Text(0)
if tx.Exec("INSERT INTO orders (customer, book_id, qty, total) VALUES (" + postgres.Quote(d, customer) + ", " + str.FromInt(book) + ", " + str.FromInt(qty) + ", " + total + ")") != 0:
tx.Rollback()
ret false
end
if tx.Commit() != 0:
ret false
end
println customer, "bought", qty, "of book", book, "for", total
ret true
end
func main:
d := postgres.Open("host=127.0.0.1 user=postgres password=secret dbname=bookshop")
defer postgres.Close(d)
_ := placeOrder(d, "Sara", 1, 2)
_ := placeOrder(d, "Omid", 5, 1)
_ := placeOrder(d, "Lena", 5, 1)
_ := placeOrder(d, "Sara", 4, 3)
println "orders:", postgres.QueryInt(d, "SELECT count(*) FROM orders"), "takings:", postgres.QueryText(d, "SELECT sum(total) FROM orders")
println "Dune left:", postgres.QueryInt(d, "SELECT stock FROM books WHERE id = 1"), "Neuromancer left:", postgres.QueryInt(d, "SELECT stock FROM books WHERE id = 5")
endUPDATE ... WHERE stock >= qty RETURNING ... checks and takes the stock in one statement, so two customers ordering the last copy at the same time cannot both get it: the second one waits for the first transaction to finish and then finds the stock is 0.
Check every statement inside a transaction. After a failed statement PostgreSQL ignores the rest of the transaction, and a later Commit rolls back instead. Commit then returns -1, so a 0 from it means your changes really were saved.
9. Errors
Every call reports failure the same way. Exec returns -1, ExecCount returns -1, and Query returns a Rows whose Ok is false and whose Next returns false at once. Error then has the server's message, with the DETAIL line when there is one; it is cleared when the next statement starts.
import db.postgres
import str
func attempt(d: postgres.Database, sql: str):
if postgres.Exec(d, sql) != 0:
print postgres.Error(d)
end
end
func main:
d := postgres.Open("host=127.0.0.1 user=postgres password=secret dbname=bookshop")
defer postgres.Close(d)
attempt(d, "INSERT INTO books (title, author, price) VALUES ('Dune', 'Someone', 5)")
attempt(d, "UPDATE books SET stock = stock - 50 WHERE id = 1")
attempt(d, "INSERT INTO orders (customer, book_id, qty, total) VALUES ('Omid', 99, 1, 1)")
attempt(d, "SELEC title FROM books")
r := postgres.Query(d, "SELECT nope FROM books")
println "query ok:", r.Ok(), "rows:", r.RowCount(), "next:", r.Next()
println "ExecCount on error:", postgres.ExecCount(d, "DELETE FROM nowhere")
print postgres.Error(d)
tx := d.Begin()
_ := tx.Exec("UPDATE books SET price = 0 WHERE id = 1")
_ := tx.Exec("INSERT INTO books (title) VALUES ('Untitled')")
print postgres.Error(d)
_ := tx.Exec("UPDATE books SET price = 1 WHERE id = 2")
print postgres.Error(d)
tx.Rollback()
println "Dune still costs", postgres.QueryText(d, "SELECT price FROM books WHERE id = 1")
killer := postgres.Open("host=127.0.0.1 user=postgres password=secret dbname=bookshop")
defer postgres.Close(killer)
pid := postgres.QueryInt(d, "SELECT pg_backend_pid()")
_ := postgres.Exec(killer, "SELECT pg_terminate_backend(" + str.FromInt(pid) + ")")
println "after the server dropped us, ping:", postgres.Ping(d)
postgres.Reset(d)
println "after Reset, ping:", postgres.Ping(d), postgres.QueryInt(d, "SELECT count(*) FROM books")
endThe second part shows the transaction rule from step 8 in action: after the failed insert, the next statement is refused with current transaction is aborted until Rollback. The last part shows a dropped connection, here cut by the server with pg_terminate_backend, as also happens when the server restarts. Ping notices it, and Reset connects again with the same settings.
A Query must return rows. An UPDATE or DELETE without RETURNING run through Query does its work, but the Rows it gives back is not Ok; use Exec or ExecCount for those.
10. A pool, and one connection per thread
A connection handles one statement at a time. A server that answers many requests keeps a few connections open and lends one to each request. db.Pool from the db package does the bookkeeping: Add puts connections in, Acquire lends one out as a lease number, Conn gives the connection for that lease and Release returns it. postgres.Conn turns a Database into the shared Connection the pool stores.
import db
import db.postgres
import mem
import thread
const DSN := "host=127.0.0.1 user=postgres password=secret dbname=bookshop"
struct Job:
pub author: str = ""
pub sold: int = 0
end
func countSold(j: Job*):
d := postgres.Open(DSN)
defer postgres.Close(d)
j[0].sold = postgres.QueryInt(d, "SELECT coalesce(sum(o.qty), 0) FROM orders o JOIN books b ON b.id = o.book_id WHERE b.author = " + postgres.Quote(d, j[0].author))
end
func main:
mut pool := db.NewPool(3)
defer db.Free(pool)
repeat 3:
_ := db.Add(pool, postgres.Conn(postgres.Open(DSN)))
end
println "pool:", db.Size(pool), "connections,", db.Idle(pool), "idle"
a := db.Acquire(pool)
b := db.Acquire(pool)
println "two requests in flight:", db.InUse(pool), "in use"
println "books:", db.QueryInt(db.Conn(pool, a), "SELECT count(*) FROM books"), "orders:", db.QueryInt(db.Conn(pool, b), "SELECT count(*) FROM orders")
db.Release(pool, a)
db.Release(pool, b)
mut jobs := Vector {} as Vector<Job*>
defer jobs.free()
mut group := Vector {} as Vector<thread.Thread>
defer group.free()
each name in ["Frank Herbert", "William Gibson", "Antoine de Saint-Exupéry", "Jane Austen"]:
j := mem.Allocate(sizeof(Job)) as Job*
j[0] = Job { author = name }
jobs.push(j)
group = thread.Collect(group, spawn(countSold, j))
end
thread.JoinAll(group)
each j in jobs:
println j[0].author, "sold", j[0].sold
mem.Free(j as void*)
end
endThe pool itself is not locked, so lease and release from one thread. For work spread over threads, the simplest safe rule is the one the second half follows: every thread opens its own connection, uses it and closes it. Never use one Database from two threads at the same time.
11. PostgreSQL features
Everything PostgreSQL can do is one SQL string away. This step uses a few features that save a lot of program code: ON CONFLICT ... DO UPDATE inserts a row or updates the one that is already there, jsonb operators read and search inside JSON documents, unnest turns an array column into rows, full text search finds words in any form, and a window function ranks the authors by takings.
import db.postgres
func show(d: postgres.Database, title: str, sql: str):
r := postgres.Query(d, sql)
defer r.Close()
println title
until r.Next():
mut line := " "
repeat r.ColumnCount() in i:
line = line + " " + r.Text(i)
end
println line
end
end
func main:
d := postgres.Open("host=127.0.0.1 user=postgres password=secret dbname=bookshop")
defer postgres.Close(d)
_ := postgres.Exec(d, `INSERT INTO books (title, author, year, price, stock) VALUES
('Dune', 'Frank Herbert', 1965, 12.50, 5),
('Foundation', 'Isaac Asimov', 1951, 8.75, 4)
ON CONFLICT (title) DO UPDATE SET stock = books.stock + EXCLUDED.stock`)
show(d, "restock, insert or add:", "SELECT title, stock FROM books WHERE title IN ('Dune', 'Foundation') ORDER BY title")
_ := postgres.Exec(d, `UPDATE books SET info = '{"pages": 412, "series": "Dune Chronicles", "awards": ["Hugo", "Nebula"]}' WHERE title = 'Dune';
UPDATE books SET info = '{"pages": 255, "series": "Foundation"}', tags = '{scifi,classic}' WHERE title = 'Foundation'`)
show(d, "jsonb fields:", "SELECT title, info->>'series', (info->>'pages')::int FROM books WHERE info ? 'series' ORDER BY title")
show(d, "won a Hugo:", `SELECT title FROM books WHERE info->'awards' @> '["Hugo"]'`)
show(d, "arrays:", "SELECT tag, count(*) FROM books, unnest(tags) AS tag GROUP BY tag ORDER BY count(*) DESC, tag")
show(d, "full text:", "SELECT title FROM books WHERE to_tsvector('english', title) @@ plainto_tsquery('english', 'guides galaxy')")
show(d, "best sellers:", `SELECT b.author, sum(o.total), rank() OVER (ORDER BY sum(o.total) DESC)
FROM orders o JOIN books b ON b.id = o.book_id GROUP BY b.author`)
endEvery value arrives as text in PostgreSQL's own format: arrays as {scifi,classic}, jsonb as JSON text and booleans as t or f (Int reads those as 1 and 0). When a result has structure, let the SQL build JSON and decode it in Salam, as the JSON guide below shows.
More
- Bulk loading:
COPY FROM STDINis not supported. Insert many rows with one multi rowINSERTper few hundred rows, inside a transaction. QueryIntandQueryTextread the first column of the first row in one call, for counts, sums and settings.- Code written against
db.Connectionruns unchanged on SQLite and MySQL;db.QueryInt,db.RunAllanddb.InTxwork on all of them. ServerVersionreturns the major version times 10000 plus the minor version, so 170011 is 17.11; compare it with a number to use features of newer servers.- Results are held in memory in full. For a very large table, read it in pages with
ORDER BY id LIMIT 1000andWHERE id > last.
Guide: schema migrations
A real application changes its schema over time, and every copy of the database has to follow. db.Migrator from the db package keeps numbered steps, each with SQL to apply it and SQL to undo it, and records the applied ones in a schema_migrations table. Each step runs in its own transaction, which in PostgreSQL covers CREATE TABLE and CREATE INDEX too, so a failed step leaves no half made change behind.
import db
import db.postgres
func main:
d := postgres.Open("host=127.0.0.1 user=postgres password=secret dbname=bookshop")
defer postgres.Close(d)
c := postgres.Conn(d)
mut m := db.NewMigrator()
defer db.FreeMigrator(m)
db.AddMigration(m, 1, "reviews", `CREATE TABLE reviews (
id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
book_id bigint NOT NULL REFERENCES books (id),
stars int NOT NULL CHECK (stars BETWEEN 1 AND 5),
body text)`, "DROP TABLE reviews")
db.AddMigration(m, 2, "orders by customer", "CREATE INDEX orders_customer ON orders (customer)", "DROP INDEX orders_customer")
println "schema version", db.SchemaVersion(m, c), "pending", db.PendingCount(m, c)
println "applied", db.MigrateUp(m, c)
println "schema version", db.SchemaVersion(m, c), "pending", db.PendingCount(m, c)
println "applied", db.MigrateUp(m, c), "on a second run"
println "rollback:", db.Rollback(m, c), "now at", db.SchemaVersion(m, c)
println "index exists:", postgres.QueryInt(d, "SELECT count(*) FROM pg_indexes WHERE indexname = 'orders_customer'")
println "migrate to 2:", db.MigrateTo(m, c, 2)
println postgres.QueryText(d, "SELECT string_agg(version || ' ' || name, ', ' ORDER BY version) FROM schema_migrations")
endRun MigrateUp when the program starts: it applies only the steps the database has not seen, so running it twice is safe. Rollback undoes the newest step and MigrateTo moves to a chosen version.
Guide: settings from the environment
Passwords do not belong in source code. Like libpq, Open starts from the environment variables PGHOST, PGPORT, PGUSER, PGPASSWORD, PGDATABASE, PGSSLMODE and PGSSLROOTCERT, then applies whatever the connection string sets. So Open("") connects with the environment alone, and the program below works with either a single DATABASE_URL or the PG... variables.
import db.postgres
import os
func main:
d := postgres.Open(os.Env("DATABASE_URL"))
defer postgres.Close(d)
if not postgres.Ok(d):
print "database: " + postgres.Error(d)
ret 1
end
println "database", postgres.DatabaseName(d), "on", postgres.HostName(d), "as", postgres.UserName(d)
println postgres.QueryInt(d, "SELECT count(*) FROM books"), "books"
endThe output above came from running it with the PG... variables set:
PGHOST=127.0.0.1 PGUSER=postgres PGPASSWORD=secret PGDATABASE=bookshop salam run app.salam
DATABASE_URL=postgresql://postgres:secret@127.0.0.1:5432/bookshop salam run app.salam
When no user is given anywhere, the USER environment variable is used, and when no database is given, the database with the same name as the user.
Guide: nested results as JSON
A list of books, each with its orders, would take one query per book, or one join whose rows you regroup by hand. PostgreSQL can build the whole nested document in one query with json_agg and json_build_object, and encoding/json reads it straight into Salam structs whose field names match the JSON keys.
import db.postgres
import encoding.json
import encoding.encerr
struct Order:
pub customer: str = ""
pub qty: int = 0
end
struct Book:
pub title: str = ""
pub price: f64 = 0.0
pub orders: Vector<Order> = Vector {}
end
struct Shelf:
pub books: Vector<Book> = Vector {}
end
func main:
d := postgres.Open("host=127.0.0.1 user=postgres password=secret dbname=bookshop")
defer postgres.Close(d)
text := postgres.QueryText(d, `SELECT json_build_object('books', coalesce(json_agg(b ORDER BY b.title), '[]'))
FROM (SELECT bk.title, bk.price,
coalesce((SELECT json_agg(json_build_object('customer', o.customer, 'qty', o.qty) ORDER BY o.id)
FROM orders o WHERE o.book_id = bk.id), '[]') AS orders
FROM books bk WHERE bk.id IN (SELECT book_id FROM orders)) b`)
mut shelf := Shelf {}
mut e := encerr.NoError()
if not json.UnmarshalLenient(text, shelf, e):
println "bad json:", e.message
ret 1
end
each b in shelf.books:
println b.title, b.price, "orders:", b.orders.len()
each o in b.orders:
println " " + o.customer, "x", o.qty
end
end
endcoalesce(..., '[]') turns "no orders" into an empty list instead of NULL. The same pattern works the other way: send a JSON document as one quoted string and unpack it on the server with jsonb_to_recordset.
Guide: which function for which job
| You want to | Use |
|---|---|
| Connect | Open with key=value pairs or a postgresql:// URI, then Ok |
| Run DDL, or a write whose count you do not need | Exec (0 is success) |
| Know how many rows a write changed | ExecCount |
| Get generated ids or changed rows back | Query with INSERT/UPDATE/DELETE ... RETURNING |
| Read many rows | Query, then until r.Next(): and r.Close() |
| Read one number or one string | QueryInt, QueryText |
| Put user text into SQL | Quote, always |
| Tell NULL from empty | r.IsNull(col) |
| Change several things at once or not at all | d.Begin(), then Commit or Rollback |
| Recover after the server dropped the connection | Ping, then Reset |
| Write code that works on several engines | Conn and the db helpers |
Types
Database
struct Database| Name | Type | Description |
|---|---|---|
handle | void* | The connection handle; null when there is none. |
affected | i64 = 0 | The number of rows changed by the last statement. |
import db
import db.postgres
func main:
if not postgres.Available():
println "the Postgres client is not available"
ret
end
println postgres.ClientVersion()
d := postgres.Open("host=127.0.0.1 port=5432 dbname=shop user=app password=s3cret")
if not postgres.Ok(d):
println postgres.Error(d)
ret
end
println postgres.ServerVersion(d), postgres.DatabaseName(d), postgres.UserName(d), postgres.HostName(d), postgres.Port(d)
println postgres.Ping(d)
_ := postgres.Exec(d, "CREATE TABLE IF NOT EXISTS items (id SERIAL PRIMARY KEY, name TEXT, price NUMERIC)")
_ := postgres.Exec(d, "INSERT INTO items (name, price) VALUES (" + postgres.Quote(d, "tea") + ", 12.5)")
println postgres.ExecCount(d, "UPDATE items SET price = price * 1.1")
rows := postgres.Query(d, "SELECT id, name, price FROM items ORDER BY id")
println rows.Ok(), rows.ColumnCount(), rows.RowCount(), rows.ColumnName(1)
until rows.Next():
println rows.Int(0), rows.Text(1), rows.Float(2), rows.Int64(0), rows.IsNull(2), rows.ColumnType(1)
end
_ := rows.Close()
println postgres.QueryInt(d, "SELECT COUNT(*) FROM items"), postgres.QueryText(d, "SELECT version()")
c := postgres.Conn(d)
tx := c.Begin()
_ := tx.Exec("DELETE FROM items")
_ := tx.Rollback()
println db.QueryInt(c, "SELECT COUNT(*) FROM items"), d.Kind()
postgres.Reset(d)
_ := postgres.Close(d)
_ := postgres.OpenFull("db.example.com", 5432, "app", "s3cret", "shop")
endKind
func Kind(): strThe example for this item is shown with the ones above: db-postgres-Client.salam
Ok
func Ok(): boolThe example for this item is shown with the ones above: db-postgres-Client.salam
Ping
func Ping(): boolThe example for this item is shown with the ones above: db-postgres-Client.salam
Exec
func Exec(sql: str): intChanges.The example for this item is shown with the ones above: db-postgres-Client.salam
Query
func Query(sql: str): dyn db.RowsOk on the result is false when it failed or returned no rows set.The example for this item is shown with the ones above: db-postgres-Client.salam
Begin
func Begin(): dyn db.TxBEGIN; the transaction is not Ok when that failed.The example for this item is shown with the ones above: db-postgres-Client.salam
LastInsertId
func LastInsertId(): i64SELECT lastval()); 0 when there is none.The example for this item is shown with the ones above: db-postgres-Client.salam
Changes
func Changes(): i64Exec called on this value changed.The example for this item is shown with the ones above: db-postgres-Client.salam
Error
func Error(): strThe example for this item is shown with the ones above: db-postgres-Client.salam
Quote
func Quote(s: str): strE'...' form.The example for this item is shown with the ones above: db-postgres-Client.salam
Close
func Close(): intThe example for this item is shown with the ones above: db-postgres-Client.salam
Rows
struct Rows| Name | Type | Description |
|---|---|---|
handle | void* | The result handle; null when the query failed. |
row | int = -1 | The index of the current row; -1 before the first Next. |
count | int = 0 | The number of rows. |
The example for this item is shown with the ones above: db-postgres-Client.salam
Ok
func Ok(): boolThe example for this item is shown with the ones above: db-postgres-Client.salam
Next
func Next(): boolThe example for this item is shown with the ones above: db-postgres-Client.salam
ColumnCount
func ColumnCount(): intThe example for this item is shown with the ones above: db-postgres-Client.salam
RowCount
func RowCount(): intThe example for this item is shown with the ones above: db-postgres-Client.salam
ColumnName
func ColumnName(col: int): strThe example for this item is shown with the ones above: db-postgres-Client.salam
ColumnType
func ColumnType(col: int): intTYPE_... constants in db: integers and booleans are TYPE_INT, real, double precision and numeric are TYPE_FLOAT, bytea is TYPE_BLOB, NULL is TYPE_NULL and the rest is TYPE_TEXT.The example for this item is shown with the ones above: db-postgres-Client.salam
IsNull
func IsNull(col: int): boolThe example for this item is shown with the ones above: db-postgres-Client.salam
Text
func Text(col: int): strThe example for this item is shown with the ones above: db-postgres-Client.salam
Int
func Int(col: int): intt and f read as 1 and 0, NULL as 0.The example for this item is shown with the ones above: db-postgres-Client.salam
Int64
func Int64(col: int): i64t and f read as 1 and 0, NULL as 0.The example for this item is shown with the ones above: db-postgres-Client.salam
Float
func Float(col: int): f64The example for this item is shown with the ones above: db-postgres-Client.salam
Close
func Close(): intThe example for this item is shown with the ones above: db-postgres-Client.salam
Txn
struct Txn| Name | Type | Description |
|---|---|---|
handle | void* | The connection pointer. |
The example for this item is shown with the ones above: db-postgres-Client.salam
Ok
func Ok(): boolBEGIN worked.The example for this item is shown with the ones above: db-postgres-Client.salam
Exec
func Exec(sql: str): intThe example for this item is shown with the ones above: db-postgres-Client.salam
Query
func Query(sql: str): dyn db.RowsThe example for this item is shown with the ones above: db-postgres-Client.salam
Commit
func Commit(): intThe example for this item is shown with the ones above: db-postgres-Client.salam
Rollback
func Rollback(): intThe example for this item is shown with the ones above: db-postgres-Client.salam
Functions
Available
func Available(): boolThe example for this item is shown with the ones above: db-postgres-Client.salam
ClientVersion
func ClientVersion(): intThe example for this item is shown with the ones above: db-postgres-Client.salam
ServerVersion
func ServerVersion(d: Database): intThe example for this item is shown with the ones above: db-postgres-Client.salam
Open
func Open(conninfo: str): Databasehost=... port=... user=... password=... dbname=..., or a postgresql:// URI. Unset values come from PGHOST, PGPORT, PGUSER, PGPASSWORD, PGDATABASE and PGSSLMODE. It always returns a Database; check Ok.The example for this item is shown with the ones above: db-postgres-Client.salam
OpenFull
func OpenFull(host: str, port: int, user: str, pass: str, dbname: str): DatabaseOpen for a value with spaces.The example for this item is shown with the ones above: db-postgres-Client.salam
Ok
func Ok(d: Database): boolThe example for this item is shown with the ones above: db-postgres-Client.salam
Ping
func Ping(d: Database): boolThe example for this item is shown with the ones above: db-postgres-Client.salam
Reset
func Reset(d: Database)The example for this item is shown with the ones above: db-postgres-Client.salam
Close
func Close(d: Database): intThe example for this item is shown with the ones above: db-postgres-Client.salam
Error
func Error(d: Database): strThe example for this item is shown with the ones above: db-postgres-Client.salam
Exec
func Exec(d: Database, sql: str): intThe example for this item is shown with the ones above: db-postgres-Client.salam
ExecCount
func ExecCount(d: Database, sql: str): i64The example for this item is shown with the ones above: db-postgres-Client.salam
Query
func Query(d: Database, sql: str): RowsOk on the result is false when it failed or returned no rows set.The example for this item is shown with the ones above: db-postgres-Client.salam
Quote
func Quote(d: Database, s: str): strThe example for this item is shown with the ones above: db-postgres-Client.salam
DatabaseName
func DatabaseName(d: Database): strThe example for this item is shown with the ones above: db-postgres-Client.salam
UserName
func UserName(d: Database): strThe example for this item is shown with the ones above: db-postgres-Client.salam
HostName
func HostName(d: Database): strThe example for this item is shown with the ones above: db-postgres-Client.salam
Port
func Port(d: Database): strThe example for this item is shown with the ones above: db-postgres-Client.salam
QueryInt
func QueryInt(d: Database, sql: str): intThe example for this item is shown with the ones above: db-postgres-Client.salam
QueryText
func QueryText(d: Database, sql: str): strThe example for this item is shown with the ones above: db-postgres-Client.salam
Conn
func Conn(d: Database): dyn db.ConnectionConnection.The example for this item is shown with the ones above: db-postgres-Client.salam