net/tcp

TCP is the base of most internet protocols: a reliable, ordered stream of bytes between two programs. A server listens on a port with Bind and takes each new client with Accept; a client connects with Dial.

Accept and Read wait until something arrives, so a server that talks to several clients hands each connection to a thread, as the worker guide shows. Every example runs on 127.0.0.1 and needs no network.

Use this package for your own protocols and for talking to servers that have no client in the standard library. For HTTP use net/http; for messages that must not be split or ordered, net/udp; for live two-way traffic with browsers, net/websocket. tls adds encryption on top of a connection from here.

What trips people up

• An empty Read means three things. The peer closed the connection, the read deadline passed, or an error happened; all return "". A message that starts with a zero byte also comes back empty, and text after a zero byte is lost, so use ReadBytes for binary data.
• Write may send only part of the text. It makes one system call and returns how many bytes went out. For large data, compare the result with the length, or use WriteBytes, which loops until everything is sent.
• Messages are not kept apart. TCP is a byte stream; frame your messages with a delimiter or a length, as the framing guide does.
• Bind listens on every IPv4 interface, not only on 127.0.0.1, so a test server is reachable from the network unless a firewall stops it. IPv6 clients cannot reach it; Dial itself tries every address a name resolves to, IPv6 included.
• Nothing stops the program on failure. Bind, Accept and Dial return a value with fd -1; check Ok or ConnOk.

A request and reply protocol

Most small protocols take turns: the client sends one request, the server sends one reply, and so on until the client hangs up. The server below reads in a loop and stops when Read returns an empty string, which is how TCP tells you the other side closed the connection. The client waits for each reply before it sends the next request, so every Read sees exactly one message.

import net.tcp
import str

mut server := tcp.Server {}

func reply(line: str): str:
    words := line.split(" ")
    command := words.get(0)
    if command == "ADD" and words.len() == 3:
        ret str.FromInt(words.get(1).to_int() + words.get(2).to_int())
    end
    if command == "LEN" and words.len() == 2:
        ret str.FromInt(words.get(1).char_count())
    end
    ret "ERR unknown command: " + command
end

func serve():
    conn := tcp.Accept(server)
    tcp.SetReadDeadline(conn, 2000)
    mut served := 0
    until true:
        request := tcp.Read(conn, 1024)
        if request == "":
            break
        end
        _ := tcp.Write(conn, reply(request))
        served += 1
    end
    tcp.Close(conn)
    println "server: answered", served, "requests"
end

func main:
    server = tcp.Bind(0)
    t := spawn(serve)
    conn := tcp.Dial("127.0.0.1", tcp.Port(server))
    tcp.SetReadDeadline(conn, 2000)
    each request in ["ADD 2 40", "LEN salam", "PING"]:
        _ := tcp.Write(conn, request)
        println request, "->", tcp.Read(conn, 1024)
    end
    tcp.Close(conn)
    join(t)
    tcp.CloseServer(server)
end

The server runs in its own thread with spawn and the main program waits for it with join. A real server would call Accept in a loop and hand each connection to a worker; while one client is served, the others wait in the queue of the listening socket.

Serving several clients with worker threads

A server that reads from one client at a time stalls everyone behind a slow client. The usual shape is one thread that only accepts, a chan that carries the new connections, and a few worker threads that serve them. A thread receives the connection as its socket number, conn.fd, and turns it back into a Conn with ConnFromFd.

import net.tcp
import chan
import str

mut server := tcp.Server {}
mut jobs := chan.Chan {} as chan.Chan<i64>

func acceptor():
    repeat 3:
        conn := tcp.Accept(server)
        if tcp.ConnOk(conn): _ := jobs.send(conn.fd) end
    end
    jobs.close()
end

func worker():
    mut ok := true
    until true:
        fd := jobs.recv(ok)
        if not ok: break end
        conn := tcp.ConnFromFd(fd)
        tcp.SetReadDeadline(conn, 2000)
        request := tcp.Read(conn, 1024)
        _ := tcp.Write(conn, "HELLO " + str.Upper(request))
        tcp.Close(conn)
    end
end

func main:
    server = tcp.Bind(0)
    jobs.init(8 as i64)
    a := spawn(acceptor)
    w1 := spawn(worker)
    w2 := spawn(worker)

    names := ["sara", "reza", "mina"]
    mut conns := Vector {} as Vector<tcp.Conn>
    repeat names.len():
        c := tcp.Dial("127.0.0.1", tcp.Port(server))
        tcp.SetReadDeadline(c, 2000)
        conns.push(c)
    end
    each i, c in conns:
        _ := tcp.Write(c, names[i])
    end
    each i, c in conns:
        println names[i], "->", tcp.Read(c, 1024)
        tcp.Close(c)
    end
    join(a)
    join(w1)
    join(w2)
    tcp.CloseServer(server)
end

The three clients connect before any of them sends a word, and only two workers exist, so the third connection waits in the channel until a worker is free; it is still answered. Closing the channel after the last Accept makes recv report ok false, which ends the worker loops so join returns. In a long-running server the acceptor loops forever, and the number of workers caps how many clients are served at the same time.

Messages in a byte stream

TCP moves bytes, not messages. Two writes can arrive as one read, and one write can arrive split over several reads. The lock step in the previous section hides that, but as soon as the other side sends several messages in a row you need a frame: a delimiter such as a newline, or a length in front of each message. The program below sends four lines in three uneven writes and puts them back together by keeping the unfinished part of the last read in pending.

import net.tcp

mut server := tcp.Server {}

func sender():
    conn := tcp.Accept(server)
    _ := tcp.Write(conn, "apple\nban")
    _ := tcp.Write(conn, "ana\ncherry\ndate")
    _ := tcp.Write(conn, "\n")
    tcp.Close(conn)
end

func main:
    server = tcp.Bind(0)
    t := spawn(sender)
    conn := tcp.Dial("127.0.0.1", tcp.Port(server))
    tcp.SetReadDeadline(conn, 2000)
    mut pending := ""
    mut count := 0
    until true:
        chunk := tcp.Read(conn, 4096)
        if chunk == "":
            break
        end
        pending = pending + chunk
        mut nl := pending.find("\n")
        until nl >= 0:
            count += 1
            println count, pending.substr(0, nl)
            pending = pending.substr(nl + 1, pending.len() - nl - 1)
            nl = pending.find("\n")
        end
    end
    println "left over:", pending.len(), "bytes"
    tcp.Close(conn)
    join(t)
    tcp.CloseServer(server)
end

For binary data, send a fixed size header with the length and read until you have that many bytes. ReadBytes and WriteBytes work on raw memory for that, and they do not stop at a zero byte the way a str does.

Timeouts and failed connections

Without a deadline, Read waits forever for a peer that never answers. SetReadDeadline limits how long one read may wait, SetWriteDeadline does the same for writes, and SetDeadline sets both. When the time runs out the read returns an empty string, the same as a closed connection, so treat both as the end of the conversation. Dial does not stop the program when nothing listens on the port: it returns a connection for which ConnOk is false.

import net.tcp

func main:
    silent := tcp.Bind(0)
    conn := tcp.Dial("127.0.0.1", tcp.Port(silent))
    println "connected:", tcp.ConnOk(conn)
    tcp.SetReadDeadline(conn, 300)
    answer := tcp.Read(conn, 1024)
    if answer == "":
        println "no answer within 300 ms, giving up"
    end
    tcp.Close(conn)
    tcp.CloseServer(silent)

    refused := tcp.Dial("127.0.0.1", tcp.Port(silent))
    if not tcp.ConnOk(refused):
        println "nothing listens there any more"
    end
end

A tiny HTTP server

HTTP/1.1 is text over TCP, so a few lines are enough for a status page or a health check. The request starts with a line such as GET /hello HTTP/1.1; WriteHTTP writes the status line, Content-Type, Content-Length and Connection: close for you. The client side shows the other half: it reads until the server closes the connection and splits the head from the body at the first empty line.

import net.tcp

mut server := tcp.Server {}

func handle():
    conn := tcp.Accept(server)
    tcp.SetDeadline(conn, 2000)
    request := tcp.Read(conn, 4096)
    first := request.split("\r\n").get(0)
    path := first.split(" ").get(1)
    if path == "/hello":
        tcp.WriteHTTP(conn, "200 OK", "text/plain", "hello from Salam")
    else:
        tcp.WriteHTTP(conn, "404 Not Found", "text/plain", "no page at " + path)
    end
    tcp.Close(conn)
end

func fetch(path: str):
    t := spawn(handle)
    conn := tcp.Dial("127.0.0.1", tcp.Port(server))
    tcp.SetDeadline(conn, 2000)
    _ := tcp.Write(conn, "GET " + path + " HTTP/1.1\r\nHost: localhost\r\n\r\n")
    mut response := ""
    until true:
        chunk := tcp.Read(conn, 4096)
        if chunk == "":
            break
        end
        response = response + chunk
    end
    tcp.Close(conn)
    join(t)
    split_at := response.find("\r\n\r\n")
    println response.split("\r\n").get(0)
    println "  body:", response.substr(split_at + 4, response.len() - split_at - 4)
end

func main:
    server = tcp.Bind(0)
    fetch("/hello")
    fetch("/missing")
    tcp.CloseServer(server)
end

This is fine for a debug endpoint. For a real web service use net/http, which handles keep alive, chunked bodies, headers and routing.

Encrypted connections

DialTLS connects and runs the TLS handshake in one call and returns a tls.Conn. From then on you read and write with the functions of the tls package, not with tcp.Read, because the bytes on the socket are encrypted. This example needs the internet; it prints the TLS version and the first line of the answer, which is HTTP/1.1 200 OK for example.com.

import net.tcp
import tls

func main:
    mut conn := tcp.DialTLS("example.com", 443)
    if not tls.ConnOk(conn):
        println "TLS failed:", tls.ConnError(conn)
        ret 1
    end
    println "protocol:", tls.Version(conn)
    _ := tls.Write(conn, "GET / HTTP/1.1\r\nHost: example.com\r\nConnection: close\r\n\r\n")
    head := tls.Read(conn, 4096)
    println head.split("\r\n").get(0)
    tls.Close(conn)
    ret 0
end

Protocols such as SMTP and IMAP start in plain text and switch to TLS later with a STARTTLS command. For those, Dial first, talk until the server agrees, then call UpgradeTLS on the same connection.

Which function for which job

Types

struct Server

import net.tcp

mut server := tcp.Server {}

func responder():
    conn := tcp.Accept(server)
    message := tcp.Read(conn, 1024)
    _ := tcp.Write(conn, "echo: " + message)
    tcp.Close(conn)
end

func main:
    server = tcp.Bind(0)
    println tcp.Ok(server), tcp.Port(server) > 0
    t := spawn(responder)
    conn := tcp.Dial("127.0.0.1", tcp.Port(server))
    println tcp.ConnOk(conn)
    tcp.SetDeadline(conn, 2000)
    tcp.SetKeepAlive(conn, true)
    tcp.SetNoDelay(conn, true)
    println tcp.Write(conn, "hello")
    println tcp.Read(conn, 1024)
    tcp.Close(conn)
    join(t)
    tcp.CloseServer(server)
    println tcp.ConnOk(tcp.Dial("127.0.0.1", 1))
end

struct Conn

struct PoolEntry

import net.tcp

func main:
    s := tcp.Bind(0)
    port := tcp.Port(s)
    mut pool := tcp.NewPool()
    conn := tcp.Get(pool, "127.0.0.1", port, 1000)
    println tcp.ConnOk(conn), tcp.PoolLen(pool)
    tcp.Put(pool, "127.0.0.1", port, conn, 1000)
    println tcp.PoolLen(pool)
    again := tcp.Get(pool, "127.0.0.1", port, 2000)
    println again.fd == conn.fd, tcp.PoolLen(pool)
    tcp.Put(pool, "127.0.0.1", port, again, 2000)
    fresh := tcp.Get(pool, "127.0.0.1", port, 2000 + pool.idle_ttl_ms + 1)
    println tcp.ConnOk(fresh), tcp.PoolLen(pool)
    tcp.Close(fresh)
    tcp.CloseServer(s)
end

struct PoolEntries

struct Pool

Functions

func Bind(port: int): Server

func Ok(srv: Server): bool

func Port(srv: Server): int

func Accept(srv: Server): Conn

func CloseServer(srv: Server)

func ConnOk(c: Conn): bool

func Read(c: Conn, max: int): str

func Write(c: Conn, data: str): int

func ReadBytes(c: Conn, buf: void*, max: int): int

import net.tcp
import mem

mut server := tcp.Server {}

func responder():
    conn := tcp.Accept(server)
    tcp.SetReadDeadline(conn, 2000)
    tcp.SetWriteDeadline(conn, 2000)
    _ := tcp.Read(conn, 1024)
    tcp.WriteHTTP(conn, "200 OK", "text/plain", "hello")
    tcp.Close(conn)
end

func main:
    server = tcp.Bind(0)
    t := spawn(responder)
    conn := tcp.Dial("127.0.0.1", tcp.Port(server))
    request := "GET / HTTP/1.0\r\n\r\n"
    println tcp.WriteBytes(conn, request as void*, request.len())
    b := mem.AllocateZeroed(256 as u64)
    mut total := 0
    until true:
        r := tcp.ReadBytes(conn, b, 255)
        if r <= 0:
            break
        end
        total += r
    end
    println total > 0
    again := tcp.ConnFromFd(conn.fd)
    println again.fd == conn.fd
    tcp.Close(conn)
    join(t)
    tcp.CloseServer(server)
    mem.Free(b)
end

func WriteBytes(c: Conn, ptr: void*, n: int): int

func Close(c: Conn)

func Dial(host: str, port: int): Conn

func DialTimeout(host: str, port: int, millis: int): Conn

func DialTLS(host: str, port: int): tls.Conn

import net.tcp
import tls

func main:
    _ := tcp.DialTLS("example.com", 443)
    plain := tcp.Dial("example.com", 443)
    _ := tcp.UpgradeTLS(plain, tls.NewConfig("example.com"))
end

func UpgradeTLS(c: Conn, cfg: tls.Config): tls.Conn

func ConnFromFd(fd: i64): Conn

func WriteHTTP(c: Conn, status_line: str, content_type: str, body: str)

func SetReadDeadline(c: Conn, millis: int)

func SetWriteDeadline(c: Conn, millis: int)

func SetDeadline(c: Conn, millis: int)

func SetKeepAlive(c: Conn, enable: bool)

func SetNoDelay(c: Conn, enable: bool)

func NewPool(): Pool

func Get(p &: Pool, host: str, port: int, now_ms: i64): Conn

func Put(p &: Pool, host: str, port: int, c: Conn, now_ms: i64)

func PoolLen(p: Pool): int