net.utls

A firewall that cannot read your traffic can still read your first message. The ClientHello of a TLS connection lists the cipher suites, the extensions and their order, the curves and the signature algorithms the client supports, and every TLS library lists them in its own way. That list is a fingerprint (JA3 is the best known way to write it down), and a proxy client that uses the default list of its TLS library has a fingerprint no browser has. This package builds the ClientHello itself, from a description that you can read and change, so that it has the layout of a browser.

Profile returns a Spec for a client family. Build turns a spec and some Options (the server name, the ALPN list, whether to shuffle the extensions) into a Hello: the message bytes, with a fresh random, a fresh x25519 key pair and a random session id. WrapClient then runs a TLS 1.3 handshake on a connected socket with that hello and returns an ordinary tls.Conn. JA3 writes down the fingerprint of any hello, and Record wraps it into a TLS record, for example to hand it to the fragmenting helpers of net.censor. net.reality uses this package for the hello it seals its proof into.

The profiles

These are approximations written from memory of what these clients send, not captures. A comparison with a capture of the real browser was not possible here. The structure was checked in two ways: the hellos parse with net.censor.ParseClientHello, and they complete a real TLS 1.3 handshake with the tls server. Browsers change their hello with each release, so treat a profile as a starting point and edit the Spec (it is plain text lists) to match the browser you want to look like.

How a spec is written

The lists in a Spec are text of hexadecimal numbers separated by spaces. g0 to g4 are GREASE slots that become a random reserved value (0a0a, 1a1a and so on); pad marks where the padding extension goes. extensions is the order on the wire; 0000 is the server name, 0010 ALPN, 0033 the key share, fe0d ECH GREASE and so on. key_shares says which shares to send: 001d is x25519 with the key pair of the hello, 0017 is a secp256r1 share with random bytes, 11ec is X25519MLKEM768.

What trips people up

• The key shares other than x25519 have no key behind them. The secp256r1 share is random bytes, and the ML-KEM part of the hybrid share is a well formed random encapsulation key. A server that picks one of those groups cannot finish the handshake with you; it can for x25519. Pass pq = false to leave the hybrid group out of the hello completely.
• The hello offers ALPN h2 by default. If the server picks it you are talking to something that expects HTTP/2. For a proxy transport set alpn = "http/1.1", or "none" to leave the extension out.
• Some bytes are random on purpose. GREASE values, the ECH length, the random, the session id and the key shares change with every Build. JA3 ignores GREASE and key bytes, so the JA3 of a profile is stable.
• Only TLS 1.3, and no HelloRetryRequest. The handshake from a custom hello fails with a clear message if the server wants a different group or only speaks TLS 1.2.
• Free the hello. A Hello owns its buffers: call FreeHello when you are done.

Building a hello and reading its JA3

The example builds one hello for each of three profiles with the hybrid share left out, and prints the JA3 strings. A JA3 string is the version, the cipher suites, the extension ids, the curves and the point formats, each as decimal numbers, with GREASE removed. The order of the extension ids is the clearest difference between the profiles.

import net.utls

func main:
    println "profiles:", utls.ProfileNames()
    mut o := utls.Options {}
    o.server_name = "www.example.com"
    o.pq = false
    each name in ["chrome", "firefox", "safari"]:
        mut h := utls.Build(utls.Profile(name), o)
        println name, "hello is larger than 400 bytes:", h.raw.len > 400
        println utls.JA3(h.raw)
        utls.FreeHello(h)
    end
    println "unknown profile is ok:", utls.Profile("netscape").ok
end

A handshake from a profile

WrapClient takes a connected socket, a spec, the options and a tls.Config. The certificate is checked the normal way. The result is a tls.Conn, so every tls function and the TLS support of net.transport work with it.

import net.tcp
import net.utls
import tls

func main:
    c := tcp.Dial("www.example.com", 443)
    mut o := utls.Options {}
    o.server_name = "www.example.com"
    o.alpn = "http/1.1"
    cfg := tls.NewConfig("www.example.com")
    mut t := utls.WrapClient(c.fd, utls.Profile("chrome"), o, cfg)
    if not tls.ConnOk(t):
        println "failed:", tls.ConnError(t)
        ret 1
    end
    tls.Write(t, "GET / HTTP/1.1\r\nHost: www.example.com\r\nConnection: close\r\n\r\n")
    println tls.Read(t, 200)
    tls.Close(t)
end

Limits

• The profiles are approximations, not captures. They were not compared with a live browser, and the real clients change from release to release.
• There is no ML-KEM. The hybrid share is a placeholder (see above), and a server that selects it breaks the handshake.
• No session resumption and no pre-shared key extension. The hello is always a first connection.
• The ECH extension is GREASE. It has the shape of a real one and nothing is encrypted in it.
• The record layer and the handshake are the library's. The first record has the legacy version 0x0301 as browsers do, but the lengths and timing of what comes after the hello are those of the tls package, not of a browser.
• TLS 1.3 only. A server that answers with TLS 1.2 fails the handshake.

Types

struct Bytes

struct Spec

struct Options

struct Hello

Functions

func JA3(b: Bytes): str

func FreeBytes(b &: Bytes)

func FreeHello(h &: Hello)

func ToHex(b: Bytes): str

func SessionIdOffset(): int

func SetSessionId(h &: Hello, sid: void*)

func Build(sp: Spec, o: Options): Hello

func Profile(name: str): Spec

func ProfileNames(): str

func LegacyProfileNames(): str

func Record(h: Hello): Bytes

func PqSeed(h: Hello): void*

func WrapClient(fd: i64, sp: Spec, o: Options, cfg: tls.Config): tls.Conn