Standard library · Machine learning

tensor

Multi dimensional arrays and numeric operations for machine learning.

import tensor

On this page

Types
Constants and variables
Functions

A tensor is a multi dimensional array of numbers: a vector is one dimensional, a matrix two, and a batch of images four. The data is 32 bit floats in row order. The +, -, * and / operators work element by element and take tensors of the same shape, or shapes that broadcast.

This package is arithmetic, with no gradients. To train a network, use autograd and nn. Matrix multiplication and convolution go multi threaded with SetThreads.

Types

`Tensor`

struct Tensor

A multi dimensional array of 32 bit floats; the base of every neural network computation.

Name	Type	Description
`data`	`Vector`	The elements in row order.
`shape`	`Vector`	The dimensions.

tensor-Create.salam

import tensor
import compress

func main:
    mut shape := tensor.Shape2(2, 3)
    mut a := tensor.Arange(6)
    mut b := tensor.Reshape(a, shape)
    println tensor.ShapeString(b), b.rank(), b.numel(), b.dim(0), b.dim(1), b.is_empty()
    println tensor.ToString(b)
    println b.at2(1, 2), b.at(4)
    b.set_at2(0, 0, 9.0)
    b.set_at(1, 8.0)
    b.add_at(2, 0.5)
    tensor.Print(b)

    mut z := tensor.Zeros(shape)
    println tensor.ShapeString(z), z.same_shape(b), tensor.SameShape(z, b)
    mut z1 := tensor.Zeros1(3)
    mut z2 := tensor.Zeros2(2, 2)
    mut z3 := tensor.Zeros3(2, 2, 2)
    mut z4 := tensor.Zeros4(1, 2, 2, 2)
    println z1.numel(), z2.numel(), z3.numel(), z4.rank()
    mut ones := tensor.Ones(tensor.Shape1(3))
    mut full := tensor.Full(tensor.Shape3(1, 1, 2), 7.0)
    mut scalar := tensor.Scalar(2.5)
    println tensor.ToString(ones), tensor.ToString(full), scalar.rank(), scalar.at(0)
    println tensor.Shape4(1, 2, 3, 4).len(), tensor.Empty().is_empty()

    mut payload := Vector {} as Vector<f32>
    payload.push(1.0 as f32)
    payload.push(2.0 as f32)
    mut from_array := tensor.FromArray(tensor.Shape1(2), payload)
    mut copy := tensor.Clone(from_array)
    tensor.SetAll(copy, 5.0)
    tensor.CopyInto(z1, ones)
    println tensor.ToString(from_array), tensor.ToString(copy), tensor.ToString(z1)

    mut shape2 := tensor.Shape3(3, 1, 2)
    println b.reshape_(shape2), tensor.ShapeString(b)
    tensor.SqueezeInPlace(b)
    println tensor.ShapeString(b)
    tensor.UnsqueezeInPlace(b, 0)
    println tensor.ShapeString(b)
    mut order := Vector {} as Vector<int>
    order.push(2)
    order.push(1)
    order.push(0)
    mut permuted := tensor.Permute(b, order)
    println tensor.ShapeString(permuted)
    mut m := tensor.Arange(6)
    mut m2 := tensor.Reshape(m, shape)
    println tensor.ToString(tensor.SliceAxis0(m2, 1, 1)), tensor.ToString(tensor.Narrow(m2, 1, 1, 2))
    println tensor.ShapeString(tensor.Concat2(m2, m2, 0)), tensor.ShapeString(tensor.Stack2(m2, m2))
    println tensor.ShapeString(tensor.Expand(tensor.Scalar(1.0), shape))

    as_matrix := tensor.ToMatrix(m2)
    println as_matrix.rows, as_matrix.cols, tensor.ShapeString(tensor.FromMatrix(as_matrix))

    mut buffer := compress.NewByteBuf()
    println tensor.WriteTensor(buffer, "w", m2)
    mut reader := compress.ReaderFromBuf(buffer)
    mut name := ""
    mut read := tensor.Empty()
    println tensor.ReadTensor(reader, name, read), name, tensor.Equal(read, m2)
    compress.BufFree(buffer)
    println tensor.TSR_TOL
    tensor.Free(a)
    b.free()
end

Output2x3 2 6 2 3 false tensor(2x3)[0 1 2 3 4 5] 5 4 tensor(2x3)[9 8 2.5 3 4 5] 2x3 true true 3 4 8 4 tensor(3)[1 1 1] tensor(1x1x2)[7 7] 1 2.5 4 true tensor(2)[1 2] tensor(2)[5 5] tensor(3)[1 1 1] true 3x1x2 3x2 1x3x2 2x3x1 tensor(1x3)[3 4 5] tensor(2x2)[1 2 4 5] 4x3 2x2x3 2x3 2 3 2x3 true true w true 1e-05

`rank`

func rank(): int

The number of dimensions.

The example for this item is shown with the ones above: tensor-Create.salam

`numel`

func numel(): int

The total number of elements.

The example for this item is shown with the ones above: tensor-Create.salam

`dim`

func dim(i: int): int

The size of dimension i.

The example for this item is shown with the ones above: tensor-Create.salam

`is_empty`

func is_empty(): bool

It has no elements.

The example for this item is shown with the ones above: tensor-Create.salam

`same_shape`

func same_shape(o &: Tensor): bool

Its shape matches another tensor's.

The example for this item is shown with the ones above: tensor-Create.salam

`at`

func at(i: int): f64

Element number i in row order.

The example for this item is shown with the ones above: tensor-Create.salam

`set_at`

func set_at(i: int, v: f64)

Sets element i.

The example for this item is shown with the ones above: tensor-Create.salam

`add_at`

func add_at(i: int, v: f64)

Adds to element i.

The example for this item is shown with the ones above: tensor-Create.salam

`at2`

func at2(i: int, j: int): f64

The element at row i and column j of a two dimensional tensor.

The example for this item is shown with the ones above: tensor-Create.salam

`set_at2`

func set_at2(i: int, j: int, v: f64)

Sets the element at row i and column j.

The example for this item is shown with the ones above: tensor-Create.salam

`reshape_`

func reshape_(shape &: Vector<int>): bool

Changes the shape in place, when the number of elements matches.

The example for this item is shown with the ones above: tensor-Create.salam

`operator_add`

func operator_add(other: Tensor): Tensor

The + operator, element by element with broadcasting.

The example for this item is shown with the ones above: tensor-Create.salam

`operator_sub`

func operator_sub(other: Tensor): Tensor

The - operator.

The example for this item is shown with the ones above: tensor-Create.salam

`operator_mul`

func operator_mul(other: Tensor): Tensor

The * operator, elementwise multiplication.

The example for this item is shown with the ones above: tensor-Create.salam

`operator_div`

func operator_div(other: Tensor): Tensor

The / operator.

The example for this item is shown with the ones above: tensor-Create.salam

`free`

func free()

Frees the memory.

The example for this item is shown with the ones above: tensor-Create.salam

`Rng`

struct Rng

A deterministic random number generator; one seed always gives the same sequence.

Name	Type	Description
`s`	`u64 = 0`	The internal state.

tensor-Random.salam

import tensor

func rounded(t &: tensor.Tensor): str:
    mut text := ""
    repeat t.numel() in i:
        if i > 0:
            text += " "
        end
        text += "" + ((t.at(i) * 100.0 + 10000.5) as int - 10000)
    end
    ret text
end

func main:
    mut rng := tensor.NewRng(42)
    println rng.next_bits() > 0, rng.next_float() < 1.0, rng.next_normal() < 10.0
    println tensor.DeriveSeed(42, 1) != tensor.DeriveSeed(42, 2)
    mut v := tensor.Zeros1(5)
    tensor.FillUniform(v, 0.0, 1.0, rng)
    println tensor.MinAll(v) >= 0.0, tensor.MaxAll(v) < 1.0
    tensor.FillNormal(v, 0.0, 1.0, rng)
    tensor.FillBernoulli(v, 0.5, rng)
    println tensor.SumAll(v) <= 5.0
    mut t1 := tensor.RandUniform(tensor.Shape1(3), -1.0, 1.0, 7)
    mut t2 := tensor.RandUniform(tensor.Shape1(3), -1.0, 1.0, 7)
    println tensor.Equal(t1, t2), rounded(t1)
    mut n := tensor.RandNormal(tensor.Shape1(2), 7)
    println n.numel()

    tensor.SetThreads(2)
    println tensor.Threads()

    println tensor.ConvOutSize(5, 3, 1, 1), tensor.ConvOutSize(5, 3, 2, 0)
    mut image := tensor.Reshape(tensor.Arange(16), tensor.Shape4(1, 1, 4, 4))
    mut kernel := tensor.Ones(tensor.Shape4(1, 1, 2, 2))
    mut bias := tensor.Zeros1(1)
    mut conv := tensor.Conv2DForward(image, kernel, bias, 2, 2, 0, 0)
    println tensor.ShapeString(conv), rounded(conv)
    mut argmax := tensor.Empty()
    mut pooled := tensor.MaxPool2DForward(image, 2, 2, 2, 2, argmax)
    println rounded(pooled), rounded(argmax)
    println rounded(tensor.AvgPool2DForward(image, 2, 2, 2, 2))
    mut cols := tensor.Im2Col(image, 0, 2, 2, 2, 2, 0, 0)
    println tensor.ShapeString(cols)
    mut grad := tensor.Zeros4(1, 1, 4, 4)
    println tensor.Col2ImAcc(cols, grad, 0, 2, 2, 2, 2, 0, 0), tensor.Equal(grad, image)

    mut m := tensor.Reshape(tensor.Arange(6), tensor.Shape2(2, 3))
    mut mt := tensor.TransposeRaw(m.data.ref(0), 2, 3)
    println tensor.ShapeString(mt), rounded(mt)
    mut product := tensor.Zeros2(2, 2)
    tensor.GemmAcc(m.data.ref(0), mt.data.ref(0), product.data.ref(0), 2, 3, 2)
    println rounded(product)
end

Outputtrue true true true true true true true -32 47 55 2 2 5 2 1x1x2x2 1000 1800 4200 5000 500 700 1300 1500 500 700 1300 1500 250 450 1050 1250 4x4 true true 3x2 0 300 100 400 200 500 500 1400 1400 5000

`next_bits`

func next_bits(): u64

64 random bits.

The example for this item is shown with the ones above: tensor-Random.salam

`next_float`

func next_float(): f64

A float between zero and one.

The example for this item is shown with the ones above: tensor-Random.salam

`next_normal`

func next_normal(): f64

A number from the standard normal distribution.

The example for this item is shown with the ones above: tensor-Random.salam

Constants and variables

`TSR_TOL`

const TSR_TOL: f64 = 1e-05

The default comparison tolerance.

The example for this item is shown with the ones above: tensor-Create.salam

Functions

`Transpose2D`

func Transpose2D(x &: Tensor): Tensor

The transpose of a matrix.

tensor-Math.salam

import tensor

func show(name: str, t &: tensor.Tensor):
    println name, tensor.ToString(t)
end

func show_rounded(name: str, t &: tensor.Tensor):
    mut text := ""
    repeat t.numel() in i:
        rounded := (t.at(i) * 1000.0 + 0.5) as int
        text += " " + rounded
    end
    println name + " x1000:" + text
end

func main:
    mut a := tensor.Arange(4)
    mut b := tensor.Full(tensor.Shape1(4), 2.0)
    mut sum := a + b
    show("+", sum)
    mut mixed := a * b - b / b
    show("*-/", mixed)
    mut added := tensor.Add(a, b)
    mut subtracted := tensor.Sub(a, b)
    mut multiplied := tensor.Mul(a, b)
    mut divided := tensor.Div(a, b)
    show("Add", added)
    show("Sub", subtracted)
    show("Mul", multiplied)
    show("Div", divided)
    mut out := tensor.Zeros1(4)
    println tensor.AddInto(a, b, out), tensor.SubInto(a, b, out), tensor.MulInto(a, b, out), tensor.DivInto(a, b, out)
    show("DivInto", out)
    show("+1", tensor.AddScalar(a, 1.0))
    show("-1", tensor.SubScalar(a, 1.0))
    show("x3", tensor.Scale(a, 3.0))
    show("/2", tensor.DivScalar(a, 2.0))
    show("Neg", tensor.Neg(a))

    mut d := tensor.Clone(a)
    tensor.AddInPlace(d, b)
    tensor.SubInPlace(d, a)
    tensor.MulInPlace(d, b)
    tensor.ScaleInPlace(d, 0.5)
    tensor.AddScalarInPlace(d, 1.0)
    tensor.AddScaled(d, a, 2.0)
    show("in place", d)

    show_rounded("Exp", tensor.Clip(tensor.Exp(a), 0.0, 10.0))
    show_rounded("Log", tensor.Log(b))
    show("Tanh", tensor.Tanh(tensor.Zeros1(2)))
    show_rounded("Sigmoid", tensor.Sigmoid(tensor.Zeros1(2)))
    show("Relu", tensor.Relu(tensor.Neg(a)))
    show_rounded("Gelu", tensor.Gelu(tensor.Zeros1(2)))
    show_rounded("Silu", tensor.Silu(tensor.Zeros1(2)))
    show("Sqrt", tensor.Sqrt(tensor.Mul(b, b)))
    show("Abs", tensor.Abs(tensor.Neg(a)))
    show("Square", tensor.Square(a))
    show("PowScalar", tensor.PowScalar(a, 3.0))

    println tensor.SumAll(a), tensor.MeanAll(a), tensor.MaxAll(a), tensor.MinAll(a), tensor.ArgMaxAll(a)
    mut m := tensor.Reshape(tensor.Arange(6), tensor.Shape2(2, 3))
    show("SumAxis", tensor.SumAxis(m, 0, false))
    show("MeanAxis", tensor.MeanAxis(m, 1, true))
    show("MaxAxis", tensor.MaxAxis(m, 1, false))
    show("ArgMaxAxis", tensor.ArgMaxAxis(m, 1, false))
    show("Cumsum", tensor.Cumsum(m, 1))
    println tensor.SumAll(tensor.Softmax(m)) > 1.99, tensor.MaxAll(tensor.LogSoftmax(a)) < 0.0

    mut t := tensor.Transpose2D(m)
    show("Transpose2D", t)
    show("MatMul", tensor.MatMul(m, t))
    show("TransposeMul", tensor.TransposeMul(m, m))
    show("MulTranspose", tensor.MulTranspose(m, m))
    mut product := tensor.Zeros2(2, 2)
    println tensor.MatMulInto(m, t, product), tensor.MatMulAcc(m, t, product)
    show("MatMulAcc", product)
    show("MatVec", tensor.MatVec(m, tensor.Ones(tensor.Shape1(3))))
    mut batch := tensor.Stack2(m, m)
    mut batch_t := tensor.Stack2(t, t)
    println tensor.ShapeString(tensor.BatchMatMul(batch, batch_t)), tensor.Dot(a, b)
    println tensor.MaxAbsDiff(a, added), tensor.AllClose(added, sum, 0.001)
end

Output+ tensor(4)[2 3 4 5] *-/ tensor(4)[-1 1 3 5] Add tensor(4)[2 3 4 5] Sub tensor(4)[-2 -1 0 1] Mul tensor(4)[0 2 4 6] Div tensor(4)[0 0.5 1 1.5] true true true true DivInto tensor(4)[0 0.5 1 1.5] +1 tensor(4)[1 2 3 4] -1 tensor(4)[-1 0 1 2] x3 tensor(4)[0 3 6 9] /2 tensor(4)[0 0.5 1 1.5] Neg tensor(4)[0 -1 -2 -3] in place tensor(4)[3 5 7 9] Exp x1000: 1000 2718 7389 10000 Log x1000: 693 693 693 693 Tanh tensor(2)[0 0] Sigmoid x1000: 500 500 Relu tensor(4)[0 0 0 0] Gelu x1000: 0 0 Silu x1000: 0 0 Sqrt tensor(4)[2 2 2 2] Abs tensor(4)[0 1 2 3] Square tensor(4)[0 1 4 9] PowScalar tensor(4)[0 1 8 27] 6 1.5 3 0 3 SumAxis tensor(3)[3 5 7] MeanAxis tensor(2x1)[1 4] MaxAxis tensor(2)[2 5] ArgMaxAxis tensor(2)[2 2] Cumsum tensor(2x3)[0 1 3 3 7 12] true true Transpose2D tensor(3x2)[0 3 1 4 2 5] MatMul tensor(2x2)[5 14 14 50] TransposeMul tensor(3x3)[9 12 15 12 17 22 15 22 29] MulTranspose tensor(2x2)[5 14 14 50] true true MatMulAcc tensor(2x2)[10 28 28 100] MatVec tensor(2)[3 12] 2x2x2 12 2 true

`MatMul`

func MatMul(x &: Tensor, y &: Tensor): Tensor

Matrix multiplication.

The example for this item is shown with the ones above: tensor-Math.salam

`MatMulInto`

func MatMulInto(x &: Tensor, y &: Tensor, out &: Tensor): bool

Matrix multiplication into a ready tensor; with no allocation.

The example for this item is shown with the ones above: tensor-Math.salam

`MatMulAcc`

func MatMulAcc(x &: Tensor, y &: Tensor, out &: Tensor): bool

Adds the product to the output.

The example for this item is shown with the ones above: tensor-Math.salam

`TransposeMul`

func TransposeMul(x &: Tensor, y &: Tensor): Tensor

The transpose of x times y, without building the transpose.

The example for this item is shown with the ones above: tensor-Math.salam

`MulTranspose`

func MulTranspose(x &: Tensor, y &: Tensor): Tensor

x times the transpose of y.

The example for this item is shown with the ones above: tensor-Math.salam

`BatchMatMul`

func BatchMatMul(x &: Tensor, y &: Tensor): Tensor

Matrix multiplication for every member of a batch.

The example for this item is shown with the ones above: tensor-Math.salam

`MatVec`

func MatVec(a &: Tensor, x &: Tensor): Tensor

A matrix times a vector.

The example for this item is shown with the ones above: tensor-Math.salam

`TransposeRaw`

func TransposeRaw(px: f32*, n: int, m: int): Tensor

The transpose from a raw pointer to the data.

The example for this item is shown with the ones above: tensor-Random.salam

`GemmAcc`

func GemmAcc(px: f32*, py: f32*, pr: f32*, n: int, k: int, m: int)

Accumulating matrix multiplication over raw pointers; the fast kernel the other multiplications are built on.

The example for this item is shown with the ones above: tensor-Random.salam

`Dot`

func Dot(x &: Tensor, y &: Tensor): f64

The dot product of two vectors.

The example for this item is shown with the ones above: tensor-Math.salam

`SetThreads`

func SetThreads(n: int)

The number of threads for matrix multiplication and convolution.

The example for this item is shown with the ones above: tensor-Random.salam

`Threads`

func Threads(): int

The current number of threads.

The example for this item is shown with the ones above: tensor-Random.salam

`Add`

func Add(x &: Tensor, y &: Tensor): Tensor

Addition with broadcasting.

The example for this item is shown with the ones above: tensor-Math.salam

`Sub`

func Sub(x &: Tensor, y &: Tensor): Tensor

Subtraction with broadcasting.

The example for this item is shown with the ones above: tensor-Math.salam

`Mul`

func Mul(x &: Tensor, y &: Tensor): Tensor

Elementwise multiplication with broadcasting.

The example for this item is shown with the ones above: tensor-Math.salam

`Div`

func Div(x &: Tensor, y &: Tensor): Tensor

Elementwise division with broadcasting.

The example for this item is shown with the ones above: tensor-Math.salam

`AddInto`

func AddInto(x &: Tensor, y &: Tensor, out &: Tensor): bool

Addition into a ready tensor.

The example for this item is shown with the ones above: tensor-Math.salam

`SubInto`

func SubInto(x &: Tensor, y &: Tensor, out &: Tensor): bool

Subtraction into a ready tensor.

The example for this item is shown with the ones above: tensor-Math.salam

`MulInto`

func MulInto(x &: Tensor, y &: Tensor, out &: Tensor): bool

Elementwise multiplication into a ready tensor.

The example for this item is shown with the ones above: tensor-Math.salam

`DivInto`

func DivInto(x &: Tensor, y &: Tensor, out &: Tensor): bool

Elementwise division into a ready tensor.

The example for this item is shown with the ones above: tensor-Math.salam

`AddScalar`

func AddScalar(x &: Tensor, s: f64): Tensor

Adds a number to every element.

The example for this item is shown with the ones above: tensor-Math.salam

`SubScalar`

func SubScalar(x &: Tensor, s: f64): Tensor

Subtracts a number from every element.

The example for this item is shown with the ones above: tensor-Math.salam

`Scale`

func Scale(x &: Tensor, s: f64): Tensor

Multiplies every element by a number.

The example for this item is shown with the ones above: tensor-Math.salam

`DivScalar`

func DivScalar(x &: Tensor, s: f64): Tensor

Divides every element by a number.

The example for this item is shown with the ones above: tensor-Math.salam

`Neg`

func Neg(x &: Tensor): Tensor

The negation.

The example for this item is shown with the ones above: tensor-Math.salam

`AddInPlace`

func AddInPlace(dst &: Tensor, src &: Tensor)

Adds in place.

The example for this item is shown with the ones above: tensor-Math.salam

`SubInPlace`

func SubInPlace(dst &: Tensor, src &: Tensor)

Subtracts in place.

The example for this item is shown with the ones above: tensor-Math.salam

`MulInPlace`

func MulInPlace(dst &: Tensor, src &: Tensor)

Multiplies elementwise in place.

The example for this item is shown with the ones above: tensor-Math.salam

`ScaleInPlace`

func ScaleInPlace(dst &: Tensor, s: f64)

Multiplies by a number in place.

The example for this item is shown with the ones above: tensor-Math.salam

`AddScalarInPlace`

func AddScalarInPlace(dst &: Tensor, s: f64)

Adds a number in place.

The example for this item is shown with the ones above: tensor-Math.salam

`AddScaled`

func AddScaled(dst &: Tensor, src &: Tensor, alpha: f64)

Adds alpha times the source to the destination.

The example for this item is shown with the ones above: tensor-Math.salam

`Exp`

func Exp(x &: Tensor): Tensor

The exponential, element by element.

The example for this item is shown with the ones above: tensor-Math.salam

`Log`

func Log(x &: Tensor): Tensor

The natural logarithm, element by element.

The example for this item is shown with the ones above: tensor-Math.salam

`Tanh`

func Tanh(x &: Tensor): Tensor

The hyperbolic tangent, element by element.

The example for this item is shown with the ones above: tensor-Math.salam

`Sigmoid`

func Sigmoid(x &: Tensor): Tensor

The sigmoid, element by element.

The example for this item is shown with the ones above: tensor-Math.salam

`Relu`

func Relu(x &: Tensor): Tensor

The rectifier, the greater of zero and x, element by element.

The example for this item is shown with the ones above: tensor-Math.salam

`Gelu`

func Gelu(x &: Tensor): Tensor

GELU, element by element.

The example for this item is shown with the ones above: tensor-Math.salam

`Sqrt`

func Sqrt(x &: Tensor): Tensor

The square root, element by element.

The example for this item is shown with the ones above: tensor-Math.salam

`Abs`

func Abs(x &: Tensor): Tensor

The absolute value, element by element.

The example for this item is shown with the ones above: tensor-Math.salam

`Square`

func Square(x &: Tensor): Tensor

The square, element by element.

The example for this item is shown with the ones above: tensor-Math.salam

`Silu`

func Silu(x &: Tensor): Tensor

SiLU, meaning x times the sigmoid of x, element by element.

The example for this item is shown with the ones above: tensor-Math.salam

`PowScalar`

func PowScalar(x &: Tensor, p: f64): Tensor

Raises the elements to a power.

The example for this item is shown with the ones above: tensor-Math.salam

`Clip`

func Clip(x &: Tensor, lo: f64, hi: f64): Tensor

Holds the elements within a range.

The example for this item is shown with the ones above: tensor-Create.salam

`BroadcastShape`

func BroadcastShape(a &: Tensor, b &: Tensor, out &: Vector<int>): bool

The shape two tensors broadcast to; false when they do not fit together.

tensor-Index.salam

import tensor

func show(name: str, t &: tensor.Tensor):
    println name, tensor.ToString(t)
end

func main:
    mut a := tensor.Arange(4)
    mut b := tensor.Full(tensor.Shape1(4), 1.5)
    show("Greater", tensor.Greater(a, b))
    show("GreaterEqual", tensor.GreaterEqual(a, tensor.Clone(a)))
    show("Less", tensor.Less(a, b))
    show("LessEqual", tensor.LessEqual(a, b))
    show("EqualMask", tensor.EqualMask(a, a))
    show("NotEqualMask", tensor.NotEqualMask(a, b))
    show("GreaterScalar", tensor.GreaterScalar(a, 1.0))
    show("LessScalar", tensor.LessScalar(a, 1.0))
    show("EqualScalar", tensor.EqualScalar(a, 2.0))
    mut mask := tensor.Greater(a, b)
    show("Where", tensor.Where(mask, a, tensor.Neg(a)))
    show("MaskedFill", tensor.MaskedFill(a, mask, -1.0))

    mut m := tensor.Reshape(tensor.Arange(6), tensor.Shape2(3, 2))
    mut indices := Vector {} as Vector<int>
    indices.push(2)
    indices.push(0)
    show("IndexSelect", tensor.IndexSelect(m, 0, indices))
    show("TakeRows", tensor.TakeRows(m, indices))
    mut places := tensor.Zeros2(3, 1)
    places.set_at(0, 1.0)
    show("Gather", tensor.Gather(m, 1, places))
    mut target := tensor.Zeros2(3, 2)
    println tensor.ScatterAddRows(target, indices, tensor.Ones(tensor.Shape2(2, 2)))
    show("ScatterAddRows", target)
    show("OneHot", tensor.OneHot(indices, 3))

    mut scores := tensor.Reshape(tensor.Arange(5), tensor.Shape1(5))
    mut top_places := tensor.Empty()
    show("TopK", tensor.TopK(scores, 2, true, top_places))
    show("TopK places", top_places)
    mut square := tensor.Ones(tensor.Shape2(3, 3))
    show("Tril", tensor.Tril(square, 0))
    show("Triu", tensor.Triu(square, 1))
    mut broadcast := Vector {} as Vector<int>
    println tensor.BroadcastShape(m, tensor.Ones(tensor.Shape1(2)), broadcast), broadcast.len(), broadcast.get(0)
    show("Pad2D", tensor.Pad2D(tensor.Ones(tensor.Shape4(1, 1, 1, 1)), 1, 0, 0, 1))
end

OutputGreater tensor(4)[0 0 1 1] GreaterEqual tensor(4)[1 1 1 1] Less tensor(4)[1 1 0 0] LessEqual tensor(4)[1 1 0 0] EqualMask tensor(4)[1 1 1 1] NotEqualMask tensor(4)[1 1 1 1] GreaterScalar tensor(4)[0 0 1 1] LessScalar tensor(4)[1 0 0 0] EqualScalar tensor(4)[0 0 1 0] Where tensor(4)[0 -1 2 3] MaskedFill tensor(4)[0 1 -1 -1] IndexSelect tensor(2x2)[4 5 0 1] TakeRows tensor(2x2)[4 5 0 1] Gather tensor(3x1)[1 2 4] true ScatterAddRows tensor(3x2)[1 1 0 0 1 1] OneHot tensor(2x3)[0 0 1 1 0 0] TopK tensor(2)[4 3] TopK places tensor(2)[4 3] Tril tensor(3x3)[1 0 0 1 1 0 1 1 1] Triu tensor(3x3)[0 1 1 0 0 1 0 0 0] true 2 3 Pad2D tensor(1x1x2x2)[0 0 1 0]

`Expand`

func Expand(x &: Tensor, shape &: Vector<int>): Tensor

Expands the tensor to a larger shape by broadcasting.

The example for this item is shown with the ones above: tensor-Create.salam

`Greater`

func Greater(a &: Tensor, b &: Tensor): Tensor

A mask of ones and zeros for a greater than b.

The example for this item is shown with the ones above: tensor-Index.salam

`GreaterEqual`

func GreaterEqual(a &: Tensor, b &: Tensor): Tensor

A mask for greater or equal.

The example for this item is shown with the ones above: tensor-Index.salam

`Less`

func Less(a &: Tensor, b &: Tensor): Tensor

A mask for less than.

The example for this item is shown with the ones above: tensor-Index.salam

`LessEqual`

func LessEqual(a &: Tensor, b &: Tensor): Tensor

A mask for less or equal.

The example for this item is shown with the ones above: tensor-Index.salam

`EqualMask`

func EqualMask(a &: Tensor, b &: Tensor): Tensor

A mask for equality.

The example for this item is shown with the ones above: tensor-Index.salam

`NotEqualMask`

func NotEqualMask(a &: Tensor, b &: Tensor): Tensor

A mask for inequality.

The example for this item is shown with the ones above: tensor-Index.salam

`GreaterScalar`

func GreaterScalar(x &: Tensor, s: f64): Tensor

A mask for the elements greater than a number.

The example for this item is shown with the ones above: tensor-Index.salam

`LessScalar`

func LessScalar(x &: Tensor, s: f64): Tensor

A mask for the elements less than a number.

The example for this item is shown with the ones above: tensor-Index.salam

`EqualScalar`

func EqualScalar(x &: Tensor, s: f64): Tensor

A mask for the elements equal to a number.

The example for this item is shown with the ones above: tensor-Index.salam

`Where`

func Where(cond &: Tensor, a &: Tensor, b &: Tensor): Tensor

Where the condition is one take from a, otherwise from b.

The example for this item is shown with the ones above: tensor-Index.salam

`MaskedFill`

func MaskedFill(x &: Tensor, mask &: Tensor, value: f64): Tensor

Fills the masked places with a value; negative infinity in an attention mask, for instance.

The example for this item is shown with the ones above: tensor-Index.salam

`IndexSelect`

func IndexSelect(x &: Tensor, axis: int, ids &: Vector<int>): Tensor

Parts of one dimension, by index.

The example for this item is shown with the ones above: tensor-Index.salam

`TakeRows`

func TakeRows(x &: Tensor, ids &: Vector<int>): Tensor

Rows by index; as an embedding table does.

The example for this item is shown with the ones above: tensor-Index.salam

`Gather`

func Gather(x &: Tensor, axis: int, idx &: Tensor): Tensor

Along one dimension, takes the elements named by a tensor of indices.

The example for this item is shown with the ones above: tensor-Index.salam

`ScatterAddRows`

func ScatterAddRows(dst &: Tensor, ids &: Vector<int>, src &: Tensor): bool

Adds the rows of the source to the indexed rows of the destination.

The example for this item is shown with the ones above: tensor-Index.salam

`OneHot`

func OneHot(ids &: Vector<int>, classes: int): Tensor

One hot vectors for a set of indices.

The example for this item is shown with the ones above: tensor-Index.salam

`TopK`

func TopK(x &: Tensor, k: int, largest: bool, idx_out &: Tensor): Tensor

The k greatest or smallest values and where they are.

The example for this item is shown with the ones above: tensor-Index.salam

`Tril`

func Tril(x &: Tensor, diagonal: int): Tensor

The lower triangle; the rest zero.

The example for this item is shown with the ones above: tensor-Index.salam

`Triu`

func Triu(x &: Tensor, diagonal: int): Tensor

The upper triangle.

The example for this item is shown with the ones above: tensor-Index.salam

`Cumsum`

func Cumsum(x &: Tensor, axis: int): Tensor

The cumulative sum along one dimension.

The example for this item is shown with the ones above: tensor-Math.salam

`ConvOutSize`

func ConvOutSize(size: int, k: int, s: int, p: int): int

The convolution output size for a size, kernel, stride and padding.

The example for this item is shown with the ones above: tensor-Random.salam

`Im2Col`

func Im2Col(x &: Tensor, sample: int, kh: int, kw: int, sh: int, sw: int, ph: int, pw: int): Tensor

Turns the convolution windows of one sample into columns, so the convolution can be done as a matrix multiplication.

The example for this item is shown with the ones above: tensor-Random.salam

`Col2ImAcc`

func Col2ImAcc(cols &: Tensor, dx &: Tensor, sample: int, kh: int, kw: int, sh: int, sw: int, ph: int, pw: int): bool

The reverse, accumulating; for the convolution gradient.

The example for this item is shown with the ones above: tensor-Random.salam

`Conv2DForward`

func Conv2DForward(x &: Tensor, w &: Tensor, bias &: Tensor, sh: int, sw: int, ph: int, pw: int): Tensor

A two dimensional convolution over an NCHW tensor with weights, bias, strides and padding.

The example for this item is shown with the ones above: tensor-Random.salam

`MaxPool2DForward`

func MaxPool2DForward(x &: Tensor, kh: int, kw: int, sh: int, sw: int, argmax_out &: Tensor): Tensor

Two dimensional max pooling; the places of the maxima come back too.

The example for this item is shown with the ones above: tensor-Random.salam

`AvgPool2DForward`

func AvgPool2DForward(x &: Tensor, kh: int, kw: int, sh: int, sw: int): Tensor

Two dimensional average pooling.

The example for this item is shown with the ones above: tensor-Random.salam

`SumAll`

func SumAll(x &: Tensor): f64

The sum of everything.

The example for this item is shown with the ones above: tensor-Math.salam

`MeanAll`

func MeanAll(x &: Tensor): f64

The mean of everything.

The example for this item is shown with the ones above: tensor-Math.salam

`MaxAll`

func MaxAll(x &: Tensor): f64

The greatest of everything.

The example for this item is shown with the ones above: tensor-Math.salam

`MinAll`

func MinAll(x &: Tensor): f64

The smallest of everything.

The example for this item is shown with the ones above: tensor-Math.salam

`ArgMaxAll`

func ArgMaxAll(x &: Tensor): int

Where the maximum is.

The example for this item is shown with the ones above: tensor-Math.salam

`SumAxis`

func SumAxis(x &: Tensor, axis: int, keepdim: bool): Tensor

The sum along one dimension; with keepdim that dimension stays, with size one.

The example for this item is shown with the ones above: tensor-Math.salam

`MeanAxis`

func MeanAxis(x &: Tensor, axis: int, keepdim: bool): Tensor

The mean along one dimension.

The example for this item is shown with the ones above: tensor-Math.salam

`MaxAxis`

func MaxAxis(x &: Tensor, axis: int, keepdim: bool): Tensor

The maximum along one dimension.

The example for this item is shown with the ones above: tensor-Math.salam

`ArgMaxAxis`

func ArgMaxAxis(x &: Tensor, axis: int, keepdim: bool): Tensor

Where the maximum is along one dimension; for taking the predicted class.

The example for this item is shown with the ones above: tensor-Math.salam

`Softmax`

func Softmax(x &: Tensor): Tensor

The softmax over the last dimension; each row of the output is a set of probabilities.

The example for this item is shown with the ones above: tensor-Math.salam

`LogSoftmax`

func LogSoftmax(x &: Tensor): Tensor

The log of the softmax, numerically stable.

The example for this item is shown with the ones above: tensor-Math.salam

`Reshape`

func Reshape(x &: Tensor, shape &: Vector<int>): Tensor

A copy with a new shape.

The example for this item is shown with the ones above: tensor-Create.salam

`SqueezeInPlace`

func SqueezeInPlace(x &: Tensor)

Removes the dimensions of size one in place.

The example for this item is shown with the ones above: tensor-Create.salam

`UnsqueezeInPlace`

func UnsqueezeInPlace(x &: Tensor, axis: int)

Adds a dimension of size one at the given place.

The example for this item is shown with the ones above: tensor-Create.salam

`Permute`

func Permute(x &: Tensor, perm &: Vector<int>): Tensor

Changes the order of the dimensions.

The example for this item is shown with the ones above: tensor-Create.salam

`SliceAxis0`

func SliceAxis0(x &: Tensor, start: int, len: int): Tensor

A range of the first dimension.

The example for this item is shown with the ones above: tensor-Create.salam

`Narrow`

func Narrow(x &: Tensor, axis: int, start: int, len: int): Tensor

A range of one dimension.

The example for this item is shown with the ones above: tensor-Create.salam

`Concat2`

func Concat2(a &: Tensor, b &: Tensor, axis: int): Tensor

Puts two tensors one after the other along one dimension.

The example for this item is shown with the ones above: tensor-Create.salam

`Stack2`

func Stack2(a &: Tensor, b &: Tensor): Tensor

Stacks two tensors of the same shape along a new first dimension.

The example for this item is shown with the ones above: tensor-Create.salam

`Pad2D`

func Pad2D(x &: Tensor, pt: int, pb: int, pl: int, pr: int): Tensor

Pads the last two dimensions with zeros: top, bottom, left and right.

The example for this item is shown with the ones above: tensor-Index.salam

`ShapeString`

func ShapeString(x &: Tensor): str

The shape as text, such as 2x3.

The example for this item is shown with the ones above: tensor-Create.salam

`ToString`

func ToString(x &: Tensor): str

The tensor as text.

The example for this item is shown with the ones above: tensor-Create.salam

`Print`

func Print(x &: Tensor)

Prints the tensor.

The example for this item is shown with the ones above: tensor-Create.salam

`MaxAbsDiff`

func MaxAbsDiff(x &: Tensor, y &: Tensor): f64

The greatest difference between the elements.

The example for this item is shown with the ones above: tensor-Math.salam

`AllClose`

func AllClose(x &: Tensor, y &: Tensor, tol: f64): bool

Every element is equal within a tolerance.

The example for this item is shown with the ones above: tensor-Math.salam

`Equal`

func Equal(x &: Tensor, y &: Tensor): bool

The shapes and the elements match.

The example for this item is shown with the ones above: tensor-Create.salam

`Empty`

func Empty(): Tensor

An empty tensor; for outputs a function fills in.

The example for this item is shown with the ones above: tensor-Create.salam

`Shape1`

func Shape1(n: int): Vector<int>

A one dimensional shape.

The example for this item is shown with the ones above: tensor-Create.salam

`Shape2`

func Shape2(a: int, b: int): Vector<int>

A two dimensional shape.

The example for this item is shown with the ones above: tensor-Create.salam

`Shape3`

func Shape3(a: int, b: int, c: int): Vector<int>

A three dimensional shape.

The example for this item is shown with the ones above: tensor-Create.salam

`Shape4`

func Shape4(a: int, b: int, c: int, d: int): Vector<int>

A four dimensional shape, such as batch by channel by height by width.

The example for this item is shown with the ones above: tensor-Create.salam

`Zeros`

func Zeros(shape &: Vector<int>): Tensor

A tensor of zeros with a shape.

The example for this item is shown with the ones above: tensor-Create.salam

`Zeros1`

func Zeros1(n: int): Tensor

A vector of zeros.

The example for this item is shown with the ones above: tensor-Create.salam

`Zeros2`

func Zeros2(r: int, c: int): Tensor

A matrix of zeros.

The example for this item is shown with the ones above: tensor-Create.salam

`Zeros3`

func Zeros3(a: int, b: int, c: int): Tensor

A three dimensional tensor of zeros.

The example for this item is shown with the ones above: tensor-Create.salam

`Zeros4`

func Zeros4(a: int, b: int, c: int, d: int): Tensor

A four dimensional tensor of zeros.

The example for this item is shown with the ones above: tensor-Create.salam

`Full`

func Full(shape &: Vector<int>, x: f64): Tensor

A tensor filled with one value.

The example for this item is shown with the ones above: tensor-Create.salam

`Ones`

func Ones(shape &: Vector<int>): Tensor

A tensor of ones.

The example for this item is shown with the ones above: tensor-Create.salam

`Scalar`

func Scalar(x: f64): Tensor

A one element tensor.

The example for this item is shown with the ones above: tensor-Create.salam

`FromArray`

func FromArray(shape &: Vector<int>, data &: Vector<f32>): Tensor

A tensor from a shape and 32 bit float data.

The example for this item is shown with the ones above: tensor-Create.salam

`Arange`

func Arange(n: int): Tensor

The vector 0 to n minus one.

The example for this item is shown with the ones above: tensor-Create.salam

`Clone`

func Clone(x &: Tensor): Tensor

An independent copy.

The example for this item is shown with the ones above: tensor-Create.salam

`CopyInto`

func CopyInto(dst &: Tensor, src &: Tensor)

Copies the data into a tensor of the same shape.

The example for this item is shown with the ones above: tensor-Create.salam

`Free`

func Free(x &: Tensor)

Frees the tensor.

The example for this item is shown with the ones above: tensor-Create.salam

`SetAll`

func SetAll(x &: Tensor, v: f64)

Sets every element to one value.

The example for this item is shown with the ones above: tensor-Create.salam

`SameShape`

func SameShape(x &: Tensor, y &: Tensor): bool

The two tensors have the same shape.

The example for this item is shown with the ones above: tensor-Create.salam

`ToMatrix`

func ToMatrix(x &: Tensor): matrix.Matrix

A two dimensional tensor as a matrix.

The example for this item is shown with the ones above: tensor-Create.salam

`FromMatrix`

func FromMatrix(m &: matrix.Matrix): Tensor

A matrix as a tensor.

The example for this item is shown with the ones above: tensor-Create.salam

`WriteTensor`

func WriteTensor(buf &: compress.ByteBuf, name: str, t &: Tensor): bool

Writes the tensor with its name into a buffer.

The example for this item is shown with the ones above: tensor-Create.salam

`ReadTensor`

func ReadTensor(rd &: compress.ByteReader, name_out &: str, out &: Tensor): bool

Reads a tensor and its name from a reader.

The example for this item is shown with the ones above: tensor-Create.salam

`NewRng`

func NewRng(seed: i64): Rng

A generator from a seed.

The example for this item is shown with the ones above: tensor-Random.salam

`DeriveSeed`

func DeriveSeed(seed: i64, index: i64): i64

An independent seed for job number i, from a main seed.

The example for this item is shown with the ones above: tensor-Random.salam

`FillUniform`

func FillUniform(t &: Tensor, lo: f64, hi: f64, g &: Rng)

Fills with the uniform distribution.

The example for this item is shown with the ones above: tensor-Random.salam

`FillNormal`

func FillNormal(t &: Tensor, mean: f64, std: f64, g &: Rng)

Fills with the normal distribution.

The example for this item is shown with the ones above: tensor-Random.salam

`FillBernoulli`

func FillBernoulli(t &: Tensor, p: f64, g &: Rng)

One with probability p, zero otherwise.

The example for this item is shown with the ones above: tensor-Random.salam

`RandUniform`

func RandUniform(shape &: Vector<int>, lo: f64, hi: f64, seed: i64): Tensor

A uniform random tensor from a seed.

The example for this item is shown with the ones above: tensor-Random.salam

`RandNormal`

func RandNormal(shape &: Vector<int>, seed: i64): Tensor

A normal random tensor from a seed.

The example for this item is shown with the ones above: tensor-Random.salam