Standard library · AI

ai.tensor

Multi dimensional arrays and numeric operations for machine learning.

import ai.tensor

On this page

A linear layer by hand
Normalizing features
Finding the nearest vectors
Reshaping, slicing and masks
Things to watch for
Types
Constants and variables
Functions

A tensor is a multi dimensional array of numbers: a vector is one dimensional, a matrix two, and a batch of images four. The data is 32 bit floats in row order. The +, -, * and / operators work element by element and take tensors of the same shape, or shapes that broadcast.

This package is arithmetic, with no gradients. To train a network, use autograd and nn, which work on these tensors. For exact 64 bit linear algebra (solving systems, inverses, decompositions) use matrix; for statistics over plain lists, stats.

Large operations run on several threads by themselves: element-wise work over 262144 values or more, and matrix products above about 32 million multiply-adds. The default is one thread per CPU core, at most 8; SetThreads changes it, and SetThreads(1) keeps everything on the calling thread.

A linear layer by hand

Most of a neural network is one line: multiply the inputs by a weight matrix and add a bias. Here three samples with two features each, a 3x2 matrix, meet a 2x3 weight matrix and give a score for each of three classes. The bias has shape 1x3 and is added to every row by broadcasting: a dimension of size one stretches to match the other operand.

tensor-guide-linear.salam

import ai.tensor

func matrix(rows: int, cols: int, values: f64[]): tensor.Tensor:
    mut data := Vector {} as Vector<f32>
    each v in values:
        data.push(v as f32)
    end
    mut shape := tensor.Shape2(rows, cols)
    ret tensor.FromArray(shape, data)
end

func main:
    mut x := matrix(3, 2, [1.0, 2.0, 0.0, 1.0, 3.0, 0.0])
    mut w := matrix(2, 3, [1.0, 0.0, -1.0, 0.5, 1.0, 2.0])
    b := matrix(1, 3, [0.0, -1.0, 0.5])
    mut scores := tensor.MatMul(x, w) + b
    println "scores", tensor.ShapeString(scores), tensor.ToString(scores)
    mut hidden := tensor.Relu(scores)
    println "relu", tensor.ToString(hidden)
    mut best := tensor.ArgMaxAxis(scores, 1, false)
    println "best class per row", tensor.ToString(best)
end

Outputscores 3x3 tensor(3x3)[2 1 3.5 0.5 0 2.5 3 -1 -2.5] relu tensor(3x3)[2 1 3.5 0.5 0 2.5 3 0 0] best class per row tensor(3)[2 2 0]

ArgMaxAxis with axis 1 picks, in each row, the column with the largest score, which is the predicted class. Axis 0 runs down the rows and axis 1 along them, so a reduction over axis 1 leaves one number per sample.

Normalizing features

Features measured in different units train badly together, so they are usually shifted to mean zero and scaled to spread one. MeanAxis with keepdim set to true keeps the reduced axis with size one, giving a 1x2 row that broadcasts back over all eight samples. Without keepdim the result is a plain vector of two, as the last line shows.

tensor-guide-normalize.salam

import ai.tensor

func main:
    mut data := Vector {} as Vector<f32>
    each v in [2.0, 20.0, 4.0, 40.0, 4.0, 40.0, 4.0, 40.0, 5.0, 50.0, 5.0, 50.0, 7.0, 70.0, 9.0, 90.0]:
        data.push(v as f32)
    end
    mut shape := tensor.Shape2(8, 2)
    mut samples := tensor.FromArray(shape, data)
    mut mean := tensor.MeanAxis(samples, 0, true)
    println "mean", tensor.ShapeString(mean), tensor.ToString(mean)
    mut centered := samples - mean
    mut spread := tensor.Sqrt(tensor.MeanAxis(tensor.Square(centered), 0, true))
    println "spread", tensor.ToString(spread)
    mut scaled := centered / spread
    println "first column", tensor.ToString(tensor.Narrow(scaled, 1, 0, 1))
    println "columns now match:", tensor.AllClose(tensor.Narrow(scaled, 1, 0, 1), tensor.Narrow(scaled, 1, 1, 1), 0.0001)
    println "new mean", tensor.ToString(tensor.MeanAxis(scaled, 0, false))
end

Outputmean 1x2 tensor(1x2)[5 50] spread tensor(1x2)[2 20] first column tensor(8x1)[-1.5 -0.5 -0.5 -0.5 0 0 1 2] columns now match: true new mean tensor(2)[0 0]

The second column is the first one times ten, and after scaling the two are equal: normalization removes the unit. AllClose compares within a tolerance, which is the right test for floats; Equal demands every bit to match.

Finding the nearest vectors

Search by meaning works on vectors: each item is a row of numbers, and the item closest to a query is the one with the largest dot product. MatVec scores every row against the query in one call and TopK returns the best scores, writing their row numbers into the tensor you pass as the last argument.

tensor-guide-search.salam

import ai.tensor

func main:
    names := ["cat", "dog", "car", "bus"]
    mut data := Vector {} as Vector<f32>
    each v in [0.9, 0.1, 0.0, 0.8, 0.2, 0.0, 0.0, 0.1, 0.9, 0.1, 0.0, 0.8]:
        data.push(v as f32)
    end
    mut shape := tensor.Shape2(4, 3)
    mut vectors := tensor.FromArray(shape, data)
    mut query_data := Vector {} as Vector<f32>
    each v in [0.7, 0.3, 0.0]:
        query_data.push(v as f32)
    end
    mut qshape := tensor.Shape1(3)
    mut query := tensor.FromArray(qshape, query_data)
    mut scores := tensor.MatVec(vectors, query)
    mut order := tensor.Empty()
    top := tensor.TopK(scores, 2, true, order)
    repeat 2 in i:
        index := order.at(i) as int
        println names[index], (top.at(i) * 100.0 + 0.5) as int
    end
end

Outputcat 66 dog 62

The rows here are already of similar length, so the dot product is a fair measure. For real embeddings, divide each row by its length first, which turns the dot product into cosine similarity. Scores are 32 bit floats, so the program rounds them before printing.

Reshaping, slicing and masks

Data often arrives flat. Reshape returns a copy of the same numbers, in the same order, with a new shape: twelve meter readings become three days of four periods. Reductions then answer questions along either axis, SliceAxis0 and Narrow cut out rows or columns, and the comparison functions build masks of 0 and 1 that MaskedFill and Where use to pick values.

tensor-guide-shape.salam

import ai.tensor

func main:
    mut readings := Vector {} as Vector<f32>
    each kwh in [1.5, 3.5, 4.25, 2.0, 1.0, 2.75, 5.5, 2.25, 0.75, 3.0, 3.25, 1.75]:
        readings.push(kwh as f32)
    end
    mut flat := tensor.FromArray(tensor.Shape1(12), readings)
    mut days_shape := tensor.Shape2(3, 4)
    mut days := tensor.Reshape(flat, days_shape)
    println "days x periods", tensor.ShapeString(days)
    println "per day", tensor.ToString(tensor.SumAxis(days, 1, false))
    println "per period", tensor.ToString(tensor.SumAxis(days, 0, false))
    println "running total of day 2", tensor.ToString(tensor.Cumsum(tensor.SliceAxis0(days, 1, 1), 1))
    mut peaks := tensor.GreaterScalar(days, 4.0)
    println "peak readings", tensor.ToString(tensor.MaskedFill(days, tensor.LessScalar(days, 4.0), 0.0))
    println "how many peaks", tensor.SumAll(peaks)
    println "periods as rows", tensor.ShapeString(tensor.Transpose2D(days))
    mut wrong_shape := tensor.Shape2(5, 2)
    println "reshape 12 into 5x2:", tensor.ShapeString(tensor.Reshape(flat, wrong_shape))
end

Outputdays x periods 3x4 per day tensor(3)[11.25 11.5 8.75] per period tensor(4)[3.25 9.25 13 6] running total of day 2 tensor(1x4)[1 3.75 9.25 11.5] peak readings tensor(3x4)[0 0 4.25 0 0 0 5.5 0 0 0 0 0] how many peaks 2 periods as rows 4x3 reshape 12 into 5x2: empty

A mask is an ordinary tensor of 0 and 1, so SumAll on it counts the matches. The last line shows the failure mode from the list below: twelve values cannot be shaped into 5x2, and instead of an error you get an empty tensor.

Things to watch for

Shapes must match or broadcast. A mismatch does not stop the program: the result is an empty tensor, ShapeString prints empty, and the mistake travels on silently. Check is_empty() on results while you are unsure of the shapes.
The data is f32. Fill a Vector<f32> and cast with as f32 when you build a tensor from your own numbers.
Every operation makes a new tensor. In a long loop, use the ...Into and ...InPlace forms, or call Free on results you no longer need.
The ...InPlace and ...Into functions do not broadcast. When the shapes differ, ...InPlace does nothing at all and ...Into returns false. Use the operators when one side needs broadcasting.
Printing goes through f64, so a value such as 0.1 prints as 0.10000000149011612, the exact f32 value. Round with conv.FormatFloatPrec before showing numbers to people.

Types

`Tensor`

struct Tensor

A multi dimensional array of 32 bit floats; the base of every neural network computation.

Name	Type	Description
`data`	`Vector`	The elements in row order.
`shape`	`Vector`	The dimensions.

ai-tensor-Create.salam

import ai.tensor
import compress

func main throws:
    mut shape := tensor.Shape2(2, 3)
    mut a := tensor.Arange(6)
    mut b := tensor.Reshape(a, shape)
    println tensor.ShapeString(b), b.rank(), b.numel(), b.dim(0), b.dim(1), b.is_empty()
    println tensor.ToString(b)
    println b.at2(1, 2), b.at(4)
    b.set_at2(0, 0, 9.0)
    b.set_at(1, 8.0)
    b.add_at(2, 0.5)
    tensor.Print(b)

    mut z := tensor.Zeros(shape)
    println tensor.ShapeString(z), z.same_shape(b), tensor.SameShape(z, b)
    mut z1 := tensor.Zeros1(3)
    mut z2 := tensor.Zeros2(2, 2)
    mut z3 := tensor.Zeros3(2, 2, 2)
    mut z4 := tensor.Zeros4(1, 2, 2, 2)
    println z1.numel(), z2.numel(), z3.numel(), z4.rank()
    mut ones := tensor.Ones(tensor.Shape1(3))
    mut full := tensor.Full(tensor.Shape3(1, 1, 2), 7.0)
    mut scalar := tensor.Scalar(2.5)
    println tensor.ToString(ones), tensor.ToString(full), scalar.rank(), scalar.at(0)
    println tensor.Shape4(1, 2, 3, 4).len(), tensor.Empty().is_empty()

    mut payload := Vector {} as Vector<f32>
    payload.push(1.0 as f32)
    payload.push(2.0 as f32)
    mut from_array := tensor.FromArray(tensor.Shape1(2), payload)
    mut copy := tensor.Clone(from_array)
    tensor.SetAll(copy, 5.0)
    tensor.CopyInto(z1, ones)
    println tensor.ToString(from_array), tensor.ToString(copy), tensor.ToString(z1)

    mut shape2 := tensor.Shape3(3, 1, 2)
    println b.reshape_(shape2), tensor.ShapeString(b)
    tensor.SqueezeInPlace(b)
    println tensor.ShapeString(b)
    tensor.UnsqueezeInPlace(b, 0)
    println tensor.ShapeString(b)
    mut order := Vector {} as Vector<int>
    order.push(2)
    order.push(1)
    order.push(0)
    mut permuted := tensor.Permute(b, order)
    println tensor.ShapeString(permuted)
    mut m := tensor.Arange(6)
    mut m2 := tensor.Reshape(m, shape)
    println tensor.ToString(tensor.SliceAxis0(m2, 1, 1)), tensor.ToString(tensor.Narrow(m2, 1, 1, 2))
    println tensor.ShapeString(tensor.Concat2(m2, m2, 0)), tensor.ShapeString(tensor.Stack2(m2, m2))
    println tensor.ShapeString(tensor.Expand(tensor.Scalar(1.0), shape))

    mut as_matrix := tensor.ToMatrix(m2)
    println as_matrix.rows, as_matrix.cols, tensor.ShapeString(tensor.FromMatrix(as_matrix))

    mut buffer := compress.NewByteBuf()
    println tensor.WriteTensor(buffer, "w", m2)
    mut reader := compress.ReaderFromBuf(buffer)
    mut name := ""
    mut read := tensor.Empty()
    try tensor.ReadTensor(reader, name, read)
    println name, tensor.Equal(read, m2)
    compress.BufFree(buffer)
    println tensor.TSR_TOL
    tensor.Free(a)
    b.free()
end

Output2x3 2 6 2 3 false tensor(2x3)[0 1 2 3 4 5] 5 4 tensor(2x3)[9 8 2.5 3 4 5] 2x3 true true 3 4 8 4 tensor(3)[1 1 1] tensor(1x1x2)[7 7] 1 2.5 4 true tensor(2)[1 2] tensor(2)[5 5] tensor(3)[1 1 1] true 3x1x2 3x2 1x3x2 2x3x1 tensor(1x3)[3 4 5] tensor(2x2)[1 2 4 5] 4x3 2x2x3 2x3 2 3 2x3 true w true 1e-05

`rank`

func rank(): int

The number of dimensions.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`numel`

func numel(): int

The total number of elements.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`dim`

func dim(i: int): int

The size of dimension i.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`is_empty`

func is_empty(): bool

It has no elements.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`same_shape`

func same_shape(o &: Tensor): bool

Its shape matches another tensor's.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`at`

func at(i: int): f64

Element number i in row order.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`set_at`

func set_at(i: int, v: f64)

Sets element i.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`add_at`

func add_at(i: int, v: f64)

Adds to element i.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`at2`

func at2(i: int, j: int): f64

The element at row i and column j of a two dimensional tensor.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`set_at2`

func set_at2(i: int, j: int, v: f64)

Sets the element at row i and column j.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`reshape_`

func reshape_(shape &: Vector<int>): bool

Changes the shape in place, when the number of elements matches.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`operator_add`

func operator_add(other: Tensor): Tensor

The + operator, element by element with broadcasting.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`operator_sub`

func operator_sub(other: Tensor): Tensor

The - operator.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`operator_mul`

func operator_mul(other: Tensor): Tensor

The * operator, elementwise multiplication.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`operator_div`

func operator_div(other: Tensor): Tensor

The / operator.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`free`

func free()

Frees the memory.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`Rng`

struct Rng

A deterministic random number generator; one seed always gives the same sequence.

Name	Type	Description
`s`	`u64 = 0`	The internal state.

ai-tensor-Random.salam

import ai.tensor

func rounded(t &: tensor.Tensor): str:
    mut text := ""
    repeat t.numel() in i:
        if i > 0:
            text += " "
        end
        text += "" + ((t.at(i) * 100.0 + 10000.5) as int - 10000)
    end
    ret text
end

func main:
    mut rng := tensor.NewRng(42)
    println rng.next_bits() > 0, rng.next_float() < 1.0, rng.next_normal() < 10.0
    println tensor.DeriveSeed(42, 1) != tensor.DeriveSeed(42, 2)
    mut v := tensor.Zeros1(5)
    tensor.FillUniform(v, 0.0, 1.0, rng)
    println tensor.MinAll(v) >= 0.0, tensor.MaxAll(v) < 1.0
    tensor.FillNormal(v, 0.0, 1.0, rng)
    tensor.FillBernoulli(v, 0.5, rng)
    println tensor.SumAll(v) <= 5.0
    mut t1 := tensor.RandUniform(tensor.Shape1(3), -1.0, 1.0, 7)
    mut t2 := tensor.RandUniform(tensor.Shape1(3), -1.0, 1.0, 7)
    println tensor.Equal(t1, t2), rounded(t1)
    mut n := tensor.RandNormal(tensor.Shape1(2), 7)
    println n.numel()

    tensor.SetThreads(2)
    println tensor.Threads()

    println tensor.ConvOutSize(5, 3, 1, 1), tensor.ConvOutSize(5, 3, 2, 0)
    mut image := tensor.Reshape(tensor.Arange(16), tensor.Shape4(1, 1, 4, 4))
    mut kernel := tensor.Ones(tensor.Shape4(1, 1, 2, 2))
    mut bias := tensor.Zeros1(1)
    mut conv := tensor.Conv2DForward(image, kernel, bias, 2, 2, 0, 0)
    println tensor.ShapeString(conv), rounded(conv)
    mut argmax := tensor.Empty()
    mut pooled := tensor.MaxPool2DForward(image, 2, 2, 2, 2, argmax)
    println rounded(pooled), rounded(argmax)
    println rounded(tensor.AvgPool2DForward(image, 2, 2, 2, 2))
    mut cols := tensor.Im2Col(image, 0, 2, 2, 2, 2, 0, 0)
    println tensor.ShapeString(cols)
    mut grad := tensor.Zeros4(1, 1, 4, 4)
    println tensor.Col2ImAcc(cols, grad, 0, 2, 2, 2, 2, 0, 0), tensor.Equal(grad, image)

    mut m := tensor.Reshape(tensor.Arange(6), tensor.Shape2(2, 3))
    mut mt := tensor.TransposeRaw(m.data.ref(0), 2, 3)
    println tensor.ShapeString(mt), rounded(mt)
    mut product := tensor.Zeros2(2, 2)
    tensor.GemmAcc(m.data.ref(0), mt.data.ref(0), product.data.ref(0), 2, 3, 2)
    println rounded(product)
end

Outputtrue true true true true true true true -32 47 55 2 2 5 2 1x1x2x2 1000 1800 4200 5000 500 700 1300 1500 500 700 1300 1500 250 450 1050 1250 4x4 true true 3x2 0 300 100 400 200 500 500 1400 1400 5000

`next_bits`

func next_bits(): u64

64 random bits.

The example for this item is shown with the ones above: ai-tensor-Random.salam

`next_float`

func next_float(): f64

A float between zero and one.

The example for this item is shown with the ones above: ai-tensor-Random.salam

`next_normal`

func next_normal(): f64

A number from the standard normal distribution.

The example for this item is shown with the ones above: ai-tensor-Random.salam

Constants and variables

`TSR_TOL`

const TSR_TOL: f64 = 1e-05

The default comparison tolerance.

The example for this item is shown with the ones above: ai-tensor-Create.salam

Functions

`SetThreads`

func SetThreads(n: int)

The number of threads for large element-wise operations, matrix products and convolutions; 0 means one per core, at most 8.

The example for this item is shown with the ones above: ai-tensor-Random.salam

`Threads`

func Threads(): int

The number of threads in use: the value given to SetThreads, or the core count capped at 8.

The example for this item is shown with the ones above: ai-tensor-Random.salam

`WriteTensor`

func WriteTensor(buf &: compress.ByteBuf, name: str, t &: Tensor): bool

Writes the tensor with its name into a buffer.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`ReadTensor`

func ReadTensor(rd &: compress.ByteReader, name_out &: str, out &: Tensor): Throws<bool>

Reads a tensor and its name from a reader. Throws (context tensor) on an empty name, a dtype other than f32, a size that does not match the shape, or data that ends early; out and name_out are left alone then.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`ConvOutSize`

func ConvOutSize(size: int, k: int, s: int, p: int): int

The convolution output size for a size, kernel, stride and padding.

The example for this item is shown with the ones above: ai-tensor-Random.salam

`Im2Col`

func Im2Col(x &: Tensor, sample: int, kh: int, kw: int, sh: int, sw: int, ph: int, pw: int): Tensor

Turns the convolution windows of one sample into columns, so the convolution can be done as a matrix multiplication.

The example for this item is shown with the ones above: ai-tensor-Random.salam

`Col2ImAcc`

func Col2ImAcc(cols &: Tensor, dx &: Tensor, sample: int, kh: int, kw: int, sh: int, sw: int, ph: int, pw: int): bool

The reverse, accumulating; for the convolution gradient.

The example for this item is shown with the ones above: ai-tensor-Random.salam

`Conv2DForward`

func Conv2DForward(x &: Tensor, w &: Tensor, bias &: Tensor, sh: int, sw: int, ph: int, pw: int): Tensor

A two dimensional convolution over an NCHW tensor with weights, bias, strides and padding.

The example for this item is shown with the ones above: ai-tensor-Random.salam

`MaxPool2DForward`

func MaxPool2DForward(x &: Tensor, kh: int, kw: int, sh: int, sw: int, argmax_out &: Tensor): Tensor

Two dimensional max pooling; the places of the maxima come back too.

The example for this item is shown with the ones above: ai-tensor-Random.salam

`AvgPool2DForward`

func AvgPool2DForward(x &: Tensor, kh: int, kw: int, sh: int, sw: int): Tensor

Two dimensional average pooling.

The example for this item is shown with the ones above: ai-tensor-Random.salam

`SumAll`

func SumAll(x &: Tensor): f64

The sum of everything.

ai-tensor-Math.salam

import ai.tensor

func show(name: str, t &: tensor.Tensor):
    println name, tensor.ToString(t)
end

func show_rounded(name: str, t &: tensor.Tensor):
    mut text := ""
    repeat t.numel() in i:
        rounded := (t.at(i) * 1000.0 + 0.5) as int
        text += " " + rounded
    end
    println name + " x1000:" + text
end

func main:
    mut a := tensor.Arange(4)
    mut b := tensor.Full(tensor.Shape1(4), 2.0)
    mut sum := a + b
    show("+", sum)
    mut mixed := a * b - b / b
    show("*-/", mixed)
    mut added := tensor.Add(a, b)
    mut subtracted := tensor.Sub(a, b)
    mut multiplied := tensor.Mul(a, b)
    mut divided := tensor.Div(a, b)
    show("Add", added)
    show("Sub", subtracted)
    show("Mul", multiplied)
    show("Div", divided)
    mut out := tensor.Zeros1(4)
    println tensor.AddInto(a, b, out), tensor.SubInto(a, b, out), tensor.MulInto(a, b, out), tensor.DivInto(a, b, out)
    show("DivInto", out)
    show("+1", tensor.AddScalar(a, 1.0))
    show("-1", tensor.SubScalar(a, 1.0))
    show("x3", tensor.Scale(a, 3.0))
    show("/2", tensor.DivScalar(a, 2.0))
    show("Neg", tensor.Neg(a))

    mut d := tensor.Clone(a)
    tensor.AddInPlace(d, b)
    tensor.SubInPlace(d, a)
    tensor.MulInPlace(d, b)
    tensor.ScaleInPlace(d, 0.5)
    tensor.AddScalarInPlace(d, 1.0)
    tensor.AddScaled(d, a, 2.0)
    show("in place", d)

    show_rounded("Exp", tensor.Clip(tensor.Exp(a), 0.0, 10.0))
    show_rounded("Log", tensor.Log(b))
    show("Tanh", tensor.Tanh(tensor.Zeros1(2)))
    show_rounded("Sigmoid", tensor.Sigmoid(tensor.Zeros1(2)))
    show("Relu", tensor.Relu(tensor.Neg(a)))
    show_rounded("Gelu", tensor.Gelu(tensor.Zeros1(2)))
    show_rounded("Silu", tensor.Silu(tensor.Zeros1(2)))
    show("Sqrt", tensor.Sqrt(tensor.Mul(b, b)))
    show("Abs", tensor.Abs(tensor.Neg(a)))
    show("Square", tensor.Square(a))
    show("PowScalar", tensor.PowScalar(a, 3.0))

    println tensor.SumAll(a), tensor.MeanAll(a), tensor.MaxAll(a), tensor.MinAll(a), tensor.ArgMaxAll(a)
    mut m := tensor.Reshape(tensor.Arange(6), tensor.Shape2(2, 3))
    show("SumAxis", tensor.SumAxis(m, 0, false))
    show("MeanAxis", tensor.MeanAxis(m, 1, true))
    show("MaxAxis", tensor.MaxAxis(m, 1, false))
    show("ArgMaxAxis", tensor.ArgMaxAxis(m, 1, false))
    show("Cumsum", tensor.Cumsum(m, 1))
    println tensor.SumAll(tensor.Softmax(m)) > 1.99, tensor.MaxAll(tensor.LogSoftmax(a)) < 0.0

    mut t := tensor.Transpose2D(m)
    show("Transpose2D", t)
    show("MatMul", tensor.MatMul(m, t))
    show("TransposeMul", tensor.TransposeMul(m, m))
    show("MulTranspose", tensor.MulTranspose(m, m))
    mut product := tensor.Zeros2(2, 2)
    println tensor.MatMulInto(m, t, product), tensor.MatMulAcc(m, t, product)
    show("MatMulAcc", product)
    show("MatVec", tensor.MatVec(m, tensor.Ones(tensor.Shape1(3))))
    mut batch := tensor.Stack2(m, m)
    mut batch_t := tensor.Stack2(t, t)
    println tensor.ShapeString(tensor.BatchMatMul(batch, batch_t)), tensor.Dot(a, b)
    println tensor.MaxAbsDiff(a, added), tensor.AllClose(added, sum, 0.001)
end

Output+ tensor(4)[2 3 4 5] *-/ tensor(4)[-1 1 3 5] Add tensor(4)[2 3 4 5] Sub tensor(4)[-2 -1 0 1] Mul tensor(4)[0 2 4 6] Div tensor(4)[0 0.5 1 1.5] true true true true DivInto tensor(4)[0 0.5 1 1.5] +1 tensor(4)[1 2 3 4] -1 tensor(4)[-1 0 1 2] x3 tensor(4)[0 3 6 9] /2 tensor(4)[0 0.5 1 1.5] Neg tensor(4)[0 -1 -2 -3] in place tensor(4)[3 5 7 9] Exp x1000: 1000 2718 7389 10000 Log x1000: 693 693 693 693 Tanh tensor(2)[0 0] Sigmoid x1000: 500 500 Relu tensor(4)[0 0 0 0] Gelu x1000: 0 0 Silu x1000: 0 0 Sqrt tensor(4)[2 2 2 2] Abs tensor(4)[0 1 2 3] Square tensor(4)[0 1 4 9] PowScalar tensor(4)[0 1 8 27] 6 1.5 3 0 3 SumAxis tensor(3)[3 5 7] MeanAxis tensor(2x1)[1 4] MaxAxis tensor(2)[2 5] ArgMaxAxis tensor(2)[2 2] Cumsum tensor(2x3)[0 1 3 3 7 12] true true Transpose2D tensor(3x2)[0 3 1 4 2 5] MatMul tensor(2x2)[5 14 14 50] TransposeMul tensor(3x3)[9 12 15 12 17 22 15 22 29] MulTranspose tensor(2x2)[5 14 14 50] true true MatMulAcc tensor(2x2)[10 28 28 100] MatVec tensor(2)[3 12] 2x2x2 12 2 true

`MeanAll`

func MeanAll(x &: Tensor): f64

The mean of everything.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`MaxAll`

func MaxAll(x &: Tensor): f64

The greatest of everything.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`MinAll`

func MinAll(x &: Tensor): f64

The smallest of everything.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`ArgMaxAll`

func ArgMaxAll(x &: Tensor): int

Where the maximum is.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`SumAxis`

func SumAxis(x &: Tensor, axis: int, keepdim: bool): Tensor

The sum along one dimension; with keepdim that dimension stays, with size one.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`MeanAxis`

func MeanAxis(x &: Tensor, axis: int, keepdim: bool): Tensor

The mean along one dimension.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`MaxAxis`

func MaxAxis(x &: Tensor, axis: int, keepdim: bool): Tensor

The maximum along one dimension.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`ArgMaxAxis`

func ArgMaxAxis(x &: Tensor, axis: int, keepdim: bool): Tensor

Where the maximum is along one dimension; for taking the predicted class.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`Softmax`

func Softmax(x &: Tensor): Tensor

The softmax over the last dimension; each row of the output is a set of probabilities.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`LogSoftmax`

func LogSoftmax(x &: Tensor): Tensor

The log of the softmax, numerically stable.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`Empty`

func Empty(): Tensor

An empty tensor; for outputs a function fills in.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`Shape1`

func Shape1(n: int): Vector<int>

A one dimensional shape.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`Shape2`

func Shape2(a: int, b: int): Vector<int>

A two dimensional shape.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`Shape3`

func Shape3(a: int, b: int, c: int): Vector<int>

A three dimensional shape.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`Shape4`

func Shape4(a: int, b: int, c: int, d: int): Vector<int>

A four dimensional shape, such as batch by channel by height by width.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`Zeros`

func Zeros(shape &: Vector<int>): Tensor

A tensor of zeros with a shape.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`Zeros1`

func Zeros1(n: int): Tensor

A vector of zeros.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`Zeros2`

func Zeros2(r: int, c: int): Tensor

A matrix of zeros.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`Zeros3`

func Zeros3(a: int, b: int, c: int): Tensor

A three dimensional tensor of zeros.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`Zeros4`

func Zeros4(a: int, b: int, c: int, d: int): Tensor

A four dimensional tensor of zeros.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`Full`

func Full(shape &: Vector<int>, x: f64): Tensor

A tensor filled with one value.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`Ones`

func Ones(shape &: Vector<int>): Tensor

A tensor of ones.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`Scalar`

func Scalar(x: f64): Tensor

A one element tensor.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`FromArray`

func FromArray(shape &: Vector<int>, data &: Vector<f32>): Tensor

A tensor from a shape and 32 bit float data.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`Arange`

func Arange(n: int): Tensor

The vector 0 to n minus one.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`Clone`

func Clone(x &: Tensor): Tensor

An independent copy.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`CopyInto`

func CopyInto(dst &: Tensor, src &: Tensor)

Copies the data into a tensor of the same shape.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`Free`

func Free(x &: Tensor)

Frees the tensor.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`SetAll`

func SetAll(x &: Tensor, v: f64)

Sets every element to one value.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`SameShape`

func SameShape(x &: Tensor, y &: Tensor): bool

The two tensors have the same shape.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`ToMatrix`

func ToMatrix(x &: Tensor): matrix.Matrix

A two dimensional tensor as a matrix.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`FromMatrix`

func FromMatrix(m &: matrix.Matrix): Tensor

A matrix as a tensor.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`Reshape`

func Reshape(x &: Tensor, shape &: Vector<int>): Tensor

A copy with a new shape; an empty tensor when the element counts differ.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`SqueezeInPlace`

func SqueezeInPlace(x &: Tensor)

Removes the dimensions of size one in place.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`UnsqueezeInPlace`

func UnsqueezeInPlace(x &: Tensor, axis: int)

Adds a dimension of size one at the given place.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`Permute`

func Permute(x &: Tensor, perm &: Vector<int>): Tensor

Changes the order of the dimensions.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`SliceAxis0`

func SliceAxis0(x &: Tensor, start: int, len: int): Tensor

A range of the first dimension.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`Narrow`

func Narrow(x &: Tensor, axis: int, start: int, len: int): Tensor

A range of one dimension.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`Concat2`

func Concat2(a &: Tensor, b &: Tensor, axis: int): Tensor

Puts two tensors one after the other along one dimension.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`Stack2`

func Stack2(a &: Tensor, b &: Tensor): Tensor

Stacks two tensors of the same shape along a new first dimension.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`Pad2D`

func Pad2D(x &: Tensor, pt: int, pb: int, pl: int, pr: int): Tensor

Pads the last two dimensions with zeros: top, bottom, left and right.

ai-tensor-Index.salam

import ai.tensor

func show(name: str, t &: tensor.Tensor):
    println name, tensor.ToString(t)
end

func main:
    mut a := tensor.Arange(4)
    mut b := tensor.Full(tensor.Shape1(4), 1.5)
    show("Greater", tensor.Greater(a, b))
    show("GreaterEqual", tensor.GreaterEqual(a, tensor.Clone(a)))
    show("Less", tensor.Less(a, b))
    show("LessEqual", tensor.LessEqual(a, b))
    show("EqualMask", tensor.EqualMask(a, a))
    show("NotEqualMask", tensor.NotEqualMask(a, b))
    show("GreaterScalar", tensor.GreaterScalar(a, 1.0))
    show("LessScalar", tensor.LessScalar(a, 1.0))
    show("EqualScalar", tensor.EqualScalar(a, 2.0))
    mut mask := tensor.Greater(a, b)
    show("Where", tensor.Where(mask, a, tensor.Neg(a)))
    show("MaskedFill", tensor.MaskedFill(a, mask, -1.0))

    mut m := tensor.Reshape(tensor.Arange(6), tensor.Shape2(3, 2))
    mut indices := Vector {} as Vector<int>
    indices.push(2)
    indices.push(0)
    show("IndexSelect", tensor.IndexSelect(m, 0, indices))
    show("TakeRows", tensor.TakeRows(m, indices))
    mut places := tensor.Zeros2(3, 1)
    places.set_at(0, 1.0)
    show("Gather", tensor.Gather(m, 1, places))
    mut target := tensor.Zeros2(3, 2)
    println tensor.ScatterAddRows(target, indices, tensor.Ones(tensor.Shape2(2, 2)))
    show("ScatterAddRows", target)
    show("OneHot", tensor.OneHot(indices, 3))

    mut scores := tensor.Reshape(tensor.Arange(5), tensor.Shape1(5))
    mut top_places := tensor.Empty()
    show("TopK", tensor.TopK(scores, 2, true, top_places))
    show("TopK places", top_places)
    mut square := tensor.Ones(tensor.Shape2(3, 3))
    show("Tril", tensor.Tril(square, 0))
    show("Triu", tensor.Triu(square, 1))
    mut broadcast := Vector {} as Vector<int>
    println tensor.BroadcastShape(m, tensor.Ones(tensor.Shape1(2)), broadcast), broadcast.len(), broadcast.get(0)
    show("Pad2D", tensor.Pad2D(tensor.Ones(tensor.Shape4(1, 1, 1, 1)), 1, 0, 0, 1))
end

OutputGreater tensor(4)[0 0 1 1] GreaterEqual tensor(4)[1 1 1 1] Less tensor(4)[1 1 0 0] LessEqual tensor(4)[1 1 0 0] EqualMask tensor(4)[1 1 1 1] NotEqualMask tensor(4)[1 1 1 1] GreaterScalar tensor(4)[0 0 1 1] LessScalar tensor(4)[1 0 0 0] EqualScalar tensor(4)[0 0 1 0] Where tensor(4)[0 -1 2 3] MaskedFill tensor(4)[0 1 -1 -1] IndexSelect tensor(2x2)[4 5 0 1] TakeRows tensor(2x2)[4 5 0 1] Gather tensor(3x1)[1 2 4] true ScatterAddRows tensor(3x2)[1 1 0 0 1 1] OneHot tensor(2x3)[0 0 1 1 0 0] TopK tensor(2)[4 3] TopK places tensor(2)[4 3] Tril tensor(3x3)[1 0 0 1 1 0 1 1 1] Triu tensor(3x3)[0 1 1 0 0 1 0 0 0] true 2 3 Pad2D tensor(1x1x2x2)[0 0 1 0]

`Transpose2D`

func Transpose2D(x &: Tensor): Tensor

The transpose of a matrix.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`MatMul`

func MatMul(x &: Tensor, y &: Tensor): Tensor

Matrix multiplication.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`MatMulInto`

func MatMulInto(x &: Tensor, y &: Tensor, out &: Tensor): bool

Matrix multiplication into a ready tensor; with no allocation.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`MatMulAcc`

func MatMulAcc(x &: Tensor, y &: Tensor, out &: Tensor): bool

Adds the product to the output.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`TransposeMul`

func TransposeMul(x &: Tensor, y &: Tensor): Tensor

The transpose of x times y, without building the transpose.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`MulTranspose`

func MulTranspose(x &: Tensor, y &: Tensor): Tensor

x times the transpose of y.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`BatchMatMul`

func BatchMatMul(x &: Tensor, y &: Tensor): Tensor

Matrix multiplication for every member of a batch.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`MatVec`

func MatVec(a &: Tensor, x &: Tensor): Tensor

A matrix times a vector.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`TransposeRaw`

func TransposeRaw(px: f32*, n: int, m: int): Tensor

The transpose from a raw pointer to the data.

The example for this item is shown with the ones above: ai-tensor-Random.salam

`GemmAcc`

func GemmAcc(px: f32*, py: f32*, pr: f32*, n: int, k: int, m: int)

Accumulating matrix multiplication over raw pointers; the fast kernel the other multiplications are built on.

The example for this item is shown with the ones above: ai-tensor-Random.salam

`Dot`

func Dot(x &: Tensor, y &: Tensor): f64

The dot product of two vectors.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`ShapeString`

func ShapeString(x &: Tensor): str

The shape as text, such as 2x3.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`ToString`

func ToString(x &: Tensor): str

The tensor as text.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`Print`

func Print(x &: Tensor)

Prints the tensor.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`MaxAbsDiff`

func MaxAbsDiff(x &: Tensor, y &: Tensor): f64

The greatest difference between the elements.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`AllClose`

func AllClose(x &: Tensor, y &: Tensor, tol: f64): bool

Every element is equal within a tolerance.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`Equal`

func Equal(x &: Tensor, y &: Tensor): bool

The shapes and the elements match.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`BroadcastShape`

func BroadcastShape(a &: Tensor, b &: Tensor, out &: Vector<int>): bool

The shape two tensors broadcast to; false when they do not fit together.

The example for this item is shown with the ones above: ai-tensor-Index.salam

`Expand`

func Expand(x &: Tensor, shape &: Vector<int>): Tensor

Expands the tensor to a larger shape by broadcasting.

The example for this item is shown with the ones above: ai-tensor-Create.salam

`Greater`

func Greater(a &: Tensor, b &: Tensor): Tensor

A mask of ones and zeros for a greater than b.

The example for this item is shown with the ones above: ai-tensor-Index.salam

`GreaterEqual`

func GreaterEqual(a &: Tensor, b &: Tensor): Tensor

A mask for greater or equal.

The example for this item is shown with the ones above: ai-tensor-Index.salam

`Less`

func Less(a &: Tensor, b &: Tensor): Tensor

A mask for less than.

The example for this item is shown with the ones above: ai-tensor-Index.salam

`LessEqual`

func LessEqual(a &: Tensor, b &: Tensor): Tensor

A mask for less or equal.

The example for this item is shown with the ones above: ai-tensor-Index.salam

`EqualMask`

func EqualMask(a &: Tensor, b &: Tensor): Tensor

A mask for equality.

The example for this item is shown with the ones above: ai-tensor-Index.salam

`NotEqualMask`

func NotEqualMask(a &: Tensor, b &: Tensor): Tensor

A mask for inequality.

The example for this item is shown with the ones above: ai-tensor-Index.salam

`GreaterScalar`

func GreaterScalar(x &: Tensor, s: f64): Tensor

A mask for the elements greater than a number.

The example for this item is shown with the ones above: ai-tensor-Index.salam

`LessScalar`

func LessScalar(x &: Tensor, s: f64): Tensor

A mask for the elements less than a number.

The example for this item is shown with the ones above: ai-tensor-Index.salam

`EqualScalar`

func EqualScalar(x &: Tensor, s: f64): Tensor

A mask for the elements equal to a number.

The example for this item is shown with the ones above: ai-tensor-Index.salam

`Where`

func Where(cond &: Tensor, a &: Tensor, b &: Tensor): Tensor

Where the condition is one take from a, otherwise from b.

The example for this item is shown with the ones above: ai-tensor-Index.salam

`MaskedFill`

func MaskedFill(x &: Tensor, mask &: Tensor, value: f64): Tensor

Fills the masked places with a value; negative infinity in an attention mask, for instance.

The example for this item is shown with the ones above: ai-tensor-Index.salam

`IndexSelect`

func IndexSelect(x &: Tensor, axis: int, ids &: Vector<int>): Tensor

Parts of one dimension, by index.

The example for this item is shown with the ones above: ai-tensor-Index.salam

`TakeRows`

func TakeRows(x &: Tensor, ids &: Vector<int>): Tensor

Rows by index; as an embedding table does.

The example for this item is shown with the ones above: ai-tensor-Index.salam

`Gather`

func Gather(x &: Tensor, axis: int, idx &: Tensor): Tensor

Along one dimension, takes the elements named by a tensor of indices.

The example for this item is shown with the ones above: ai-tensor-Index.salam

`ScatterAddRows`

func ScatterAddRows(dst &: Tensor, ids &: Vector<int>, src &: Tensor): bool

Adds the rows of the source to the indexed rows of the destination.

The example for this item is shown with the ones above: ai-tensor-Index.salam

`OneHot`

func OneHot(ids &: Vector<int>, classes: int): Tensor

One hot vectors for a set of indices.

The example for this item is shown with the ones above: ai-tensor-Index.salam

`TopK`

func TopK(x &: Tensor, k: int, largest: bool, idx_out &: Tensor): Tensor

The k greatest or smallest values and where they are.

The example for this item is shown with the ones above: ai-tensor-Index.salam

`Tril`

func Tril(x &: Tensor, diagonal: int): Tensor

The lower triangle; the rest zero.

The example for this item is shown with the ones above: ai-tensor-Index.salam

`Triu`

func Triu(x &: Tensor, diagonal: int): Tensor

The upper triangle.

The example for this item is shown with the ones above: ai-tensor-Index.salam

`Cumsum`

func Cumsum(x &: Tensor, axis: int): Tensor

The cumulative sum along one dimension.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`NewRng`

func NewRng(seed: i64): Rng

A generator from a seed.

The example for this item is shown with the ones above: ai-tensor-Random.salam

`DeriveSeed`

func DeriveSeed(seed: i64, index: i64): i64

An independent seed for job number i, from a main seed.

The example for this item is shown with the ones above: ai-tensor-Random.salam

`FillUniform`

func FillUniform(t &: Tensor, lo: f64, hi: f64, g &: Rng)

Fills with the uniform distribution.

The example for this item is shown with the ones above: ai-tensor-Random.salam

`FillNormal`

func FillNormal(t &: Tensor, mean: f64, std: f64, g &: Rng)

Fills with the normal distribution.

The example for this item is shown with the ones above: ai-tensor-Random.salam

`FillBernoulli`

func FillBernoulli(t &: Tensor, p: f64, g &: Rng)

One with probability p, zero otherwise.

The example for this item is shown with the ones above: ai-tensor-Random.salam

`RandUniform`

func RandUniform(shape &: Vector<int>, lo: f64, hi: f64, seed: i64): Tensor

A uniform random tensor from a seed.

The example for this item is shown with the ones above: ai-tensor-Random.salam

`RandNormal`

func RandNormal(shape &: Vector<int>, seed: i64): Tensor

A normal random tensor from a seed.

The example for this item is shown with the ones above: ai-tensor-Random.salam

`Add`

func Add(x &: Tensor, y &: Tensor): Tensor

Addition with broadcasting.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`Sub`

func Sub(x &: Tensor, y &: Tensor): Tensor

Subtraction with broadcasting.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`Mul`

func Mul(x &: Tensor, y &: Tensor): Tensor

Elementwise multiplication with broadcasting.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`Div`

func Div(x &: Tensor, y &: Tensor): Tensor

Elementwise division with broadcasting.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`AddInto`

func AddInto(x &: Tensor, y &: Tensor, out &: Tensor): bool

Addition into a ready tensor; all three must have the same shape, or it returns false.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`SubInto`

func SubInto(x &: Tensor, y &: Tensor, out &: Tensor): bool

Subtraction into a ready tensor.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`MulInto`

func MulInto(x &: Tensor, y &: Tensor, out &: Tensor): bool

Elementwise multiplication into a ready tensor.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`DivInto`

func DivInto(x &: Tensor, y &: Tensor, out &: Tensor): bool

Elementwise division into a ready tensor.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`AddScalar`

func AddScalar(x &: Tensor, s: f64): Tensor

Adds a number to every element.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`SubScalar`

func SubScalar(x &: Tensor, s: f64): Tensor

Subtracts a number from every element.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`Scale`

func Scale(x &: Tensor, s: f64): Tensor

Multiplies every element by a number.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`DivScalar`

func DivScalar(x &: Tensor, s: f64): Tensor

Divides every element by a number.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`Neg`

func Neg(x &: Tensor): Tensor

The negation.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`AddInPlace`

func AddInPlace(dst &: Tensor, src &: Tensor)

Adds in place; does nothing when the shapes differ.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`SubInPlace`

func SubInPlace(dst &: Tensor, src &: Tensor)

Subtracts in place.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`MulInPlace`

func MulInPlace(dst &: Tensor, src &: Tensor)

Multiplies elementwise in place.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`ScaleInPlace`

func ScaleInPlace(dst &: Tensor, s: f64)

Multiplies by a number in place.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`AddScalarInPlace`

func AddScalarInPlace(dst &: Tensor, s: f64)

Adds a number in place.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`AddScaled`

func AddScaled(dst &: Tensor, src &: Tensor, alpha: f64)

Adds alpha times the source to the destination.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`Exp`

func Exp(x &: Tensor): Tensor

The exponential, element by element.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`Log`

func Log(x &: Tensor): Tensor

The natural logarithm, element by element.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`Tanh`

func Tanh(x &: Tensor): Tensor

The hyperbolic tangent, element by element.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`Sigmoid`

func Sigmoid(x &: Tensor): Tensor

The sigmoid, element by element.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`Relu`

func Relu(x &: Tensor): Tensor

The rectifier, the greater of zero and x, element by element.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`Gelu`

func Gelu(x &: Tensor): Tensor

GELU, element by element.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`Sqrt`

func Sqrt(x &: Tensor): Tensor

The square root, element by element.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`Abs`

func Abs(x &: Tensor): Tensor

The absolute value, element by element.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`Square`

func Square(x &: Tensor): Tensor

The square, element by element.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`Silu`

func Silu(x &: Tensor): Tensor

SiLU, meaning x times the sigmoid of x, element by element.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`PowScalar`

func PowScalar(x &: Tensor, p: f64): Tensor

Raises the elements to a power.

The example for this item is shown with the ones above: ai-tensor-Math.salam

`Clip`

func Clip(x &: Tensor, lo: f64, hi: f64): Tensor

Holds the elements within a range.

The example for this item is shown with the ones above: ai-tensor-Create.salam