Tensor contractions

EINSUM expresses a dense contraction with an explicit-output pattern, so a calculation states its index meaning instead of hiding it in transposes, reshapes and reductions.

product = EINSUM("ik,kj->ij", lhs, rhs)
diagonal = EINSUM("ii->i", matrix)
trace = EINSUM("ii->", matrix)
variance = EINSUM("i,ij,j->", weights, covariance, weights)

EINSUM is an ordinary pure function. Assignment binds its result and ordinary reactive evaluation computes it; there is no declaration, job or accumulator.

Patterns

The pattern has one input term per operand, exactly one -> and an explicit output term. Letters a–z and A–Z are distinct single-character axis labels; ... stands for a group of anonymous axes; spaces are ignored.

A named label has exactly equal extents wherever it occurs; named axes never broadcast. Ellipsis groups align from the right and broadcast where one extent is one. A label repeated within one operand selects its diagonal. Labels absent from the output are reduced. An output label must occur in an input and may appear only once.

Values and results

Operands are finite numeric scalars (rank zero), native numeric Lists (rank one), rectangular worksheet arrays (rows and columns, never squeezed), rectangular nested Lists, or explicitly shaped f32/f64 Tensors. Booleans, text, blanks, errors, ragged shapes, integer/complex Tensors and nonfinite numbers are rejected, not coerced (EINSUM_TYPE, EINSUM_RAGGED, EINSUM_NUMERIC). Use an explicit numeric projection for a Frame.

With ordinary operands, rank-zero output is a Number, rank one a native List, and rank two a rectangular array. Higher ranks, or a contraction involving a Tensor, use an immutable Tensor carrier at every rank. Its exact shape, precision and element tag survive assignment, composition and typed transport (grid.tensor.value.v1), including empty axes in any position. A zero-sized reduced axis yields zeros; scalar shape [] contains one value. The explicit display projection produces nested numeric arrays when bounded, and otherwise retains the compact shape/data object.

The default order: :reference enumerates reduced indices in a specified order and sums products starting at zero, so results are reproducible. An optional final options record {order: :optimized, atol, rtol} permits reassociation within explicit tolerances; reference order refuses tolerance fields, and optimized order requires both. A bounded deterministic pair planner can reassociate multi-operand f64 contractions. It retains an axis until every operand using it has participated, including axes shared by three operands. Every candidate is compared with the complete reference result for that input generation. A candidate that exceeds workspace, produces nonfinite values or misses the requested tolerance falls back to that reference result. Reference errors remain errors even if reassociation could avoid them. Optimization is a numerical permission, not a speed guarantee: validation cost is included.

Admission bounds are 32 operands, rank eight, a 4,096-byte pattern, 10,000,000 elements per operand or result and a bounded multiplication budget (200 million indexed terms). Pinned inputs, admission copies, output and selected intermediates must fit a conservative 128 MiB workspace bound. Ordinary array publication also bounds structural prefixes; shaped empty Tensors retain dimensions without allocating nested empty containers. Oversized outputs fail with EINSUM_RESOURCE before output allocation.

Reports

EINSUM.REPORT(pattern, operands…[, options]) performs the same single execution and returns grid.tensor.contraction-report.v1: status, outcome (OK(value) or ERR({code, message, stage})), the pattern, input and output shapes, the axis extents, precision, numerical contract, physical path and estimated work. Static pattern mistakes report stage :pattern. planningVersion, backend, estimatedPeakBytes and referenceValidation describe the selected implementation and conservative workspace admission; they are not measured peak memory or device-performance evidence.

patternPreparation is :admitted when a literal pattern was prepared with the model, or :fallback when parsed for this execution. planReuse describes the shape/stride plan: :built, :reused, :fallback, :contended-fallback, :budget-fallback, or :not-planned when admission failed before a plan was obtained. These fields describe physical execution; they do not change the numerical contract.

Literal patterns retain one shape plan per call site. Same-shaped value edits reuse it, while changed shapes rebuild it. Every execution still checks fresh operand types and numeric values. Preparation is limited to 64 sites per model and 64 KiB per site; excess sites and unprepared calls use the same reference kernel. Structural model edits discard preparation. Plain EINSUM avoids constructing report-only metadata.

Current scope

Ordinary inputs compute in f64. Explicit f32 Tensor operands compute in f32 when all operands are f32; mixed inputs or an explicit f64 computation widen before multiplication. A result annotation alone does not narrow the operands' arithmetic. Declared Tensor operand shapes are retained; an ordinary one-row array remains rank two.

Shaped static-pattern expressions can lower to a resident Tensor contraction on CPU (f32/f64), Metal (f32), or CUDA (f32/f64). The portable operation carries the pattern and operand handles; shape plans are bounded and unchanged input generations reuse resident results. The same indexed arithmetic implements resident and general execution. Candle's training graph differentiates contractions using reverse scatter: diagonals receive diagonal gradients, broadcast axes accumulate and repeated use of one variable adds all contributions. Resident storage supports arbitrary empty-axis positions. Mutable shaped inputs import the current Workbook value without overwriting input/default/STATE ownership. Content generations preserve unchanged upstream contractions after downstream-only edits. Imports validate and hash their data at operation ingress; this is selected work, not zero-copy Frame/device ingestion.

Precision compatibility is selected for the connected tensor component at model admission. Metal components containing f64/i64 handles retain CPU placement; CUDA contracts support f32/f64. Unsupported precision or device-index ranges select the exact CPU component fallback. Explicit unsupported requests refuse. Direct resident-output requests compute the required ingress values before native dispatch, so checking a cached contraction does not first repeat its ordinary calculation. Mixed requests retain normal targeted evaluation.

Homogeneous physical-unit operands normalize compatible scales before multiplication. Output dimensions multiply across operands and reduction adds like-dimensional terms. Mixed dimensions, affine absolute temperatures and unsupported refinements/compound currencies require an explicit supported conversion. Unit-bearing source expressions use the general native path.

Static pattern preparation recognizes literals, immutable aliases, local LET bindings, and constant parameters/captures in local or imported helpers. It specializes bounded copies of helper bodies while preserving their original arguments and captures. Dynamic patterns, mutable bindings and expansions beyond the specialization budget retain the checked general path.

Metal reference execution uses integer IEEE-754 f32 arithmetic to preserve rounding order, signed zero and subnormals. Primal values and reverse-mode intermediates remain resident; bounded dispatches check a four-byte numeric status and cancellation before publishing a replacement generation. Pipelines are compiled once per selected runtime and indexing metadata is retained with the shape plan. The existing Candle backward traversal selects exact gradient accumulation for graphs containing a contraction, including repeated operands. Unsupported GPU precision or device combinations refuse explicitly. The historical target-machine Performance budgets remain separate from local development measurements.

CUDA uses explicit round-to-nearest f32/f64 operations with fusion and flush-to-zero disabled, the same bounded plan metadata, resident reverse gradients and conditional gradient accumulation. Its Rust adapter and host execution of the production indexing algorithm have been checked locally. CUDA compilation and execution on an NVIDIA device remain unverified because that hardware is unavailable. Local Apple M4 results do not establish the retained Apple M1 latency or peak-memory targets.

See the executable weighted covariance, attention and rank-four mechanics examples.