External Functions

External functions cross the runtime boundary. They run in worker processes — not in Redis Lua and not in the in-process formula evaluator — and their results are written back asynchronously into your model.

This is the user-facing guide. For the formal queue + writeback contract, see docs/specs/external_functions.md.

1. The Built-In External Functions

All built-ins are registered in src/external/registry.ts. Tier 1 deployments may also register Python-backed inference functions from a model-host manifest, for example CHURN_SCORE(features). These functions are still normal external_async functions: Grid queues the call, a worker invokes the localhost Python model host, and Node writes the typed result back through the external writeback path. See docs/specs/python_model_host.md.

2. Calling An External Function

Just like any other function:

A1 = FX_RATE("EUR", "USD")
A2 = HTTP_JSON("https://api.example.com/widgets/42")
A3 = ML_SCORE([0.1, 0.2, 0.3, 0.4])
A4 = AI_PROMPT("Summarize this note", { temperature: 0.2 })
A5 = SELECT id, amount FROM finance.public.orders WHERE amount >= 100 ORDER BY id DESC LIMIT 25

Or with named arguments:

A4 = FX_RATE(base := "GBP", quote := "USD")

The cell value will eventually contain the worker's result. Until then, the cell goes through a defined status lifecycle.

PostgreSQL table references are pull-based in v1. Grid reruns the query when normal Grid dependencies change or when the model is explicitly refreshed; it does not subscribe to external database row changes. Mutations use PG_MUTATE behind a guarded worker route and are disabled unless the deployment explicitly enables the write path and configures database-side idempotency.

3. The Status Lifecycle

A cell that depends on an external function moves through these states:

   ┌─────────┐  input/dep change   ┌─────────┐  worker claims   ┌─────────┐
   │  dirty  │ ──────────────────► │ queued  │ ───────────────► │ running │
   └─────────┘                     └─────────┘                  └─────────┘
                                                                      │
                                                       ┌──────────────┼──────────────┐
                                                       │              │              │
                                                       ▼              ▼              ▼
                                                  ┌────────┐    ┌────────┐    ┌────────┐
                                                  │ ready  │    │ stale  │    │ failed │
                                                  └────────┘    └────────┘    └────────┘

You can read these via GET /api/models/:id/resolve/:symbol.

4. Eager Vs Lazy External Calls

Both work; the difference is when work gets queued.

4.1 Eager (=)

A1 = FX_RATE("EUR", "USD")

A job is queued immediately when the model deploys. Downstream cells remain stale until the worker writes back. After writeback, the runtime resumes downstream recalculation.

Use eager when:

• The value is critical and you want it as fresh as possible.
• Downstream cells are eager and need a value to make progress.

4.2 Lazy (~=)

A2 ~= ML_SCORE([0.1, 0.2, 0.3, 0.4])

No job is queued until something reads the cell. Reads come from:

• A direct API call: GET /api/models/:id/resolve/A2.
• A downstream eager cell that references A2.

Use lazy when:

• The call is expensive and rarely consulted.
• The model is large and you want to defer non-critical work.

4.3 Mixing With Fallbacks

External calls are network-dependent and can fail. Pair them with DEFAULT so downstream cells stay computable:

A1 = FX_RATE("EUR", "USD")
B1 = ROUND(A1 DEFAULT 1.08, 4)        # uses 1.08 if A1 isn't ready

DEFAULT matches both BLANK and any error:* typeTag.

For longer chains, use WITH:

A1 = WITH rate = FX_RATE("EUR", "USD"), price = base * rate
THEN ROUND(price, 2) ELSE 0

The bindings list may span lines (when each , is at end-of-line), but THEN <expr> ELSE <fallback> must be on the same line.

If any step fails, WITH returns the ELSE value.

5. The ~= Operator In Detail

~= makes a cell lazy. Three things change:

1. No work is enqueued at deploy time. The cell sits in dirty state until something reads it.
2. Reads materialize on demand. A read triggers enqueue (if external) or evaluation (if local).
3. Caching is the same. Once computed, the value is cached until an input changes; after invalidation the cell goes back to dirty.

~= is not limited to external calls. It works for any expensive local computation:

A1 ~= LARGE_MATRIX_INVERT(B1:Z100)

But it shines for external functions because it eliminates the "deploy creates a thundering herd of jobs" problem.

6. Synthetic Symbols (Compiler Lowering)

When an external call is nested inside a larger formula, the compiler lowers it to a deterministic synthetic symbol:

# You write:
A1 = ROUND(FX_RATE("EUR", "USD") + 0.01, 4)
 
# The compiler emits (conceptually):
__ext__:A1:9f23c8 = FX_RATE("EUR", "USD")
A1 = ROUND(__ext__:A1:9f23c8 + 0.01, 4)

Consequences:

• The external call's result is cached independently of the wrapping expression.
• ROUND(...) re-runs cheaply when the cached value is reused.
• Retries and stale handling attach to the synthetic target, not the outer formula.

You will never write a synthetic symbol yourself. They appear in:

• API responses (status, jobs).
• Generated Lua source.
• Audit streams.

See docs/specs/external_functions.md for the deterministic naming rule.

7. Caching, TTL, And Refresh

Each external function declares a cache policy in its registry entry:

For the built-ins:

Status (v1): Cache policy is declared by every external function and propagated through the job envelope, but the workers and runtime do not currently consult these fields. There is no automatic background refresh, no TTL-based eviction, and no staleness-based blocking refresh. A value is recomputed only when an upstream dependency changes (or when an explicit resolve call arrives).

Treat the cache policy fields as a forward-compatible contract that tools may surface (and that future runtime versions will honor), not as an active behavior in v1.

In practice this means:

• Eager external cells (A1 = FX_RATE(...)) compute once at deploy time and stay stable until an input dependency changes.
• Lazy external cells (A2 ~= FX_RATE(...)) compute the first time they're read and cache thereafter.
• For "refresh on a schedule" semantics today, you would need a client-side mechanism (cron job, webhook trigger) that bumps an input the external depends on. There is no in-engine periodic refresh in v1.

8. Failure Handling

When the worker reports a failure, the runtime applies this policy:

1. If a stale cached value exists and the function allows fallback, the cell becomes stale with last_error populated.
2. Otherwise the cell becomes failed with last_error populated.

You can detect either via:

ISERROR(A1)            # TRUE if the cell holds an error value
A1 IS ERROR            # same, predicate form

Your model should never depend on an external value being immediately available. Always pair with DEFAULT, IFERROR, or WITH ... ELSE.

9. Worked Example

MODEL "External Enrichment"
RUNTIME "lua_generated"
VERSION "1.0.0"
 
# Inputs
A1 as currency = 250000
A2 = "EUR"
A3 = "USD"
 
# Eager external — queued at deploy
B1 = FX_RATE(base := A2, quote := A3)
 
# Lazy external — queued only when read
B2 ~= ML_SCORE([0.12, 0.18, 0.27, 0.43])
 
# Fallback-safe analytics
C1 as currency = ROUND(A1 * (B1 DEFAULT 1.08), 2)
C2 = B2 DEFAULT 0
C3 = C2 > 0.35 THEN "manual-review" ELSE "auto-approve"
 
END MODEL

While B1 is queued, C1 is computed using the fallback (1.08). When B1 becomes ready, C1 recomputes with the real rate.

B2 stays in dirty state until the API or another cell reads it — no work happens at deploy.

10. Operational Reads

To inspect status from outside the model:

# Resolve a single symbol
curl http://localhost:3000/api/models/my-model/resolve/B1
 
# Sample response (ready):
{
  "status": "ready",
  "value": { "kind": "number", "value": 1.0823, "typeTag": "fx_rate" },
  "request_rev": 5,
  "resolved_rev": 5
}
 
# Sample response (stale):
{
  "status": "stale",
  "value": { "kind": "number", "value": 1.0810, "typeTag": "fx_rate" },
  "request_rev": 6,
  "resolved_rev": 5,
  "pending_job_id": "job_01J..."
}

To inspect a specific job:

curl http://localhost:3000/api/models/my-model/jobs/job_01J...

11. Restrictions

12. See Also

• docs/specs/external_functions.md — formal queue, writeback, and revision contract.
• assignments.md — ~= operator details.
• errors.md — handling failed external calls.
• docs/specs/deployment_tiers.md — how workers are deployed in each tier.