Evaluation

Kookie Flow can run your graph, but it never knows what a node does. You give it one function, onEvaluate, that turns an entity's inputs into its outputs. The canvas decides when to call it, carries each result along the wires, cancels runs whose inputs have changed, and draws each entity's progress on its outline.

onEvaluate receives an entity's id, its type, its inputs and a context object. Return the outputs as an object keyed by output socket id. The function can be synchronous or async.

'use client';import { useEffect, useRef } from 'react';import { KookieFlow, useGraph, type FlowObject, type KookieFlowInstance, type OnEvaluate } from '@kushagradhawan/kookie-flow';const onEvaluate: OnEvaluate = async (id, type, inputs) => {  switch (type) {    case 'number':      return { out: Number(inputs.value ?? 0) };    case 'add':      return { sum: Number(inputs.a ?? 0) + Number(inputs.b ?? 0) };    case 'uppercase':      return { text: String(inputs.text ?? '').toUpperCase() };  }};export function Board({ graph }: { graph: FlowObject }) {  const flowRef = useRef<KookieFlowInstance>(null);  const { entities, edges, onEntitiesChange, onEdgesChange, onConnect } = useGraph({    initialEntities: graph.entities,    initialEdges: graph.edges,  });  // A graph you mount with has never run, so run all of it once.  useEffect(() => {    flowRef.current?.evaluateAll();  }, []);  return (    <KookieFlow      ref={flowRef}      entities={entities}      edges={edges}      onEntitiesChange={onEntitiesChange}      onEdgesChange={onEdgesChange}      onConnect={onConnect}      onEvaluate={onEvaluate}    />  );}

The function's type is OnEvaluate, and it takes four arguments.

EvaluationContext has three fields.

What you return decides what the outputs hold:

• An object writes each key to the output socket with that id. The values can be anything: a number, a string, an object, an image URL.
• An output you leave out keeps the value it already had. Returning nothing keeps all of them, and the run still succeeds.
• A thrown error fails the run. The entity's status becomes error, and the error's message is shown under the node.

The canvas works out every input at the moment a run starts, one socket at a time.

1. A connected input takes the value on the output at the other end of its wire. It's undefined when the entity upstream hasn't produced that output yet. When two wires reach one input, the first of them in edges is used.
2. An unconnected input takes the value its widget shows. That's the value a person just set, even before your onWidgetChange has written it back. After that comes data.values[socketId] on the entity, and then the socket's defaultValue.

An app inspector can update evaluation inputs by writing to data.values. The Sockets and widgets chapter shows that write.

Output values are kept by the canvas, not in your entities, so a result never passes through your React state. Read one with getSocketValue(entityId, socketId) on the ref.

Every entity has a status, of type EvaluationStatus: idle, dirty, running, success or error. An entity is dirty when its inputs have changed and nothing has run on them yet.

A change marks the entity it touches, and everything downstream of it, dirty at once. The canvas then starts each dirty entity whose inputs are ready, which means nothing it reads from is dirty, running or failed. A chain therefore runs front to back, two branches run at the same time, and a node where two branches meet runs once, after both have finished. A pass starts once the code that made the change has returned, so a paste of ten nodes starts one pass rather than ten.

These changes mark an entity dirty:

• A person changes a widget on it.
• Your app gives its data.values a new object, as an undo or a preset does. Changing a value inside the existing object isn't noticed.
• A wire into it is added, removed or rewired, including edits that keep the edge id.
• Its resolved type inputs or outputs change.
• It's muted or unmuted.
• It arrives through addElements, insertOnEdge or a replacement entities array.

Moving, resizing, selecting and renaming an entity don't mark it. To mark entities after a change the canvas can't see, call markDirty on the store from a component inside KookieFlow.

The graph you mount with starts idle until you mark it or run it from code. That's why the board above calls evaluateAll() once after it mounts. Newly added computation nodes are marked dirty automatically; manual types still wait for their trigger. Content kinds such as images, text and frames are excluded from evaluation.

A failed entity holds its chain. Everything downstream stays dirty rather than running on an output the failed run never replaced. The entity runs again when its inputs change, or when you call evaluate on it. Entities that form a loop can never be ready, so they stay dirty.

Set evaluation: 'manual' on an entity type to make its entities wait. A change still marks a manual entity dirty, but it doesn't run, and everything downstream of it stays dirty too. The type is EvaluationMode, and its other value, 'reactive', is the default.

import type { EntityTypeDefinition } from '@kushagradhawan/kookie-flow';const entityTypes: Record<string, EntityTypeDefinition> = {  blur: {    type: 'blur',    label: 'Blur',    inputs: [{ id: 'image', name: 'Image', type: 'image' }, { id: 'radius', name: 'Radius', type: 'float' }],    outputs: [{ id: 'image', name: 'Image', type: 'image' }],  },  generate: {    type: 'generate',    label: 'Generate',    evaluation: 'manual',    inputs: [{ id: 'prompt', name: 'Prompt', type: 'string' }],    outputs: [{ id: 'image', name: 'Image', type: 'image' }],  },};

Make a type manual when a run is slow or costs money: a model call, a render, a paid API. This is a judgment about what people expect. Dragging a slider upstream should update the cheap nodes on every step, not start a generation on every step.

evaluate(id) runs a manual entity, and reactive entities downstream follow it as usual. evaluateDirty() and evaluateAll() run manual entities too.

restoreAll() is for opening a saved graph. Reactive entities run, and manual ones are called with ctx.restore set to true. Return a result you already have, such as one your server stored, or return nothing. An entity that returns nothing stays dirty, and so does everything downstream of it, until something runs it for real.

'use client';import { useCallback, useEffect, useRef } from 'react';import { Button, Flex, Stack, Text } from '@kushagradhawan/kookie-ui-react';import { KookieFlow, useGraph, type Edge, type Entity, type EntityTypeDefinition, type KookieFlowInstance, type OnEvaluate } from '@kushagradhawan/kookie-flow';const float = (id: string, name: string) => ({ id, name, type: 'float', min: 0, max: 10, step: 1 });const entityTypes: Record<string, EntityTypeDefinition> = {  number: { type: 'number', label: 'Number', inputs: [float('value', 'Value')], outputs: [float('out', 'Out')] },  add: { type: 'add', label: 'Add', inputs: [float('a', 'A'), float('b', 'B')], outputs: [float('sum', 'Sum')] },  // Manual: a change upstream marks it stale, and it waits to be run.  render: { type: 'render', label: 'Render', evaluation: 'manual', inputs: [float('in', 'In')], outputs: [float('out', 'Out')] },};const node = (id: string, type: string, x: number, y: number, value?: number): Entity =>  ({ id, type, position: { x, y }, data: value === undefined ? {} : { values: { value } } });const wire = (source: string, sourceSocket: string, target: string, targetSocket: string): Edge =>  ({ id: `${source}-${target}`, source, sourceSocket, target, targetSocket });const wait = (ms: number, signal: AbortSignal) => new Promise((resolve, reject) => {  const timer = setTimeout(resolve, ms);  signal.addEventListener('abort', () => { clearTimeout(timer); reject(signal.reason); });});const onEvaluate: OnEvaluate = async (id, type, inputs, ctx) => {  const n = (key: string) => Number(inputs[key] ?? 0);  if (type === 'number') return { out: n('value') };  if (type === 'add') return { sum: n('a') + n('b') };  if (type === 'render') {    for (let step = 1; step <= 20; step++) {      await wait(100, ctx.signal); // a changed input rejects here, and the run is dropped      ctx.progress(step / 20);    }    return { out: n('in') * 10 };  }};export default function EvaluationExample() {  const { entities, edges, onEntitiesChange, onEdgesChange, onConnect } = useGraph({    initialEntities: [node('a', 'number', 0, 0, 2), node('b', 'number', 0, 160, 3), node('add', 'add', 300, 80), node('render', 'render', 600, 80)],    initialEdges: [wire('a', 'out', 'add', 'a'), wire('b', 'out', 'add', 'b'), wire('add', 'sum', 'render', 'in')],  });  const flowRef = useRef<KookieFlowInstance>(null);  const readoutRef = useRef<HTMLSpanElement>(null);  const frameRef = useRef(0);  // Once a frame, into the DOM rather than into state, so dragging a slider re-renders nothing here.  const onStatusChange = useCallback(() => {    if (frameRef.current) return;    frameRef.current = requestAnimationFrame(() => {      frameRef.current = 0;      const flow = flowRef.current;      if (!flow || !readoutRef.current) return;      const [sum, out] = [flow.getSocketValue('add', 'sum'), flow.getSocketValue('render', 'out')];      readoutRef.current.textContent = `Sum ${sum ?? '–'} · Render ${out ?? '–'} (${flow.getEvaluationStatus('render')})`;    });  }, []);  useEffect(() => {    flowRef.current?.fitView({ padding: 40, maxZoom: 1 });    // Nothing runs on mount. Run the two inputs once: Add follows them, Render waits.    flowRef.current?.evaluate('a');    flowRef.current?.evaluate('b');    return () => cancelAnimationFrame(frameRef.current);  }, []);  return (    <Stack gap="3">      <Flex gap="3" align="center">        <Button size="2" emphasis="loud" onClick={() => flowRef.current?.evaluate('render')}>Run Render</Button>        <Text size="2" ref={readoutRef}>Sum – · Render –</Text>      </Flex>      <div style={{ width: '100%', height: 360 }}>        <KookieFlow          ref={flowRef}          entities={entities}          edges={edges}          entityTypes={entityTypes}          onEntitiesChange={onEntitiesChange}          onEdgesChange={onEdgesChange}          onConnect={onConnect}          onEvaluate={onEvaluate}          onStatusChange={onStatusChange}        />      </div>    </Stack>  );}

Drag either slider. Number and Add run on every step, while Render dims and waits. Press Run Render and its outline fills as the run reports progress. Move a slider during the run and the run is cancelled. Render goes back to waiting, because a manual entity whose inputs change mid-run waits for the next trigger rather than starting again.

When your entityTypes change so that a manual type becomes reactive, the entities waiting under it run.

When an entity's inputs change during a run, the canvas aborts ctx.signal, sets the entity back to dirty, and ignores whatever the run later returns or throws. A reactive entity then runs again with the new inputs. Calling evaluate on an entity that's already running also cancels the run in progress and starts over. Deleting an entity cancels its run and releases its outputs, and unmounting KookieFlow cancels every run.

You don't have to honour the signal, because a cancelled result is thrown away either way. Honour it anyway when the work is expensive, so nobody pays for a result that will never be used. Pass it to fetch, or check signal.aborted inside a loop.

import type { OnEvaluate } from '@kushagradhawan/kookie-flow';const onEvaluate: OnEvaluate = async (id, type, inputs, ctx) => {  if (type === 'weather') {    const city = encodeURIComponent(String(inputs.city ?? ''));    const response = await fetch(`/api/weather?city=${city}`, { signal: ctx.signal });    if (!response.ok) throw new Error(`The weather service answered ${response.status}`);    return { forecast: await response.json() };  }};

A cancelled fetch rejects with an AbortError. The canvas ignores errors from cancelled runs, so you don't need to catch it.

Call ctx.progress(fraction) with a number from 0 to 1. The entity's outline fills with the accent colour to that fraction, starting at the top. The drawn outline catches up with each report smoothly, so reporting every tenth still reads as one continuous motion. A run that never reports shows a short arc travelling round the outline instead.

import type { OnEvaluate } from '@kushagradhawan/kookie-flow';const onEvaluate: OnEvaluate = async (id, type, inputs, ctx) => {  if (type === 'translate') {    const lines = String(inputs.text ?? '').split('\n');    const translated: string[] = [];    for (const line of lines) {      const response = await fetch('/api/translate', { method: 'POST', body: line, signal: ctx.signal });      translated.push(await response.text());      ctx.progress(translated.length / lines.length);    }    return { text: translated.join('\n') };  }};

A value below 0 or above 1 is clamped. Reports from a cancelled run are ignored.

Each status draws on the entity's own outline.

SUCCESS_HOLD_MS is exported, and it's 1500 milliseconds. The success outline fades rather than staying on, because an outline held in the accent colour is also what a selected node looks like. To keep a node marked as done, set its status yourself, as described below.

onStatusChange(entityId, status, message) is called on every transition the canvas makes, including the return to idle after the success hold. message is set when the status is error. Its type is OnStatusChange. To ask about one entity at any moment, call getEvaluationStatus(entityId) on the ref.

A slider drag produces several transitions for every step, so keep onStatusChange cheap. The example above schedules one update for the next animation frame and writes its readout into a DOM node rather than into React state, so the drag doesn't re-render the component.

The store holds more detail as an EvaluationRecord, which you read with getEvaluationRecord(entityId) through the flow store. It returns undefined for an entity that has never been marked.

data.status on an entity draws a status of your own, and data.statusMessage shows a line of text under the node. The type of data.status is EntityStatus: 'dirty', 'error', 'warning', 'running' or 'success'. Use it for work that happens outside onEvaluate, such as a job on your server.

import type { Entity } from '@kushagradhawan/kookie-flow';const training: Entity = {  id: 'train',  type: 'default',  position: { x: 0, y: 0 },  data: { label: 'Train model', status: 'warning', statusMessage: 'Dataset is 3 days old' },};

Your status wins over the canvas's while it's set, and the canvas's progress isn't drawn for that entity. 'warning' draws an amber outline, and the canvas never sets it itself. Your 'success' draws a steady green outline, because nothing ran that could fade it. Your 'running' draws the travelling arc. statusMessage shows whatever the status is, and it replaces the canvas's error message. Setting data.status changes only what is drawn: the entity still runs, and onStatusChange still reports the canvas's own status.

These methods are on the KookieFlowInstance you get from ref.

evaluate doesn't wait for dirty entities upstream. It runs with the outputs they hold right now, which may be out of date. Call evaluateDirty() to run a whole stale chain in order. On the other hand, evaluateDirty() doesn't retry a failed entity, because a failed entity isn't dirty. Call evaluate on it instead.

Use setSocketValue for a result that already exists: one you saved with the graph, or one another system produced. A preview band draws whatever value sits on its socket, so this also fills a preview in a graph that never evaluates. The Images, video and models chapter covers previews.

'use client';import { useState, type RefObject } from 'react';import type { KookieFlowInstance } from '@kushagradhawan/kookie-flow';export function RunStaleButton({ flowRef }: { flowRef: RefObject<KookieFlowInstance | null> }) {  const [running, setRunning] = useState(false);  const run = async () => {    setRunning(true);    await flowRef.current?.evaluateDirty();    setRunning(false);  };  return (    <button type="button" disabled={running} onClick={run}>      {running ? 'Running…' : 'Run stale nodes'}    </button>  );}

Comments, text, images, video, models and ink compute nothing, so adding one never marks it dirty. evaluateAll() marks every entity in the graph, though, so write onEvaluate to return nothing for a type it doesn't compute, rather than throwing. The switch in the first example does this: a type with no case falls through, returns nothing, and succeeds.

Until you pass onEvaluate, changed entities stay dirty. Reactive entities that are dirty when the function arrives run then.

A muted entity doesn't call onEvaluate. Its input values pass straight to its outputs by position: the first input to the first output, the second to the second, and so on. Muting and unmuting both mark the entity dirty, so what it feeds runs again. Mute an entity with muteEntity(entityId) and unmute it with unmuteEntity(entityId) on the flow store.

Deleting or bypassing an entity clears its runtime mute state along with its run and outputs. Reusing the id starts unmuted.