Architecture

Jaiph is a workflow system with a TypeScript CLI and a JavaScript kernel (src/runtime/kernel/) that interprets the workflow AST in process — there is no separate “workflow shell” emitted for execution.

This page describes how Jaiph is built: repository layout of major subsystems, core components, compile and run pipelines, and runtime contracts (events, artifacts on disk, distribution). It is the map of the implementation. For workflow syntax and semantics, see the Language guide; this document stays on implementation boundaries.

Why this split: the transpiler turns each script block (and inline script bodies) into real files under scripts/ with a stable layout and JAIPH_SCRIPTS, while NodeWorkflowRuntime always executes from the AST (buildRuntimeGraph). That separation keeps bash entrypoints predictable for subprocesses without duplicating workflow logic in a second language.

For how to contribute — branches, test layers, E2E assertion policy, and bash harness details — see Contributing. For the *.test.jh language and test blocks, see Testing.

System overview

Workflow authors write .jh / .test.jh modules. The toolchain turns those files into validated modules plus extracted script files, then the same AST interpreter runs workflows whether you use local jaiph run, Docker, or jaiph test.

  1. Parse source into AST (the CLI parses once up front for jaiph run metadata such as runtime config; buildRuntimeGraph and transpilation use the same parser on disk contents).
  2. Compile-time validation (validateReferences, invoked from emitScriptsForModule / buildScripts()) runs before script extraction, not inside buildRuntimeGraph() (the graph loader only parses modules and follows imports). The jaiph compile command walks the same import closure but runs validateReferences only: it parses each reachable module on disk and does not emit scripts/ (no buildScriptFiles / buildScripts), does not invoke buildRuntimeGraph(), and never spawns the workflow runner (src/cli/commands/compile.ts). For a directory argument it discovers *.jh via walkjhFiles, which skips *.test.jh; to validate a test module, pass that file explicitly. Imported modules in the closure are still validated recursively either way.
  3. CLI (dist/src/cli.js via npm, or a Bun-compiled dist/jaiph binary) prepares script executables (scripts-only), then spawns a detached child that loads node-workflow-runner.js. That child calls buildRuntimeGraph() and runs NodeWorkflowRuntime. The child’s interpreter is process.execPath of the CLI process (Node when you run node dist/src/cli.js, the standalone Bun binary when you run dist/jaiph). Script steps execute as managed subprocesses; prompt, inbox I/O, and event/summary emission are handled by the kernel under src/runtime/kernel/.
  4. Stream live events to the CLI and persist durable run artifacts.

Interactive jaiph run parses __JAIPH_EVENT__ lines from the runner’s stderr, renders the progress tree, and runs hooks. jaiph run --raw skips that shell: the child uses inherited stdio so events still land on stderr unchanged — used when embedding Jaiph or when the host wraps a container (see CLI — jaiph run and Sandboxing — Docker container isolation).

All orchestration — local jaiph run, jaiph test, and Docker jaiph run — uses the Node workflow runtime (AST interpreter). Docker containers run the same node-workflow-runner process with the compiled JS source tree and scripts mounted read-only.

Core components

Runtime vs CLI responsibilities

Runtime responsibilities (Node workflow runtime)

CLI responsibilities

Contracts

Channel transport remains file/queue based in runtime inbox logic.

Durable artifact layout

For an onboarding-style description of the same paths (what to expect in a repo, what to ignore in git), see Runtime artifacts.

The runtime persists step captures and the event timeline under a UTC-dated hierarchy:

.jaiph/runs/
  <YYYY-MM-DD>/                       # UTC date (see NodeWorkflowRuntime)
    <HH-MM-SS>-<source-basename>/       # UTC time + JAIPH_SOURCE_FILE or entry basename
      000001-module__step.out          # stdout capture per step (6-digit seq prefix)
      000001-module__step.err          # stderr capture (may be empty)
      artifacts/                       # user-published files (JAIPH_ARTIFACTS_DIR); created at run start
      inbox/                           # audit copies of routed channel payloads (optional)
      heartbeat                        # liveness: epoch ms, refreshed about every 10s
      return_value.txt                 # when `jaiph run` default workflow returns a value (success only)
      run_summary.jsonl                # durable event timeline

Step sequence numbers are monotonic and unique per run: RuntimeEventEmitter allocates them in memory (allocStepSeq) when opening each step’s capture files (%06d-<safe_name>.out|.err). There is no .seq file in the run directory.

Channels and hooks in context

Channels are validated at compile time (validateReferences / send RHS rules) and executed via in-memory queue and dispatch in the Node runtime; durable inbox/ files under the run directory appear only for routed sends (audit — see Inbox & Dispatch). Hooks are CLI-only: they load from hooks.json and run as shell commands with JSON on stdin, driven by the same __JAIPH_EVENT__ stream as the progress UI — see Hooks.

Test runner integration (*.test.jh in the kernel)

How jaiph test wires into the same stack as jaiph run: *.test.jh files are parsed in the CLI; runTestFile() drives blocks in-process. buildRuntimeGraph(testFile) is called once per runTestFile invocation and the resulting graph is reused across all blocks and test_run_workflow steps (the import closure is constant for a given test file within a single process run). Each test_run_workflow step resolves mocks against that cached graph, then constructs NodeWorkflowRuntime with mockBodies / mock prompt env, passing suppressLiveEvents: true so RuntimeEventEmitter skips writing __JAIPH_EVENT__ lines to stderr while still appending run_summary.jsonl for that run. Without this flag, every workflow event would print to the test process’s stderr and swamp node --test reporter output. Mock prompts, workflows, rules, and scripts are supported through the runtime’s mock infrastructure.

Before that, the CLI prepares script executables via buildScripts(testFileAbs, tmpDir, workspaceRoot) — the same buildScripts helper as jaiph run, with the test file as the entrypoint. That walks the test module and its import closure (transitive import edges), runs validateReferences / emitScriptsForModule per reachable file, and writes scripts/ so imported workflows have paths under JAIPH_SCRIPTS. Unrelated *.jh files elsewhere in the repo are not compiled unless imported.

Authoring rules, fixtures, and mock syntax for *.test.jh are documented in Testing, not here.

CLI progress reporting pipeline

The progress UI combines a static step tree derived from the workflow AST (src/cli/run/progress.ts) with live updates from the runtime event stream. Event wiring: src/cli/run/events.ts and src/cli/run/stderr-handler.ts parse __JAIPH_EVENT__ lines; src/cli/run/emitter.ts bridges into the renderer. Line-oriented formatting (formatStartLine, formatHeartbeatLine, formatCompletedLine) lives primarily in src/cli/run/display.ts, which shares some display helpers with progress.ts. Async branch numbering (subscript ₁₂₃… prefixes) is driven by async_indices on step and log events — the runtime propagates a chain of 1-based branch indices through AsyncLocalStorage, and the stderr handler renders them at the appropriate indent level. const steps whose value is a match_expr are walked for nested run/ensure arms; matched targets appear as child items in the step tree (e.g. ▸ script safe_name under the const row). This pipeline does not apply to jaiph run --raw.

Distribution: Node vs Bun standalone

Mermaid architecture diagram

flowchart TD
    U[User / CI] --> CLI[CLI: Node or Bun jaiph]

    subgraph Transpile["Per-module: emitScriptsForModule()"]
        PARSE[parsejaiph]
        VAL[validateReferences]
        EMIT[Emit atomic script files under scripts/]
        PARSE --> VAL
        VAL -->|compile errors| ERR[Deterministic compile errors]
        VAL --> EMIT
    end

    CLI -->|jaiph run| BS1[buildScripts]
    BS1 --> Transpile

    CLI -->|jaiph test| BS2[buildScripts(entry .test.jh)]
    BS2 --> Transpile
    BS2 --> TR[Node Test Runner in-process]

    Transpile -->|jaiph run local| RW[Node workflow runner child]
    Transpile -->|jaiph run Docker| DC[Container runs node-workflow-runner]

    RW --> G[buildRuntimeGraph parse-only + imports]
    G --> GRAPH[RuntimeGraph]
    RW --> RT[NodeWorkflowRuntime]
    RT --> GRAPH

    DC --> G
    DC --> RT

    TR -->|test_run_workflow| G
    TR --> RT

    RT -->|script steps| SCRIPT[Managed script subprocesses]
    RT -->|prompt steps| KERNEL[Kernel libs: prompt, events, inbox, stream, schema, mock]

    RT -->|live events| EV["__JAIPH_EVENT__ stderr only"]
    EV --> CLI
    CLI --> PR[Progress rendering]

    RT -->|channels files / queues| INBOX[Inbox under .jaiph/runs]
    RT -->|durable artifacts| SUM[.jaiph/runs + run_summary.jsonl]
    CLI --> HK[Hook dispatcher via event stream]
    HK --> HPROC[Hook shell commands]

Emit artifacts: buildScripts() persists only extracted script bodies under scripts/. No workflow-level shell modules or jaiph_stdlib.sh are produced.

Sequence diagram: regular flow (*.jh)

Interactive jaiph run (no --raw): banner, progress tree, hooks, and PASS/FAIL footer.

sequenceDiagram
    participant User
    participant CLI as CLI jaiph run
    participant Prep as buildScripts
    participant TF as emitScriptsForModule per module
    participant Runner as node-workflow-runner
    participant Graph as buildRuntimeGraph
    participant Runtime as NodeWorkflowRuntime
    participant Kernel as JS kernel
    participant Report as Artifacts (.jaiph/runs)

    User->>CLI: jaiph run main.jh args...
    Note over CLI: parse once for metadata config only
    CLI->>Prep: buildScripts(input)
    Prep->>TF: loop: parse + validateReferences + emit
    TF-->>Prep: scripts/ atomic only
    Prep-->>CLI: scriptsDir + env JAIPH_SCRIPTS
    alt local
        CLI->>Runner: spawn detached node-workflow-runner
    else Docker
        CLI->>CLI: prepareImage (pull --quiet + verify jaiph)
        Note over CLI: runs before banner so pull doesn't interleave
        CLI->>Runner: spawn container running node-workflow-runner
        Note over CLI: CLI parses events on stderr only
    end
    Runner->>Graph: buildRuntimeGraph(sourceAbs) parse-only
    Graph-->>Runner: RuntimeGraph
    Runner->>Runtime: runDefault(run args)
    Runtime->>Kernel: prompt / managed scripts / emit / inbox
    Runtime-->>CLI: __JAIPH_EVENT__ on stderr
    Runtime->>Report: run_summary.jsonl + step artifacts
    Runner-->>CLI: exit + meta file with run_dir paths
    CLI-->>User: live progress
    CLI-->>User: PASS/FAIL

Docker: the inner container command is jaiph run --raw … (see Sandboxing): no banner or progress UI inside the container; __JAIPH_EVENT__ lines still appear on stderr for the host CLI to parse.

Sequence diagram: jaiph test flow

sequenceDiagram
    participant User
    participant CLI as CLI jaiph test
    participant Parser as parsejaiph
    participant Prep as buildScripts(test file)
    participant TestRunner as runTestFile / runTestBlock
    participant Graph as buildRuntimeGraph
    participant Runtime as NodeWorkflowRuntime
    participant Report as Artifacts

    User->>CLI: jaiph test flow.test.jh
    CLI->>Parser: parse test file
    Parser-->>CLI: jaiphModule + tests[] blocks
    CLI->>Prep: buildScripts(test path, tmp) import closure
    Prep-->>CLI: scriptsDir
    CLI->>TestRunner: runTestFile(test path workspace scriptsDir blocks)
    TestRunner->>Graph: buildRuntimeGraph(test file) once per file
    Graph-->>TestRunner: RuntimeGraph cached
    loop each test block
        TestRunner->>TestRunner: mocks / shell steps / expectations
        opt test_run_workflow step
            TestRunner->>Runtime: new runtime mockBodies from block (reuses cached graph)
            Runtime->>Runtime: runNamedWorkflow(ref args)
            Runtime-->>TestRunner: status output returnValue error
        end
    end
    Runtime->>Report: artifacts when workflows ran
    TestRunner-->>CLI: aggregate PASS/FAIL
    CLI-->>User: exit code

Summary

© 2026 Jaiph — Open Source · Powerful · Friendly!