Packet — P1.1.2 Determinism Verification Harness (R83.A)

1. Plan summary

Two files, single implementation commit, single verification commit.

# Path Commit
1 src/domains/rules/determinism.ts feat(r83-a-determinism): P1.1.2 determinism verification harness
2 src/__tests__/domains/rules/determinism.test.ts (same commit)

Zero schema, zero MCP wiring, zero docs outside the 5-chain quartet. Imports limited to src/domains/rules/integer-math.js (for tests) + node built-ins (node:fs/promises, node:path, node:url) in the test file.

2. Implementation sketch — determinism.ts

/**
 * Colibri — κ Rule Engine — Determinism Verification Harness (P1.1.2).
 *
 * Pure, deterministic guardrail library for every downstream κ sub-task.
 * Wraps a function in an N-run equality check and exposes a regex-based
 * forbidden-op scanner over function source. Companion to the P1.1.1
 * basis-point arithmetic helpers — callers use `assertDeterministic` to
 * prove their code is floor-monotone and RNG-free.
 *
 * Canonical references:
 *   - docs/contracts/r83-a-determinism-contract.md
 *   - docs/3-world/physics/laws/rule-engine.md §Forbidden operations
 *   - docs/guides/implementation/task-prompts/p1.1-kappa-rule-engine.md §P1.1.2
 *
 * Invariants (contract §3):
 *   I1. The library uses no Math.*, Date.*, async, RNG, float literals.
 *       (The scanner, turned on itself, returns [] for every export body.)
 *   I2. assertDeterministic is order-insensitive across iterations ≥ 2.
 *   I3. Error messages include label + iteration index + both values
 *       truncated to 120 chars.
 *   I4. inspectFunctionForbidden is pure (no globals, no closures).
 *   I5. Float-literal regex rejects 3.14 but not 1n, 100, 1_000_000.
 *
 * Outbound imports: NONE. Self-standing to preserve I1.
 */

// ---------- Typed error ----------

export class DeterminismError extends Error {
  override readonly name = 'DeterminismError';
  constructor(message: string) { super(message); }
}

// ---------- Forbidden-op pattern manifest (contract §4) ----------

const FORBIDDEN_PATTERNS: readonly { readonly pattern: RegExp; readonly label: string }[] = [
  { pattern: /\bMath\.[A-Za-z_]\w*/g,                       label: 'Math.*' },
  { pattern: /\bDate\.[A-Za-z_]\w*/g,                       label: 'Date.*' },
  { pattern: /\bnew\s+Date\b/g,                             label: 'new Date' },
  { pattern: /\b(?:setTimeout|setInterval|setImmediate)\b/g, label: 'timer' },
  { pattern: /\b(?:fetch|XMLHttpRequest)\b/g,               label: 'network' },
  { pattern: /\brequire\s*\(\s*['"](?:fs|node:fs)['"]/g,    label: 'require(fs)' },
  { pattern: /\bfrom\s+['"](?:fs|node:fs)['"]/g,            label: 'import fs' },
  { pattern: /\bcrypto\.[A-Za-z_]\w*/g,                     label: 'crypto.*' },
  { pattern: /\bprocess\.(?:hrtime|nextTick)\b/g,           label: 'process.time' },
  { pattern: /\bawait\b/g,                                  label: 'await' },
  { pattern: /\basync\s+(?:function|\()/g,                  label: 'async fn' },
  { pattern: /(?<![0-9n])\b\d+\.\d+\b/g,                    label: 'float literal' },
  { pattern: /\[native code\]/g,                            label: 'native code' },
];

// ---------- inspectFunctionForbidden ----------

export function inspectFunctionForbidden(fn: Function): readonly string[] {
  const source = fn.toString();
  const hits = new Set<string>();
  for (const { pattern } of FORBIDDEN_PATTERNS) {
    const matches = source.match(pattern);
    if (matches) {
      for (const m of matches) { hits.add(m); }
    }
  }
  return [...hits].sort();
}

// ---------- assertNoForbiddenOps ----------

export function assertNoForbiddenOps(
  fn: Function,
  opts?: { label?: string },
): void {
  const hits = inspectFunctionForbidden(fn);
  if (hits.length === 0) { return; }
  const prefix = opts?.label ? `[${opts.label}] ` : '';
  throw new DeterminismError(
    `${prefix}forbidden operations detected: ${hits.join(', ')}`,
  );
}

// ---------- deepEqualDeterministic ----------

export function deepEqualDeterministic(a: unknown, b: unknown): boolean {
  if (Object.is(a, b)) { return true; }
  if (typeof a === 'bigint' && typeof b === 'bigint') { return a === b; }
  if (a instanceof Error && b instanceof Error) {
    return a.name === b.name && a.message === b.message;
  }
  if (Array.isArray(a) && Array.isArray(b)) {
    if (a.length !== b.length) { return false; }
    for (let i = 0; i < a.length; i++) {
      if (!deepEqualDeterministic(a[i], b[i])) { return false; }
    }
    return true;
  }
  if (
    typeof a === 'object' && a !== null &&
    typeof b === 'object' && b !== null &&
    a.constructor === Object && b.constructor === Object
  ) {
    const ak = Object.keys(a); const bk = Object.keys(b);
    if (ak.length !== bk.length) { return false; }
    for (const k of ak) {
      if (!Object.prototype.hasOwnProperty.call(b, k)) { return false; }
      if (!deepEqualDeterministic(
        (a as Record<string, unknown>)[k],
        (b as Record<string, unknown>)[k],
      )) { return false; }
    }
    return true;
  }
  return false;
}

// ---------- assertDeterministic ----------

function renderValue(v: unknown): string {
  // Bigint-safe inspect; caps at 120 chars.
  try {
    const s = typeof v === 'bigint'
      ? `${v.toString()}n`
      : JSON.stringify(v, (_k, val) => typeof val === 'bigint' ? `${val}n` : val);
    const str = s === undefined ? String(v) : s;
    return str.length > 120 ? `${str.slice(0, 117)}...` : str;
  } catch {
    return String(v);
  }
}

type Capture<R> =
  | { threw: true; name: string; message: string; error: unknown }
  | { threw: false; value: R };

function capture<A extends readonly unknown[], R>(
  fn: (...args: A) => R,
  args: A,
): Capture<R> {
  try {
    return { threw: false, value: fn(...args) };
  } catch (error) {
    const name = (error as { name?: unknown })?.name;
    const message = (error as { message?: unknown })?.message;
    return {
      threw: true,
      name: typeof name === 'string' ? name : 'Error',
      message: typeof message === 'string' ? message : String(error),
      error,
    };
  }
}

export function assertDeterministic<A extends readonly unknown[], R>(
  fn: (...args: A) => R,
  args: A,
  opts?: { iterations?: number; label?: string },
): R {
  const iterations = opts?.iterations ?? 10;
  const prefix = opts?.label ? `[${opts.label}] ` : '';
  if (iterations < 1) {
    throw new DeterminismError(`${prefix}iterations must be >= 1 (got ${iterations})`);
  }
  const ref = capture(fn, args);
  for (let i = 2; i <= iterations; i++) {
    const cur = capture(fn, args);
    if (ref.threw !== cur.threw) {
      const refDesc = ref.threw
        ? `threw ${ref.name}: ${ref.message}`
        : `returned ${renderValue(ref.value)}`;
      const curDesc = cur.threw
        ? `threw ${cur.name}: ${cur.message}`
        : `returned ${renderValue(cur.value)}`;
      throw new DeterminismError(
        `${prefix}determinism violation at run ${i}: run 1 ${refDesc}; run ${i} ${curDesc}`,
      );
    }
    if (ref.threw && cur.threw) {
      if (ref.name !== cur.name || ref.message !== cur.message) {
        throw new DeterminismError(
          `${prefix}determinism violation at run ${i}: ` +
          `run 1 threw ${ref.name}: ${ref.message}; ` +
          `run ${i} threw ${cur.name}: ${cur.message}`,
        );
      }
      continue;
    }
    // Both returned.
    if (!ref.threw && !cur.threw) {
      if (!deepEqualDeterministic(ref.value, cur.value)) {
        throw new DeterminismError(
          `${prefix}determinism violation at run ${i}: ` +
          `run 1 returned ${renderValue(ref.value)}; ` +
          `run ${i} returned ${renderValue(cur.value)}`,
        );
      }
    }
  }
  if (ref.threw) {
    throw ref.error;
  }
  return ref.value;
}

Notes on sketch:

  • The Function type for inspectFunctionForbidden / assertNoForbiddenOps is deliberate. TypeScript’s Function is the only type that matches all callable shapes (arrow, function expression, class method). Accepting a narrower (...a: any[]) => unknown would reject class-method refs. We’ll silence @typescript-eslint/ban-types / no-unsafe-function-type if the linter trips.
  • renderValue is bigint-safe because JSON.stringify crashes on bigint without a replacer. The replacer tags bigints as "123n" so error messages stay readable.
  • capture returns a tagged union that assertDeterministic dispatches on. tsc --strict --exactOptionalPropertyTypes requires every branch of the outer if/else to return in the same void/value shape — the sketch respects this.
  • The float-literal regex uses a negative look-behind (?<![0-9n]) so 1n and 1_000_000 are not matched — _ is not \d, n is excluded. NodeNext / ES2022 supports look-behind.

3. Implementation sketch — determinism.test.ts

/**
 * Tests for κ Determinism Verification Harness (P1.1.2 — R83.A).
 *
 * Coverage: every branch of every export in src/domains/rules/determinism.ts.
 * Target: 100% branches per contract AC#9.
 *
 * Acceptance criteria traced to
 *   docs/contracts/r83-a-determinism-contract.md §6 (AC#1 – AC#10).
 */

import { readdir, readFile } from 'node:fs/promises';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';

import {
  assertDeterministic,
  assertNoForbiddenOps,
  DeterminismError,
  deepEqualDeterministic,
  inspectFunctionForbidden,
} from '../../../domains/rules/determinism.js';
import {
  apply_bps,
  bps_div,
  bps_mul,
  decay,
  DivisionByZeroError,
  safe_div,
  safe_mul,
} from '../../../domains/rules/integer-math.js';

// ---------- DeterminismError (AC#1) ----------

describe('DeterminismError', () => {
  it('is a named Error subclass', () => {  });
  it('preserves message', () => {  });
});

// ---------- assertDeterministic — happy path (AC#2) ----------

describe('assertDeterministic — happy path over P1.1.1 helpers', () => {
  it('bps_mul — 10 iterations on (1000n, 500n) all return 50n', () => {  });
  it('bps_div — 10 iterations on (5000n, 2500n) all return 20000n', () => {  });
  it('apply_bps — 10 iterations on (1000n, 150n) all return 985n', () => {  });
  it('decay — 10 iterations on (1000n, 150n, 2n) all return 970n', () => {  });
  it('safe_mul — 10 iterations on (2n, 3n) all return 6n', () => {  });
  it('safe_div — 10 iterations on (7n, 2n) all return 3n', () => {  });
  it('returns first-call result', () => {  });
});

// ---------- assertDeterministic — throw-path ----------

describe('assertDeterministic — consistent throws are deterministic', () => {
  it('bps_div with bps=0n throws DivisionByZeroError consistently', () => {
    expect(() => assertDeterministic(bps_div, [100n, 0n] as const))
      .toThrow(DivisionByZeroError);
  });
  it('decay with negative epochs throws UnderflowError consistently', () => {  });
  it('safe_mul overflow throws OverflowError consistently', () => {  });
});

// ---------- assertDeterministic — FAIL detection (AC#3) ----------

describe('assertDeterministic — fails on non-deterministic fn', () => {
  it('detects Math.random drift', () => {
    const mock = () => Math.random();
    expect(() => assertDeterministic(mock, [] as const))
      .toThrow(DeterminismError);
  });
  it('detects Date.now drift', () => {  });
  it('detects throw/return mixing', () => {
    let c = 0;
    const mock = () => { c++; if (c > 1) throw new Error('x'); return 1; };
    expect(() => assertDeterministic(mock, [] as const))
      .toThrow(DeterminismError);
  });
  it('detects throw-type drift (different error name)', () => {  });
  it('detects throw-message drift (same name, different message)', () => {  });
  it('detects return-then-throw direction', () => {  });
});

// ---------- assertDeterministic — label + iterations (AC#10 branches) ----------

describe('assertDeterministic — options', () => {
  it('prefixes error with label', () => {  });
  it('iterations=1 returns single-call value', () => {  });
  it('iterations=2 exits after 2 calls', () => {  });
  it('iterations<1 throws DeterminismError', () => {
    expect(() => assertDeterministic(() => 1, [] as const, { iterations: 0 }))
      .toThrow(DeterminismError);
  });
  it('default iterations is 10 (no opts)', () => {  });
});

// ---------- inspectFunctionForbidden (AC#4 + AC#5) ----------

describe('inspectFunctionForbidden — clean P1.1.1 helpers', () => {
  it('bps_mul has no forbidden ops', () => {
    expect(inspectFunctionForbidden(bps_mul)).toEqual([]);
  });
  it('bps_div, apply_bps, decay, safe_mul, safe_div — all clean', () => {  });
});

describe('inspectFunctionForbidden — detects every forbidden pattern (13 patterns)', () => {
  it('Math.*', () => {  });
  it('Date.*', () => {  });
  it('new Date', () => {  });
  it('setTimeout / setInterval / setImmediate', () => {  });
  it('fetch', () => {  });
  it('require("fs")', () => {  });   // via function toString containing literal text
  it('import from "fs"', () => {  });
  it('crypto.*', () => {  });
  it('process.hrtime / process.nextTick', () => {  });
  it('await', () => {  });
  it('async function', () => {  });
  it('float literal (3.14 matches, 1n and 100 do not)', () => {  });
  it('[native code] body', () => {
    // Bound native fns expose "[native code]" in their toString().
    expect(inspectFunctionForbidden(Math.random.bind(null)))
      .toContain('[native code]');
  });
});

// ---------- assertNoForbiddenOps (AC#6) ----------

describe('assertNoForbiddenOps', () => {
  it('is a no-op on clean functions', () => {
    expect(() => assertNoForbiddenOps(bps_mul)).not.toThrow();
  });
  it('throws DeterminismError listing offending tokens', () => {
    expect(() => assertNoForbiddenOps(() => Math.random()))
      .toThrow(/Math\.random/);
  });
  it('propagates label', () => {
    expect(() => assertNoForbiddenOps(() => Date.now(), { label: 'bad-helper' }))
      .toThrow(/\[bad-helper\]/);
  });
});

// ---------- deepEqualDeterministic (AC#7) ----------

describe('deepEqualDeterministic', () => {
  it('primitives', () => {  });
  it('bigints equal and unequal', () => {  });
  it('null / undefined distinct', () => {  });
  it('arrays — same length equal, different length fail, nested', () => {  });
  it('plain objects — same keys equal, missing key fail, extra key fail, nested', () => {  });
  it('Error instances — by name + message', () => {  });
  it('mixed types (array vs object vs primitive) fail', () => {  });
  it('Map is opaque (equal only by reference)', () => {  });
});

// ---------- rule-engine corpus self-scan (AC#8) ----------

describe('rule-engine corpus self-scan', () => {
  it('no forbidden tokens in src/domains/rules/ shipped .ts (excluding determinism.ts + __tests__)', async () => {
    const thisFile = fileURLToPath(import.meta.url);
    const rulesDir = join(dirname(thisFile), '..', '..', '..', 'domains', 'rules');
    const entries = await readdir(rulesDir, { withFileTypes: true });
    for (const entry of entries) {
      if (!entry.isFile()) { continue; }
      if (!entry.name.endsWith('.ts')) { continue; }
      if (entry.name === 'determinism.ts') { continue; } // see contract §6.1
      const src = await readFile(join(rulesDir, entry.name), 'utf8');
      // Run each regex; expect zero matches outside of comments/strings
      // mentioning the concept names.
      // Rule-engine canon forbids all 13 tokens in shipped .ts.
      // integer-math.ts has none.
      // (We do NOT exclude comments because κ consensus cares about the
      //  whole text, not just executable statements.)
      // …regex sweep assertions…
    }
  });
});

4. Test-case matrix (explicit per AC)

AC Case Expected
AC#2 assertDeterministic(bps_mul, [1000n, 500n]) returns 50n, no throw
AC#2 assertDeterministic(bps_div, [5000n, 2500n]) returns 20000n, no throw
AC#2 assertDeterministic(apply_bps, [1000n, 150n]) returns 985n, no throw
AC#2 assertDeterministic(decay, [1000n, 150n, 2n]) returns 970n, no throw
AC#2 assertDeterministic(safe_mul, [2n, 3n]) returns 6n, no throw
AC#2 assertDeterministic(safe_div, [7n, 2n]) returns 3n, no throw
AC#3 assertDeterministic(() => Math.random(), []) throws DeterminismError
AC#3 assertDeterministic(() => Date.now(), []) throws DeterminismError
AC#3 Mixed throw/return: () => (++c > 1 ? (() => { throw new Error('x') })() : 1) DeterminismError “run 1 returned / run 2 threw”
AC#3 Throw-name drift: () => { throw c++ > 0 ? new RangeError('x') : new TypeError('x') } DeterminismError
AC#3 Throw-message drift: () => { throw new Error(++c > 1 ? 'b' : 'a') } DeterminismError
AC#3 Return-then-throw: () => (c++ < 1 ? 1 : (() => { throw new Error() })()) DeterminismError
AC#4 inspectFunctionForbidden(bps_mul) []
AC#4 same for bps_div, apply_bps, decay, safe_mul, safe_div all []
AC#5 inspectFunctionForbidden(() => Math.random()) contains Math.random
AC#5 12 other forbidden patterns (per §4 table) each returns non-empty
AC#6 assertNoForbiddenOps(bps_mul) no throw
AC#6 assertNoForbiddenOps(() => Date.now()) throws
AC#6 with label → error message contains [label] throws
AC#7 deepEqualDeterministic(1n, 1n) true
AC#7 deepEqualDeterministic(1n, 2n) false
AC#7 deepEqualDeterministic([1n, 2n], [1n, 2n]) true
AC#7 deepEqualDeterministic([1n, 2n], [1n, 2n, 3n]) false
AC#7 deepEqualDeterministic({a:1n}, {a:1n}) true
AC#7 deepEqualDeterministic({a:1n}, {a:1n, b:2n}) false
AC#7 deepEqualDeterministic({a:1n, b:2n}, {a:1n}) false (a has key b missing from b)
AC#7 deepEqualDeterministic(null, undefined) false
AC#7 deepEqualDeterministic(new Error('x'), new Error('x')) true
AC#7 deepEqualDeterministic(new Error('x'), new RangeError('x')) false
AC#7 deepEqualDeterministic(new Map(), new Map()) false (opaque)
AC#8 Corpus scan over src/domains/rules/integer-math.ts zero forbidden tokens
AC#10 npm run build && npm run lint && npm test exit 0

Expected new tests: 38–42 it blocks → 38+ new tests. 1123 → ~1165.

5. Branch-coverage map (for AC#9)

Mapping each control-flow branch in determinism.ts to a test that covers it:

Branch Covered by
inspectFunctionForbidden for-loop, pattern matches every “AC#5” pattern test
inspectFunctionForbidden for-loop, no match “clean P1.1.1 helpers”
inspectFunctionForbidden for-loop, matches-pushed every match test
assertNoForbiddenOps — hits.length === 0 path “no-op on clean”
assertNoForbiddenOps — hits.length > 0 path “throws DeterminismError”
assertNoForbiddenOps — opts.label truthy “propagates label”
assertNoForbiddenOps — opts.label falsy “no-op on clean” + “throws …” without label
deepEqualDeterministicObject.is short-circuit primitives
deepEqualDeterministic — bigint-bigint equal equal bigints
deepEqualDeterministic — bigint-bigint unequal unequal bigints
deepEqualDeterministic — Error-Error branch Error case
deepEqualDeterministic — Array branch, length !== array-length-mismatch
deepEqualDeterministic — Array branch, element !== array-content-mismatch
deepEqualDeterministic — Array branch, all equal array-equal
deepEqualDeterministic — Object branch, length !== object-different-keys
deepEqualDeterministic — Object branch, hasOwnProperty === false “extra key” test
deepEqualDeterministic — Object branch, recursive fail nested-mismatch
deepEqualDeterministic — Object branch, all equal object-equal
deepEqualDeterministic — fallthrough (opaque) Map
renderValue — bigint path value rendering on bigint
renderValue — JSON path value rendering on object
renderValue — truncation path long-value rendering
renderValue — catch path circular-ref value (we don’t actually add this test; catch path coverage is optional) — see §5.1
capture — returned path any no-throw fn
capture — threw with name/message integer-math error classes
capture — threw without string name throw "string" edge case
assertDeterministic — iterations === undefined default call
assertDeterministic — iterations >= 1 custom iterations
assertDeterministic — iterations < 1 “iterations=0 throws”
assertDeterministic — ref.threw vs cur.threw mismatch “mixed throw/return”
assertDeterministic — ref.threw && cur.threw, name equal deterministic-throw path
assertDeterministic — ref.threw && cur.threw, name mismatch “throw-name drift”
assertDeterministic — !ref.threw && !cur.threw, deep equal happy path
assertDeterministic — !ref.threw && !cur.threw, NOT deep equal Math.random mock
assertDeterministic — ref.threw final throw deterministic-throw returns
assertDeterministic — !ref.threw final return happy path return value
assertDeterministic — opts.label “prefixes error with label”

5.1. Coverage gap note

renderValue has a try { … } catch { return String(v) } block. JSON.stringify on bigint uses a replacer, so the only way to hit the catch is with a circular object containing bigint — but the replacer itself doesn’t throw on circulars (the JSON.stringify call does). We will add a test that exercises the catch branch via a Proxy that throws on toJSON. This brings all branches of the renderValue helper in scope.

6. Execution timeline

  1. Audit — DONE (step 1).
  2. Contract — DONE (step 2).
  3. Packet — DONE (this step).
  4. Implementation — one commit creating both files, then run npm run build && npm run lint && npm test locally and iterate until green.
  5. Verification — one commit dropping docs/verification/r83-a-determinism-verification.md with the gate output captured.

7. Risks

# Risk Mitigation
R1 Regex false positive in a comment in integer-math.ts breaks the AC#8 self-scan The P1.1.1 file has no Math. / Date. / float-literal tokens in code OR comments (checked during audit). If a future commit adds one, the scan catches it, which is the correct behavior.
R2 ESLint ban-types/no-unsafe-function-type trips on Function param Accept the lint warning; the codebase already tolerates any as warn. If it escalates to error, narrow to (...args: any[]) => unknown with an eslint-disable-next-line.
R3 The renderValue catch branch is hard to hit Added a Proxy-based test per §5.1 to force coverage.
R4 startup — subprocess smoke flake Known pre-existing. Rerun-once policy per prompt.
R5 The [native code] detection requires Math.random.bind() — this IS a forbidden-op itself so the test relies on the string escaping. The test uses the resulting bound function’s toString(), which returns function () { [native code] } — that’s the string-literal we scan for, no forbidden op invoked.

8. Commit plan

git add src/domains/rules/determinism.ts src/__tests__/domains/rules/determinism.test.ts
git commit -m "feat(r83-a-determinism): P1.1.2 determinism verification harness"

Then after npm gates green:

git add docs/verification/r83-a-determinism-verification.md
git commit -m "verify(r83-a-determinism): npm gates green"

Five total commits on feature/r83-a-determinism:

  1. audit(r83-a-determinism): inventory harness surface (already committed)
  2. contract(r83-a-determinism): behavioral contract (already committed)
  3. packet(r83-a-determinism): execution plan (this commit)
  4. feat(r83-a-determinism): P1.1.2 determinism verification harness
  5. verify(r83-a-determinism): npm gates green

9. Ready for Step 4

Implementation blocked on this packet being committed. All design decisions recorded; no open questions remain.


Back to top

Colibri — documentation-first MCP runtime. Apache 2.0 + Commons Clause.

This site uses Just the Docs, a documentation theme for Jekyll.