Validators

API reference for validator functions

The SDK provides 9 boolean validator functions for asserting tool call behavior in tests, plus evaluateToolCalls — a richer, configurable matcher that returns structured results.

Import

import {
  matchToolCalls,
  matchToolCallsSubset,
  matchAnyToolCall,
  matchToolCallCount,
  matchNoToolCalls,
  matchToolCallWithArgs,
  matchToolCallWithPartialArgs,
  matchToolArgument,
  matchToolArgumentWith,
  // Structured matcher
  evaluateToolCalls,
} from "@mcpjam/sdk";

The matchers module is also available as a standalone browser-safe entry point:

import { evaluateToolCalls } from "@mcpjam/sdk/matchers";

---

Tool Name Validators

matchToolCalls()

Exact match - all tools in exact order.

matchToolCalls(expected: string[], actual: string[]): boolean

Parameters

Parameter	Type	Description
expected	string[]	Expected tool names in order
actual	string[]	Actual tool names (from result.toolsCalled())

Returns

boolean - true if exact match.

Example

const result = await agent.run("Add 2+3, then multiply by 4");
// toolsCalled() = ["add", "multiply"]

matchToolCalls(["add", "multiply"], result.toolsCalled()); // true
matchToolCalls(["multiply", "add"], result.toolsCalled()); // false (wrong order)
matchToolCalls(["add"], result.toolsCalled());             // false (missing multiply)

---

matchToolCallsSubset()

Checks if all expected tools were called (any order, extras allowed).

matchToolCallsSubset(expected: string[], actual: string[]): boolean

Parameters

Parameter	Type	Description
expected	string[]	Tools that must be present
actual	string[]	Actual tool names

Returns

boolean - true if all expected tools are present.

Example

// toolsCalled() = ["add", "multiply"]

matchToolCallsSubset(["add"], result.toolsCalled());             // true
matchToolCallsSubset(["multiply", "add"], result.toolsCalled()); // true (order doesn't matter)
matchToolCallsSubset(["divide"], result.toolsCalled());          // false

---

matchAnyToolCall()

Checks if at least one expected tool was called.

matchAnyToolCall(expected: string[], actual: string[]): boolean

Parameters

Parameter	Type	Description
expected	string[]	Any of these tools should be called
actual	string[]	Actual tool names

Returns

boolean - true if at least one match.

Example

// toolsCalled() = ["add"]

matchAnyToolCall(["add", "subtract"], result.toolsCalled()); // true
matchAnyToolCall(["multiply", "divide"], result.toolsCalled()); // false

---

matchToolCallCount()

Checks if a tool was called exactly N times.

matchToolCallCount(
  toolName: string,
  actual: string[],
  count: number
): boolean

Parameters

Parameter	Type	Description
toolName	string	Tool to count
actual	string[]	Actual tool names
count	number	Expected count

Returns

boolean - true if count matches.

Example

// toolsCalled() = ["add", "add", "add"]

matchToolCallCount("add", result.toolsCalled(), 3); // true
matchToolCallCount("add", result.toolsCalled(), 2); // false

---

matchNoToolCalls()

Checks that no tools were called.

matchNoToolCalls(actual: string[]): boolean

Parameters

Parameter	Type	Description
actual	string[]	Actual tool names

Returns

boolean - true if empty.

Example

const result = await agent.run("What is the capital of France?");
matchNoToolCalls(result.toolsCalled()); // true (if no tools called)

---

Argument Validators

matchToolCallWithArgs()

Checks if a tool was called with exactly the specified arguments.

matchToolCallWithArgs(
  toolName: string,
  expectedArgs: Record<string, unknown>,
  toolCalls: ToolCall[]
): boolean

Parameters

Parameter	Type	Description
toolName	string	Tool to check
expectedArgs	Record<string, unknown>	Expected arguments (exact match)
toolCalls	ToolCall[]	From result.getToolCalls()

Returns

boolean - true if exact argument match.

Example

const result = await agent.run("Add 2 and 3");

matchToolCallWithArgs("add", { a: 2, b: 3 }, result.getToolCalls()); // true
matchToolCallWithArgs("add", { a: 2 }, result.getToolCalls());       // false (missing b)
matchToolCallWithArgs("add", { a: 3, b: 2 }, result.getToolCalls()); // false (wrong values)

---

matchToolCallWithPartialArgs()

Checks if a tool was called with arguments containing the expected subset.

matchToolCallWithPartialArgs(
  toolName: string,
  expectedArgs: Record<string, unknown>,
  toolCalls: ToolCall[]
): boolean

Parameters

Parameter	Type	Description
toolName	string	Tool to check
expectedArgs	Record<string, unknown>	Arguments to match (subset)
toolCalls	ToolCall[]	From result.getToolCalls()

Returns

boolean - true if all expected args present with matching values.

Example

// Tool called with { name: "Task", priority: "high", project: "default" }

matchToolCallWithPartialArgs("createTask", { name: "Task" }, calls);     // true
matchToolCallWithPartialArgs("createTask", { priority: "high" }, calls); // true
matchToolCallWithPartialArgs("createTask", { status: "open" }, calls);   // false

---

matchToolArgument()

Checks if a specific argument has the expected value.

matchToolArgument(
  toolName: string,
  argName: string,
  expectedValue: unknown,
  toolCalls: ToolCall[]
): boolean

Parameters

Parameter	Type	Description
toolName	string	Tool to check
argName	string	Argument name
expectedValue	unknown	Expected value
toolCalls	ToolCall[]	From result.getToolCalls()

Returns

boolean - true if argument matches.

Example

matchToolArgument("add", "a", 5, result.getToolCalls());  // true if a=5
matchToolArgument("add", "b", 10, result.getToolCalls()); // true if b=10

---

matchToolArgumentWith()

Checks if an argument passes a custom predicate.

matchToolArgumentWith(
  toolName: string,
  argName: string,
  predicate: (value: unknown) => boolean,
  toolCalls: ToolCall[]
): boolean

Parameters

Parameter	Type	Description
toolName	string	Tool to check
argName	string	Argument name
predicate	(value: unknown) => boolean	Custom validation function
toolCalls	ToolCall[]	From result.getToolCalls()

Returns

boolean - true if predicate returns true.

Example

// Type check
matchToolArgumentWith(
  "echo",
  "message",
  (v) => typeof v === "string",
  result.getToolCalls()
);

// Content check
matchToolArgumentWith(
  "echo",
  "message",
  (v) => String(v).includes("hello"),
  result.getToolCalls()
);

// Range check
matchToolArgumentWith(
  "add",
  "a",
  (v) => typeof v === "number" && v > 0 && v < 100,
  result.getToolCalls()
);

// Email validation
matchToolArgumentWith(
  "sendEmail",
  "to",
  (v) => /^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(String(v)),
  result.getToolCalls()
);

---

Usage with Test Frameworks

Jest / Vitest

import { describe, it, expect } from "vitest";
import { matchToolCalls, matchToolCallWithArgs } from "@mcpjam/sdk";

describe("Calculator", () => {
  it("calls add for addition", async () => {
    const result = await agent.run("Add 5 and 3");
    expect(matchToolCalls(["add"], result.toolsCalled())).toBe(true);
  });

  it("passes correct arguments", async () => {
    const result = await agent.run("Add 10 and 20");
    expect(
      matchToolCallWithArgs("add", { a: 10, b: 20 }, result.getToolCalls())
    ).toBe(true);
  });
});

With EvalTest

import { EvalTest, matchToolCallsSubset } from "@mcpjam/sdk";

const test = new EvalTest({
  name: "multi-step",
  test: async (agent) => {
    const result = await agent.run("Add 2+3, then multiply by 4");
    return matchToolCallsSubset(["add", "multiply"], result.toolsCalled());
  },
});

---

evaluateToolCalls()

A configurable matcher that returns a structured result instead of a plain boolean. Use this when you need to know why a match failed (missing tools, argument mismatches, wrong order, extra calls) or when you want to control matching behavior per test case.

evaluateToolCalls(
  expected: EvalToolCall[],
  actual: EvalToolCall[],
  options?: EvalMatchOptions & { isNegativeTest?: boolean }
): EvalToolCallMatchResult

Parameters

Parameter	Type	Description
expected	EvalToolCall[]	Expected tool calls (each has toolName and optional arguments)
actual	EvalToolCall[]	Actual tool calls from the trace
options	EvalMatchOptions	Matching behavior overrides (see below)

EvalMatchOptions

Option	Type	Default	Description
toolCallOrder	"ignore" | "strict"	"ignore"	"strict" requires matched calls to appear in the same relative order as expected.
allowExtraToolCalls	boolean	true	When false, any actual call beyond what’s expected fails the test.
argumentMatching	"partial" | "exact" | "ignore"	"partial"	"partial" checks only expected keys and supports type placeholders ("string", "number", "any", etc.). "exact" requires deep equality. "ignore" skips argument comparison.

EvalToolCallMatchResult

Property	Type	Description
passed	boolean	Whether the match succeeded.
missing	EvalToolCall[]	Expected calls that were not found in actual.
extra	EvalToolCall[]	Actual calls beyond what was expected.
outOfOrder	EvalOutOfOrderToolCall[]	Matched calls that appeared in the wrong order (only populated when toolCallOrder: "strict").
argumentMismatches	EvalArgumentMismatch[]	Calls matched by name but with incompatible arguments.

Example

import { evaluateToolCalls } from "@mcpjam/sdk";

const result = await agent.run("Create a task called 'Deploy' with high priority");

const match = evaluateToolCalls(
  [{ toolName: "createTask", arguments: { priority: "high" } }],
  result.getToolCalls(),
  { argumentMatching: "partial" },
);

if (!match.passed) {
  console.log("Missing:", match.missing);
  console.log("Arg mismatches:", match.argumentMismatches);
}

Argument type placeholders

When argumentMatching is "partial" (the default), string values in expected arguments are interpreted as type checks:

Placeholder	Matches
"any"	Any non-undefined value
"string"	typeof value === "string"
"number"	typeof value === "number"
"boolean"	typeof value === "boolean"
"object"	Non-null, non-array object
"array"	Array
"null"	null

evaluateToolCalls(
  [{ toolName: "sendEmail", arguments: { to: "string", subject: "any" } }],
  result.getToolCalls(),
);

Layered match options

resolveMatchOptions merges suite → case → run-override layers on top of defaults. Import it from @mcpjam/sdk/matchers when building custom eval pipelines:

import { resolveMatchOptions } from "@mcpjam/sdk/matchers";

const resolved = resolveMatchOptions(
  { allowExtraToolCalls: false },   // suite-level
  { argumentMatching: "exact" },    // case-level
  undefined,                        // run-level override
);
// { toolCallOrder: "ignore", allowExtraToolCalls: false, argumentMatching: "exact" }

---

Quick Reference

Validator	Use Case
evaluateToolCalls	Structured match with detailed diagnostics
matchToolCalls	Exact sequence of tools
matchToolCallsSubset	Required tools (any order)
matchAnyToolCall	At least one of these tools
matchToolCallCount	Tool called N times
matchNoToolCalls	No tools called
matchToolCallWithArgs	Exact argument match
matchToolCallWithPartialArgs	Subset of arguments
matchToolArgument	Single argument value
matchToolArgumentWith	Custom validation

---

Related

• Testing with LLMs - Conceptual guide
• PromptResult Reference - Get tool calls
• EvalTest Reference - Use in evaluations
• Saving Eval Results - Per-result matchOptions on EvalResultInput