@swarmmachina/benchkit v0.3.1

benchkit

Benchmark / load generation

Zero-runtime-dependency tools for reproducible Node.js benchmarks: HTTP/1.1 and WebSocket load generators, latency histograms, CPU, ELU, memory, and regression guards.

Use it to measure services and guard against regressions. Load generators run in worker threads; run the target service separately so target measurements do not mix with generator process metrics.

Benchmarkbenchkit workersHTTP / WebSocketTarget service
22 / 24
Node.js
0
runtime deps
HTTP + WS
load generation
MPL-2.0
license

Installation and quick start

Shellinstall
npm install --save-dev @swarmmachina/benchkit
JavaScriptquick-start.js
import { runHttp1Load } from '@swarmmachina/benchkit/load/http1'

const result = await runHttp1Load({
  name: 'health',
  url: 'http://127.0.0.1:3000/health',
  connections: 100,
  pipelining: 10,
  workers: 4,
  warmupMs: 2_000,
  durationMs: 10_000
})

console.log({
  rps: result.requests.averagePerSecond,
  p95Ms: result.latencyMs.p95Ms,
  p99Ms: result.latencyMs.p99Ms,
  errors: result.errors.total,
  dropped: result.transport.rateDropped
})
  • Start the target service first; this example expects GET /health on port 3000.
  • Omit rate for closed-loop saturation. rate sets the aggregate request rate; capacity exhaustion increases rateDropped rather than creating an unbounded queue.
  • Warmup preserves workers and connections; counters and histograms reset before measurement.
  • Record environment parameters, errors, dropped requests, and generator load alongside throughput and p95/p99.

Package capabilities

HTTP/1.1
HTTP/HTTPS, keep-alive, pipelining, fixed-rate scheduling, and coordinated-omission correction.
WebSocket
Built-in Node.js client, bounded in-flight messages, and send-buffer backpressure.
Measurements and regressions
Bounded latency histograms, CPU, ELU, process memory, paired comparisons, and absolute/relative guards.
Orchestration
Local and SSH targets with explicit deadlines and lifecycle; benchkit-agent CLI for remote orchestration.

Runtime and platforms

EnvironmentStatusNote
Node.js ^22.13.0 / 24.xSupportedNative ESM and worker threads; Node.js 26 is outside the engine range.
HTTP / HTTPSHTTP/1.1Close-delimited responses and HTTP upgrades are rejected.
WebSocket targetEcho / request-responseEvery response completes the oldest in-flight message; unsolicited messages are protocol errors.
SSH orchestrationOptionalRequires ssh, tar, key-based authentication, and direct access to the load port.

API map

Load generators

Separate subpath exports for load scenarios.

runHttp1Load()runWebSocketLoad()@swarmmachina/benchkit/load/http1@swarmmachina/benchkit/load/websocket

Measurements / reporting

Measurements, statistics, and result artifacts.

./measurement./statistics./profiling./reporting./results

Regression / orchestration

Guards and local/remote target lifecycle.

./regression./baseline./control./target./target-provider./orchestration

Complete signatures, types, and examples are available in the source repository README and TypeScript declarations.

Measurements, pressure, and environment boundaries

Results depend on the service, the generator, and the runtime environment. Retain all three parts of that context.

Separate the target from the generator

Generator CPU and RSS include unrelated work in the caller process. Use a separate process or host for the target.

Account for fixed-rate drops

When connections × pipelining or connections × maxInFlight is exhausted, new arrivals are dropped and counted in rateDropped.

Check the response contract

The HTTP generator supports Content-Length, chunked, interim 1xx, HEAD, 204, and 304. WebSocket targets must behave as echo/request-response endpoints.

Inspect the generator itself

Read loadGenerator CPU, worker ELU, and memory together with errors and scheduling lag. High RPS alone does not establish service headroom.

Generate WebSocket echo load

Use the separate WebSocket generator for an echo endpoint. maxInFlight bounds outstanding messages per connection.

JavaScriptwebsocket-load.js
import { runWebSocketLoad } from '@swarmmachina/benchkit/load/websocket'

const result = await runWebSocketLoad({
  url: 'ws://127.0.0.1:3000/echo',
  message: '{"type":"ping"}',
  connections: 100,
  maxInFlight: 4,
  workers: 4,
  warmupMs: 2_000,
  durationMs: 10_000
})

console.log(result.messages.averagePerSecond, result.latencyMs.p99Ms)
Need an HTTP/WebSocket service to measure?Open swm-core