HTTP endpoints

The agent is an HTTP server and nothing else: it opens no outbound connection. Every path, parameter, status code and line format below is read from internal/agent/http.go, internal/agent/capture.go, internal/agent/source.go and internal/sample/sample.go.

Listen address

The agent binds ADDR:PORT. install writes the agent’s own veth address into ADDR and 9123 into PORT, so the default is http://172.30.10.2:9123. An agent started with ADDR unset binds :PORT, every address inside its container’s network namespace. How a host reaches that address, directly, through the relay or through --expose, is on Network access.

The server gives a client 5 s to send its request headers and 30 s to take a response (ReadHeaderTimeout and WriteTimeout in internal/agent/agent.go).

Authentication

With TOKEN unset, every path is open. With it set, every path but /healthz needs:

Authorization: Bearer <token>

The header’s value is compared with the token after an optional "Bearer " prefix is removed, so a header holding the bare token is accepted too. A request without it, or with a different token, gets 401, the body token required and WWW-Authenticate: Bearer. /healthz stays open on purpose: it carries the board string an operator is asked to send when their board has no port map, and a token should not be needed for that.

The relay transport cannot present a token. /tool fetch runs on the router and sends no Authorization header (internal/transport/transport.go), so the relay reaches /healthz and nothing else on an agent that has one. --expose makes the token mandatory; Expose on the LAN says why.

Endpoints

Method	Path	Token	Returns
GET	/healthz	no	JSON: liveness, sequence numbers, clocks, rate, slipped ticks, capability hash, board
GET	/capabilities	yes	JSON: the kernel, the board, the sources, the device’s own ceilings and each source’s cadence
GET	/snapshot	yes	NDJSON, then closes: the last N seconds, or up to N samples after a sequence number
GET	/stream	yes	NDJSON, chunked and open: backfill from a sequence number, then live
GET	/sampler	yes	JSON: what only the agent can count about itself — ticks, slips, triggers, captures
GET	/captures	yes	JSON: the capture index
GET	/captures/{id}	yes	NDJSON: one capture header line, then the sample lines verbatim
DELETE	/captures/{id}	yes	204, and the capture’s bytes return to the budget
POST	/capture	yes	JSON: arms a capture now

Any other path is 404; a known path with another method is refused by Go’s router with 405.

GET /healthz

Field	Type	Meaning
ok	bool	always true when the agent answers
seq	integer	sequence number of the newest sample in the ring; 0 when the ring is empty
oldest_seq	integer	oldest sequence number still held; 0 when empty
wall_ns	integer	the router’s wall clock at the reply, ns since the epoch
mono_ns	integer	the agent’s monotonic clock at the reply, ns
uptime_s	float	seconds since the HTTP server was set up
rate_hz	integer	configured sampler rate
slipped	integer	ticks whose read finished after the next tick was due
capabilities_hash	string	eight hex digits over the kernel, the core count and the enabled sources
version	string	the agent’s build identity
board	string	the device tree’s model string; omitted when the board has none
boot_id	string	the kernel’s boot id, read at start; omitted when the kernel does not publish it

What the CLI reads from it:

• wall_ns, against its own clock, to measure the skew;
• seq and oldest_seq, to plan a backfill;
• capabilities_hash, to notice that the kernel or the source set under it changed;
• in doctor, board to classify kernel-log records, and seq with oldest_seq to read the whole ring, or its newest 10 000 samples when it holds more;
• in forward, boot_id, to tell a router that rebooted from an agent whose container alone restarted: the container shares the router’s kernel, so only a reboot gives it a new id (reboot).

No CLI code reads mono_ns. The probe after install prints this reply as:

direct transport ok: agent <version>, <rate> Hz, seq <n>, <n> slipped, <rtt> round trip

<version> is the CLI’s own version, which it stamps into the agent it builds (dev when the CLI itself was built without one).

GET /capabilities

What the agent established at start about the kernel and the board, with no RouterOS API. Device info is this object as the collector hands it to every sink.

Field	Meaning
kernel	the kernel version string from /proc/version
board	the device tree’s model; omitted when absent
ports	kernel interface name to RouterOS default name, for a board in the port table; omitted otherwise
ports_from	how that map was established; omitted with it
cores	core count, from the first read of /proc/stat
user_hz	USER_HZ, the tick counters’ unit
sources	source name to true when the file opened and the source is enabled: stat, meminfo, loadavg, softnet, softirqs, interrupts, vmstat, psi, schedstat, self, yaffs, diskstats, slabinfo, kmsg, thermal, cpufreq, perf, buddyinfo, mtd
namespaced	what the agent deliberately does not read as router data: net/dev, net/snmp, net/netstat, sys/net/netfilter/nf_conntrack_count
cgroup	true when cgroup2 cpu.stat is readable, so the self-cost is exact
privileged	true when both /proc/slabinfo and /dev/kmsg opened
limits	the device’s own ceilings, read once at start (below)
cadences	source name to {"hz", "reason"} for each level source (below)
hash	the same value as /healthz’s capabilities_hash

limits has no JSON tags in the code, so its keys are the Go field names: ThermalCriticalMilliC (zone to the lowest critical trip, m°C), ThermalPollingMS (zone to its polling_delay), CPUFreqMinKHz, CPUFreqMaxKHz, CPUFreqStepsKHz (core to the ladder), CPUFreqGovernor, CPUFreqRelated (core to the cores that change frequency with it), CgroupMemoryMaxBytes and ConntrackMax. A ceiling the device does not publish is empty or 0.

A cadence’s reason is one of rate (read at the sampler rate), declared (the device publishes its own refresh cadence), policy (a userspace cpufreq governor means the clock cannot move without a write), budget (a measured parse cost), change (read every tick, stored on change) or override (FLOOR_HZ). Counters never appear here, because counters are never floored. Source cadences explains each floor.

GET /snapshot

Writes NDJSON and closes. Two forms, chosen by whether since is present.

Parameter	Default	Accepted	Meaning
seconds	1	1–3600	without since: the newest seconds × rate samples the ring holds, oldest first
since	absent	a sequence number	up to max samples with a sequence number above it, oldest first; since=0 starts from the oldest held
max	20	1–10000	with since: the most samples one reply carries

A value outside its range is 400 with a one-line reason. The since form is what record and forward pull, over both transports, and what doctor reads the ring with (since one below oldest_seq, or seq − 10 000 when the ring holds more than 10 000 samples, max=10000, direct transport only, with the bearer token when TOKEN is set). It adds two line kinds the seconds form does not: a gap line first when since is older than the ring, and a trigger line before each sample a capture condition fired on. max exists for the relay, because /tool fetch output=user returns at most 64 512 B and truncates the rest silently; the relay asks for at most 13 lines.

A large /snapshot is not a free way to read the cost

A 60 s /snapshot at 10 Hz makes the agent hand over about 600 lines, around 1.9 MB, and the self.cpu_us inside those samples includes the cost of serving them. Read the agent’s cost from the collector’s /metrics instead: Agent cost has the procedure.

GET /stream

Chunked NDJSON with Cache-Control: no-store, held open until the client leaves or the server’s 30 s WriteTimeout closes it, whichever comes first.

• since absent or 0: starts live. The first sample written is the next one produced after the request; the newest sample already in the ring is not written.
• since=N: backfills every held sample after N first, then follows. A since older than the ring writes one gap line first.
• The ring is checked every half period and every new sample is written as it arrives, with trigger lines before the samples they fired on.
• Every 5 s a comment line # heartbeat seq=<newest> is written. A consumer skips lines that start with #.

A since that is not a number is 400.

The server’s 30 s WriteTimeout applies to every response, and Go’s HTTP server does not lift it for a handler that keeps writing, so a /stream connection ends about 30 s after it opens however much it still has to say: a connection lives 30.01–30.06 s and carries 304–306 lines at 10 Hz. A consumer of /stream has to reconnect, with the last seq it saw as since so the gap is backfilled. The CLI does not depend on /stream: record and forward pull /snapshot?since=.

Line types

Apart from /stream’s heartbeat comment, every line /snapshot, /stream and /captures/{id} write is one JSON object and a newline. Four kinds exist, told apart by their first key:

Line	Where	Meaning
a sample, starting {"seq":	all three	one tick, below
{"gap":{"from":F,"to":T}}	/snapshot?since=, /stream	samples F to T are no longer in the ring
{"trigger":{…}}	/snapshot?since=, /stream	a capture condition fired on the sample that follows
{"capture":{…}}	first line of /captures/{id}	the capture’s header

A consumer that does not know a kind skips it, so a line kind it has never seen costs it nothing.

Sample line

Every numeric field is a delta since the previous sample unless the table says it is a level. A source the kernel does not have, or that the deployment cannot read, is omitted, never written as zero. The busy ticks of a core are u + n + s + q + sq + st; idle and iowait are not busy.

Field	Kind	Content
seq	—	sequence number, from 1
mono_ns	—	the agent’s monotonic clock at the read
wall_ns	—	the router’s wall clock at the read, ns since the epoch
dt_ns	—	the real interval since the previous sample; every delta is over this, not over the nominal period
cpu	delta	one object per core: u user, n nice, s system, i idle, w iowait, q irq, sq softirq, st steal, in USER_HZ ticks
cpu_total	delta	the same keys, from /proc/stat’s own cpu summary line
ctxt, intr	delta	context switches and interrupts from /proc/stat
forks	delta	the processes line of /proc/stat
procs_blocked	level	tasks in uninterruptible sleep
psi	delta	cpu_some, mem_some, mem_full, io_some, io_full, µs; only on a kernel with PSI
sched	delta	per CPU run_ns, wait_ns; only on a kernel with /proc/schedstat
softnet	delta	per CPU p processed, d dropped, ts time squeeze
softirq	delta	softirq kind to an array of per-CPU counts
irq	delta	the IRQ_TOP_K busiest interrupt lines this tick (8 by default): id, name, cpu array
irq_total	delta	every interrupt line summed, the top-K’s denominator
irq_err	delta	the Err row of /proc/interrupts; omitted when zero
mem	level	/proc/meminfo in kB, keyed by Go field name: MemTotal, MemFree, MemAvailable, Buffers, Cached, Dirty, Shmem, Slab, SReclaimable, CommittedAS, Writeback, SUnreclaim, AnonPages, Mapped, KernelStack, PageTables, CommitLimit, Active, Inactive
load	level	/proc/loadavg: Load1, Load5, Load15, Running, Total, LastPID
vm	delta	/proc/vmstat events: pgfault, pgmajfault, and when non-zero pgscan_kswapd, pgscan_direct, pgsteal_kswapd, pgsteal_direct, pgalloc, pgfree, allocstall, compact_stall, oom_kill, pswpin, pswpout
vmg	level	nr_free_pages, nr_dirty, nr_writeback, nr_slab_reclaimable, nr_slab_unreclaimable, in pages
self	mixed	cpu_us (delta, µs), rss (level, bytes), cg_mem (level, omitted when zero), cg (true when cgroup2 was read), and throttled, throttled_us, oom_kill (deltas); the last four are omitted when zero or false; wake_ns (how late the loop woke after its ticker, ns) and read_ns (how long reading every due source took, ns), each omitted when zero
thermal	level	per zone type, mc (m°C) and Celsius; at the zone’s declared cadence, every tick when no zone declares one, or at FLOOR_HZ
freq_khz	level	per core, kHz; on change or on the 60 s heartbeat
thermal_critical	level	zone to critical trip in m°C, on the rows that carry thermal
freq_max_khz	level	core to cpufreq ceiling in kHz, on the rows that carry freq_khz
cgroup_mem_max	level	the container’s memory.max in bytes, on the first tick and once per heartbeat
flash	mixed	per YAFFS device dev, pw, pr, er, gcc, gc (deltas) and bad, free (levels); a device with no operation and unchanged free chunks is omitted
disk	mixed	per block device name, r, rs, w, ws, io_ms (deltas) and inflight (level); an idle device is omitted
slab	level	cache to active objects; needs privileged=yes; on change, at a budget floor
slab_limit	level	cache to its published ceiling, today only nf_conntrack
perf	delta	per hardware counter name, cpu array, and enabled_ns, running_ns; needs privileged=yes, and only counters the CPU implements
buddy	level	per zone node, zone, free array indexed by order; on change or heartbeat
mtd	level	per partition dev, name, corr, fail, bad, bbt, bitflip_threshold, ecc_strength; needs privileged=yes; kernel’s since-boot counts
events	—	kernel-log records this tick: prio, lvl (0 emerg … 7 debug), fac, seq, us (µs since boot), msg, and, when the text names a port, iface, ros_iface and kind (link-up, link-down, stp-<state>, own-address or other)
events_dropped	—	kernel-log loss events this tick, not records: one if the tick hit the 64-record cap, one per kernel ring overrun (which can stand for many records); while non-zero, events is a lower bound; omitted when zero
resets	—	monotonic counters that went backwards without a 32-bit wrap this tick; omitted when zero

The perf counter names the agent tries to open are cycles, instructions, cache-references, cache-misses, branch-instructions, branch-misses and bus-cycles. The slab caches it keeps are nf_conntrack, skbuff_head_cache, skbuff_fclone_cache, TCP, UDP, TCPv6, UDPv6, sock_inode_cache, dst_cache, ip_dst_cache, kmalloc-1k and kmalloc-2k, where the kernel has them.

GET /sampler

What only the agent can count about itself, as JSON: ticks taken, ticks slipped, and what the trigger evaluator has fired, suppressed and refused, with what it is holding right now and the budget those captures pin. Every configured condition is present from the first read, at 0 until it fires — a counter that appears with its first event reads as a gap in the series rather than as a quiet router.

The collector reads it at start and then on the same one-minute cadence it re-measures the clock skew on, and hands it to every sink. These figures are not per-tick, so they cannot ride in a sample.

The agent serves no /metrics. The Prometheus exposition is the collector’s, built from the samples and from this endpoint, and it carries every family the dashboards ask for: Prometheus metrics. The agent is a sampler and a ring; it folds no sample into counters or histograms, which is memory it does not spend.

Captures

All four capture paths answer 404 with captures disabled (CAPTURE_MB=0) when the budget is 0.

GET /captures

Field	Meaning
policy	first or last
budget_bytes	the pinned-bytes budget, CAPTURE_MB in bytes
bytes	bytes the held captures pin
pending	the capture still collecting its post-fire window; omitted when there is none
captures	the held captures’ headers, oldest first
triggers	the configured conditions, each name and threshold

GET /captures/{id}

One {"capture":{…}} header line, then the sample lines exactly as /snapshot would have written them, so no new parser is needed. The bytes served are counted in mikroscope_capture_bytes_served_total: a download runs on the same core as the sampler. An id that is not a number is 400; an unknown one is 404 with no such capture.

Field	Meaning
id	capture number, from 1
cause	the condition’s name as configured, or manual
condition	the condition as configured, for example busy>=0.95; empty for a manual capture
field	what was compared, for example cpu[2].busy_ratio; for a manual capture, the reason
value	the value that tripped it
threshold	the configured threshold
fire_seq	the sample it fired on
fire_mono_ns, fire_wall_ns	the two clocks at the fire
first_seq, last_seq	the window held
samples, bytes	its size
complete	false when it holds fewer than pre + post + 1 samples: the ring did not reach back far enough, or the agent stopped first

The trigger line in /snapshot and /stream carries id, cause, field (omitted when empty), value, threshold, seq and wall_ns. The agent keeps the last 64 of them.

DELETE /captures/{id}

Frees the capture’s bytes and answers 204 with no body. Unknown or non-numeric ids answer as for GET.

POST /capture

Arms a capture at the newest sample, as if a condition had fired:

curl -X POST "http://172.30.10.2:9123/capture?reason=queue-tree-applied"

reason defaults to operator. The reply is {"id":N,"armed":true}. If another capture is still collecting its window, or a manual capture fired within the refractory window, the reply is 409 and nothing is armed. The conditions, the budget and what a capture cannot show are on Triggered capture.

See also

• Network access: the three ways a host gets to these paths.
• Prometheus metrics: what the collector’s /metrics carries.
• Triggered capture: the conditions behind /captures.
• Expose on the LAN: when the token becomes mandatory.