Triggered capture

Everything on /metrics is a lossy summary, and a recording only exists if someone started it before the moment. This page answers what the agent does instead: which conditions make it keep the samples around a moment at full rate, how to configure them, how to fetch what it kept, what the collector does with the news, and what a capture cannot tell you.

What a capture is, and what it is not

The one lossless thing the agent can do on its own is keep the samples that already exist, at full rate, around the moment they matter — and only the agent can, because only the agent has every sample. The ring already holds the last 300 s by default, so the seconds before a fire cost nothing to keep; the seconds after cost only the wait.

It decides nothing about meaning. A condition is a comparison you configured. The field it compared and the value that tripped it travel in the capture’s header, so you can see what was compared. The capture is the same raw delta lines /snapshot ships. Nothing is turned into a percentage or a verdict.

A capture does not copy those lines. The ring stores each sample as a pre-encoded, immutable line, and a capture pins the lines it needs, so firing costs a copy of the entry headers once per fire and nothing per tick. The design estimates that at about 3 µs for a 10 s window at 10 Hz; it has not been measured on the device.

Conditions are evaluated in the sampler’s own loop, between reading a sample and pushing it into the ring, never in a second goroutine. There is no expression language, on purpose: a parser is a dependency and an attack surface, and an operator-writable expression on the sampler’s hot path is a way to make the router slow.

Configuring it

The agent reads six variables from its container’s envlist. Two of them have an install flag.

Agent variable	install flag	Default	Accepted	What it sets
TRIGGERS	--triggers	softnet-drop,oom,kmsg<=3,reset,irq-err,flash-bad	the conditions below, comma-separated	Which conditions arm a capture.
CAPTURE_MB	--capture-mb	4	0–256	The budget of pinned ring bytes, in MiB. 0 turns the feature off.
CAPTURE_PRE_S	none	5	1–60	Seconds kept before the sample that fired.
CAPTURE_POST_S	none	5	1–60	Seconds kept after it.
CAPTURE_POLICY	none	first	first, last	On a full budget: first refuses the new capture, last evicts the oldest.
TRIGGER_REFRACTORY_S	none	10	0–3600	Quiet time per condition after it fires.

install always writes CAPTURE_MB, and writes TRIGGERS only when --triggers is given; without it the agent uses its default set. It writes none of the other four, so an installed agent runs with their defaults. mikroscope plan shows the envlist entries before anything is written.

--triggers goes through the agent’s own parser before the first connection: Finish hands it to agent.ParseTriggers, so an unknown condition, a threshold out of range, a quote or a semicolon fails the verb with exit status 2 and writes nothing. The agent parses the list again when it starts, because an envlist can be edited by hand on the router; a value it rejects there makes it refuse to run, with one line on its standard output, which RouterOS puts in its log.

The conditions

The default set is the conditions that take no threshold except squeeze — softnet-drop, oom, reset, irq-err, flash-bad, each firing when the kernel counts something it normally does not — plus kmsg<=3. The level conditions are not in it: their thresholds are yours to choose.

Condition	Fires when	field in the header	Threshold	In the default set
softnet-drop	any softnet queue dropped a packet in the sample	softnet[N].dropped	none	yes
squeeze	any softnet queue ran out of budget (time_squeeze) in the sample	softnet[N].time_squeeze	none	no
oom	the kernel OOM-killed something (vm.oom_kill moved)	vm.oom_kill	none	yes
reset	a counter went backwards in a way that is not a 32-bit wrap	resets	none	yes
irq-err	the Err row of /proc/interrupts moved	irq_err	none	yes
flash-bad	a YAFFS partition’s bad-block count rose since the previous sample	flash[<device>].bad_blocks	none	yes
kmsg<=N	a kernel-log record at severity N or more severe (0 is emergency, 3 error)	events.level	0–7	kmsg<=3
busy>=X	any core’s busy ratio is at or above X	cpu[N].busy_ratio	0.05–1	no
slip>=X	the sample’s interval was at least X sampler periods	dt_ns/period	1.1–100	no
memfall>=N	MemAvailable fell by N or more in one tick	mem.MemAvailable fall (MB)	1–100000	no

Where a condition covers several cores, queues or partitions, the header names the first one that matched. memfall compares /proc/meminfo’s kB divided by 1 024, so its N is in MiB although the field calls it MB. kmsg<=N needs the kernel log, which the agent can read only in a privileged container (what privileged buys); flash-bad needs a YAFFS partition. A condition whose source is absent never fires.

squeeze is available but not a default. On the reference RB5009 time squeezes are background: a rule that fires on any squeeze fires 92 times in twenty minutes there (RouterOS 7.24.2, 2026-09-15). The collector’s microburst rule therefore asks for an episode of three deviations.

A capture can also be armed by hand, with POST /capture (below). Its cause is manual and its field is the reason you gave.

How a fire becomes a capture

1. A condition is true on sample S. If that condition fired fewer than TRIGGER_REFRACTORY_S × rate samples ago (that many seconds at the nominal rate), the fire is suppressed (reason refractory). If another capture is still collecting its window, the fire is suppressed (reason pending): one capture collects at a time, whatever condition armed it.
2. Otherwise a capture is armed for the window from S − CAPTURE_PRE_S × rate to S + CAPTURE_POST_S × rate, and a {"trigger":{…}} line is queued for the stream.
3. When the sample at the end of the window is in the ring, the capture pins the ring’s lines for that window. If the ring no longer holds any of them, the capture is refused (empty).
4. If the window’s bytes alone exceed the budget, it is refused (budget). If the budget is full, first refuses it (budget) and last evicts the oldest captures until it fits.

A capture whose window holds fewer than (pre + post) × rate + 1 samples is kept with complete: false rather than silently short. That happens when the ring did not hold the whole window: a fire within CAPTURE_PRE_S of the agent starting, or a ring (BUFFER_S) shorter than the window. When the agent stops, it collects a pending capture with what the ring holds, but it cannot serve it: captures are in memory, and the HTTP server stops with the agent.

Captures live in the agent’s memory. Nothing writes them to disk, so a restart of the container, an upgrade or a reboot loses the ones not yet downloaded.

What a capture weighs

A capture’s size is its window’s sample count times the line size. The mean line measured on the RB5009 (RouterOS 7.24.2, 10 Hz, every source of that date, 2026-09-12) was 2 439 B; lines at the default floors were not measured. That gives, by arithmetic and not by measuring captures:

Rate	Default window (5 s + 5 s)	Captures in the 4 MiB default
10 Hz	101 samples, about 250 kB	17
50 Hz	501 samples, about 1.2 MB	3
100 Hz	1 001 samples, about 2.4 MB	1

A bigger board — more cores, more interrupt lines — has longer lines. The bytes field of each capture is the real figure.

The budget is memory the agent holds beyond its ring: a pinned line stays alive after the ring has moved past it. So the agent counts it at start in the same check as the ring. When the agent can read the container’s memory.max and about rate × buffer × 2.56 kB plus CAPTURE_MB exceeds it, the agent refuses to start, naming the three settings to lower or --memory-max to raise. When MEM_LIMIT_MB is above 0, it warns when twice that exceeds its Go soft memory limit; the agent’s own default for MEM_LIMIT_MB is 14, and install writes 40. The cost of the observer explains why that second ratio matters.

Reading captures over HTTP

Four endpoints on the agent. Each needs the bearer token when the agent has one, and each answers 404 with captures disabled (CAPTURE_MB=0) when the feature is off.

Request	Answer
GET /captures	The index, as JSON.
GET /captures/<id>	One {"capture":{…}} header line, then the sample lines verbatim, as NDJSON.
DELETE /captures/<id>	Frees that capture’s share of the budget; 204.
POST /capture?reason=…	Arms a manual capture at the newest sample: {"id":N,"armed":true}, or 409 when a capture is pending or the manual trigger is in its refractory window. The reason defaults to operator.

curl -s -H "Authorization: Bearer $MIKROSCOPE_TOKEN" http://172.30.10.2:9123/captures
curl -s -H "Authorization: Bearer $MIKROSCOPE_TOKEN" http://172.30.10.2:9123/captures/3 > cap3.jsonl
mikroscope plot --in cap3.jsonl
curl -s -X DELETE -H "Authorization: Bearer $MIKROSCOPE_TOKEN" http://172.30.10.2:9123/captures/3

The index carries the policy, budget_bytes, the bytes held, the capture still pending if there is one, the configured triggers, and one entry per capture: id, cause, condition, field, value, threshold, fire_seq, fire_mono_ns, fire_wall_ns, first_seq, last_seq, samples, bytes and complete.

The sample lines of GET /captures/<id> are byte-identical to what /snapshot serves for the same samples, so a tool that reads a snapshot needs no new parser; plot skips the header line. The CLI has no verb for captures: use any HTTP client.

Captures need the direct path

The relay transport pulls through /tool fetch, which returns at most 64 512 B and sends no token. A capture at the defaults is about 250 kB. Fetch captures from a host that reaches the agent directly, or through --expose: reaching the agent covers both.

The trigger line, and what the collector does with it

When a capture is armed, the agent places one line before the sample it fired on, in /stream and in /snapshot?since= (not in /snapshot?seconds=):

{"trigger":{"id":3,"cause":"busy>=0.95","field":"cpu[2].busy_ratio","value":1,"threshold":0.95,"seq":48213,"wall_ns":1789000000000000000}}

The values above are illustrative. The line is a kind of its own, like the {"gap":…} line, not a field on the sample, so the sample schema is unchanged. The agent keeps the last 64 of them for pullers, so a puller more than 64 fires behind never sees the older ones; a suppressed fire produces none. A manual capture fires on the newest sample already in the ring, so a puller that has already received that sample gets no trigger line for it; read /captures instead.

forward recognises the line, never mistakes it for a sample, counts it, and hands it to every sink as an annotation: the mikroscope_trigger measurement in InfluxDB and table in SQL, mikroscope_collector_triggers_total{cause} on the collector’s Prometheus exposition, and the line itself in the file sink. Both Grafana dashboards carry a triggers annotation, off by default in the toggle bar: on InfluxDB it reads the mikroscope_trigger rows, on Prometheus the agent’s mikroscope_trigger_fired_total. The capture itself stays on the agent, under /captures/<id>.

record does not recognise the line yet — Record, mark, plot says what it does with it.

Counting what was not captured

The agent’s /metrics carries the families that say how much the captures did not see. Every condition and reason pair is rendered from the start, at 0 until it happens, so a dashboard can show “0 so far”.

Family	Type	Meaning
mikroscope_trigger_fired_total{condition}	counter	Times each condition armed a capture; condition="manual" appears once a manual capture has been armed.
mikroscope_trigger_suppressed_total{condition,reason}	counter	Times a condition was true and nothing was armed: refractory or pending.
mikroscope_capture_refused_total{reason}	counter	Captures collected and then not kept: budget or empty.
mikroscope_captures_held	gauge	Captures currently retained.
mikroscope_capture_bytes	gauge	Ring bytes the retained captures pin.
mikroscope_capture_budget_bytes	gauge	The budget, from CAPTURE_MB.
mikroscope_capture_bytes_served_total	counter	Bytes handed out over /captures/<id>.

The collector cannot recompute these from the samples, so the Prometheus dashboard expects a scrape job on the agent itself that keeps only the agent-only families; Prometheus has the job.

What it cannot do

Stated because each of these will happen:

• The capture set is a sample of events, never a census. The refractory window and the byte budget bound a trigger storm, and one capture collects at a time. mikroscope_trigger_suppressed_total and mikroscope_capture_refused_total are how much was not seen.
• Full rate is not full detail. A capture holds samples at the sampler’s rate: at 10 Hz nothing shorter than 100 ms is reliably visible, and a busy ratio still moves in the kernel’s tick steps. The resolution floor sets that limit, not the capture.
• A window can be short. One the ring did not wholly hold is served with complete: false. One cut off by the agent stopping is collected but never served, because the captures stop with the agent.
• Downloading costs the router. A download runs on the same core as the sampler, and is counted in mikroscope_capture_bytes_served_total the way any puller is charged.

Not measured, so not claimed

The cost of a fire on the device — 3 µs is the design’s estimate. How the agent behaves under a sustained trigger storm. Any capture at 50 Hz or 100 Hz. The capture sizes in the table above, which are arithmetic from the line size, not the sizes of captures taken on the RB5009.

See also

• Record, mark, plot: a recording you start, with markers, and the chart plot draws from a capture too.
• The agent’s HTTP endpoints: /captures, /stream and /snapshot beside the rest.
• Prometheus metric families: the trigger and capture families with every other one the agent exposes.
• The resolution floor is the kernel’s: what full rate can and cannot resolve.