# Record, mark, plot How to pull a window of full-rate samples to your machine, mark the moments in it, add the router’s own log, and draw it as a deterministic SVG. Source: https://jmrplens.github.io/mikroscope/record/ The collector keeps a router under watch; `record` is for the other question — _I am about to change something, what does it actually do?_ This page answers how to take a recording from your machine, what the four files it writes hold, how markers get into it and in whose clock, how the router's log becomes part of it, and what `plot` draws. ## Three verbs ```sh mikroscope record --for 5m --out cap # cap.jsonl, cap.csv, cap.markers.csv, cap.meta.json; type lines to mark mikroscope mark --out cap "queue tree applied" # a note stamped with the current time (see below) mikroscope mark --out cap --log-markers # the router's own log lines, over the API mikroscope plot --in cap # cap.svg, deterministic ``` All three verbs share one flag set, so every flag below is accepted by each of them; the notes say which verb acts on it. | Flag | Default | What it does | | --------------- | ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--out` | `capture-` | Output prefix: `.jsonl`, `.csv`, `.markers.csv`, `.meta.json`. `mark` needs the prefix of an existing recording. | | `--for` | `0` | `record`: stop after this long. `0` records until Ctrl-C. | | `--from-start` | off | `record`: backfill everything the agent's ring holds before going live. | | `--poll` | `500ms` | How often to pull the agent's ring. | | `--batch` | `0` | Samples per pull. `0` is twice what one `--poll` interval produces at the agent's rate, at least 20; the relay caps a pull at 18. | | `--transport` | `auto` | `auto`, `direct` (HTTP to the veth) or `relay` (`/tool fetch` over the RouterOS API). | | `--log-markers` | off | `record`: adds the router log lines of the window once the recording stops. `mark`: adds the log lines of the recording's window. | | `--topics` | `system,interface,container` | Log topics kept as markers; add `firewall` or `script` when their lines are the story. | | `--router-tz` | `Local` | IANA zone the router's clock shows. RouterOS log times carry no zone. | | `--api` | `MIKROSCOPE_API_ADDR` | RouterOS API `host:port`, for the relay and for `--log-markers`. | | `--api-user` | `MIKROSCOPE_API_USER` | The API user. The password is read only from `MIKROSCOPE_API_PASSWORD`; there is no flag for it. | | `--token` | `MIKROSCOPE_TOKEN` | The agent's bearer token. | | `--port` | `9123` | The agent's HTTP port. | | `--subnet` | `MIKROSCOPE_SUBNET`, else `172.30.10.0/30` | The agent's /30; its address is `.2`. | | `--in` | none | `plot`: a recording prefix, or the path of a `.jsonl` file. | | `--svg` | `.svg` | `plot`: the output file. | | `--title` | the prefix | `plot`: the chart's title. | ## What a recording has been measured to deliver Measured on RB5009UG+S+ · 4 × 1.4 GHz Cortex-A72 · RouterOS 7.24.2 · 2026-09-12 · a 60 s `record` at 10 Hz, 600 samples over 59.9 s, a RouterOS script loop started over ssh and the router log added as markers That recording yielded exactly 600 samples, 0 gaps and a clock skew of −7 ms. In it, a scripted RouterOS loop (`:for … 400 000`) showed as one core's worth of load at 100 % from t = 21.0 to 25.8 s, with the sample at 25.8 s reading about 60 %, onset and offset resolved to 100 ms, and the router's log markers explained a 2 s plateau at 15–17 s that nobody had caused: the `ensure-ipv6-nd-prefix` scheduler. [Five minutes with a router](/mikroscope/start/walkthrough/) runs `record` and `plot` end to end on a different capture: 70 s at 10 Hz with the router otherwise at rest, with three notes typed into `record`'s terminal. > **Not measured, so not claimed** > > A recording above 10 Hz. The lossless 50 Hz and 100 Hz runs on [the rate > ceiling](/mikroscope/cost/rate-ceiling/) were the collector's, not `record`'s. Both use the same > batch sizing, but a recording at those rates has not been measured on its own. Nor has a recording > through the relay at any rate. ## How `record` reaches the agent `--transport auto` tries the direct path first: a `GET /healthz` to `http://<.2 of --subnet>:<--port>`. If that does not answer, it opens the RouterOS API and asks the router to fetch from the agent itself. If `--api` or `--api-user` is missing, it stops with an error that names `--api`, `--api-user` and `MIKROSCOPE_API_PASSWORD` and suggests `install --expose`; a missing password surfaces instead as a login error, `api : …`. The `--expose` hint helps other HTTP clients, not `record`: `record` always dials the `.2` of `--subnet`, and has no flag for the router's LAN address. `direct` and `relay` force one path and fail rather than fall back. - **direct** is plain HTTP from your machine to the veth. It sends `--token` as a bearer token. - **relay** runs `/tool fetch output=user` on the router over the binary API, so the API user needs the `read,api,test` policies. Each relayed call takes either about 3 ms or about 1 s. A reply is capped at 64 512 B, so a relayed pull asks for at most 18 samples, and a reply that reaches the cap is refused rather than parsed truncated. The router-side fetch carries no header: an agent installed with a token can only be recorded over the direct path. > **A full relayed pull can still reach the cap** > > By arithmetic, read from the code on 2026-09-15 and not measured. The cap of 18 samples comes from > the 64 512 B fetch limit, a mean line of 2 560 B, and 134 % headroom; > the mean is the 2 439 B measured on the RB5009 (RouterOS 7.24.2, > 2026-09-12), rounded up. A full pull of mean lines is about 46 kB. A full pull whose lines average > more than 134 % of that mean would still reach the limit and be refused, and how long lines run at > today's default per-source floors was not measured. At 18 samples a pull and the default 500 ms > `--poll` the relay carries 36 samples a second; above that, use the direct path. [Reaching the agent](/mikroscope/install/reaching-the-agent/) covers which path a network allows. The API user is described on [its own page](/mikroscope/security/api-user/). Once connected, `record` reads `/healthz`. The agent's wall clock minus your machine's is the **clock skew**, printed on stderr with the agent's version, rate, newest sequence number, the oldest one the ring still holds, and the transport. The recording then starts live from the newest sample, or from the oldest the ring holds with `--from-start`. Every `--poll` it pulls `/snapshot?since=&max=`. While a pull comes back full it asks again, up to 100 times per poll, so a ring that got ahead is drained rather than followed at a fixed pace. A pull that fails is logged as `pull: …` and the recording carries on; the next pull asks from the same sequence number. When `--for` elapses or you press Ctrl-C, one last pull drains what arrived meanwhile, and `record` prints how many samples it kept, their sequence range, the gaps, the markers, the transport and the files. ## The four files Every file is created with mode `0600`, and an existing file with the same prefix is overwritten. - **`.jsonl`** — each sample line exactly as the agent sent it: raw tick and counter deltas, with every source the agent has. This is the recording; the other files are views of it. - **`.csv`** — one wide row per sample, for a spreadsheet. The column set is sized from the first sample: one group per core and one per softnet queue. - **`.markers.csv`** — `wall_ns,wall_utc,seq,kind,label`, one row per marker. - **`.meta.json`** — written at the start, so that `mark` and `plot` can run later: `started_utc`, `skew_ns` (agent wall minus host wall), `agent` (its version), `rate_hz`, `transport`, and `capabilities` — what the agent established about the board — when the transport could fetch it. The CSV holds a fixed subset of each sample. Everything else a sample carries — kernel-log events, PMU counters, temperatures, interrupts per line and the rest — is in the `.jsonl` only. | Columns | Content | | ------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------- | | `seq`, `wall_ns`, `wall_utc`, `dt_ns` | The sample's sequence number, agent wall clock, and real interval. | | `busy_total` | The mean of the per-core busy ratios, three decimals, computed by the CLI from the ticks. | | `c_busy`, `c_user`, `c_nice`, `c_system`, `c_idle`, `c_iowait`, `c_irq`, `c_softirq` | Per core: the busy ratio, then the raw tick deltas. | | `ctxt`, `intr`, `irq_total` | Context switches and interrupts from `/proc/stat`, and the delta summed over every row of `/proc/interrupts`. | | `softnet_processed`, `softnet_dropped`, `softnet_time_squeeze` | Per softnet queue, the receive path's deltas. | | `mem_free_kb`, `mem_available_kb`, `mem_cached_kb`, `mem_slab_kb` | Memory levels, in kB. | | `load1`, `threads_running`, `threads_total` | Load average and thread counts. | | `pgfault`, `pgmajfault` | Page-fault deltas. | | `self_cpu_us`, `self_rss_bytes` | The agent's own CPU time during the sample, in µs, and its resident memory at that moment, in bytes. | The busy ratio in the CSV is the one number the CLI computes, and it inherits the kernel's floor: a tick is 10 ms, so one core in a 100 ms sample resolves to 10 % steps. [The resolution floor](/mikroscope/limits/) explains why, and why the ratio is capped at 1. ## Markers, and whose clock they are in A marker is a row in `.markers.csv` of one of three kinds: - **`note`** — a line of text you added. While `record` runs in a terminal it prints `type a line and press Enter to add a marker; Ctrl-C stops`, and every non-empty line you type becomes a note. When standard input is not a terminal, nothing is read from it. - **`gap`** — written when the agent reports that samples were no longer in its ring, labelled `samples .. lost`. The same gaps are listed in the summary `record` prints at the end. - **`log`** — a line of the router's own log, added with `--log-markers` (below). Notes and gaps are stamped with your machine's clock plus the skew measured at the start, so they sit on the agent's time axis, not your machine's. `mark --out cap "text"` does the same with the skew stored in `cap.meta.json`: it stamps the current time in the agent's clock, and no flag sets another. On a finished recording the note therefore falls after the last sample, and `plot` does not draw it. `mark` needs that file and an existing `cap.markers.csv`, and its row carries `seq` 0. > **Mark while the recording runs, not after** > > `record` does not hold `.markers.csv` open: it opens the file, appends and closes it for > each marker, which is the same thing `mark --out "text"` does from a second shell. So a > marker typed in one terminal and a marker added from another land in the file in the order they > were made, on Linux, macOS and Windows alike. What a text `mark` > cannot do is mark a moment in a recording that has already stopped: it stamps the current time, > which falls after the last sample, and `plot` draws nothing there. ## The router's own log as markers The log often explains a transient you did not cause: scheduler runs, errors, interface events. `--log-markers` turns the log lines of the recording's window into `log` markers, labelled `: `. ```sh export MIKROSCOPE_API_ADDR=192.168.88.1:8728 MIKROSCOPE_API_USER=mikroscope MIKROSCOPE_API_PASSWORD=… mikroscope record --for 60s --out burst mikroscope mark --out burst --log-markers --router-tz Europe/Madrid ``` - `mark --log-markers` uses the window from `started_utc` in the meta file, shifted by its `skew_ns` into the agent's clock, to the last sample in the `.jsonl`, and prints how many lines it added, the window and the topics. `started_utc` is stored to the second. It appends without checking what is already there: running it twice adds the same lines again. - `record --log-markers` asks for the log once the recording has stopped, for the window from its start to its end in the agent's clock, and appends the lines through the same path `mark` uses, so they land in `.markers.csv` and the summary's marker total counts what the file holds. If the log request itself fails, `record` prints `log markers: …` and keeps the recording. The CLI asks the router only for the window, with a `?>time=` query starting one second before it, and only for the `time`, `topics` and `message` fields: the RB5009 held 66 217 log rows, because its `dns` topic logs to disk. It then keeps the lines inside the window whose topics include one of `--topics`. A router with several logging actions on one topic carries each event once per action, prefixed with the action's name (`[INFO]: …`, `[SYSTEM]: …`); the prefix is stripped and the duplicates are kept once. RouterOS log times carry no zone. RouterOS 7.24.2 prints a full date over the API; the parser also accepts the shorter forms without a year or without a date, and fills them in from the end of the window. They are read in `--router-tz` and stamped as the agent's wall clock — the router's own clock — so no skew is applied to them. The default `Local` is your machine's zone: if the router's differs, pass it, or every log marker lands off by the difference. Log times have one-second resolution, so a log marker places its event to the second, not to the sample. ## The chart `plot --in cap` reads `cap.jsonl`, and `cap.markers.csv` when it exists, and writes `cap.svg` (or `--svg`). It prints the file name with its sample and marker counts. The same input always produces the same bytes. The chart is 1 200 units wide, with a title (`--title`, or the prefix) and a line giving the sample count, the duration and the core count. Three panels share one time axis, in seconds since the recording started: 1. **CPU busy per core, %** — 0 to 100, one line per core, each labelled at its end. The palette has eight colours, so the first eight cores are drawn. 2. **softnet per second, all CPUs** — `dropped` and `time_squeeze`, summed over every queue and bucketed into whole seconds. Drops are drawn in red. 3. **memory available, MiB** — `MemAvailable` per sample, with the y-axis bounded to the data. Every marker is a dashed vertical line across all three panels, grey for notes and log lines and red for gaps, with a label chip above the first panel. Chips are laid out in up to six rows to keep them apart; a label longer than 40 characters is shortened. Consecutive log markers, in time order, that fall in the same second are folded into one chip, `× : `, so a chatty scheduler does not bury the chart; a note or gap between them breaks the fold, and notes and gaps are never folded. Markers outside the recording's time span are not drawn. With a single sample the SVG says `not enough samples to draw`; with none, `plot` stops with `record: no samples` and writes no SVG. The palette is the chart's own, validated for its light background: adjacent-pair colour-vision-deficiency ΔE 9.1, normal-vision ΔE 22.9. There is no dark variant. `plot` also reads a capture the agent kept on a trigger: save `GET /captures/` to a `.jsonl` file and pass it to `--in`. The capture's header line carries no sequence number and is skipped. ## Triggers during a recording When a trigger fires on the agent, a `{"trigger":{…}}` line rides in the same pull as the samples, just before the sample it fired on. The collector recognises it; `record` does not. > **A trigger line in a recording** > > `record` reads a trigger line as if it were a sample with sequence number 0. The line is written > verbatim to the `.jsonl`, where `plot` skips it, and as a row of zeros with `seq` 0 to the `.csv`; > it is counted once in the number of samples `record` reports. If it is the first line of the > recording, the CSV header is sized for zero cores. `record` does not add it as a marker. To see > what fired during a window, read `/captures` on the agent. [Triggered capture](/mikroscope/record/triggers/) explains what fires, and how to fetch the full-rate window the agent kept around it. ## See also - [Five minutes with a router](/mikroscope/start/walkthrough/): a real recording from the RB5009, its markers and its chart, step by step. - [Triggered capture](/mikroscope/record/triggers/): the samples around a condition, kept at full rate by the agent without a recording running. - [Reaching the agent](/mikroscope/install/reaching-the-agent/): which of the direct and relay paths your network allows. - [The API user](/mikroscope/security/api-user/): the RouterOS user the relay and `--log-markers` need.