Skip to content

How to read what it shows

This section answers the question that comes after installing the agent: the numbers are arriving — what does a fault look like in them? Each page is one shape, read from data the agent collected on the reference router, with the commands that produced it so you can reproduce it on your own device.

Every number in these pages comes from one campaign on the reference router, an aarch64 kernel on a board with 1 GB of RAM:

Measured on RB5009UG+S+ · 4 × 1.4 GHz Cortex-A72 · RouterOS 7.24.2 · Linux 5.6.3 · · agent at 10 Hz in an ephemeral privileged container

Where a later page adds a figure from another date, it says so beside the figure. Where a fault was provoked deliberately, the page says exactly how.

The scenarios were chosen so that nothing they do can break the router’s uplink or cut an administrator’s own path to it. Keep that property on your own device: find out which port you are connected through first,

/interface/bridge/host/print where mac-address="<your machine's MAC>"

and leave that port, the WAN port and anything carrying a service alone.

Read the idle shape first. Without it, every other page looks like an anomaly.

Page How it came about Where it shows Signature
The shape of an idle router 60 s at idle per-core busy, time_squeeze, events 6–8 % busy across four cores, squeeze never zero, zero kernel events
A loop only the kernel could see real, found by accident on the production router events (the kmsg source) events at 1.49 /s, 2.00–2.01 s apart; the API reported a healthy device
RouterOS ports and kernel names a dead port toggled off and on events the kernel prints eth5 where RouterOS says ether6
A CPU-bound workload a console loop that ends by itself per-core busy, temperature, frequency ~29 % total, one core at 99.8 %, ~20 s of migration before it settles
A packet flood ping -f at the router’s own LAN address, 10 s switch0 interrupts, softirqs, time_squeeze switch0 5.5 k → 34 k per 5 s bucket, all on the one core the IRQ is pinned to
Flash wear nothing provoked; the router writes on its own the yaffs source, MTD ECC counters 2 page writes per 30 s at idle, traced to the dns topic logging to disk
Conntrack without the API a cross-check against the API, no storm the nf_conntrack slab cache the router’s real connection count where the container’s own namespace reports 0

The sampler was not starved. mikroscope_slipped_total should be 0 over the window you are reading. A slipped tick is one whose read finished after the next tick was due, and the sampler’s own accounting is then the first thing to distrust.

The kernel log was kept whole. mikroscope_kmsg_dropped_total counts loss events, not records: one per tick that hit the agent’s cap of 64 records, and one per kernel ring overrun, which can stand for many records. While it is non-zero, the per-level counts in mikroscope_kmsg_records_total are a lower bound, and so is an event rate taken from them.

Measure the observer, and measure it honestly — including the part where measuring changes the answer.

Pulling a 60-second /snapshot is not free: the agent has to hand over ~600 pre-encoded lines, about 1.5 MB, and self.cpu_us inside those samples includes the cost of serving them. Reading the agent’s cost out of a large snapshot therefore overstates it, and doing it repeatedly overstates it more.

Use /metrics instead. It is small, its counters are cumulative, and it is independent of who scrapes it and when — scrape it twice and divide:

Terminal window
U=http://172.30.10.2:9123/metrics
get() { curl -s "$U" | awk -v k="$1" '$1==k{print $2}'; }
c0=$(get mikroscope_self_cpu_usec_total); t0=$(date +%s)
sleep 180
c1=$(get mikroscope_self_cpu_usec_total); t1=$(date +%s)
echo "$c0 $c1 $t0 $t1" | awk '{printf "%.2f %% of one core\n", 100*($2-$1)/1e6/($4-$3)}'

Two things to expect:

  • Cost and memory rise until the ring fills. With the default 300 s ring at 10 Hz the agent holds 3 000 pre-encoded samples; a figure taken in the first minute after install is measured on a nearly empty heap and will be optimistic. Wait out BUFFER_S before quoting a steady-state number.
  • mikroscope_slipped_total is the number that actually matters. A sampler that costs a little more but never slips is telling you the truth; one that slips is not.

For scale: at the install default the agent costs 2.85 % of one core and 31.3 MiB RSS. That figure was measured on 2026-09-15 with the full source set and three sinks at once, not during the campaign these readings come from:

Measured on RB5009UG+S+ · 4 × 1.4 GHz Cortex-A72 · RouterOS 7.24.2 · · 60 s windows at steady state (ring full), full source set, collector forwarding to a file, a Prometheus exposition and InfluxDB 3 at once