Five dashboards, one panel list

dashboards/ holds one Grafana dashboard per store, both generated by mikroscope dashboards gen from a single panel list in internal/dashboards. This page answers what is on them: which sections, which panels in each, what the Overview shows before anything is expanded, and which panels exist on only one of the two stores. Importing them and checking them against a live Grafana is on Import and check; the alert rules generated beside them are on Alert rules.

What gen writes

• Directorydashboards/

  • mikroscope-influxdb.json 171 panels, InfluxDB 3 (SQL)
  • mikroscope-prometheus.json 133 panels, Prometheus
  • mikroscope-postgres.json 156 panels, PostgreSQL / TimescaleDB
  • mikroscope-graphite.json 41 panels, Graphite
  • mikroscope-elasticsearch.json 30 panels, Elasticsearch
  • mikroscope-alerts-influxdb.yaml 10 rules
  • mikroscope-alerts-prometheus.yaml 11 rules
  • mikroscope-alerts-postgres.yaml 8 rules

mikroscope dashboards gen                    # writes the four files into ./dashboards
mikroscope dashboards gen --out /tmp/dash    # or into another directory

All five dashboards are in Grafana’s shareable export format: the datasource is a ${DS_MIKROSCOPE} placeholder declared in __inputs, there is no id, and the uid is fixed — mikroscope-influxdb and mikroscope-prometheus — so a re-import updates the same dashboard in place instead of creating a second one. A test asserts that two generations of the InfluxDB dashboard are identical, and regenerating on 2026-09-15 reproduced all four committed files byte for byte.

The InfluxDB dashboard queries InfluxDB 3 in SQL, and every query is bounded by $__timeFilter, because InfluxDB 3 Core refuses unbounded scans; where a column is named cluster the SQL quotes it, because cluster is a reserved word in DataFusion. The Prometheus dashboard queries a Prometheus that scrapes the collector’s /metrics — which carries everything the agent’s does, recomputed from the samples, plus the collector’s own derived and detection families — and the agent itself for the few families only the sampler produces. Prometheus 3 renders a histogram’s zero bucket boundary as le="0.0", so a query reading that bucket matches le=~"0|0.0". The two scrape jobs are on Import and check.

What it looks like

One capture per section of the InfluxDB dashboard, in the order the dashboard puts them. Each is a link to the file at full size.

These are a demonstration database, not a router

Every capture below is the InfluxDB dashboard over a run of the same canned fake agent the end-to-end suites use, written into a container store by test/e2e/docker’s fill step and photographed by site/scripts/gen-dashboard-captures.mjs. Nothing in them comes from a real device, the host is called rb5009 because the fake agent imitates that board’s captured /proc, and the figures are whatever the fake publishes — read them as “this is the shape of the page”, never as a measurement. A panel that is empty in a capture is one the fake agent produces nothing for; on a real router with the API tier running, several of them fill in, and two whole sections that the store probe moved into “not available” come back.

Overview — 12 panels

CPU and scheduler — 11 panels

Memory and load — 9 panels

Connections — 9 panels

Detections and captures — 5 panels

Network receive path — 9 panels

Forwarding cost (derived) — 4 panels

Interrupts and softirqs — 12 panels

Temperature and clock — 10 panels

Kernel log — 7 panels

CPU: how long a core stayed busy — 3 panels

Memory: fragmentation — 3 panels

Memory: reclaim and page faults — 8 panels

Memory: detail and cross-checks — 5 panels

Hardware counters (PMU) — 11 panels

Flash wear — 7 panels

NAND health (ECC) — 2 panels

The observer — 11 panels

The observer: sampler timing and self events — 3 panels

This device — 3 panels

One list, two stores

Every panel is declared once, with its SQL and its PromQL side by side, so a panel added to one store is added to the other. Where a panel has no query for one store, the generator drops it from that store’s dashboard rather than shipping it to read “No data” forever. A section all of whose panels are dropped emits no row at all.

That is why the counts differ, and they differ in both directions. Most of the thermal and clock family, the flash wear section, the slab census and several memory panels have SQL and no PromQL, so they exist only on InfluxDB. The sampler’s timing histograms, the trigger suppressions and the capture budget live on the agent’s /metrics only. The busy run still in progress and the age of each held reading are Prometheus exposition families, on the collector’s /metrics too, with no InfluxDB field. Neither group is written to InfluxDB, so the panels reading them exist only on Prometheus. Each panel below that is on one store only says which.

Panels per section, per store

Section (Grafana row)	InfluxDB 3	Prometheus	PostgreSQL	Graphite	Elasticsearch
Overview	12	12	12	10	6
CPU and scheduler	11	9	11	1	no row
Memory and load	9	9	9	7	5
Connections	9	3	9	1	1
Interface traffic	13	11	9	2	no row
Detections and captures	5	4	5	no row	no row
Network receive path	9	7	9	3	3
Forwarding cost (derived)	4	4	4	no row	no row
Interrupts and softirqs	12	10	12	2	2
Temperature and clock	10	2	10	2	2
Kernel log	7	6	no row	no row	no row
CPU: how long a core stayed busy	3	4	3	no row	no row
Memory: fragmentation	3	1	no row	no row	no row
Memory: reclaim and page faults	8	6	8	2	1
Memory: detail and cross-checks	5	3	5	1	1
Hardware counters (PMU)	11	7	11	no row	no row
Flash wear	7	no row	7	1	1
NAND health (ECC)	2	2	2	no row	no row
RouterOS API cross-checks — CPU and memory	9	9	9	1	no row
The observer	11	9	10	3	3
The observer: sampler timing and self events	3	7	3	no row	no row
This device	3	3	3	no row	no row
Not available on this device	5	5	5	5	5
Total	171	133	156	41	30

The counts are those of the committed files, which carry the compiled defaults. import and check ask the datasource what it holds first and can move panels into or out of the last row; see Not available on this device below.

Reading order

The sections are ordered by how often they are opened, not by taxonomy (owner, 2026-09-13: “usually the first thing you want to see is CPU, RAM, connections and the like”; in the original, “normalmente lo primero que se quiere ver es cpu, RAM, conexiones y cosas así”). First the four questions an operator arrives with — how busy is it, how much memory is left, how many connections is it holding, how much traffic is moving — then what the collector and the agent flagged, then the families that explain the first four when one looks wrong, then the deep tiers a reader goes to deliberately, and last what mikroscope costs the router it is measuring.

Every section but the Overview ships collapsed. Grafana keeps a collapsed row’s panels inside the row object and runs none of their queries until someone expands it, so the first render asks the store for the Overview’s twelve panels and not for all 171.

The defaults are a 3-hour range (now-3h) and a 5-minute refresh. The 5-minute refresh is kept for the case where someone expands a section: the slab census and the PMU and per-sample cost heatmaps ship one row per sample, and maxDataPoints does not apply to raw SQL. The 3-hour range is kept because now-15m opened on 108 empty panels with the agent stopped, and with it running a 15-minute window of 100 ms samples drew unreadable walls of noise. A live investigation sets its own range and refresh.

The Overview

The one open section. It answers “is this router healthy right now” and, before that, “can I believe these numbers”. No panel in it is a heatmap or a join across measurements, and every panel returns an aggregate; three tiles (“Reboots in the window”, “Sample continuity” and “Ticks never delivered, this window”) compute it with a window function over the window’s raw samples. Most of them are copies of panels that also live in a later section, because a Grafana panel belongs to exactly one row. “Connections tracked right now”, “Load average (1 min) against the core count”, “Reboots in the window” and “Detections in the window” live only here, as does “OOM kills in the window” on Prometheus, where the Reclaim section’s copy has no PromQL and is dropped. The copies share their SQL; on Prometheus the Overview’s “OOM kills in the window” and the Observer’s “Ticks never delivered, this window” differ from their counterparts (the Observer’s carries a third query, the sampler’s slipped ticks).

1. CPU busy per core, Memory in use, against the kernel’s own total (a gauge of (MemTotal − MemAvailable) / MemTotal, orange at 75 %, red at 90 %) and Connections tracked right now (the nf_conntrack slab’s active objects; blank on an unprivileged container, and the tile says so).
2. Interface throughput — rx above, tx below (API tier; blank with --api-mode off, and the tile names the flag), Die temperature by zone and Load average (1 min) against the core count, where the core count is measured from the store rather than assumed.
3. Sample continuity, full width: each bin, never narrower than a minute, classified as continuous, ticks missing or agent restarted from the first difference of the sample sequence number. On Prometheus, which has no sequence number, the lane approximates this from the collector’s gap counter and resets of the agent’s sample counter, and its description says so. Every other panel on the dashboard should be read against this lane.
4. Packets dropped in the kernel RX path (window total), OOM kills in the window, Reboots in the window and Detections in the window — each zero on a healthy device and colored on its own thresholds.
5. Ticks never delivered, this window, beside the count of agent restarts.

One screen, measured once

On 2026-09-14 an eleven-panel Overview measured 1 052 px tall in a 1 080 px browser viewport — one desktop screen with nothing cut off. The Overview carries twelve panels, and the grid wraps “Ticks never delivered” onto its own full-width line; that height has not been measured in a browser, though the committed JSON of both dashboards makes it 31 grid units, row header included. On a phone it is not one screen: Grafana stacks a 24-column row into one column below about 768 px, and an eight-panel Overview rendered 2 188 px tall against an 844 px viewport on 2026-09-12.

The four everyday questions

CPU and scheduler

/proc/stat ticks and the sampler’s own cadence. Every per-core series comes from the data (GROUP BY cpu, by (cpu)), so a 2-core and an 8-core board each draw their own lines.

• CPU busy per core
• Busy per core, p95 of one-second means
• Worst sample interval, relative to the window's median
• Per-core busy as states — which core paid, and when
• Where the ticks went — device share by mode
• softirq share of busy time, per core
• Busy-tick distribution per sample
• Share of samples the tick counter called completely idle
• Cycles retired in samples /proc/stat called idle — InfluxDB only
• nice, irq and iowait ticks in the window
• Tick accounting closes — InfluxDB only

Memory and load

The levels from /proc/meminfo and /proc/loadavg, averaged or maxed over a bin and never summed. Every “percent of RAM” divides by the kernel’s own emitted total.

• Memory in use, against the kernel's own total
• Load average, all three windows
• Runnable threads out of total
• Memory by category, as a share of total
• Free memory — three definitions, against the ceiling
• Commit headroom — Committed_AS as a share of CommitLimit
• Load average (1 min) per core
• Threads on the whole router
• Slab — the conntrack and route tables the netns hides

Connections

The router’s connection table from the global slab allocator (/proc/slabinfo, privileged only), next to the API tier’s count where it is polled. An empty section here means an unprivileged agent, not an idle router; the “Slab caches reporting” tile exists to say which.

• Connection table, two ways — slab objects against the RouterOS API count
• Connection churn floor — peak-to-trough swing inside each bin — InfluxDB only
• Packet-buffer and large-allocation caches — InfluxDB only
• Every slab cache, normalized to its own window minimum — InfluxDB only
• Connection count distribution over time (nf_conntrack) — InfluxDB only
• Slab census — latest, min, max and spread per cache — InfluxDB only
• Connection table occupancy
• Connections as RouterOS counts them (API poll)
• Slab caches reporting (is this a privileged deployment?) — InfluxDB only

Interface traffic

/interface/monitor-traffic, the MAC’s per-port counters and /system/health from the RouterOS API — the only per-interface counters mikroscope has, because the container’s /proc/net/dev describes its own veth — beside the configuration the API tier reads at start and every --labels-every: what each interface is. The rates arrive already per second and are never summed, and neither are the per-port counters, because RouterOS counts a different thing on each type. A switch port counts its wire, hardware-forwarded frames included; the bridge counts its CPU side; a VLAN or a PPPoE counts what the CPU sent and received. On the reference RB5009 (RouterOS 7.24.2, 2026-09-16) ether1 had received 255.8 GB on the wire and 29.7 GB of it had reached the CPU: ether1 and bridge are different planes, and neither is a subset of the other. The rate panels’ interface set is whatever --interfaces selected, so an interface missing from one may be idle, unconfigured or not selected, and the three look the same.

• Interface throughput — rx above, tx below
• Interface packet rate — rx above, tx below
• Mean packet size per interface
• CPU cost of forwarding: cpu-load against the busiest interface's packet rate — InfluxDB only
• Port errors per bin — typed, from the MAC counters
• Where a port's receive bytes went — switched in hardware, CPU fast path, CPU slow path — InfluxDB only
• Link flaps — link-downs per interface, per bin
• Frame size mix over the window, per Ethernet port
• Interface drops — rx, tx and tx-queue
• What each interface is: type, role, bridge and label
• Interface inventory and window summary
• /system/health sensors, as RouterOS reads them
• API tier coverage — which sources delivered, and when

“What each interface is: type, role, bridge and label” is the table to read before two interface series are compared: one row per interface the router has — RouterOS type, interface lists, the bridge it is a port of, its comment, its default name and its MTU — from three configuration reads, never per poll. “Frame size mix over the window, per Ethernet port” is one row per port and carries no total: each tx-rx-* counter holds both directions of its own port, so a frame switched from ether1 to ether8 is counted on both, and a sum over ports counts every switched frame twice. “Interface inventory and window summary” carries no error columns, because monitor-traffic on RouterOS 7.24.2 returns no error rate; the MAC’s typed error counters have their own panel.

What was flagged

Detections and captures

Events, not levels: what the collector’s derive stage and the agent’s triggered capture said about the window. Empty is the healthy state, and the event panels are marked known-empty so that check does not fail on it.

• Detections per bin, by rule
• Memory pressure state
• Detections in this window — InfluxDB only
• Trigger fires and suppressions per bin
• Captures held on the agent, and the budget they pin — Prometheus only
• Trigger markers in this window — InfluxDB only

In “Trigger fires and suppressions per bin”, the suppressions are on Prometheus only: they live on the agent’s /metrics.

The red dashed lines on every panel

A dashboard with detections in its window draws a vertical red dashed line, with a small triangle at the foot of the axis, across every panel at the instant of each one. They are annotations, not data: not a gap in the record, not a slipped tick, not a break in the series. They are there so that whatever panel you happen to be reading can be read next to what the collector flagged at that moment — a memory-stall regime on one core, a microburst on one queue, a link that flapped — without having to scroll to the Detections section to find out that anything happened at all.

Two layers ship with every dashboard:

Layer	Colour	Default	What each marker is
detections	red	on	one row of mikroscope_detection: the rule, its key and its message, from the collector’s derive stage
triggers	orange	off	one capture marker from the agent: the condition that fired, the field and the value

Hovering a marker shows the message the row carries, so the line answers what as well as when. The triggers layer is off by default so that a quiet dashboard stays quiet; turn it on when you are working with triggered captures.

Turning them off. Both layers are checkboxes in the submenu row under the dashboard title — the same row a dashboard’s variables sit in, which on the InfluxDB, Prometheus and PostgreSQL dashboards holds nothing else. Click the layer’s name to hide its markers. That is a view setting: it lasts for the session, and saving the dashboard keeps it. Nothing about the underlying rows changes, and the Detections section still counts them.

On InfluxDB each annotation is one event row with its message; on Prometheus it is increase(…[1m]) > 0 at a 1-minute step, so an annotation there marks the minute, not the instant.

The section captures further up are taken with both layers off, because the canned fake agent that fills the demonstration database fires a detection every few seconds and twenty minutes of that photographs as a solid red wash. This is the same overview with the detections layer on, over a demonstration store carrying three of them — the density a real deployment produces, and the tile at the bottom right counts the same three:

The detections layer, on: one red dashed line per detection, on every panel at once.

Why one of the four looks the way it does

Network receive path

/proc/net/softnet_stat, which is global even inside the container’s network namespace. No absolute packet-rate band survives here: the squeeze regime is banded as a multiple of the window’s own median.

• RX path: packets processed per second, per core
• Squeeze rate: softirq budget exhaustions per second, per core
• Squeeze pressure: budget exhaustions per 1 000 packets
• Receive-path balance across cores
• Packets dropped in the kernel RX path (window total)
• Squeeze regime per core, as a multiple of this window's median
• Burst distribution: packets per sample (all cores) — InfluxDB only
• Squeeze against throughput (1 s points, whole window) — InfluxDB only
• Packets per NET_RX poll, per core

Forwarding cost (derived)

The per-packet ratios and the fast-path share the collector derives across subsystems — PMU against softnet, softnet against interrupts, RouterOS port counters against each other.

The fast-path share is the share of the traffic an interface hands the CPU, not a share of the wire: fp-rx-byte over driver-rx-byte on a switch port, over rx-byte on a software interface, with hardware-switched frames in neither number. That split is the “Where a port’s receive bytes went” panel’s, from the raw counters. On the reference RB5009 (RouterOS 7.24.2, 2026-09-16) the switch ports read about 100 % — fp-rx-byte equals driver-rx-byte there to within a few kB — while bridge had fast-pathed 211.9 GB of 663.0 GB since boot (32 %) and PPPoE_DIGI 99.97 %. The tx line is usually absent: fp-tx-byte stayed 0 on every interface of that router after hundreds of GB transmitted, so the collector withholds the tx share while the cumulative counter is 0 rather than draw a 0 % it did not measure. Per bin the share is bytes-weighted — the bin’s fast-path bytes over the bin’s bytes — because an interface moving a few packets per poll swings 0–100 % from poll to poll, as that router showed on 2026-09-15. The Prometheus form is the collector’s per-poll gauge and keeps that noise.

• Cycles, instructions and cache misses per packet
• Packets per device interrupt (NAPI coalescing depth)
• Fast-path share of the traffic each interface hands the CPU
• Sub-sample bursts: squeezed samples whose packet count looked ordinary

Interrupts and softirqs

/proc/interrupts and /proc/softirqs deltas. The device name is recovered from the raw /proc/interrupts text in the query, not matched against a driver name.

• Interrupts per second by source (top-K)
• Per-line interrupt load on the device with the most lines
• Receive-path IRQ imbalance (max / mean across a device's lines)
• Which core takes each interrupt
• Top-K interrupt coverage
• Interrupt mix right now
• Top-K membership — which interrupt sources the agent was watching — InfluxDB only
• NET_RX softirq invocations per core
• Softirq invocations by kind (all CPUs)
• NET_RX burst size distribution per sample — InfluxDB only
• µs of softirq CPU per invocation
• NET_TX and TASKLET softirq invocations per second

Temperature and clock

/sys/class/thermal and cpufreq. Every thermal threshold is measured from the zone’s own critical trip point, read from the board: the headroom panel’s thresholds sit at 10 °C and 5 °C of remaining margin below it. No panel carries a temperature the board did not publish.

• Die temperature by zone
• Temperature by source — kernel zones, and the RouterOS sensor when it is polled — InfluxDB only
• Thermal slope, °C per minute by zone
• Sensor step dwell — where each zone actually sat — InfluxDB only
• Latest temperature by zone — InfluxDB only
• Thermal headroom — how far each zone is from its own critical trip — InfluxDB only
• Gap between the hottest and coolest thermal zone — InfluxDB only
• Core clock per core — governor state — InfluxDB only
• Is the clock pinned? — InfluxDB only
• Clock-weighted work: effective Hz per core — InfluxDB only

Kernel log

/dev/kmsg, privileged only. The log is not absent when it is empty, it is silent, and silence is the healthy state.

• Worst kernel-log severity in each bin
• Kernel-log records per second by severity
• Warning-or-worse kernel-log rate now
• Kernel records in the window, by severity
• Kernel records per sample — burst distribution against the per-tick cap — InfluxDB only
• Port events from the kernel log, per port and kind
• Port events in the window, per port

Two of them read the records that name a network port. Each such record carries a kind — link-up, link-down, stp-<state>, own-address (the bridge receiving a frame with its own MAC as the source, the layer-2 loop signature) or other — classified by the agent as it reads /dev/kmsg, and by the collector for records from an agent that does not classify. The collector then replaces the board’s default port name with the interface’s current RouterOS name and attaches its comment and interface lists from the API tier’s inventory; without the API tier the record keeps the default name and gets no label. “Port events from the kernel log, per port and kind” counts them per bin, one series per port and kind, and “Port events in the window, per port” is the census beside it: one row per port, with the label, the role and a column per kind. This is the per-port view the container has for free — the kernel log costs the router nothing and dates each transition to the microsecond, where the RouterOS per-port counters need an API read every poll — and it is blind to anything the kernel never hears of, the hardware-switched frames included. A link-up is followed by stp-blocking, stp-learning and stp-forwarding on its bridge port: four records, not four faults. Both panels are marked known-empty, because a quiet set of ports is the healthy state. The Prometheus form reads the collector’s /metrics, where every port record carries a kind because the collector classifies whatever reaches it unclassified; the agent’s own exposition carries the label only when the agent itself classifies, which the one on the reference router does not on 2026-09-16.

On InfluxDB the mikroscope_kmsg table is created by the first kernel record written, and its kind column by the first port record written with one. On a router whose kernel has not spoken since the collector started, the table does not exist and InfluxDB 3 refuses the query when it plans it; the collapsed section keeps that query from running until someone opens it. The code calls this a mitigation, not a fix. import with the probe routes those panels into the not-available row instead.

The deep tiers

CPU: how long a core stayed busy

Contiguity, which no per-sample histogram can recover: a 2 s plateau and twenty scattered spikes bin the same. On InfluxDB the run lengths are a query over the raw samples; on Prometheus they are the upper edge of the largest populated bucket of mikroscope_cpu_busy_run_seconds, clamped at 60 s, so 60 reads as “longer than a minute”, not as a measurement.

• Longest run at or above 90 % busy, per cpu
• Longest run at or above 50 % busy, per cpu
• Busy run in progress right now — Prometheus only
• Blocked tasks and forks

Memory: fragmentation

/proc/buddyinfo, the one memory number /proc/meminfo cannot give.

• Free memory by block order (pages) — InfluxDB only
• Free blocks per order (count)
• Largest block order with any free block — InfluxDB only

Memory: reclaim and page faults

The /proc/vmstat deltas, summed or rated over a bin and never averaged — a separate section from the levels so the two kinds of reducer cannot be mixed.

• Reclaim efficiency — pgsteal ÷ pgscan
• Did the kernel have to reclaim at all?
• Allocation distress — stalls and swap
• OOM kills in the window — InfluxDB only
• Page churn — allocate, free, and the net
• Page faults — minor and major per second
• Minor-fault bursts — distribution per sample — InfluxDB only
• Context switches and all interrupts per second

Memory: detail and cross-checks

Writeback, the LRU, the small levels, and two panels that validate the instrument itself. These are read once per new device, not during an incident.

• Writeback backlog — dirty pages and pages in flight
• LRU balance — active vs inactive
• Mapped, kernel stacks and page tables
• vmstat pages vs meminfo kB — unit cross-check — InfluxDB only
• Kernel stack per thread — InfluxDB only

Hardware counters (PMU)

perf_event_open from the privileged container. Every panel divides two raw counts; the clock-normalized ones divide by the frequency the kernel reported for that core in that sample.

• IPC per core (instructions retired / cycles)
• Beneath the tick floor: PMU cycles against /proc/stat busy ticks — InfluxDB only
• Instructions retired while /proc/stat reported the core idle — InfluxDB only
• Work the jiffie threw away (selected range) — InfluxDB only
• Core cycles per bus cycle
• Unhalted-cycle fraction of the clock, per core
• Distribution of cycles per sample, as a share of the clock (all cores pooled) — InfluxDB only
• Cache-miss rate per core (misses / references)
• Cache misses per 1 000 instructions (MPKI), per core
• Branch mispredictions per 1 000 instructions per core
• PMU counters this CPU actually opened

Flash wear

/proc/yaffs, the only NAND wear signal on a RouterBOARD. InfluxDB only: the section has no row on the Prometheus dashboard. No panel can show a share of the partition, because the agent does not parse the partition’s block range, so there is no denominator for one.

• Flash page traffic per YAFFS partition (pages/s)
• Write amplification — GC copies per page write
• Flash housekeeping — erasures, garbage collections and GC copies per bin
• YAFFS free-chunk drift within the window (chunks, relative to the first sample)
• YAFFS partition state over the window
• Bad blocks retired during the window, per YAFFS partition
• Page writes and erasures per day, at the window's rate

NAND health (ECC)

The MTD ECC counters under /sys/class/mtd, privileged only — the flash’s leading indicator, where the YAFFS bad-block count is the post-mortem.

• ECC corrections since boot, per partition, against the bitflip threshold
• Uncorrectable ECC failures and bad blocks, per partition

RouterOS API cross-checks — CPU and memory

/system/resource and /system/resource/cpu: the independent tier the kernel figures are checked against. RouterOS recomputes these once per second, so every field is 1 Hz at best and integer-quantized; no panel here claims a sub-second reading.

• RouterOS cpu-load vs kernel busy — do the two tiers agree?
• Cross-tier residual: cpu-load − kernel busy, distribution
• Per-core load, RouterOS's own accounting
• Per-core mean over the window: RouterOS load next to kernel busy
• Per-core IRQ time as RouterOS accounts it, against kernel softirq+system
• Per-core disk time (RouterOS) — max over the window
• RAM used, as RouterOS accounts it
• Free memory: RouterOS free-memory vs the kernel's two answers
• RouterOS uptime

What observing costs, and whether it ran

The observer

What mikroscope costs the router it is measuring, and whether it was running. On InfluxDB, continuity is derived from the sequence number of every sample (Prometheus approximates it, as in the Overview), which found 4 493 missing ticks and one restart in the 2026-09-11/12 capture that the collector’s gap record said nothing about.

• Agent CPU cost against its 2 % budget
• Observer effect: agent share of all busy CPU on the router
• CPU per sample: mean and worst tick
• Where the agent's cost actually lives (per-sample distribution over time) — InfluxDB only
• Agent memory against the container cap
• Headroom under the container memory cap
• CPU budget used (window mean)
• Ticks the agent took vs ticks the store received
• Sample continuity
• Ticks never delivered, this window
• Gaps and restarts in this window — InfluxDB only

The observer: sampler timing and self events

The sampler’s own smear — how late it woke and how long the read took — and the cgroup events the agent records about itself. The three heatmaps read the agent’s histograms, which are never shipped as samples, so they exist only on Prometheus.

• Tick interval distribution, relative to the nominal period — Prometheus only
• Wake latency: how late the sampler ran after its ticker — Prometheus only
• Read duration: how long every source took to read — Prometheus only
• Counter resets the agent saw
• The container's own throttling and OOM kills
• How each level source is read
• Age of each held reading — Prometheus only

This device

The device-info stream: what the agent established about the board at start with no RouterOS API — identity, ceilings, the frequency ladder.

• This device, as the agent established it
• Thermal zones: the board's own trip points and polling cadence
• CPU clock: range, ladder, governor and clusters

Not available on this device

The last row is titled “Not available on this device — measurements this kernel or board does not produce (open to read why)” and is collapsed. A panel whose measurement the store does not hold is moved out of its own section into this row, where its description says what it is waiting for.

In the committed files — the compiled defaults, which are what plain gen writes and what a manual upload into Grafana gets — the row holds the five panels the reference RB5009 (RouterOS 7.24.2, kernel 5.6.3 arm64) cannot produce:

• Pressure stall (PSI), where the kernel exposes it
• Block-device queue depth (requests in flight)
• Block-device requests per second (reads and writes completed)
• Block-device busy percent (io_s / wall time)
• Block-device throughput (sectors → bytes per second)

The PSI panel is empty there because that kernel has no /proc/pressure. The block-device panels are empty there because the agent drops a block device whose reads, writes and in-flight count are all zero in a tick, and on that router every listed device stays at zero, so no sink ever creates the table.

Two sections are declared for these panels, Pressure stall (PSI) and Block devices, and ship no row while every panel in them is absent. On a kernel built with PSI, or a board with USB or eMMC storage that moves, import finds the measurement and the section appears in its place with no edit to the generator. import also works the other way: on a store that lacks a measurement the reference device had — or a field added after that store was first written — the panel moves into this row with its query removed, so it shows its explanation and not a red error badge. How the probe decides is on Import and check.

How the dashboards handle ratios, holes and figures

• Ratios are never the agent’s. The agent ships raw counters only, never percentages. A ratio on these dashboards is computed either in the panel’s own query or by the collector’s derive stage (the Forwarding cost section and the mikroscope_derived* families). The API tier’s interface rates are the exception: they arrive from RouterOS already computed.
• They do not draw short holes. A line is broken where two neighbouring points are more than 5 minutes apart. The threshold has to exceed the widest bin a reader selects — at a 2-day range a 12-column panel’s bin is about 4 minutes — so a hole shorter than 5 minutes is still drawn as an interpolated line. Read holes from Sample continuity, not from the shape of a line.
• Their figures are one router’s. Descriptions that quote a figure attribute it to the reference RB5009 with the date it was measured and say yours will differ. No title, query or threshold names a device, its core count, its interfaces or its memory size; the only fixed numbers are mikroscope’s own budget targets (2 % of one core, 16 MiB).

Grafana versions

The dashboards were checked on Grafana 12.3.2 (2026-09-12) and Grafana 13.2.1 (browser passes on 2026-09-12 and 2026-09-14, check and render on 2026-09-15, check again on 2026-09-16). The panel options are written to the schema Grafana 13.2.1 expects — the xychart’s mark, for one, moved between Grafana 11 and 13 — and __requires declares Grafana 11.0.0. No version other than those two has been tried.

See also

• Import and check: the datasource, the two Prometheus scrape jobs, and what check does and does not verify.
• Alert rules: the rules generated beside the dashboards, and where each threshold comes from.
• Detections: the events behind the Detections section and the annotations.
• Prometheus metric families: what each Prometheus panel reads.