OpenTelemetry

The OTLP sink pushes ghchronicle’s metrics to an OpenTelemetry collector, or straight into Prometheus through its OTLP receiver.

sinks:
  otlp:
    endpoint: http://collector:4318/v1/metrics
    service: ghchronicle
    headers:
      Authorization: Bearer ${OTLP_TOKEN}
    raw: false
    batch: 2000

The wire contract

OTLP over HTTP with the JSON encoding, which every receiver worth using accepts on the same endpoint. That is a deliberate trade: JSON is larger on the wire, and it keeps a code generator, a protobuf runtime and their transitive dependencies out of the binary. Every sink but PostgreSQL, which needs its driver, speaks its protocol with the standard library alone.

endpoint is the full URL of the metrics path, not a base. headers go with every request, which is where an API key or a tenant id belongs. service becomes the resource attribute the backend groups by. batch is how many data points go in one request, 2000 unless it is lowered for a receiver with a smaller body limit.

Metric names use dots, following the OpenTelemetry convention: github.workflow.run.duration.seconds. A receiver exporting onward to Prometheus converts them itself; going the other way it cannot.

raw

raw: false (default)

Sends the same reduced current values as the Prometheus exporter, made by the same reduction, so each mean is taken over the items that carried the field. Safe with any backend, including Prometheus’s own OTLP receiver, because nothing in the payload is older than the sweep.

raw: true

Sends the dated points. Whether they survive is entirely the backend’s decision: OTLP data points carry an explicit timestamp, so a store that accepts old ones keeps the history, and one that does not rejects them.

Prometheus's OTLP receiver is one that does not

Measured: against Prometheus 3.14 with --web.enable-otlp-receiver and out_of_order_time_window: 30m, a sample dated two days back comes back as HTTP 400. With raw: true pointed at Prometheus, roughly half of every sweep is refused. Use raw: false there, or send to a store that keeps the dates.

repeat

A gauge asserted once and then never again fades from the dashboards, and a family that runs every twelve hours would be a flat line with a dot in it. With repeat set, the sink keeps asserting the newest value of every series it has seen until the next sweep replaces it.

sinks:
  otlp:
    endpoint: http://collector:4318/v1/metrics
    repeat: 1m

This matters for a backend that answers an instant query from the last sample within a lookback window.

A collector in front

The usual arrangement is a collector that receives from here and exports onward, which is what makes this sink worth having at all: the tool speaks one protocol and the pipeline decides where the data ends up.

receivers:
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:4318

exporters:
  prometheusremotewrite:
    endpoint: http://prometheus:9090/api/v1/write

service:
  pipelines:
    metrics:
      receivers: [otlp]
      exporters: [prometheusremotewrite]

With that pipeline, keep raw: false: the exporter at the far end is Prometheus, and the constraint follows the data rather than the protocol.

Nothing to migrate

The sink pushes today’s values, in the shape of the release that runs it, so -migrate has nothing to do here. What the receiver keeps of an old series ends where the new one begins.

Where to go next

• Choosing a store compares OpenTelemetry with the others, and holds the write ledger every one of them shares.
• The dashboards says which of the five is drawn against which store, and what a panel a store cannot answer becomes.