Resource Benchmark

Nothing else published about this server says what it costs to run. This page does: resident memory, processor time, goroutine count and per-method latency, measured on both transports across the settings that move them.

At a glance

• HTTP mode sizes by credentials that are calling. An idle process holds no tool catalog; the first credential of a configuration builds one, and it is then shared by every other credential of that configuration.
• Holding a credential and serving one differ by three orders of magnitude. The concurrency series reports both at every step: 30 to 53 KiB of settled live heap per credential held, 0.81 to 4.27 MiB of peak resident set per credential calling.
• --max-http-clients is therefore not a memory setting. Its default of 100 bounds about five mebibytes of tenancy. Size from concurrent callers.
• The first call of a configuration pays for registration, and it is a third of a second to a second and a third. No later call pays it again, though on individual a warm tools/list still costs 0.38 seconds on a quiet stdio process and 2.4 seconds with 64 credentials calling at once.
• The tool surface does not change what a credential costs to hold, and decides everything about the responses: a tools/list is 12 KB on dynamic and 3.2 MB on individual.
• These are this release’s numbers. The reference host is re-measured for every release and the values here are replaced, with no comparison against what they were.

How this was measured

The real binary, built from the current checkout and started the way a client starts it, on both transports. GitLab is stood in by an HTTP server inside the harness, so a run needs no instance, no credentials and no network, and two people re-measuring compare this server rather than their GitLab installations. The tool surface is passed explicitly rather than read from the environment, so a developer machine and CI measure the same thing.

Everything below is regenerated by one command, make bench-resources, which runs the benchmark and redraws the charts from the record it writes. The machine, the build and the method are named in the text under the heading and along the bottom edge of every chart, because a resident-set figure without them is a number nobody can act on.

This page is a snapshot, and only a snapshot. Every release re-measures on the reference host and replaces these values wholesale: no “before” column, no ratio against an earlier run, no history. A figure compared against a run whose code no longer exists says nothing about the instance you are sizing.

The point scenarios run 64 credentials on HTTP and 8 processes on stdio, admitting them one at a time and then loading all of them at once. 64 sits past the point where the host saturates and below the pool’s own default bound of 100, so the scenario runs the shipped default and measures real contention; 8 on stdio, because each one is a whole process with its own catalog and eight concurrent clients is a busy developer machine.

A shared deployment meets hundreds, so the benchmark also runs a concurrency series per surface: one HTTP process, given 1, 2, 5, 10, 20, 50, 100, 200, 500 and 1000 distinct credentials in turn, each warmed with a tools/list, and measured over a ten-second steady phase at each count in which every credential keeps calling tools/call and tools/list. Each step records the resident set (mean and peak), processor time per call, the p50 and p99 of both methods and the goroutine count, and captures a CPU and a heap profile of the server through its --pprof-addr listener, which the analysis reads and the site does not publish.

Each step is then measured a second time, with the load stopped. The two readings answer different questions: the resident set over the steady phase is what N credentials cost while all of them are calling, and the settled reading, taken after the load ends with a collection forced, is what they cost to hold. The first is measured in megabytes per credential and the second in kilobytes, so one is never a substitute for the other. The settled resident set is recorded beside the settled heap and lags it, because freeing a heap does not hand the pages back: Go’s scavenger returns them on its own schedule.

A series stops early rather than take the host down: before each step the resident set it would reach is estimated from the steps so far, and the rest of the list is skipped when the estimate exceeds the memory budget (80% of the host’s available memory unless one is given); a step whose tools/call p99 passes 30 seconds is the last one run. Where a series stopped, and why, is printed under its table. In the run published here none of them stopped: all three reached a thousand credentials.

Read these as shapes, not as constants

One machine, one operating system, one Go toolchain. The relationships between the numbers should carry to your host; the absolute values will not. The resident set also includes memory Go’s collector has not returned yet, which is the honest thing to size a limit against and not the same as the live heap.

Measurements

Measured on Intel(R) Core(TM) i5-14400, 16 logical CPUs, 62 GiB RAM, linux/amd64, kernel 6.12.105-production+truenas, go1.27.1, build 2.7.6-0.20260906081503-18bed59da189 (18bed59d), 2026-09-06T08:24:43Z. 3 rounds per method, resident set sampled every 100 ms.

Memory, goroutines and processor time per scenario

Scenario	Clients	Idle	One client	All clients	Per extra client	Peak	Goroutines	CPU, % of one core
stdio, dynamic	8	n/a	106	849	106	995	34	1039%
stdio, meta	8	n/a	107	857	107	1018	36	794%
stdio, individual	8	n/a	277	2134	265	2291	36	1120%
stdio, dynamic, telemetry	8	n/a	111	893	112	1048	43	1080%
http, dynamic	64	35	108	118	0	375	291	974%
http, meta	64	37	112	118	0	324	293	928%
http, individual	64	36	282	248	-1	1422	165	1147%
http, dynamic, telemetry	64	38	113	122	0	383	300	1018%

What a client waits for, per scenario

Scenario	Process ready	First tools/list	Warm tools/list (p50)	tools/list payload
stdio, dynamic	0.26	351	1.9	12 KB
stdio, meta	0.17	341	42	598 KB
stdio, individual	0.18	1279	379	3.2 MB
stdio, dynamic, telemetry	0.24	333	1.7	12 KB
http, dynamic	59	319	21	12 KB
http, meta	51	324	270	599 KB
http, individual	51	1277	2394	3.2 MB
http, dynamic, telemetry	51	319	20	12 KB

Latency percentiles per method

Scenario	Method	Call	p50	p90	p99	Max
stdio, dynamic	resources/list	smallest listing	0.65	7.6	7.7	7.7
stdio, dynamic	tools/call	gitlab_find_action	30	41	49	49
stdio, dynamic	tools/list	whole surface	1.9	6.0	6.2	6.2
stdio, meta	resources/list	smallest listing	0.64	0.99	1.2	1.2
stdio, meta	tools/call	gitlab_server (status)	1.7	40	41	41
stdio, meta	tools/list	whole surface	42	51	55	55
stdio, individual	resources/list	smallest listing	0.57	0.78	0.93	0.93
stdio, individual	tools/call	gitlab_server_status	0.87	1.4	1.8	1.8
stdio, individual	tools/list	whole surface	379	405	424	424
stdio, dynamic, telemetry	resources/list	smallest listing	0.79	0.87	0.95	0.95
stdio, dynamic, telemetry	tools/call	gitlab_find_action	30	47	50	50
stdio, dynamic, telemetry	tools/list	whole surface	1.7	2.3	2.6	2.6
http, dynamic	resources/list	smallest listing	33	48	80	85
http, dynamic	tools/call	gitlab_find_action	213	307	314	314
http, dynamic	tools/list	whole surface	21	35	38	38
http, meta	resources/list	smallest listing	10	55	62	66
http, meta	tools/call	gitlab_server (status)	23	31	33	34
http, meta	tools/list	whole surface	270	388	403	404
http, individual	resources/list	smallest listing	9.8	13	30	30
http, individual	tools/call	gitlab_server_status	8.4	16	18	18
http, individual	tools/list	whole surface	2394	2855	2929	2949
http, dynamic, telemetry	resources/list	smallest listing	14	53	59	61
http, dynamic, telemetry	tools/call	gitlab_find_action	213	301	314	315
http, dynamic, telemetry	tools/list	whole surface	20	35	36	37

Concurrency series

http, dynamic surface: 4 in flight per credential, 10 s per step, memory budget 35000 MiB

Credentials	Resident, mean	Resident, peak	Settled heap	Settled resident	CPU per call	Calls	tools/call p50	tools/call p99	tools/list p50	tools/list p99	Goroutines
1	143	154	39.0	106	7.394	6895	11	14	0.96	3.4	18
2	172	192	39.1	107	7.633	11070	13	18	1.1	4.7	22
5	217	232	39.2	109	7.910	13684	23	43	3.9	16	36
10	241	277	39.5	110	7.971	13660	43	118	7.5	53	56
20	289	364	40.0	115	7.843	14254	75	274	11	147	96
50	414	481	41.5	122	7.629	16777	151	766	11	471	216
100	547	608	44.0	133	7.639	17482	269	1578	15	956	416
200	869	1070	48.9	157	7.674	17851	406	3572	20	1757	816
500	1958	3089	64.0	232	7.851	17187	911	9313	52	4188	2015
1000	897	1454	88.3	288	9.176	16119	3106	8857	2216	6357	4015

Fitted across these steps: the peak resident set under load grows 1.92 MiB per credential, and the settled live heap, read with the load stopped and a collection forced, grows 50.6 KiB per credential. The first is what a credential costs while it and every other one is calling; the second is what it costs to hold. The settled resident set lags both, because Go returns freed pages to the operating system on its own schedule. Every planned step ran, up to 1000 credentials.

http, meta surface: 4 in flight per credential, 10 s per step, memory budget 35000 MiB

Credentials	Resident, mean	Resident, peak	Settled heap	Settled resident	CPU per call	Calls	tools/call p50	tools/call p99	tools/list p50	tools/list p99	Goroutines
1	134	146	32.6	93.0	8.014	6323	0.97	4.0	11	16	20
2	162	179	32.7	94.7	8.001	9544	1.4	5.3	15	20	24
5	185	203	32.8	94.7	8.512	10810	6.9	20	29	49	36
10	199	226	33.1	96.0	8.489	11025	16	53	51	117	56
20	232	257	33.6	100	8.234	11486	34	146	91	247	96
50	298	388	35.2	107	7.953	12021	138	451	165	530	216
100	295	406	37.9	116	8.517	11842	349	775	348	700	416
200	376	531	43.2	135	8.458	12427	747	1481	648	1250	816
500	514	689	59.5	184	8.599	14008	1414	3252	1763	3077	2015
1000	749	1009	83.5	243	8.330	16130	1791	6808	3180	6028	4015

Fitted across these steps: the peak resident set under load grows 0.81 MiB per credential, and the settled live heap, read with the load stopped and a collection forced, grows 52.6 KiB per credential. The first is what a credential costs while it and every other one is calling; the second is what it costs to hold. The settled resident set lags both, because Go returns freed pages to the operating system on its own schedule. Every planned step ran, up to 1000 credentials.

http, individual surface: 2 in flight per credential, 10 s per step, memory budget 35000 MiB

Credentials	Resident, mean	Resident, peak	Settled heap	Settled resident	CPU per call	Calls	tools/call p50	tools/call p99	tools/list p50	tools/list p99	Goroutines
1	273	296	88.1	186	111.908	262	1.1	4.9	151	268	18
2	296	338	88.3	183	116.938	480	1.1	6.9	164	201	20
5	429	530	88.4	191	133.564	766	1.3	11	262	291	26
10	538	673	88.5	194	137.591	855	17	61	454	593	36
20	731	1073	88.8	200	134.659	953	38	257	784	1319	56
50	1081	1303	89.7	211	127.568	1110	68	804	1685	3502	116
100	1266	1724	91.1	215	126.672	1292	213	2572	2828	7144	216
200	2244	3274	94.0	248	122.502	1499	239	3428	5430	12328	415
500	2522	3427	103	277	110.330	2361	285	11643	11082	18163	1015
1000	3330	4677	117	326	98.265	3637	4974	16559	16796	25229	2015

Fitted across these steps: the peak resident set under load grows 4.27 MiB per credential, and the settled live heap, read with the load stopped and a collection forced, grows 29.8 KiB per credential. The first is what a credential costs while it and every other one is calling; the second is what it costs to hold. The settled resident set lags both, because Go returns freed pages to the operating system on its own schedule. Every planned step ran, up to 1000 credentials.

Sizing a deployment

stdio. One process per client, and the process builds its catalog as soon as it starts, so there is no idle state to save memory in. Between 106 and 277 MiB per process depending on the surface, with peaks under load a little higher. Nothing is shared between processes, so the cost is a straight line in the client count: a developer machine running one client will not notice, and the “all clients” column is what eight of them weigh together.

HTTP. Start from 35 to 38 MiB for a process serving nobody, then add one catalog per distinct configuration, built by the first credential that asks for it and shared by every other credential of that configuration. That is what the “per extra client” column reports: 0.15, 0.09 and -0.53 MiB on dynamic, meta and individual, which is nothing above the noise of a resident-set reading, across a ladder that admits 64 credentials one at a time. What is left to size for is the concurrent work, which is the peak column beside it: 324 to 1422 MiB with all 64 calling.

There are two axes, and only one of them is the pool. Holding a credential costs 50.6 KiB of settled live heap on dynamic, 52.6 KiB on meta and 29.8 KiB on individual, so a thousand admitted credentials are 88, 83 and 117 MiB of live heap and under a third of a gibibyte of resident set. Serving them is where an instance runs out: with every credential holding two to four requests in flight, the peak resident set grows 1.92, 0.81 and 4.27 MiB per credential, and at the thousand-credential step it stood at 1.5, 1.0 and 4.7 GiB.

A practical rule for HTTP: budget for concurrent callers, at 1 to 4 MiB each depending on the surface, on top of 40 MiB for the process and one catalog per configuration. --max-http-clients bounds pooled credentials and is not a memory setting: at 50 KiB each, its default of 100 is five mebibytes. Shorten --pool-idle-timeout if you want entries reclaimed sooner, but expect the memory back in kilobytes rather than gigabytes.

Two at-rest figures, both correct

The end-to-end test that runs on every push reports about 8 KiB per credential, and the series above reports 30 to 53 KiB. They answer different questions. The test’s clients complete their tools/list and disconnect, so it measures the pool entry with no connection open. The benchmark driver keeps its sockets open, four per credential on dynamic and meta and two on individual, so its figure also carries the buffers behind those connections, at roughly 10 KiB each. Size against the series figure: a credential you are sizing for is a connected one.

Throughput. In the series, the calls completed per step stop growing at about five credentials on dynamic and meta: a call costs 8 ms of processor time, so a sixteen-thread host completes 11,000 to 17,900 per ten-second step whatever the number of credentials, and latency past that point is queueing, with the p50 of tools/call on dynamic climbing from 11 ms at one credential to 269 ms at a hundred and 3.1 seconds at a thousand. individual is a different shape: a tools/call costs about a millisecond, but a call there averages 120 ms of processor time, because every second call is a tools/list serialising three megabytes of schemas, so a single credential with two requests in flight already keeps three cores busy.

What a client waits for

The process is ready in milliseconds and the surface behind it is not. On HTTP /health answers in 51 to 59 ms while the pool holds nothing; the first credential of a configuration builds its catalog, and that is where the wait is: 0.32 seconds on dynamic, 0.32 on meta and 1.28 on individual on the host measured. On stdio the process is executed in well under a millisecond and starts building immediately, so its first request waits on the same work, at 0.35, 0.34 and 1.28 seconds.

That wait is per configuration rather than per credential, since the built server is shared, and it returns: when every entry of a configuration has been reclaimed by --pool-idle-timeout or evicted at the pool’s size bound, the next request that needs it rebuilds from scratch.

Once warm, the picture inverts. A tools/list costs 1.9 to 21 ms on dynamic, 42 to 270 ms on meta and 0.38 to 2.4 seconds on individual, which is the cost of serializing 12 KB, 599 KB and 3.2 MB respectively, the higher end of each pair being HTTP with all 64 credentials calling. A client speaking protocol 2026-07-28 may serve repeat listings from its own cache and pay this less often, but it pays it at least once per session.

Telemetry

Exporting over OTLP costs a handful of goroutines and measurable processor time, and no material memory. It is off by default; see OpenTelemetry for what it exports and where it goes.

Reproducing

make bench-resources          # measure, then redraw everything
make bench-resources-render   # redraw from the committed record, no measuring
make check-bench-resources    # verify the charts match the record

The eight point scenarios take five to ten minutes on the reference host: the stdio half spends most of it building one catalog per process, which is the cost being measured rather than overhead of the harness, and the HTTP half spends it serving 64 credentials at once. The series then take as long as the host’s memory lets them, up to a quarter of an hour per surface. The series can also be measured on one host and rendered on another: build the driver and the server, and on the host with the memory run the driver with -binary, -json, -profiles and -no-render, which reads nothing from a repository; then copy the record back and run make bench-resources-render. The developer-facing details, including what the harness deliberately does not measure, are in the resource benchmark reference.