Troubleshooting

Connection issues

Cannot connect to MikroTik

Error: dial tcp: connection refused

The bouncer cannot reach the MikroTik API port.

1. Verify the API service is enabled

   /ip/service/print where name=api

2. Check the port matches your configuration (default: 8728, TLS: 8729)

3. Verify firewall rules allow access from the bouncer’s IP

4. If using Docker, ensure the container can reach the router network

Authentication failed

Error: cannot log in

Wrong username or password.

1. Verify credentials in your configuration file

2. Check the user exists and has the correct group

   /user/print where name=crowdsec

3. Verify the user’s group has required policies: api, read, write

4. Check allowed address restrictions on the user account

TLS certificate errors

Error: x509: certificate signed by unknown authority

TLS verification failed.

1. For self-signed certificates, set the CA path:

   crowdsec:
     ca_cert_path: "/path/to/ca.pem"

2. Verify the certificate is valid and not expired

3. Check the hostname matches the certificate’s CN or SAN

CrowdSec LAPI issues

Cannot connect to LAPI

Error: connection to CrowdSec LAPI failed

The bouncer cannot reach the CrowdSec Local API.

1. Verify the LAPI URL

   curl http://localhost:8080/v1/decisions

2. Check the bouncer API key

   cscli bouncers list

3. If CrowdSec runs in Docker, ensure network connectivity between containers

No decisions received

Bouncer starts but no IPs are banned

The bouncer connects successfully but the address list stays empty.

Possible causes:

1. No active decisions — check with cscli decisions list
2. Origin filtering too restrictive — check crowdsec.origins config
3. Scenario filtering excludes all — check crowdsec.scenarios_containing / crowdsec.scenarios_not_containing
4. CrowdSec engine not detecting threats — check CrowdSec logs

Firewall rule issues

Rules not created

Address list has entries but no firewall rules

IPs are added to the address list but no firewall rules exist.

1. Check filter and raw are enabled

   firewall:
     filter:
       enabled: true
     raw:
       enabled: true

2. Look for errors in logs

   journalctl -u cs-routeros-bouncer -f

3. Verify the user has write policy for firewall operations

Rules are not in the expected position

Firewall rules appear at the bottom

Rules may be placed after existing rules.

1. Verify rule_placement in configuration. Use "top" for highest priority or "bottom" to append. Use structured placement (position, before_comment, after_comment) when existing router rules must stay first. Check both the global setting and any firewall.ipv4.rule_placement or firewall.ipv6.rule_placement override.

2. Check for dynamic rules at position 0. The bouncer’s retry logic only applies to top and position. It tries lower positions when RouterOS refuses a dynamic or built-in target.

3. Check comment anchors for before_comment and after_comment. Matching is case-sensitive; comment_match: "contains" is a literal substring match, not a regular expression.

4. Inspect positions manually

   /ip/firewall/filter print
   /ip/firewall/raw print

5. Check bouncer logs with journalctl -u cs-routeros-bouncer -f | grep -i placement. Missing comment anchors use fallback (top by default, or bottom). Numeric placement requires position; out-of-range numeric positions are appended at the bottom.

Performance issues

High CPU during reconciliation

Router CPU spikes during bouncer startup

This is expected during initial reconciliation with large address lists.

Performance benchmarks:

| List size | Time | CPU peak | | ----------------------- | --------------------------------------------- | ----------------- | | ~28,700 IPs (full CAPI) | ~58 s wall-clock; ~35–36 s RouterOS bulk work | ~39% observed | | No-drift periodic check | ~3–4 s | Returns near idle |

To reduce impact:

1. Filter synced decisions with origins:

   crowdsec:
     origins: ["crowdsec", "cscli"]

2. Schedule bouncer restarts during low-traffic periods

CPU impact is temporary

CPU impact is temporary — only while reconciliation is actively adding or removing entries.

Sustained high RouterOS CPU after startup

Router CPU stays high after reconciliation

This is not expected from address-list entries simply remaining on the router. The bouncer should be mostly idle between CrowdSec stream updates and periodic reconciliation runs.

Check for repeated RouterOS API churn:

journalctl -u cs-routeros-bouncer --since "30 min ago" | grep -E "already have such entry|attempting reconnect|RouterOS command failed"

Signs of correct behavior (what to look for):

1. Cached duplicate ban decisions are skipped without touching RouterOS
2. RouterOS already have such entry device errors do not trigger reconnects
3. Router CPU returns near normal idle/traffic levels after reconciliation finishes

If you still see repeated duplicate errors and reconnect attempts, upgrade to the first release whose changelog includes the RouterOS device-error handling and cache fast-path fixes. Until that release is cut, build from a commit that contains that changelog entry.

Reconciliation timeout or slow startup

Large address lists can make startup reconciliation the slowest part of boot. Check whether the logs show bulk script warnings, command timeouts, or pool fallback messages.

Actions to try:

1. Increase mikrotik.command_timeout from its default "30s" to "60s".
2. Keep crowdsec.origins: ["crowdsec", "cscli"] on small routers.
3. Increase mikrotik.pool_size from its default 4 only after checking RouterOS API max-sessions.
4. Move full CAPI testing to a maintenance window.

mikrotik:
  command_timeout: "60s"
  pool_size: 6

Duplicate address messages

RouterOS can return already have such entry when an address is already present. The bouncer treats this as a device-level duplicate, finds the existing entry, and refreshes timeout/comment when needed. Occasional duplicate messages are harmless.

Repeated duplicate messages every few seconds usually mean the in-memory cache is not matching router state or the service is restarting. Check for restarts, manual address-list edits, and repeated reconciliation failures.

Bulk script fallback warnings

During reconciliation, additions are sent in RouterOS scripts of up to 100 entries. If a script fails, the bouncer falls back to individual adds for that chunk and keeps going.

If fallback happens often:

1. Check mikrotik.command_timeout.
2. Check RouterOS logs for script or policy errors.
3. Verify the API user has read, write, api, and sensitive policies.
4. Test with local-only origins to reduce list size.

IPv6 addresses appear with /128

RouterOS stores single IPv6 addresses in address lists with an explicit /128 prefix. The bouncer normalizes IPv6 single addresses this way before add, find, and reconciliation operations. This is expected and does not mean CrowdSec sent a range.

Logging and debugging

Debug logging

logging:
  level: "debug"

Or via environment variable:

LOG_LEVEL=debug cs-routeros-bouncer -c config.yml

Health check

curl http://localhost:2112/health

Metrics

curl http://localhost:2112/metrics

Router state

# View address list entries
/ip/firewall/address-list print where list=crowdsec-banned

# Count entries
/ip/firewall/address-list print count-only where list=crowdsec-banned

# View bouncer firewall rules
/ip/firewall/filter print where comment~"crowdsec-bouncer"
/ip/firewall/raw print where comment~"crowdsec-bouncer"

Docker-specific issues

Container cannot reach router

Connection timeout from Docker container

Docker networking may prevent the container from reaching the router.

1. Use host networking in Docker Compose:

   services:
     cs-routeros-bouncer:
       network_mode: host

2. Or ensure the Docker network can route to the router’s IP

3. Check DNS resolution if using hostnames

Permission denied for config file

Error: permission denied reading config

The container user cannot read the mounted config file.

1. Check file permissions: chmod 644 config.yml

2. If using Docker secrets, ensure the secret is properly mounted

3. Or use environment variables instead of a config file