Skip to content

Metrics

Metrics instruments and tags

ChokaQ exposes metrics through the .NET System.Diagnostics.Metrics API. The meter name is ChokaQ.

Metrics are a public production contract. Names, units, and tag cardinality need to be stable enough for dashboards and alerts.

Instruments

MetricTypeMeaning
chokaq.jobs.enqueuedCounterJobs accepted into the queue.
chokaq.jobs.completedCounterJobs completed successfully.
chokaq.jobs.failedCounterHandler execution failures.
chokaq.jobs.processing_durationHistogram, msHandler processing duration.
chokaq.jobs.queue_lagHistogram, msTime eligible jobs waited before processing.
chokaq.jobs.dlqCounterJobs moved to DLQ.
chokaq.jobs.retriedCounterJobs scheduled for retry.
chokaq.workers.activeUpDownCounterCurrent active processing workers.
chokaq.jobs.heartbeat_failuresCounterFailed heartbeat writes.
chokaq.jobs.state_transition_conflictsCounterWorker-owned transitions that affected no rows.
chokaq.idempotency.claimsCounterIdempotency claim outcomes.
chokaq.circuits.eventsCounterCircuit breaker state events.

Cardinality Control

Metrics tags are capped by ChokaQ:Metrics options:

  • max queue tag values;
  • max job type tag values;
  • max error tag values;
  • max failure reason tag values;
  • max tag value length;
  • unknown value label;
  • overflow value label.

When a process sees too many distinct values, new values collapse into the overflow label instead of creating unlimited time series.

Alerting Guidance

SignalUse
Queue lag p95/p99Primary saturation alert.
DLQ rateTerminal failure alert.
Retry rateDownstream instability signal.
Heartbeat failuresStorage or worker health issue.
State transition conflictsOwnership/race/recovery signal.
Circuit eventsSystemic downstream failure protection.

Architecture Decision

Why this pattern?

OpenTelemetry-compatible metrics let ChokaQ integrate with existing monitoring without taking a dependency on a specific vendor.

Trade-offs

Tag cardinality must be controlled. Queue names, job types, and error values can become unbounded if applications feed dynamic strings into them.

Alternatives considered

AlternativeBenefitCost
Vendor-specific SDKRich features.Locks users into one observability stack.
Logs-derived metricsFlexible.Slower and more expensive for alerting.
No built-in metricsSmall runtime surface.Weak production story.

Additional Questions

Why is queue lag more important than queue depth?
Depth lacks time context. Lag tells you how long eligible work is actually waiting.

Why cap tag values?
To prevent accidental high-cardinality time series from damaging the monitoring backend.

What metric indicates duplicate-protection behavior?
chokaq.idempotency.claims, split by outcome.

Apache 2.0 Licensed