Contents

Overview

A metric typically reflects the usage of a single point in your service processing multiple requests over time. A value such as the total time consumed by a given REST endpoint underscores the aggregate nature of metric values; Helidon accumulates the time from all requests in the total duration.

Tracing, on the other hand, captures the usage of multiple parts of your code as your service responds to a single request.

Metrics and tracing come together in Helidon’s support for examplars.

Note

exemplar - one that serves as a model or example

 — Merriam-Webster Dictionary

In the context of metrics, an exemplar for a given metric is a specific sample which, in some sense, made a typical contribution to the metric’s value. For example, an exemplar for a SimpleTimer might be a sample in which the duration it contributed to the value of a SimpleTimer is near the mean of the durations over all samples. The metrics output identifies the exemplar sample using the trace ID of the trace which triggered that sample.

Maven Coordinates

To enable OpenMetrics exemplar support add the following dependency to your project’s pom.xml (see Managing Dependencies).

<dependency>
    <groupId>io.helidon.metrics</groupId>
    <artifactId>helidon-metrics-trace-exemplar</artifactId>
    <scope>runtime</scope>
</dependency>
Copied

Also, include the Helidon integration module for a tracing implementation (such as Helidon Zipkin)

<dependency>
    <groupId>io.helidon.tracing</groupId>
    <artifactId>helidon-tracing-zipkin</artifactId>
</dependency>
Copied

Add the Helidon tracing component itself:

<dependency>
    <groupId>io.helidon.microprofile.tracing</groupId>
    <artifactId>helidon-microprofile-tracing</artifactId>
</dependency>
Copied

Usage

Once you add the appropriate dependencies to your project, exemplar support runs automatically as part of Helidon metrics. You do not need to change your application or configuration.

Interpreting Exemplars

Helidon automatically records a sample (label, value, and timestamp) with each update to certain metrics. When a client accesses the /metrics endpoint, Helidon adds the label, value, and timestamp to the OpenMetrics response for those metrics.

By default, Helidon adds exemplars only to counters and the counter portion of simple timers. The OpenMetrics specification allows exemplars only on counters and buckets. (Helidon’s histogram output, consistent with the MicroProfile Metrics spec, includes the quantiles but not the buckets.)

Note

Earlier releases of Helidon included exemplars on other metric types as well. If you must keep the prior behavior for compatibility reasons, see the Controlling Exemplar Output section below.

Output Format

In the OpenMetrics output, an exemplar actually appears as a comment appended to the normal OpenMetrics output.

OpenMetrics format with exemplars
metric-identifier metric-value # exemplar-label sample-timestamp
Copied

Even downstream consumers of OpenMetrics output that do not recognize the exemplar format should continue to work correctly (as long as they do recognize comments).

But some consumers, such as trace collectors and their U/Is, understand the exemplar format, and they allow you to browse metrics and then navigate directly to the trace for the metric’s exemplar.

Controlling Exemplar Output

Once you add dependencies on helidon-metrics-trace-exemplar and one of the Helidon tracing integration libraries to your project, Helidon automatically adds exemplars to those metrics which the OpenMetrics specification permits to have exemplars.

Earlier releases of Helidon added exemplars to other, non-standard metric types. If you require the former behavior for compatibility reasons, you can enable exemplars on non-standard metric types. In this context, with strict exemplar behavior Helidon adds exemplars only to those metrics allowed by the OpenMetrics spec. With lax behavior Helidon also adds exemplars to simple timer and meter output.

Note

The lax exemplar behavior is non-standard. Some downstream consumers of the resulting OpenMetrics output might reject it because it contains non-standard content.

You control strict vs. lax exemplar support separately for each registry type.

Enabling non-standard exemplar behavior in META-INF/microprofile-config.properties
metrics.registries.0.type=application
metrics.registries.0.exemplars.strict = false
Copied

The exemplars.strict setting defaults to true for all registry types.

Examples

Once you enable exemplar support you can see the exemplars in the metrics output.

# TYPE application_globalRequestTracker_total counter
# HELP application_globalRequestTracker_total
application_globalRequestTracker_total 4 # {trace_id="daf26fe35fee9917"} 0.001183992 1617725180.234000

This exemplar in this case is the most recent sample.

Additional Information

Brief discussion of exemplars in the OpenMetrics spec