OpenTelemetry

Helidon support for OpenTelemetry configuration and semantic conventions as described below is currently an @Features.Preview feature. We intend to support it going forward, but we might change its external API and behavior in backward-incompatible ways across dot releases.

Overview

Helidon supports OpenTelemetry in several important ways:

Implements the neutral Helidon tracing API using OpenTelemetry
Allows users to assign OpenTelemetry settings as follows:
- Declaratively, using Helidon config under the top-level telemetry config key
- Programmatically, using the OpenTelemetry SDK API and the Helidon OpenTelemetry API
Conforms to the OpenTelemetry semantic conventions for automatically-created spans and metrics for HTTP requests
Allows publishing Helidon metrics to backend systems using OTLP.

OpenTelemetry models observability as a set of signals.

Each signal—for example metrics, tracing, and logging—is an origin of monitoring data, and each has configurable settings which control its behavior.

Helidon’s config support for OpenTelemetry has certain config attributes which apply to OpenTelemetry as a whole, others which pertain to individual signals, and still more which describe lower-level elements within a signal.

The Helidon OpenTelemetry configuration format, the Helidon OpenTelemetry API, and this documentation all follow this hierarchy:

Top-level telemetry
- Resource attributes
- Signals

This document describes how to configure each level in the hierarchy and covers general topics related to Helidon’s support of OpenTelemetry.

API

There are two APIs that might be useful to developers working with OpenTelemetry:

The Helidon OpenTelemetry API - useful for mapping configuration sources to Helidon builders and, ultimately, OpenTelemetry objects.
The OpenTelemetry API - useful for creating OpenTelemetry objects apart from Helidon configuration sources.

The types in the Helidon OpenTelemetry API correspond closely to the configuration structures described in later sections of this document. Application code can use Helidon OpenTelemetry builders to prepare and construct each of the configurable entities to ultimately prepare an OpenTelemetry instance set up according to the application’s needs.

That said, application code can equally well use the OpenTelemetry API and its builders to prepare the OpenTelemetry instance.

Applications could even use both APIs together, reading configuration to construct a Helidon builder and then adding to that builder OpenTelemetry objects created separately using the OpenTelemetry API.

The Helidon OpenTelemetry API Javadoc page lists the various types developers can use to prepare OpenTelemetry objects programmatically. As a starting point, the OpenTelemetryConfig interface and its Builder represents the top-level configuration for OpenTelemetry. Their Javadoc contains links to other types that compose the top-level object, and so on.

Later sections in this document also describe the configuration settings available.

The OpenTelemetry SDK documentation explains its API.

Many applications do not need to use either the Helidon OpenTelemetry API or the OpenTelemetry API directly. They can instead rely completely on declarative Helidon configuration of OpenTelemetry.

Application Instance

Typically, an application uses the same OpenTelemetry instance throughout its execution. Helidon exposes the application OpenTelemetry instance and the corresponding application Tracer through the service registry.

In most cases, an application that prepares OpenTelemetry programmatically and then bypasses Helidon’s OpenTelemetry ownership should initialize both of those before either contract is requested from the registry, as shown in the following example.

Setting the service-registry OpenTelemetry and Tracer instances in Helidon:

import java.util.Map;

import io.helidon.service.registry.Services;
import io.helidon.tracing.Tracer;

import io.opentelemetry.api.GlobalOpenTelemetry;
import io.opentelemetry.api.OpenTelemetry;
import io.opentelemetry.api.trace.SpanKind;
import io.opentelemetry.api.trace.StatusCode;
import io.opentelemetry.context.Scope;
import io.opentelemetry.sdk.autoconfigure.AutoConfiguredOpenTelemetrySdk;

// Application code using the OpenTelemetry API or the Helidon OpenTelemetry API or both.
OpenTelemetry customOpenTelemetry = prepareOpenTelemetry();

// App code to build any tags to be applied to every span.
Map<String, String> tags = prepareTags();

GlobalOpenTelemetry.set(customOpenTelemetry);
Services.set(OpenTelemetry.class, customOpenTelemetry);
Services.set(Tracer.class,
             io.helidon.tracing.providers.opentelemetry.HelidonOpenTelemetry.create(
                     customOpenTelemetry,
                     customOpenTelemetry.getTracer("your-service-name"),
                     tags));

Assigning the Application Instance

Using Helidon configuration to prepare the application OpenTelemetry instance has these effects:

Registers the instance in the Helidon service registry.
Creates a Helidon Tracer using the OpenTelemetry instance and registers that tracer in the Helidon service registry.

Application code should retrieve the Helidon OpenTelemetry and Tracer instances from the service registry, for example using Services.get(OpenTelemetry.class) and Services.get(Tracer.class).

Some third-party OpenTelemetry instrumentation uses GlobalOpenTelemetry directly. For that interop case, keep global set to true so Helidon publishes the selected application OpenTelemetry instance to the OpenTelemetry global. This must happen before any other code initializes GlobalOpenTelemetry; OpenTelemetry allows the global to be assigned only once. If an OpenTelemetry global already exists, Helidon leaves it unchanged and uses the existing global as the application OpenTelemetry instance in the service registry.

The global setting applies when Helidon’s OpenTelemetry ownership path creates or adopts the application instance. If application code manually registers both OpenTelemetry and Tracer using Services.set(...), then Helidon’s ownership factories are bypassed; publish the same OpenTelemetry instance to GlobalOpenTelemetry directly before anything else initializes the OpenTelemetry global.

Maven Coordinates

To enable various aspects of OpenTelemetry Support add one or more of the following dependencies to your project’s pom.xml (see Managing Dependencies).

Using the OpenTelemetry Implementation of the Helidon Tracing API

Helidon offers an implementation of its neutral tracing API that uses OpenTelemetry. Add the following dependency to use OpenTelemetry for tracing.

pom.xml

<dependency>
  <groupId>io.helidon.tracing.providers</groupId>
  <artifactId>helidon-tracing-providers-opentelemetry</artifactId>
</dependency>

Adding OpenTelemetry Configuration and Builder Support

To allow deployers and end users to use Helidon configuration to control OpenTelemetry behavior, add the following dependency.

pom.xml

<dependency>
  <groupId>io.helidon.telemetry</groupId>
  <artifactId>helidon-telemetry-opentelemetry-config</artifactId>
</dependency>

Enabling Automatic Spans for HTTP Requests

Helidon’s tracing observability support automatically creates a new tracing span for each HTTP request if your project includes the following dependency.

pom.xml

<dependency>
  <groupId>io.helidon.webserver.observe</groupId>
  <artifactId>helidon-webserver-observe-telemetry-tracing</artifactId>
  <scope>runtime</scope>
</dependency>

Automatic HTTP spans use exact, case-sensitive standard method names for the http.request.method attribute. For an unrecognized method, including a case variant such as get, Helidon records _OTHER, preserves the received token in http.request.method_original, and uses HTTP as the method portion of the span name as required by the OpenTelemetry semantic conventions.

Enabling OpenTelemetry Metrics Semantics for HTTP Requests

pom.xml

<dependency>
  <groupId>io.helidon.webserver.observe</groupId>
  <artifactId>helidon-webserver-observe-telemetry-metrics</artifactId>
  <scope>runtime</scope>
</dependency>

Helidon limits the values recorded for the OpenTelemetry http.request.method attribute, mapping methods not in the configured known method list to _OTHER. The comparison is exact and case-sensitive. See the automatic HTTP metrics configuration for the default method list and customization details.

WebClient Support

Helidon supports the OpenTelemetry semantic conventions for outgoing traffic which uses the Helidon WebClient. See the Helidon WebClient documentation.

Additional Dependencies

Most applications need to declare other runtime dependencies on OpenTelemetry artifacts because the configuration specifies—or the application code uses—particular OpenTelemetry types packaged in other artifacts.

For example, OpenTelemetry exporters are packaged individually or as related groups. See this section below for some specific dependencies to consider adding for particular exporters.

These exporters transmit telemetry data using a different protocol. (See this OpenTelemetry page.)

Key	Type	Description
`container-id`	`String`	Identifier of the container in which the service instance is running
`service-namespace`	`String`	Namespace for the service
`container-image-repo-digests`	`List<String>`	Repository digests of the container image as reported by the container runtime
`container-image-name`	`String`	Name of the container image on which the container was built
`service-instance-id`	`String`	Identifier for this service instance
`container-image-tags`	`List<String>`	Tags of the container image on which the container was built
`attributes`	`CustomMethods`	Additional resource attributes shared by all configured signals
`deployment-environment-name`	`String`	Name of the deployment environment

Key	Type	Description
`longs`	`Map<String, Long>`	Long attributes
`strings`	`Map<String, String>`	String attributes
`doubles`	`Map<String, Double>`	Double attributes
`booleans`	`Map<String, Boolean>`	Boolean attributes

Key	Type	Description
`schedule-delay`	`Duration`	Delay between consecutive exports
`max-export-batch-size`	`Integer`	Maximum number of items batched for export together
`type`	`ProcessorType`	Processor type
`max-queue-size`	`Integer`	Maximum number of items retained before discarding excess unexported ones
`timeout`	`Duration`	Maximum time an export can run before being cancelled
`exporters`	`List<String>`	Name(s) of the exporter(s) this processor should use; specifying no names uses all configured exporters (or if no exporters are configured, the default OpenTelemetry exporter(s))

Key	Type	Default	Description
`connect-timeout`	`Duration`		Connection timeout
`headers`	`Map<String, String>`		Headers added to each export message
`endpoint`	`URI`		Endpoint of the collector to which the exporter should transmit
`memory-mode`	`MemoryMode`		Memory mode
`protocol`	`CustomMethods`	`DEFAULT`	Exporter protocol type
`internal-telemetry-version`	`InternalTelemetryVersion`		Self-monitoring telemetry OpenTelemetry should collect
`certificate`	`Resource`		Trusted certificates
`client.key`	`Resource`		TLS client key
`client.certificate`	`Resource`		TLS certificate
`compression`	`CompressionType`		Compression the exporter uses
`retry-policy`	`CustomMethods`		Retry policy
`timeout`	`Duration`		Exporter timeout