• Overview

• Maven Coordinates

• Usage

• Configuration

• Additional Information

  • Jaeger Tracing

  • Zipkin Tracing

  • OpenTelemetry Tracing

• Reference

Distributed tracing is a critical feature of microservice based applications, since it traces workflow both within a service and across multiple services. This provides insight to sequence and timing data for specific blocks of work, which helps you identify performance and operational issues. Helidon includes support for distributed tracing through its own API, backed by either OpenTelemetry, Jaeger, or Zipkin. Tracing is integrated with WebServer and Security.

To enable Helidon Tracing, add the following dependency to your project’s pom.xml (see Managing Dependencies).

<dependencies>
    <dependency>
        <groupId>io.helidon.tracing</groupId>
        <artifactId>helidon-tracing</artifactId>    
    </dependency>
    <dependency>
        <groupId>io.helidon.webserver.observe</groupId>
        <artifactId>helidon-webserver-observe-tracing</artifactId> 
    </dependency>
</dependencies>

• Helidon tracing dependency.
• Observability dependencies for tracing.

To transmit tracing data from your service to a backend, you need to add a tracing provider to your project.

For Jaeger:

<dependency>
    <groupId>io.helidon.tracing.providers</groupId>
    <artifactId>helidon-tracing-providers-jaeger</artifactId>
    <scope>runtime</scope>
</dependency>

For Zipkin:

<dependency>
    <groupId>io.helidon.tracing.providers</groupId>
    <artifactId>helidon-tracing-providers-zipkin</artifactId>
    <scope>runtime</scope>
</dependency>

For OpenTelemetry:

<dependency>
    <groupId>io.helidon.tracing.providers</groupId>
    <artifactId>helidon-tracing-providers-opentelemetry</artifactId>
    <scope>runtime</scope>
</dependency>

For OpenTracing (deprecated):

<dependency>
    <groupId>io.helidon.tracing.providers</groupId>
    <artifactId>helidon-tracing-providers-opentracing</artifactId>
</dependency>

This section explains a few concepts that you need to understand before you get started with tracing.

• In the context of this document, a service is synonymous with an application.

• A span is the basic unit of work done within a single service, on a single host. Every span has a name, starting timestamp, and duration. For example, the work done by a REST endpoint is a span. A span is associated to a single service, but its descendants can belong to different services and hosts.

• A trace contains a collection of spans from one or more services, running on one or more hosts. For example, if you trace a service endpoint that calls another service, then the trace would contain spans from both services. Within a trace, spans are organized as a directed acyclic graph (DAG) and can belong to multiple services, running on multiple hosts.

• Baggage is a collection of key-value pairs associated with a span.

• Span context captures data about a span not related to its duration, such as the tracer ID, the span ID, and baggage.

Support for specific tracers is abstracted. Your application can depend on the Helidon abstraction layer and provide a specific tracer implementation as a Java ServiceLoader service. Helidon provides such an implementation for:

• OpenTracing tracers, either using the GlobalTracer, provider resolver approach, or explicitly using Zipkin tracer

• OpenTelemetry tracers, either using the global OpenTelemetry instance, or explicitly using Jaeger tracer

Setup WebServer

Tracer tracer = TracerBuilder.create("helidon") 
        .build();

WebServer.builder()
        .addFeature(ObserveFeature.builder()
                            .addObserver(TracingObserver.create(tracer)) 
                            .build())
        .build()
        .start();

Creating Custom Spans

Span span = tracer.spanBuilder("name") 
        .tag("key", "value")
        .start();

try { 
    // do some work
    span.end();
} catch (Throwable t) { 
    span.end(t);
}

Handling Baggage

Responding to Span Lifecycle Events

// Note that callbackEnabledSpan implements OpenTelemetry Span.
io.opentelemetry.api.trace.Span nativeOtelSpan = callbackEnabledSpan.unwrap(io.opentelemetry.api.trace.Span.class);
if (nativeOtelSpan instanceof io.opentelemetry.sdk.trace.ReadableSpan readableSpan) {
    // Work with the span as a ReadableSpan
}

The following table lists all spans traced by Helidon components:

Some of these spans log to the span. These log events can be (in most cases) configured:

There are also tags that are set by Helidon components. These are not configurable.

The following configuration should be supported by all tracer implementations (if feasible)

Traced Spans Configuration

TracingConfig.builder()
        .addComponent(ComponentTracingConfig.builder("web-server")
                              .addSpan(SpanTracingConfig.builder("HTTP Request")
                                               .addSpanLog(SpanLogTracingConfig.builder("content-write")
                                                                   .enabled(false)
                                                                   .build())
                                               .build())
                              .build())
        .build();

tracing:
    components:
      web-server:
        spans:
          - name: "HTTP Request"
            logs:
              - name: "content-write"
                enabled: false

Tracer tracer = TracerBuilder.create(config.get("tracing")).build(); 
server.addFeature(ObserveFeature.builder()
                          .addObserver(TracingObserver.create(tracer)) 
                          .build());

tracing:
  paths:
    - path: "/favicon.ico"
      enabled: false
    - path: "/metrics"
      enabled: false
    - path: "/health"
      enabled: false
    - path: "/greet"
      components:
        web-server:
          spans:
          - name: "content-read"
            new-name: "read"
            enabled: false

tracing:
  components:
    web-server:
      spans:
      - name: "HTTP Request"
        new-name: "HTTP %1$s %2$s"

WebClient Span Propagation

<dependencies>
    <dependency>
        <groupId>io.helidon.webclient</groupId>
        <artifactId>helidon-webclient</artifactId>
    </dependency>
    <dependency>
        <groupId>io.helidon.webclient</groupId>
        <artifactId>helidon-webclient-tracing</artifactId>
    </dependency>
</dependencies>

WebClient client = WebClient.builder()
        .addService(WebClientTracing.create())
        .build();

String response = client.get()
        .uri(uri)
        .requestEntity(String.class);

Jaeger Tracing

<dependency>
    <groupId>io.helidon.tracing</groupId>
    <artifactId>helidon-tracing-providers-jaeger</artifactId>
</dependency>

The following is an example of a Jaeger configuration, specified in the YAML format.

tracing:
    service: "helidon-full-http"
    protocol: "https"
    host: "jaeger"
    port: 14240

Jaeger Tracing Metrics

Zipkin Tracing

<dependency>
    <groupId>io.helidon.tracing.providers</groupId>
    <artifactId>helidon-tracing-providers-zipkin</artifactId>
</dependency>

The following is an example of a Zipkin configuration, specified in the YAML format.

tracing:
  zipkin:
    service: "helidon-service"
    protocol: "https"
    host: "zipkin"
    port: 9987
    api-version: 1
    # this is the default path for API version 2
    path: "/api/v2/spans"
    tags:
      tag1: "tag1-value"
      tag2: "tag2-value"
    boolean-tags:
      tag3: true
      tag4: false
    int-tags:
      tag5: 145
      tag6: 741

Example of Zipkin trace:

OpenTelemetry Tracing

<dependency>
    <groupId>io.helidon.tracing.providers</groupId>
    <artifactId>helidon-tracing-providers-opentelemetry</artifactId>
</dependency>

tracing:
  service: helidon-otel-tracing-example 
  global: false      
  int-tags:
    example: 1       
  tags:
    direction: north 

• Specifies the OpenTelemetry service name.
• Indicates the configured tracer should not be made the global tracer (defaults to true).
• Assigns an integer-valued tag example the value 1.
• Assigns a string-valued tag direction the value north.

By default, Helidon tracing support for OpenTelemetry uses OpenTelemetry’s OTLP gRPC exporter. Alternatively, you can choose to use OpenTelemetry’s HTTP exporter using protobuf by setting exporter-type to http/proto. To use other exporters OpenTelemetry offers, use the Helidon telemetry configuration instead of tracing.

• OpenTelemetry API

• Opentracing Project (now part of OpenTelemetry)