Tracing

Tracing Support

Helidon includes support for tracing through the OpenTracing APIs. Tracing is integrated with WebServer, gRPC Server, and Security.

Support for specific tracers is abstracted. Your application can depend on the abstraction layer and provide a specific tracer implementation as a Java ServiceLoader service.

Configuring Tracing with Helidon SE

Declare the following dependency in your project to use the tracer abstraction:

Tracer Abstraction

<dependency>
    <groupId>io.helidon.tracing</groupId>
    <artifactId>helidon-tracing</artifactId>
</dependency>

Copied

Configuring Tracing with WebServer

To configure tracer with WebServer:

Configuring OpenTracing Tracer

ServerConfiguration.builder()
                   .tracer(TracerBuilder.create("my-application")                    
                                 .collectorUri(URI.create("http://10.0.0.18:9411"))  
                                 .build())
                   .build()

Copied

The name of the application (service) to associate with the tracing events
The endpoint for tracing events, specific to the tracer used, usually loaded from Config

Configuring Tracing with gRPC Server

Configuring OpenTracing Tracer

Tracer tracer = (Tracer) TracerBuilder.create("Server")
        .collectorUri(URI.create("http://10.0.0.18:9411"))   
        .build();

Copied

If using zipkin tracing system, the endpoint would be:

http://10.0.0.18:9411/api/v2/spans

GrpcTracingConfig tracingConfig = new GrpcTracingConfig.Builder()
                .withStreaming()
                .withVerbosity()
                .withTracedAttributes(ServerRequestAttribute.CALL_ATTRIBUTES,
                     ServerRequestAttribute.HEADERS,
                     ServerRequestAttribute.METHOD_NAME)
                .build();

GrpcServerConfiguration serverConfig = GrpcServerConfiguration.builder().port(0)
                .tracer(tracer)
                .tracingConfig(tracingConfig)
                .build();

Configuration using Helidon Config

There is a set of common configuration options that this section describes. In addition each tracer implementation may have additional configuration options - please see the documentation of each of them.

Each implementation may provide defaults for these options.

All common configuration options:

key	description
service	Name of the service sending the tracing information. This is usually visible in the trace data to distinguish actors in a conversation (e.g. when multiple microservices are connected together)
protocol	Protocol of the tracing collector (e.g. `http`, `https`)
host	Host of the tracing collector (e.g. `localhost`)
port	Port of the tracing collector (e.g. `9411`)
path	Path of the tracing collector service that is used to send spans to
enabled	If set to false, tracing would be disabled
tags	String tags that are to be added to each span reported (object node of string-string pairs)
boolean-tags	Boolean tags that are to be added to each span reported (object node of string-boolean pairs)
int-tags	Int tags that are to be added to each span reported (object node of string-int pairs)

Creating custom spans

To create a custom span that is a child of the WebServer request:

Span span = request.tracer()
        .buildSpan("my-operation")
        .asChildOf(request.spanContext())
                         .start();

Copied

Trace propagation across services

Automated trace propagation is supported currently only with Jersey client.

Tracing propagation with Jersey client

//...

Response response = client.target(serviceEndpoint)
    .request()
    .get();

Copied

Traced spans

The following table lists all spans traced by Helidon components:

component	span name	description
`web-server`	`HTTP Request`	The overall span of the Web Server from request intitiation until response Note that in `Zipkin` the name is replaced with `jax-rs` span name if `jax-rs` tracing is used.
`web-server`	`content-read`	Span for reading the request entity
`web-server`	`content-write`	Span for writing the response entity
`security`	`security`	Processing of request security
`security`	`security:atn`	Span for request authentication
`security`	`security:atz`	Span for request authorization
`security`	`security:response`	Processing of response security
`security`	`security:outbound`	Processing of outbound security
`jax-rs`	A generated name	Span for the resource method invocation, name is generated from class and method name
`jax-rs`	`jersey-client-call`	Span for outbound client call

Some of these spans log to the span. These log events can be (in most cases) configured:

span name	log name	configurable	enabled by default	description
`HTTP Request`	`handler.class`	YES	YES	Each handler has its class and event logged
`security`	`status`	YES	YES	Logs either "status: PROCEED" or "status: DENY"
`security:atn`	`security.user`	YES	NO	The username of the user if logged in
`security:atn`	`security.service`	YES	NO	The name of the service if logged in
`security:atn`	`status`	YES	YES	Logs the status of security response (such as `SUCCESS`)
`security:atz`	`status`	YES	YES	Logs the status of security response (such as `SUCCESS`)
`security:outbound`	`status`	YES	YES	Logs the status of security response (such as `SUCCESS`)

There are also tags that are set by Helidon components. These are not configurable.

span name	tag name	description
`HTTP Request`	`component`	name of the component - `helidon-webserver`, or `jaxrs` when using MP
`HTTP Request`	`http.method`	HTTP method of the request, such as `GET`, `POST`
`HTTP Request`	`http.status_code`	HTTP status code of the response
`HTTP Request`	`http.url`	The path of the request (for SE without protocol, host and port)
`HTTP Request`	`error`	If the request ends in error, this tag is set to `true`, usually accompanied by logs with details
`content-read`	`requested.type`	Type (class) of the requested entity (if entity is read)
`content-write`	`response.type`	Type (class) of the entity being sent (if enitty is sent)
`security`	`security.id`	ID of the security context created for this request (if security is used)
`jersey-client-call`	`http.method`	HTTP method of the client request
`jersey-client-call`	`http.status_code`	HTTP status code of client response
`jersey-client-call`	`http.url`	Full URL of the request (such as `http://localhost:8080/greet`)

Traced spans configuration

Each component and its spans can be configured using Config. The traced configuration has the following layers:

TracingConfig - the overall configuration of traced components of Helidon
ComponentTracingConfig - a component of Helidon that traces spans (such as web-server, security, jax-rs)
SpanTracingConfig - a single traced span within a component (such as security:atn)
SpanLogTracingConfig - a single log event on a span (such as security.user in span security:atn)

The components using tracing configuration use the TracingConfigUtil. This uses the io.helidon.common.Context to retrieve current configuration.

Configuration using builder

Builder approach, example that disables a single span log event:

Configure tracing using a builder

TracingConfig.builder()
     .addComponent(ComponentTracingConfig.builder("web-server")
             .addSpan(SpanTracingConfig.builder("HTTP Request")
                     .addSpanLog(SpanLogTracingConfig.builder("content-write").enabled(false).build())
                     .build())
             .build())
     .build()

Copied

Configuration using Helidon Config

Tracing configuration can be defined in a config file.

Tracing configuration

tracing:
    components:
      web-server:
        spans:
          - name: "HTTP Request"
            logs:
              - name: "content-write"
                enabled: false

Copied

Use the configuration in web server

routing.register(WebTracingConfig.create(config.get("tracing")));

Copied

Path based configuration in Helidon Web Server

For Web Server we have a path based support for configuring tracing, in addition to the configuration described above.

Configuration of path can use any path string supported by the Web Server. The configuration itself has the same possibilities as traced configuration described above. The path specific configuration will be merged with global configuration (path is the "newer" configuration, global is the "older")

Configuration in YAML

tracing:
  paths:
    - path: "/favicon.ico"
      enabled: false
    - path: "/metrics"
      enabled: false
    - path: "/health"
      enabled: false
    - path: "/greet"
      components:
        web-server:
          spans:
          - name: "content-read"
            new-name: "read"
            enabled: false

Copied

Configuration with Web Server

routingBuilder.register(WebTracingConfig.create(config.get("tracing"));

Copied

Configuration with Web Server using a builder

routingBuilder.register(WebTracingConfig.builder()
    .addPathConfig(PathTracingConfig.builder()
        .path("/metrics")
        .tracingConfig(TracingConfig.DISABLED)
        .build();
    .build());

Copied

Renaming top level span using request properties

To have a nicer overview in search pane of a tracer, you can customize the top-level span name using configuration.

Example:

Configuration in YAML

tracing:
  components:
    web-server:
      spans:
      - name: "HTTP Request"
        new-name: "HTTP %1$s %2$s"

Copied

This is supported ONLY for the span named "HTTP Request" on component "web-server".

Parameters provided: 1. Method - HTTP method 2. Path - path of the request (such as '/greet') 3. Query - query of the request (may be null)