Fault Tolerance in Helidon

Overview

Helidon Fault Tolerance support is inspired by well-established fault-tolerance patterns for retries, timeouts, circuit breakers, fallbacks, and bulkheads. The API defines the notion of a fault handler that can be combined with other handlers to improve application robustness. Handlers are created to manage error conditions (faults) that may occur in real-world application environments. Examples include service restarts, network delays, temporal infrastructure instabilities, etc.

The interaction of multiple microservices bring some new challenges from distributed systems that require careful planning. Faults in distributed systems should be compartmentalized to avoid unnecessary service interruptions. For example, if comparable information can be obtained from multiples sources, a user request should not be denied when a subset of these sources is unreachable or offline. Similarly, if a non-essential source has been flagged as unreachable, an application should avoid continuous access to that source as that would result in much higher response times.

In order to tackle the most common types of application faults, the Helidon Fault Tolerance API provides support for circuit breakers, retries, timeouts, bulkheads and fallbacks. In addition, the API makes it very easy to create and monitor asynchronous tasks that do not require explicit creation and management of threads or executors.

For more information, see Fault Tolerance API Javadocs.

Maven Coordinates

To enable Fault Tolerance, add the following dependency to your project’s pom.xml (see Managing Dependencies).

<dependency>
  <groupId>io.helidon.fault-tolerance</groupId>
  <artifactId>helidon-fault-tolerance</artifactId>
</dependency>

API

The Fault Tolerance API is blocking and based on the JDK’s virtual thread model. As a result, methods return direct values instead of promises in the form of Single<T> or Multi<T>.

In the sections that follow, we shall briefly explore each of the constructs provided by this API.

Retries

Temporal networking problems can sometimes be mitigated by simply retrying a certain task. A Retry handler is created using a RetryPolicy that indicates the number of retries, delay between retries, etc.

Retry retry = Retry.builder()
        .retryPolicy(Retry.JitterRetryPolicy.builder()
                             .calls(3)
                             .delay(Duration.ofMillis(100))
                             .build())
        .build();
T result = retry.invoke(this::retryOnFailure);

The sample code above will retry calls to the supplier this::retryOnFailure for up to 3 times with a 100-millisecond delay between them.

If the call to the supplier provided completes exceptionally, it will be treated as a failure and retried until the maximum number of attempts is reached; finer control is possible by creating a retry policy and using methods such as applyOn(Class<? extends Throwable>... classes) and skipOn(Class<? extends Throwable>... classes) to control the exceptions that must be retried and those that must be ignored.

An overload accepting Function<RetryContext, T> provides the current attempt number, elapsed time, preceding delay, and preceding failure to each invocation. If the invocation does not succeed, RetryException exposes a RetryOutcome with the termination reason and final attempt metadata. The original Supplier overload continues to propagate the terminal application exception or RetryTimeoutException.

Retry retry = Retry.builder()
        .calls(100)
        .delay(Duration.ofMillis(100))
        .delayFactor(2)
        .jitterFactor(0.2)
        .maxDelay(Duration.ofSeconds(5))
        .overallTimeout(Duration.ofMinutes(5))
        .build();

try {
    T result = retry.invoke(context -> retryOnFailure(context.attempt()));
} catch (RetryException e) {
    RetryOutcome outcome = e.outcome();
    // Inspect outcome.termination(), outcome.attempts(), and outcome.lastThrowable().
}

calls must be at least one, delay must be non-negative, and overall-timeout must be positive. Exponential delay can use either a non-negative absolute jitter duration (PT-1S disables it) or a relative jitter-factor from zero inclusive to one exclusive (-1 disables it); the two jitter options are mutually exclusive. The optional non-negative max-delay caps the final delay after jitter. Jitter is applied independently to each calculated exponential delay and does not affect later base-delay calculations. A contextual invocation can also supply a Retry.WaitStrategy to maintain an external resource while waiting or cancel another attempt by returning false. A strategy returning true must complete the requested wait and must restore the thread interrupt status if interrupted.

In Helidon 27, configuring both delay-factor and absolute jitter applies the delay factor first and then applies jitter. Earlier releases ignored jitter when both options were configured. The defaults are unchanged: three total calls, a 200 ms base delay with a factor of two, no jitter, no maximum delay, and a one-second overall timeout.

Timeouts

A request to a service that is inaccessible or simply unavailable should be bounded to ensure a certain quality of service and response time. Timeouts can be configured to avoid excessive waiting times. In addition, a fallback action can be defined if a timeout expires as we shall cover in the next section.

The following is an example of using Timeout:

T result = Timeout.create(Duration.ofMillis(10))
        .invoke(this::mayTakeVeryLong);

The example above monitors the call to method mayTakeVeryLong and reports a TimeoutException if the execution takes more than 10 milliseconds to complete.

Fallbacks

A fallback to a known result can sometimes be an alternative to reporting an error. For example, if we are unable to access a service we may fall back to the last result obtained from that service at an earlier time.

A Fallback instance is created by providing a function that takes a Throwable and produces some T to be used when the intended method failed to return a value:

T result = Fallback.createFromMethod(throwable -> lastKnownValue)
        .invoke(this::mayFail);

This example calls the method mayFail and if it produces a Throwable, it maps it to the last known value using the fallback handler.

Circuit Breakers

Failing to execute a certain task or to call another service repeatedly can have a direct impact on application performance. It is often preferred to avoid calls to non-essential services by simply preventing that logic to execute altogether. A circuit breaker can be configured to monitor such calls and block attempts that are likely to fail, thus improving overall performance.

Circuit breakers start in a closed state, letting calls to proceed normally; after detecting a certain number of errors during a pre-defined processing window, they can open to prevent additional failures. After a circuit has been opened, it can transition first to a half-open state before finally transitioning back to a closed state. The use of an intermediate state (half-open) makes transitions from open to close more progressive, and prevents a circuit breaker from eagerly transitioning to states without considering sufficient observations.

Consider the following example in which this::mayFail is monitored by a circuit breaker:

CircuitBreaker breaker = CircuitBreaker.builder()
        .volume(10)
        .errorRatio(30)
        .delay(Duration.ofMillis(200))
        .successThreshold(2)
        .build();
T result = breaker.invoke(this::mayFail);

The circuit breaker in this example defines a processing window of size 10, an error ratio of 30%, a duration to transition to half-open state of 200 milliseconds, and a success threshold to transition from half-open to closed state of 2 observations. It follows that,

• After completing the processing window, if at least 3 errors are detected, the circuit breaker will transition to the open state, thus blocking the execution of any subsequent calls.
• After 200 millis, the circuit breaker will transition back to half-open and allow calls to proceed again.
• If the next two calls after transitioning to half-open are successful, the circuit breaker will transition to closed state; otherwise, it will transition back to open state, waiting for another 200 milliseconds before attempting to transition to half-open again.

A circuit breaker will throw a io.helidon.faulttolerance.CircuitBreakerOpenException if an attempt to make an invocation takes place while it is in open state.

Bulkheads

Concurrent access to certain components may need to be limited to avoid excessive use of resources. For example, if an invocation that opens a network connection is allowed to execute concurrently without any restriction, and if the service on the other end is slow responding, it is possible for the rate at which network connections are opened to exceed the maximum number of connections allowed. Faults of this type can be prevented by guarding these invocations using a bulkhead.

A waiting queue can be associated with a bulkhead to handle tasks that are submitted when the bulkhead is already at full capacity.

Bulkhead bulkhead = Bulkhead.builder()
        .limit(3)
        .queueLength(5)
        .build();
T result = bulkhead.invoke(this::usesResources);

This example creates a bulkhead that limits concurrent execution to this:usesResources to at most 3, and with a queue of size 5. The bulkhead will report a io.helidon.faulttolerance.BulkheadException if unable to proceed with the call: either due to the limit being reached or the queue being at maximum capacity.

Asynchronous

Asynchronous tasks can be created or forked by using an Async instance. A supplier of type T is provided as the argument when invoking this handler. For example:

CompletableFuture<Thread> cf = Async.create().invoke(Thread::currentThread);
cf.thenAccept(t -> System.out.println("Async task executed in thread " + t));

The supplier () → Thread.currentThread() is executed in a new thread and the value it produces printed by the consumer and passed to thenAccept.

By default, asynchronous tasks are executed using a new virtual thread per task based on the default ExecutorService used by the Fault Tolerance module. Alternatively, an ExecutorService can be specified when building a non-standard Async instance.

Handler Composition

Method invocations can be guarded by any combination of the handlers presented above. For example, an invocation that times out can be retried a few times before resorting to a fallback value —assuming it never succeeds.

The easiest way to achieve handler composition is by using a builder in the FaultTolerance class as shown in the following example:

FaultTolerance.TypedBuilder<T> builder = FaultTolerance.typedBuilder();

Timeout timeout = Timeout.create(Duration.ofMillis(10));
builder.addTimeout(timeout);

Retry retry = Retry.builder()
        .retryPolicy(Retry.JitterRetryPolicy.builder()
                             .calls(3)
                             .delay(Duration.ofMillis(100))
                             .build())
        .build();
builder.addRetry(retry);

Fallback<T> fallback = Fallback.createFromMethod(throwable -> lastKnownValue);
builder.addFallback(fallback);

T result = builder.build().invoke(this::mayTakeVeryLong);

The exact order in which handlers are added to a builder depends on the use case, but generally the order starting from innermost to outermost should be: bulkhead, timeout, circuit breaker, retry and fallback. That is, fallback is the first handler in the chain (the last to executed once a value is returned) and bulkhead is the last one (the first to be executed once a value is returned).

Metrics

The Helidon Fault Tolerance module has support for some basic metrics to monitor certain application conditions. Metrics are disabled by default, but can be enabled programmatically as described in Enabling Metrics Programmatically, and by including an actual metrics implementation in your classpath. For more information about metrics implementations see Helidon Metrics.

The following tables list all the metrics created by the Fault Tolerance module. Each handler is identified by its name, assigned either programmatically or automatically by the API. Multiple handler instances may intentionally use the same name; they then refer to the same metric series. Use distinct names when separate per-instance series are required.

Bulkheads:

Circuit Breakers:

Retries:

Timeouts:

Enabling Metrics Programmatically

Metrics can be enabled programmatically either globally or, if disabled globally, individually for each command instance. To enable metrics globally, configure the application Config with ft.metrics.default-enabled=true and make that config available through the service registry, for example by calling io.helidon.service.registry.Services.set(Config.class, config) on application startup, before any command instances are created.

If metrics are not enabled globally, they can be enabled programmatically on each command instance using the enableMetrics(boolean) method on its corresponding builder. For example, the following snippet shows how to create a Retry instance of name my-retry with metrics support enabled.

Retry retry = Retry.builder()
        .name("my-retry")
        .enableMetrics(true)
        .build();

Examples

See API section for examples.

Additional Information

For additional information, see the Fault Tolerance API Javadocs.