Temporal networking problems can sometimes be mitigated by simply retrying a certain task. A Retry handler is created using a RetryPolicy that indicates the number of retries, delay between retries, etc.

Retry retry = Retry.builder()
        .retryPolicy(Retry.JitterRetryPolicy.builder()
                             .calls(3)
                             .delay(Duration.ofMillis(100))
                             .build())
        .build();
T result = retry.invoke(this::retryOnFailure);

The sample code above will retry calls to the supplier this::retryOnFailure for up to 3 times with a 100-millisecond delay between them.

If the call to the supplier provided completes exceptionally, it will be treated as a failure and retried until the maximum number of attempts is reached; finer control is possible by creating a retry policy and using methods such as applyOn(Class<? extends Throwable>…​ classes) and skipOn(Class<? extends Throwable>…​ classes) to control the exceptions that must be retried and those that must be ignored.

A request to a service that is inaccessible or simply unavailable should be bounded to ensure a certain quality of service and response time. Timeouts can be configured to avoid excessive waiting times. In addition, a fallback action can be defined if a timeout expires as we shall cover in the next section.

The following is an example of using Timeout:

T result = Timeout.create(Duration.ofMillis(10))
        .invoke(this::mayTakeVeryLong);

The example above monitors the call to method mayTakeVeryLong and reports a TimeoutException if the execution takes more than 10 milliseconds to complete.

A fallback to a known result can sometimes be an alternative to reporting an error. For example, if we are unable to access a service we may fall back to the last result obtained from that service at an earlier time.

A Fallback instance is created by providing a function that takes a Throwable and produces some T to be used when the intended method failed to return a value:

T result = Fallback.createFromMethod(throwable -> lastKnownValue)
        .invoke(this::mayFail);

This example calls the method mayFail and if it produces a Throwable, it maps it to the last known value using the fallback handler.

Failing to execute a certain task or to call another service repeatedly can have a direct impact on application performance. It is often preferred to avoid calls to non-essential services by simply preventing that logic to execute altogether. A circuit breaker can be configured to monitor such calls and block attempts that are likely to fail, thus improving overall performance.

Circuit breakers start in a closed state, letting calls to proceed normally; after detecting a certain number of errors during a pre-defined processing window, they can open to prevent additional failures. After a circuit has been opened, it can transition first to a half-open state before finally transitioning back to a closed state. The use of an intermediate state (half-open) makes transitions from open to close more progressive, and prevents a circuit breaker from eagerly transitioning to states without considering sufficient observations.

Consider the following example in which this::mayFail is monitored by a circuit breaker:

CircuitBreaker breaker = CircuitBreaker.builder()
        .volume(10)
        .errorRatio(30)
        .delay(Duration.ofMillis(200))
        .successThreshold(2)
        .build();
T result = breaker.invoke(this::mayFail);

The circuit breaker in this example defines a processing window of size 10, an error ratio of 30%, a duration to transition to half-open state of 200 milliseconds, and a success threshold to transition from half-open to closed state of 2 observations. It follows that,

• After completing the processing window, if at least 3 errors are detected, the circuit breaker will transition to the open state, thus blocking the execution of any subsequent calls.

• After 200 millis, the circuit breaker will transition back to half-open and allow calls to proceed again.

• If the next two calls after transitioning to half-open are successful, the circuit breaker will transition to closed state; otherwise, it will transition back to open state, waiting for another 200 milliseconds before attempting to transition to half-open again.

A circuit breaker will throw a io.helidon.faulttolerance.CircuitBreakerOpenException if an attempt to make an invocation takes place while it is in open state.

Concurrent access to certain components may need to be limited to avoid excessive use of resources. For example, if an invocation that opens a network connection is allowed to execute concurrently without any restriction, and if the service on the other end is slow responding, it is possible for the rate at which network connections are opened to exceed the maximum number of connections allowed. Faults of this type can be prevented by guarding these invocations using a bulkhead.

A waiting queue can be associated with a bulkhead to handle tasks that are submitted when the bulkhead is already at full capacity.

Bulkhead bulkhead = Bulkhead.builder()
        .limit(3)
        .queueLength(5)
        .build();
T result = bulkhead.invoke(this::usesResources);

This example creates a bulkhead that limits concurrent execution to this:usesResources to at most 3, and with a queue of size 5. The bulkhead will report a io.helidon.faulttolerance.BulkheadException if unable to proceed with the call: either due to the limit being reached or the queue being at maximum capacity.

Asynchronous tasks can be created or forked by using an Async instance. A supplier of type T is provided as the argument when invoking this handler. For example:

CompletableFuture<Thread> cf = Async.create().invoke(Thread::currentThread);
cf.thenAccept(t -> System.out.println("Async task executed in thread " + t));

The supplier () → Thread.currentThread() is executed in a new thread and the value it produces printed by the consumer and passed to thenAccept.

By default, asynchronous tasks are executed using a new virtual thread per task based on the ExecutorService defined in io.helidon.faulttolerance.FaultTolerance and configurable by an application. Alternatively, an ExecutorService can be specified when building a non-standard Async instance.

Method invocations can be guarded by any combination of the handlers presented above. For example, an invocation that times out can be retried a few times before resorting to a fallback value —assuming it never succeeds.

The easiest way to achieve handler composition is by using a builder in the FaultTolerance class as shown in the following example:

FaultTolerance.TypedBuilder<T> builder = FaultTolerance.typedBuilder();

Timeout timeout = Timeout.create(Duration.ofMillis(10));
builder.addTimeout(timeout);

Retry retry = Retry.builder()
        .retryPolicy(Retry.JitterRetryPolicy.builder()
                             .calls(3)
                             .delay(Duration.ofMillis(100))
                             .build())
        .build();
builder.addRetry(retry);

Fallback<T> fallback = Fallback.createFromMethod(throwable -> lastKnownValue);
builder.addFallback(fallback);

T result = builder.build().invoke(this::mayTakeVeryLong);

The exact order in which handlers are added to a builder depends on the use case, but generally the order starting from innermost to outermost should be: bulkhead, timeout, circuit breaker, retry and fallback. That is, fallback is the first handler in the chain (the last to executed once a value is returned) and bulkhead is the last one (the first to be executed once a value is returned).

Metrics can be enabled programmatically either globally or, if disabled globally, individually for each command instance. To enable metrics globally, call FaultTolerance.config(Config) passing a Config instance that sets ft.metrics.default-enabled=true. This must be done on application startup, before any command instances are created.

If metrics are not enabled globally, they can be enabled programmatically on each command instance using the enableMetrics(boolean) method on its corresponding builder. For example, the following snippet shows how to create a Retry instance of name my-retry with metrics support enabled.

Retry retry = Retry.builder()
        .name("my-retry")
        .enableMetrics(true)
        .build();