Asynchronous tasks can be created or forked by using an Async instance. A supplier of type T is provided as the argument when invoking this handler. For example:

Single<Thread> s = Async.create().invoke(() -> Thread.currentThread()));
s.thenAccept(t -> System.out.println("Async task executed in thread " + t));

The supplier () → Thread.currentThread() is executed in a new thread and the value it produces printed by the consumer and passed to thenAccept.

Asynchronous tasks are executed in a thread pool managed by the Helidon SE Fault Tolerance module. Thread pools are created during the initialization phase of class io.helidon.faulttolerance.FaultTolerance and can be configured for your application.

Temporal networking problems can sometimes be mitigated by simply retrying a certain task. A Retry handler is created using a RetryPolicy that indicates the number of retries, delay between retries, etc.

Retry retry = Retry.builder()
                   .retryPolicy(Retry.JitterRetryPolicy.builder()
                                     .calls(3)
                                     .delay(Duration.ofMillis(100))
                                     .build())
                   .build();
retry.invoke(this::retryOnFailure);

The sample code above will retry calls to the supplier this::retryOnFailure for up to 3 times with a 100 millisecond delay between them.

If the CompletionStage<T> returned by the method completes exceptionally, the call will be treated as a failure and retried until the maximum number of attempts is reached; finer control is possible by creating a retry policy and using methods such as applyOn(Class<? extends Throwable>…​ classes) and skipOn(Class<? extends Throwable>…​ classes) to control those exceptions on which to act and those that can be ignored.

A request to a service that is inaccessible or simply unavailable should be bounded to ensure a certain quality of service and response time. Timeouts can be configured to avoid excessive waiting times. In addition, a fallback action can be defined if a timeout expires as we shall cover in the next section.

The following is an example of using Timeout:

Single<T> s = Timeout.create(Duration.ofMillis(10)).invoke(this::mayTakeVeryLong);
s.handle((t, e) -> {
    if (e instanceof TimeoutException) {
        // Invocation has timed out!
    }
    //...
});

The example above monitors the call to method mayTakeVeryLong and reports a TimeoutException if the execution takes more than 10 milliseconds to complete.

A fallback to a known result can sometimes be an alternative to reporting an error. For example, if we are unable to access a service we may fall back to the last result obtained from that service.

A Fallback instance is created by providing a function that takes a Throwable and produces a CompletionStage<T> as shown next:

Single<T> single = Fallback.create(
    throwable -> Single.just(lastKnownValue).invoke(this::mayFail);
single.thenAccept(t -> {
    //...
});

In this example, we register a function that can produce a Single<T> (which implements CompletionStage<T>) if the call to this::mayFail completes exceptionally.

Failing to execute a certain task or call another service repeatedly can have a direct impact on application performance. It is often preferred to avoid calls to non-essential services by simply preventing that logic to execute altogether. A circuit breaker can be configured to monitor such calls and block attempts that are likely to fail, thus improving overall performance.

Circuit breakers start in a closed state, letting calls to proceed normally; after detecting a certain number of errors during a pre-defined processing window, they can open to prevent additional failures. After a circuit has been opened, it can transition first to a half-open state before finally transitioning back to a closed state. The use of an intermediate state (half-open) makes transitions from open to close more progressive, and prevents a circuit breaker from eagerly transitioning to states without considering "sufficient" observations.

Consider the following example in which this::mayFail is monitored by a circuit breaker:

CircuitBreaker breaker = CircuitBreaker.builder()
                                       .volume(10)
                                       .errorRatio(30)
                                       .delay(Duration.ofMillis(200))
                                       .successThreshold(2)
                                       .build();
Single<T> result = breaker.invoke(this::mayFail);

The circuit breaker in this example defines a processing window of size 10, an error ratio of 30%, a duration to transition to half-open state of 200 milliseconds, and a success threshold to transition from half-open to closed state of 2 observations. It follows that,

• After completing the processing window, if at least 3 errors were detected, the circuit breaker will transition to the open state, thus blocking the execution of any subsequent calls.

• After 200 millis, the circuit breaker will transition back to half-open and enable calls to proceed again.

• If the next two calls after transitioning to half-open are successful, the circuit breaker will transition to closed state; otherwise, it will transition back to open state, waiting for another 200 milliseconds before attempting to transition to half-open again.

A circuit breaker will throw a io.helidon.faulttolerance.CircuitBreakerOpenException if an attempt to make an invocation takes place while it is in open state.

Concurrent access to certain components may need to be limited to avoid excessive use of resources. For example, if an invocation that opens a network connection is allowed to execute concurrently without any restriction, and if the service on the other end is slow responding, it is possible for the rate at which network connections are opened to exceed the maximum number of connections allowed. Faults of this type can be prevented by guarding these invocations using a bulkhead.

A waiting queue can be associated with a bulkhead to handle tasks that are submitted when the bulkhead is already at full capacity.

Bulkhead bulkhead = Bulkhead.builder()
                            .limit(3)
                            .queueLength(5)
                            .build();
Single<T> single = bulkhead.invoke(this::usesResources);

This example creates a bulkhead that limits concurrent execution to this:usesResources to at most 3, and with a queue of size 5. The bulkhead will report a io.helidon.faulttolerance.BulkheadException if unable to proceed with the call: either due to the limit being reached or the queue being at maximum capacity.

Method invocations can be guarded by any combination of the handlers presented above. For example, an invocation that times out can be retried a few times before resorting to a fallback value —assuming it never succeeds.

The easiest way to achieve handler composition is by using a builder in the FaultTolerance class as shown in the following example:

FaultTolerance.TypedBuilder<T> builder = FaultTolerance.typedBuilder();

// Create and add timeout
Timeout timeout = Timeout.create(Duration.ofMillis(10));
builder.addTimeout(timeout);

// Create and add retry
Retry retry = Retry.builder()
                   .retryPolicy(Retry.JitterRetryPolicy.builder()
                                     .calls(3)
                                     .delay(Duration.ofMillis(100))
                                     .build())
                   .build();
builder.addRetry(retry);

// Create and add fallback
Fallback fallback = Fallback.create(throwable -> Single.just(lastKnownValue));
builder.addFallback(fallback);

// Finally call the method
Single<T> single = builder.build().invoke(this::mayTakeVeryLong);

The exact order in which handlers are added to a builder depends on the use case, but generally the order starting from innermost to outermost should be: bulkhead, timeout, circuit breaker, retry and fallback. That is, fallback is the first handler in the chain (the last to executed once a value is returned) and bulkhead is the last one (the first to be executed once a value is returned).

All the examples presented so far have focused on invocations returning a single value of type Single<T>. If the invocation in question can return more than one value (i.e., a Multi<T>) then all that is needed is to use the method invokeMulti instead of invoke. The supplier passed to this method must return a Flow.Publisher<T> instead of a CompletionStage<T>.

A Flow.Publisher<T> is a generalization of a Single<T> that can produce zero or more values. Note that a Flow.Publisher<T>, unlike a Single<T>, can report an error after producing one or more values, introducing additional challenges if all values must be processed transactionally, that is, in an all or nothing manner.

The following example creates an instance of Retry and invokes the invokeMulti method, it then registers a subscriber to process the results:

Retry retry = Retry.builder()
                   .retryPolicy(Retry.JitterRetryPolicy.builder()
                                     .calls(2)
                                     .build())
                   .build();
Multi<Integer> multi = retry.invokeMulti(() -> Multi.just(0, 1, 2));

IntSubscriber ts = new IntSubscriber();
multi.subscribe(ts);
ts.request(Integer.MAX_VALUE);

The call to Multi.just(0, 1, 2) simply returns a multi that produces the integers 0, 1 and 2. If an error was generated during this process, the policy will retry the call one more time —for a total of 2 calls.

• Delaying the Retry Policy

• Jitter Retry Policy

Delaying the Retry Policy

Jitter Retry Policy