The config system supports using environment variables as a config source, and is enabled by default. Since environment variable names are normally restricted to alphanumeric characters and underscore, this config source adds aliases that enable setting or overriding config entries with dotted and/or hyphenated keys.

The mapping makes it possible to set or override a config entry with a key of "foo.bar" using an environment variable named "FOO_BAR" and "foo.bar-baz" using "FOO_BAR_dash_BAZ".

One use case for this mapping is config overrides in containers, where passing environment variables directly or via Kubernetes Secrets/ConfigMaps is common. Scripts that solve the mapping problem by explicitly converting variables to system properties can also be simplified.

Aliases are produced for any environment variable name that matches all of the following:

• does not begin or end with a '_' character
• does not contain "__"
• contains one or more '_' characters

For each such name, two aliases are added with the names mapped as follows:

• Replace any "_dash_" or "_DASH_" substrings with "-", e.g. "APP_PAGE_dash_SIZE" becomes "APP_PAGE-SIZE".
• Replace '_' with '.' and add as an alias, e.g. "APP_GREETING" is added as "APP.GREETING" and "APP_PAGE-SIZE" is added as "APP.PAGE-SIZE". This mapping is added primarily to support mixed case config keys such as "app.someCamelCaseKey".
• Convert the result of step 2 to lowercase and add as an alias, e.g. "APP.GREETING" is added as "app.greeting" and "APP.PAGE-SIZE" is added as "app.page-size".

The config system supports using a file system directory as a config source. Each non-directory file in the directory becomes a config entry: the file name is the key and the contents of that file are used as the corresponding config String value.

The following example shows, for example, one way to load Kubernetes secrets mounted on the pod’s filesystem.

If the directory conf/secrets contains these two files

jose

^ery$ecretP&ssword

your application can load this as configuration as follows:

Config secrets = Config.withSources(
        ConfigSources.directory("conf/secrets")) 
        .disableEnvironmentVariablesSource()     
        .disableSystemPropertiesSource()         
        .build();

assert secrets.get("username")                   
        .asString()
        .get()
        .equals("jose");
assert secrets.get("password")                   
        .asString()
        .get()
        .equals("^ery$ecretP&ssword");

• Loads all files from the conf/secrets directory.
• No need to use environment variables or system properties as sources in building the Config.
• The loaded config maps the key username to the value jose…​
• …​and the key password to ^ery$ecretP&ssword.

Remember that your application can process the contents of a given file as configuration. See the config sources section and the ConfigSources.file JavaDoc.

The config system provides several ways to create a Config tree from data already in memory. See the ConfigSources javadoc for further details. The numerous variants of the from method construct ConfigSource or Builder<ConfigSource> instances.

Subtree of Another Config

Config anotherConfig = Config.create(classpath("application.conf"));

Config config = Config.create(
        ConfigSources.create(anotherConfig.get("data")));

Properties Object

Config config = Config.create(
        ConfigSources.create(System.getProperties()).build());                

String of a Given Media Type

Config config = Config.create(
        ConfigSources.create("app.greeting = Hi", "text/x-java-properties")); 

Map

Config config = Config.crate(
        ConfigSources.create(Map.of("app.page-size", "20"))
            .lax()             
            .build());         

ad hoc Config Nodes

Config config = Config.create(
        ConfigSources.create(ObjectNode.builder()
                                   .addList("app.basic-range", ListNode.builder()
                                           .addValue("-20")
                                           .addValue("20")
                                           .build())
                                   .build()));

Although the examples above use a single source, you can build a single Config from multiple sources.

Handling Key Collisions

greeting = "Hello"
page-size = 20
basic-range = [ -20, 20 ]

providers: [
    {
        name = "Provider1"
        class = "this.is.my.Provider1"
    },
    {
        name = "Provider2"
        class = "this.is.my.Provider2"
    }
]

Config config = Config.create(
        ConfigSources.prefixed("app",                    
                               classpath("app.conf")),   
        ConfigSources.prefixed("data",                   
                               classpath("data.conf"))); 

assert config.get("app.greeting")                        
        .asString()
        .get()
        .equals("Hello");

assert config.get("data.providers.0.name")               
        .asString()
        .get()
        .equals("Provider1");

Config config = Config.create(                                               
        ConfigSources.create(file("conf/dev.properties").optional(),         
                           file("conf/config.properties").optional())        
                .add(classpath("application.properties"))                    
                .mergingStrategy(ConfigSources.MergingStrategy.fallback())); 

As the config sources and parsers section describes, these two work together to read and translate configuration data from some external form into the corresponding in-memory config tree.

Although most applications are explicit about the config sources they use in building a Config, the config system often has to figure out what parser to use. It does so by:

1. determining, the best that it can, the media type of the source, and
2. locating a parser that can translate that media type.

Identifying the Media Type

Config config = Config.create(classpath("props")                             
                                    .mediaType("text/x-java-properties")); 

Locating a Parser

Config config = Config.create(classpath("props")                            
                                    .parser(ConfigParsers.properties())); 

A config value node might contain an entire config document in String form, but in a format different from the containing document. Your application can tell the config system to parse such a node as config in a different format and replace the String value node in the original tree with the config tree that results from parsing that String.

In this example, a YAML document contains a JSON document as a leaf.

secrets:
    username: "jose"
    password: "^ery$ecretP&ssword"

app: >                             
    {
        "greeting": "Hello",
        "page-size": 20,
        "basic-range": [ -20, 20 ]
    }

• The property app is itself formatted as a JSON document.

Specify Key-to-media-type Mapping

Config config = Config.create(
        classpath("application.yaml")
                .mediaTypeMapping(                          
                        key -> "app".equals(key.toString()) 
                                ? "application/json"
                                : null));

assert config.get("secrets.username").asString()            
        .get().equals("jose");
assert config.get("secrets.password").asString()            
        .get().equals("^ery$ecretP&ssword");

assert config.get("app").type() == Type.OBJECT;             

assert config.get("app.greeting")                           
        .asString().get().equals("Hello");
assert config.get("app.page-size")                          
        .asInt().get() == 20;
assert config.get("app.basic-range.0")                      
        .asInt().get() == -20;
assert config.get("app.basic-range.1")                      
        .asInt().get() == 20;

Specify Key-to-parser Mapping

Config config = Config.create(
        ConfigSources.classpath("application.yaml")
                .parserMapping(                                           
                        key -> "app".equals(key.toString())               
                                ? HoconConfigParserBuilder.buildDefault()
                                : null));

Instead of including code in your application to construct config trees from builders, sources, etc., you can instead prepare meta-configuration in a file that declares the sources to load and their attributes.

You can either specify the meta-config file in your application or allow the config system to search for and load meta-config from a preset list of possible sources.

Loading Config by Specifying a Meta-configuration File

Loading Config from an Implicit Meta-configuration File

Meta-configuration File Format

sources = [
    {
        type = "environment-variables"
    }
    {
        type = "system-properties"
    }
    {
        type = "directory"
        properties {
            path = "conf/secrets"
            media-type-mapping {
                yaml = "application/x-yaml"
                password = "application/base64"
            }
            polling-strategy {
                type = "regular"
                properties {
                    interval = "PT15S"
                }
            }
        }
    }
    {
        type = "url"
        properties {
            url = "http://config-service/my-config"
            media-type = "application/hocon"
            optional = true
            retry-policy {
                type = "repeat"
                properties {
                    retries = 3
                }
            }
        }
    }
    {
        type = "file"
        properties {
            path = "conf/env.yaml"
            polling-strategy {
                type = "watch"
            }
        }
    }
    {
        type = "prefixed"
        properties {
            key = "app"
            type = "classpath"
            properties {
                resource = "app.conf"
            }
        }
    }
    {
        type = "classpath"
        properties {
            resource = "application.conf"
        }
    }
]

Meta-config for Custom Source Types

{
    class = "io.helidon.config.git.GitConfigSourceBuilder"
    properties {
        path = "application.conf"
        directory = "/app-config"
    }
}

git = io.helidon.config.git.GitConfigSourceBuilder

{
    type = "git"
    properties {
        path = "application.conf"
        directory = "/app-config"
    }
}

Loading Config using Meta-configuration

ConfigSource sourceFromMetaConfig = ConfigSources.load(  
        classpath("config-meta-all.conf")).build();      

Config config = Config.create(sourceFromMetaConfig);       

Meta-config for Polling Strategies and Retry Policies

Each filter accepts a key and the value as defined in the source, and returns the value to be used. The filter can leave the value unchanged or alter it, as it sees fit.

The built-in value-resolving filter enables the token substitution described below.

See the ConfigFilter JavaDoc for more information.

The overrides feature allows you to create an external document containing key/value pairs which replace the value otherwise returned for the name, and then add that document as an override source to a config builder.

There are some key differences between overrides and filters.

• Because overrides are loaded from sources those sources can change while your application runs and so the overrides they that prescribe can change.

• The override document can use wildcards in key expressions.

• Overrides can affect only keys that already exist in the original source; filters can supply values even if the key is absent from the config source.

Each override entry consists of a Java properties-format definition. The key is an expression (which can use wildcards) to match config keys read from the current config sources, and the override value is the new value for any key matching the key expression from that entry. Order is important. The config system tests every key expression/value pair one by one in the order they appear in the overrides sources. Once the config system finds an override entry in which the key expression matches the configuration key, the system returns that entry’s value for the key being processed.

See the OverrideSource JavaDoc for more detail.

A token reference is a key token starting with $, optionally enclosed between { and }, i.e. $ref, ${ref}. Even a key composed of more than one token can be referenced in another key, i.e. ${env.ref}.

As an example use case, you can use token references to declare the default values (see resolving-tokens.yaml below), while the references may be resolved in another config source, which identifies a current environment (see env.yaml examples below). You can then use the same overrides for different environments, say test and prod. The configuration in each environment is then overridden with a different values using wildcards (see overrides.properties below).

Config config = Config.builder()
        .overrides(OverrideSources.file("conf/overrides.properties")) 
        .sources(file("conf/env.yaml"),                               
                 classpath("resolving-tokens.yaml"))                  
        .build();

• Loads overrides from the specified file.
• A deployment-specific environment configuration file.
• A default configuration containing token references that are resolved using the environment-specific override.

The two methods PollingStrategies.regular(Duration) and PollingStrategies.watch(Path) return builders for their respective strategies. Both builders expose the executor method which your application can invoke, passing a java.util.concurrent.ScheduledExecutorService instance it wants used for the polling work. By default each polling strategy instance uses a separate thread pool executor.

The following example shares the same executor for two different polling strategy instances.

ScheduledExecutorService executor = Executors.newScheduledThreadPool(2); 

Config config = Config.create(
        ConfigSources.file("conf/dev.properties")
                .pollingStrategy(
                        PollingStrategies.regular(Duration.ofSeconds(2)) 
                                .executor(executor)),                    
        ConfigSources.create("conf/config.properties")
                .pollingStrategy(
                        path -> PollingStrategies.watch(path)            
                                .executor(executor)));                   

• Prepares a thread pool executor with core pool size set 2 to be shared by all polling strategies.
• Selects the built-in periodic polling strategy.
• Tells the config system to use the specific executor to poll the dev.properties config source.
• Uses the Java filesystem WatchService to monitor the specified path.
• Tells the config system to use the same executor to monitor the path.

Recall that when a polling strategy detects a change in a source, it informs interested parties of the changes. By default each Config.Builder arranges for the resulting Config tree to use a shared executor that reuses available threads from a pool, creating new threads as needed. The same executor is used for actually reloading the source.

Your application can invoke the polling strategy builder’s changesExecutor method to tell the builder to use a different Executor. (As an aside, your application can also control the size of the buffer used for holding source change events by invoking the builder’s changesMaxBuffer method. The default is 256.)

Executor executor = Executors.newCachedThreadPool();               

Config config = Config.builder()
        .overrides(
                OverrideSources.file("conf/overrides.properties")
                        .pollingStrategy(PollingStrategies::watch)
                        .changesExecutor(executor)                 
                        .changesMaxBuffer(4))                      
        .sources(
                ConfigSources.file("conf/env.yaml")
                        .pollingStrategy(PollingStrategies::watch)
                        .changesExecutor(executor)                 
                        .changesMaxBuffer(4))                      
        .build();

• Prepares a thread pool executor to be shared by selected sources.
• Tells the builder that the resulting overrides source should use the specified Executor for notifying interested parties of changes and for reloading the override source.
• Specifies an event buffer size of 4.
• Uses the same Executor and event buffer size for the config source as for the override source above.

When your application supplies multiple sources to a config builder, as with Config.create(Supplier<ConfigSource>…​) and Config.create(List<Supplier<ConfigSource>>), the config system automatically uses a composite config source which aggregates the separate sources but also listens for changes to any of the individual sources so it can delegate the change notification. For this change detection and notification the config system, by default, uses an executor with a dedicated thread pool that is shared across all Config instances.

Your application can invoke the builder’s changesExecutor method to use a different ScheduledExecutorService instance. The builder returned by the from methods mentioned above is a CompositeBuilder which extends Config.Builder.

Because a composite source might yield more numerous change events — because of the multiple underlying sources — your application can specify a debounce timeout for the composite source by invoking the CompositeBuilder.changesDebounce(Duration) method. The composite source aggregates multiple change events within this period into a single event and broadcasts that one instead and reloads the sources at that time, not necessarily in response to every single change in any source. The default is 100 milliseconds.

ScheduledExecutorService executor = Executors.newScheduledThreadPool(1);       

Config config = Config.create(
        ConfigSources.create(file("conf/dev.properties")                         
                                   .pollingStrategy(PollingStrategies::watch),
                           file("conf/config.properties")                      
                                   .pollingStrategy(PollingStrategies::watch))
                .changesExecutor(executor)                                     
                .changesMaxBuffer(4)                                           
                .changesDebounce(Duration.ofSeconds(1)));                      

• Prepares a thread pool executor.
• ConfigSources.create(Supplier<ConfigSource>…​) creates and returns a CompositeBuilder based on the two sources.
• Specifies a particular executor for monitoring and change event notification.
• Sets the subscriber’s buffer size to 4 events. The composite source discards any events not consumed by a subscriber if it needs to create room for more recent events.
• Change events will not fire more frequently than once per a second.

A loaded config tree subscribes to change events publishes by its source(s). By default, each Config uses an executor which manages a dedicated thread pool reusing previously-created threads when they are available and creating new threads as needed. All Config instances share the dedicated thread pool.

Your application can specify a non-default Executor for a tree to use for accepting and propagating those events by invoking the changesExecutor method on the Config.Builder. Each source subscriber has a dedicated buffer for holding changes events. This defaults to 256 but you can tailor this value as needed.

Executor executor = Executors.newCachedThreadPool();        

Config config = Config.create(
        file("conf/config.properties")
                .pollingStrategy(PollingStrategies::watch))
        .changesExecutor(executor)                          
        .changesMaxBuffer(16)                               
        .build();

• Prepares a specific thread pool executor.
• Specifies the executor the Config tree will use to listen for and propagate change events.
• Sets the event subscriber buffer to 16 events.

You can control which executor a retry policy should use for its work. The RetryPolicies.repeat(int retries) method returns a RetryPolicies.Builder. Your application can invoke the retry policy builder’s executor method to specify which ScheduledExecutorService instance it should use to schedule and execute delayed retries. By default the config system uses a separate thread pool executor for each retry policy instance.

ScheduledExecutorService executor = Executors.newScheduledThreadPool(2, myThreadFactory); 

Config config = Config.create(
        ConfigSources.file("conf/dev.properties")
                .optional()                                                               
                .retryPolicy(RetryPolicies.repeat(2)                                      
                        .executor(executor)));                                            

• Prepares a thread pool executor with core pool size set to 2 and a custom java.util.concurrent.ThreadFactory.
• When the source is flagged as optional(), the loading attempt will be repeated as the retry policy defines, but an overall failure will not lead to failing the initial load or preventing the source from being polled if so configured.
• Uses the built-in repeating implementation of RetryPolicy that can be used with any config source, but typically for ones that might suffer brief, intermittent outages.
• Specifies the executor to use for loading and retries.