java – Using picoCLI to recursively parse configuration files?

Here’s my solution; maybe it’s useful to someone. The basic approach is to use a new CommandLine instance for each nested config file.

All code snippets below are from the annotated class; project-specific housekeeping, error checking, path construction etc. were removed.

Problem #1 is solved by the parseConfigFile(...) method:

@Option(names = { "configfile", "cf" } )
public void parseConfigfile(final List<String> accumulatedCfgFiles) {
    // prune the list, keeping only "new" entries that haven't been seen yet:
    List<String> newCfgFiles = this.cfgfileHelper.retainNewOptionValuesOnly(
        accumulatedCfgFiles, "configfile");
    if(newCfgFiles.isEmpty()) {
        // another picoCLI quirk: this happens even if there always are values 
    } else if(newCfgFiles.size() > 1) {
        // loop over the files if you want to allow this, or report an error
    // some path tinkering left out
    File cfgFile = new File(newCfgFiles.get(0));
    if(this.stackOfConfigFiles.contains(cfgFile)) {
        // report error because of cyclic reference
    } else {
        // task a new CommandLine instance with processing that file:
        CommandLine cmd = new CommandLine(this);
        String[] optionsFromFile = FileUtils.readLines(cfgFile); // Apache Commons               
        this.cfgfileHelper.wrapParseArgs(cmd, optionsFromFile);

The method uses an instance of NestedCfgfileHelper (see source below) which does all the nested-config-specific housekeeping to solve problem #2. One instance of that helper class per annotated class keeps track of the CommandLine instances that are currently working on your annotated class. It’s Constructor needs the names of all options that the helper class should take care of:

public final NestedCfgfileHelper cfgfileHelper = 
    new NestedCfgfileHelper(new String[] { "configfile", "section" });

The following steps make all this work:

  • identify those options that are sensitive to “spurious setter method calls” (most are not);
  • if there are any, paste NestedCfgfileHelper‘s source to your annotated class as an inner class;
  • create an instance of NestedCfgfileHelper as a public member of your annotated class, telling the constructor the names of all those “problematic” options;
  • never call yourInstanceOfCommandLine.parseArgs(...) directly, but pass it to the helper instead using instanceOfYourAnnotatedClass.cfgfileHelper.wrapParseArgs(...)
  • let setter method(s) for those “difficult” options…
    • … first get rid of “old” values ​​from previous invocations by calling retainNewOptionValuesOnly
    • … then process the remaining option value(s) normally

Finally, here’s the source of NestedCfgfileHelper:

/** NestedCfgfileHelper ensures that the values of certain options are 
 * processed just once, despite the picoCLI quirk.     */
public final class NestedCfgfileHelper {
    /** Maps an (option name|CommandLine instance) pair to the number of 
     * option values that instance has so far passed for that option. 
     * Because Java doesn't have Maps with two keys, it's implemented as 
     * a Map of Maps:         */
    private Map<String, Map<CommandLine, Integer>> mapOptionAndCLToCount =
            new HashMap<>();

    /** Constructs a helper instance and prepares it to handle the options 
     * given as parameters. 
     * @param optionNames any number of Strings, with each String denoting
     * one option whose values should be protected against being processed
     * multiple times */
    public NestedCfgfileHelper(String... optionNames) {
        // make one mapping for each option name given:
        for(String optionName: optionNames) {
            mapOptionAndCLToCount.put(optionName, new HashMap<CommandLine, Integer>());
    /** This stack keeps track of CommandLine instances that are currently 
     * working on this TFConfig instance. A stack is needed because config files
     * can be nested. Entries in the CommandLine stack correspond to entries in
     * the config file stack (maintained by the parseConfigfile() method),
     * except the lowest (first) one that gets fed the String[] coming directly
     * from the main(String[] args) call.     */
   private Stack<CommandLine> stackOfCmdLineInstances = new Stack<>();

    /** Wraps the call to {@link CommandLine#parseArgs(String...)} with some
     * housekeeping so that when an annotated setter method is being called
     * during option parsing, the helper method can look up from which 
     * CommandLine instance the call is coming.
     * Because parseArg invocations will be nested recursively for nested config
     * files, the respective CommandLine instances are kept on a stack.
     * @param cl CommandLine instance that's been about to start parsing
     * @param args options that are to be parsed     */
    public void wrapParseArgsCall(final CommandLine cl, final String[] args) {
        // the brand new CommandLine instance hasn't passed any values yet,
        // so put 0 in all maps: 
            (String s, Map<CommandLine, Integer> m) -> m.put(cl, 0));

    /** This method filters its list parameter, discarding the first n 
     * entries (assuming they've already been processed), where n is retrieved
     * from a Map instance kept for each option name. This method is intended
     * to be called by annotated setter methods.
     * @param accumulated List containing all values (old and new ones 
     * accumulated) of the option named in the other parameter.
     * @param optionName describes the option that's being parsed.
     * @return pruned list containing only the "new" values that haven't
     * been seen before.     */
    private List<String> retainNewOptionValuesOnly(
        final List<String> accumulated, 
        final String optionName) {

        // get the CommandLine instance currently working on this TFConfig instance:
        CommandLine currentCL = this.stackOfCmdLineInstances.peek();

        // get the CommandLine->int map for the option name passed:
        Map<CommandLine, Integer> map = mapOptionAndCLToCount.get(optionName);
        if(map == null) {
            throw new IllegalArgumentException("unknown option: " + optionName);
        /* Find out how many option values it has already passed to the setter.
         * For each CL, that number is stored in the map passed to this method.*/
        int n = map.get(currentCL);
        /* discard the first n entries (they have already been processed) of
         * accumulated, keeping only the "new" ones: */ 
        List<String> optionValuesNewThisTime = 
            accumulated.subList(n, accumulated.size());
        // associate the new number of patterns with the current CommandLine:
        int newNumber = n + optionValuesNewThisTime.size();
        map.put(currentCL, newNumber);
        return optionValuesNewThisTime;

Leave a Comment