Repository navigation
Behavior changes
Config file and env var lists no longer swallow subcommands or REMAINDER args
ConfigArgParse hands config file and env var values to argparse by adding them to the command line. For an option with nargs='+' or nargs='*', files = [a, b] becomes --files a b, and before 1.10.0 it was added just before the subcommand. So prog run was parsed as prog --files a b run: argparse read run as a third element in the list, and the subcommand silently never ran (files=['a', 'b', 'run'], command=None).
Now:
- If an option comes before the subcommand, the list is added before that option, which ends the list:
prog -v rungivesfiles=['a', 'b']and treatsrunas a subcommand. - Otherwise, if no other top-level options are specified, parsing stops with an error saying to set the list on the command line instead, or to specify another option before the subcommand.
The same applies to a nargs=REMAINDER positional: with files = [a, b], prog cmd x used to give files=['a', 'b', 'cmd', 'x'] and now stops with the same error.
Known issue in 1.10.0: with a subcommand, the error also happens when another option is set from a config file but the list comes from an env var or from a later config file. 1.9.0 handled those cases.
For nargs options, list elements that start with - or with an args-file prefix such as @ are now an error
For an option with nargs (eg. nargs='+'), a config file or env var list is added to the command line as separate args after the option, so files = [a, --admin] becomes --files a --admin. Argparse then reads --admin as an option rather than a file name. In 1.9.0 that silently set admin=True, even from an env var when --admin has no env var of its own. In the same way, with fromfile_prefix_chars='@', files = [a, @extra_args.txt] read more args from extra_args.txt. (A flat list for an action='append' option is passed as --files=a --files=--admin instead, and isn't affected.)
Now a list element that starts with - (or another of the parser's prefix_chars) or with one of its fromfile_prefix_chars stops parsing with an error that names the key. A lone - and negative numbers such as -2 still work.
Command-line values now win over config file and env var values in more cases
A config file or env var value is meant to be skipped when the same option is on the command line. 1.9.0 only spotted the option when it was written out in full, so it missed other forms argparse accepts and applied both values:
- An abbreviated name:
prog --ta ewithtag = cin a config file gavetag=['c', 'e']for anappendoption. Nowtag=['e']. - Bundled short flags:
prog -vvwithverbose = 2gaveverbose=4for acountoption. Nowverbose=2. - A value attached to a short option:
prog -Ifoowithinclude = bargaveinclude=['bar', 'foo']for anappendoption. Nowinclude=['foo']. - An abbreviated name in a mutually exclusive group: with
--leveland--quietin the group,prog --lev 3withquiet = truefailed with "argument --level: not allowed with argument --quiet". Now the config file'squietis skipped, as it already was forprog --level 3.
When the command line has a subcommand or a --, this also affected options that keep a single value: prog --na y run with name = x in a config file gave name='x'; now it gives name='y'. Otherwise the command-line value came last and won anyway.
Options inside an args file now win too. With fromfile_prefix_chars='@', prog @args.txt, where args.txt holds --name and y on separate lines (argparse reads each line as one arg), gave the config file's name='x'; now it gives name='y'. A config file path given in an args file the same way (-c and my.conf on separate lines) is now opened; before, it was ignored.
Options that belong to a subcommand or to REMAINDER args no longer block config file and env var values
1.9.0 treated an option as already given when argparse actually hands it to a subcommand or to REMAINDER args. With name = x in a config file (or set by an env var), 1.9.0 dropped it (name=None) in each of these cases, and 1.10.0 gives name='x':
prog run --name y, where the subcommandrunhas its own--namethat stores its value under another name (eg.dest='run_name').prog cmd --name y, where everything fromcmdon goes to anargs=REMAINDERpositional.prog --exec cmd --name y, where--exectakesnargs=REMAINDER.
Likewise, a -h after -- or inside REMAINDER args (prog -- -h, prog cmd -h) made 1.9.0 skip config files entirely, as if help had been asked for. Config files are now read.
The config file option works the same way. 1.9.0 also opened a file named by a -c in a subcommand's args or in REMAINDER args, so prog --exec grep -c pattern failed with "Unable to open config file: pattern", and prog run -c run.conf (where run has its own -c) applied run.conf to the main parser too. Now only the main parser's own -c opens a file for it.
BooleanOptionalAction values from config files and env vars
For parser.add_argument('--feature', action=argparse.BooleanOptionalAction), a config file key can be written in its no- form, but 1.9.0 did the opposite of what it says: no-feature = true gave feature=True, and no-feature = false gave feature=False. Now no-feature = true gives feature=False, like --no-feature on the command line, and no-feature = false gives feature=True.
If the option also has a short alias, as in ("-f", "--feature"), a false value from a config file or env var used to set it to true (#367); it now sets it to false.
An option with only short names, such as -f and -F, has no no- form, so nothing on the command line can set it to false. When an env var asked for false anyway, 1.9.0 set it to true, or crashed with IndexError if the option had a single name. It now stops with an error such as "FEATURE can't set feature to False: none of -f/-F does".
parse_intermixed_args() now reads config files and env vars like parse_args() does
The same goes for parse_known_intermixed_args(). In 1.9.0, depending on the Python version, these either ignored config files and env vars or let their values override the command line.
exit_on_error=False now covers ConfigArgParse's own config file and env var errors
With exit_on_error=False, errors that ConfigArgParse itself finds in config files and env vars (eg. flag = maybe, or a line it can't parse) now raise argparse.ArgumentError instead of exiting the program. Errors that argparse reports, such as a wrong type, already did.
Other changes that can turn a previously accepted input into an error
- Writing out a config file (with
write_config_file()or an arg added withis_write_out_config_file_arg=True) now raisesValueErrorin these cases, where 1.9.0 wrote a file that read back differently:- In the default format, a string in square brackets, such as
[1, 2]or the regex[a-z], was written as is and read back as a list. - With
TomlConfigParser, some strings with a backslash, such asC:\temp\xdata(a backslash followed byx), were written so that the file couldn't be read back. Other backslashes, as inC:\Users\me, are written correctly in both versions. TomlConfigParser([])andIniConfigParser([], ...), which have no section to read from, wrote files that read back as nothing.
- In the default format, a string in square brackets, such as
- A number given for a
countoption in a config file or env var must now be digits only:verbose = -1used to be ignored and+2was accepted; both are now errors.trueandfalsework as before. - Calling
YAMLConfigFileParser().serialize()directly with an object such as aPathraises an error, instead of writing a!!python/objecttag its own loader can't read.
Other reading changes
- Default config format: a key with no
=and a comment after it, such asverbose # be loud, is now read asverbose = true. 1.9.0 read the comment as the value, so it rejected that line for a flag and setname #valueto#value.name = #valueis unchanged. - Wildcards in
default_config_filesare read in alphabetical order, so the alphabetically last matching file wins. Before, the order depended on the file system. IniConfigParser(['my_tool', 'DEFAULT'], ...): the named section now wins over[DEFAULT].
New
- Python 3.15 is now tested and listed as supported. On 3.15,
parser.print_help(file)no longer writes terminal color codes intofilewhen stdout is a terminal butfileis not (eg. a log file). nargs=1options accept a list from config files and env vars (size = [5]).configargparse.already_on_command_line(), the helper that checks whether an option is already on a command line, takes an optionalparser=argument. With it, the helper follows the rules above: it recognizes abbreviations, bundled short flags and attached values, and ignores args that go to a subcommand or to REMAINDER args. Calls without it work as before.
Fixes
Writing config files
- In the default config format, list values are written as JSON (eg.
files = ["a,b", "c"]), so they read back unchanged (#366); before,['a,b', 'c']read back as three items. ConfigArgParse 1.3 and later can read these files. - Options given in abbreviated or bundled form (
--gen hg38,-vv) are written; they used to be left out. - Values that a
type=function turned into other objects, such astype=json.loadsdicts andFileTypefiles, are written as the string they came from, so they read back. - Enum members, tuple defaults of multi-value options,
nargs=1values,append_constflags (each given once), andappend/countoptions with defaults now read back with the values they had when the file was written. - Defaults that can't be read back (eg.
default=sys.stdout, empty lists), defaults of mutually exclusive options, and the config file path option (when given on the command line, by an env var or as a default) are left out of written files. - YAML writes
PathandIntEnumvalues as plain values. The default format andIniConfigParserquote values that would otherwise read back differently (tests from #368), andIniConfigParserwithsplit_ml_text_to_list=Truewrites an empty list as[]instead of losing it. - An arg that writes the config file with
type=pathlib.Pathno longer crashes after writing.
Reading config files and env vars
- Options inherited through
parents=from an argparse parser created before ConfigArgParse was imported no longer crash parsing or help output. - Config file and env var values no longer cause an error or get lost when an option's value is a subcommand name (
prog --target build build), when--follows the subcommand (prog run -- a), or when anargs=REMAINDERpositional's args include an option (prog cmd --opt). - The config file path is found in a bundle of short flags (
-vc conf), no longer confused with an option whose name starts the same (--setnext to--settings), and, withallow_abbrev=False, no longer read from an abbreviation such as--sett. IniConfigParserstrips a value written on the line afterkey =.- Default format: line numbers in "Unexpected line N" errors start at 1.
- A YAML key read as a bool or number (
yes: 1), a stray%in anIniConfigParservalue,flag = [a, b]for astore_trueoption, andverbose = lotsfor acountoption now give a clear error instead of a traceback. - A TOML file that sets
tool = "x"when the parser reads sectiontool.my_tool, and abytespath indefault_config_files, no longer crash. An empty or comment-only YAML file is now read as having no settings instead of stopping with an error.
Help and print_values()
- Calling
format_help()more than once gives the same text; later calls used to drop the config file paragraph. - Env var names from
auto_env_var_prefixshow in the help before anything is parsed. CompositeConfigParser's help labelsConfigparserConfigFileParserinstead of showing??, andIniConfigParser's help says keys are lowercased.print_values()writes to the currentsys.stdout, shows@fileargs expanded, and no longer lists a positional's default when it was given.
Full changelog: v1.9.0...v1.10.0