chore: Audit and normalize CLI options for providing configuration files

[spencergilbert]

The variety of Vector's commands and sub-commands don't share a common experience for providing configuration files to the invoked command. Non exhaustive examples:

vector:

OPTIONS:
    -c, --config <config>
            Read configuration from one or more files. Wildcard paths are supported. File format is
            detected from the file name. If zero files are specified the default config path
            `/etc/vector/vector.toml` will be targeted

            [env: VECTOR_CONFIG=]

    -C, --config-dir <config-dir>
            Read configuration from files in one or more directories. File format is detected from
            the file name.

            Files not ending in .toml, .json, .yaml, or .yml will be ignored.

            [env: VECTOR_CONFIG_DIR=]

        --config-toml <config-toml>
            Read configuration from one or more files. Wildcard paths are supported. TOML file
            format is expected

            [env: VECTOR_CONFIG_TOML=]

        --config-json <config-json>
            Read configuration from one or more files. Wildcard paths are supported. JSON file
            format is expected

            [env: VECTOR_CONFIG_JSON=]

        --config-yaml <config-yaml>
            Read configuration from one or more files. Wildcard paths are supported. YAML file
            format is expected

            [env: VECTOR_CONFIG_YAML=]

vector validate

OPTIONS:
        --config-toml <config-toml>
            Vector config files in TOML format to validate

            [env: VECTOR_CONFIG_TOML=]

        --config-json <config-json>
            Vector config files in JSON format to validate

            [env: VECTOR_CONFIG_JSON=]

        --config-yaml <config-yaml>
            Vector config files in YAML format to validate

            [env: VECTOR_CONFIG_YAML=]

    -C, --config-dir <config-dir>
            Read configuration from files in one or more directories. File format is detected from
            the file name.

            Files not ending in .toml, .json, .yaml, or .yml will be ignored.

            [env: VECTOR_CONFIG_DIR=]

vector config

# No options, does accept ENV VAR based options?

vector graph

OPTIONS:
    -c, --config <config>
            Read configuration from one or more files. Wildcard paths are supported. File format is
            detected from the file name. If zero files are specified the default config path
            `/etc/vector/vector.toml` will be targeted

            [env: VECTOR_CONFIG=]

        --config-toml <config-toml>
            Vector config files in TOML format

        --config-json <config-json>
            Vector config files in JSON format

        --config-yaml <config-yaml>
            Vector config files in YAML format

    -C, --config-dir <config-dir>
            Read configuration from files in one or more directories. File format is detected from
            the file name.

            Files not ending in .toml, .json, .yaml, or .yml will be ignored.

            [env: VECTOR_CONFIG_DIR=]

Proposal:
Move config options into a separate module such that all commands and sub-commands can share a unified set of options when configurations are needed.

[jszwedko]

Also vector test which takes the configuration as an argument rather than a flag.

[spencergilbert]

Oh I missed that, just looking at the options - it takes them as both ARGS and OPTIONS

vector test

ARGS:
    <PATHS>...
            Any number of Vector config files to test. If none are specified the default config path
            `/etc/vector/vector.toml` will be targeted

OPTIONS:
        --config-toml <config-toml>
            Vector config files in TOML format to test

        --config-json <config-json>
            Vector config files in JSON format to test

        --config-yaml <config-yaml>
            Vector config files in YAML format to test

    -C, --config-dir <config-dir>
            Read configuration from files in one or more directories. File format is detected from
            the file name.

            Files not ending in .toml, .json, .yaml, or .yml will be ignored.

            [env: VECTOR_CONFIG_DIR=]

[jszwedko]

Just noting that most subcommands that have the same --config flags defined don't actually read the global --config flags (though they could). I believe those global flags primarily exist to run vector with no subcommand. I'm wondering if we may want to think about moving the default command to vector run as not all subcommands take --config flags (like vector generate, vector top, vector tap) and so it's a bit weird that they are global.

[jszwedko]

See #14688 as well.