Adding template support for Pinot Ingestion Job Spec

[xiangfu0]

Per #5330, we need to support Pinot Ingestion job take an ingestion job spec template file and passing thru values.

This will make user to schedule daily job simpler, without generating new yaml file everyday.

Templating is based on jinjava syntax.

• Adding jinjava template render support for Pinot.
• Update pinot ingestion launcher command to take a list of parameters(in format of 'k1=v1' 'k2=v2'...) which will be the context to set in the template.
• The template ingestion spec file will be rendered to final ingestion spec with the given context values.
• Adding support for default values today, yesterday into context for dev simplicity.

Usage example:
user can define inputDirURI as 'file:///path/to/input/{{year}}/{{month}}/{{day}}/{{hour}}'
during job launcher, the new command line is :

bin/pinot-admin.sh LaunchDataIngestionJob  \
-jobSpecFile ingestionJobSpec.yaml \
-values year=2020 month=05 day=06 hour=07

After that the real ingestion spec passed to ingestion job will have inputDirURI as 'file:///path/to/input/2020/05/06/07'

[kishoreg]

Nice

[codecov-io]

Codecov Report

Merging #5341 into master will increase coverage by 0.39%.
The diff coverage is 82.02%.

@@            Coverage Diff             @@
##           master    #5341      +/-   ##
==========================================
+ Coverage   66.08%   66.47%   +0.39%     
==========================================
  Files        1072     1076       +4     
  Lines       54668    54799     +131     
  Branches     8152     8170      +18     
==========================================
+ Hits        36125    36427     +302     
+ Misses      15895    15702     -193     
- Partials     2648     2670      +22     

Impacted Files	Coverage Δ
...ava/org/apache/pinot/common/utils/SchemaUtils.java	9.85% <0.00%> (ø)
...ger/realtime/Server2ControllerSegmentUploader.java	71.42% <ø> (ø)
...edicate/BaseDictionaryBasedPredicateEvaluator.java	54.16% <ø> (ø)
...predicate/BaseRawValueBasedPredicateEvaluator.java	87.87% <ø> (ø)
...e/operator/dociditerators/MVScanDocIdIterator.java	62.50% <14.28%> (-3.61%)	⬇️
...inot/spi/ingestion/batch/IngestionJobLauncher.java	8.51% <27.27%> (+8.51%)	⬆️
...rg/apache/pinot/broker/routing/RoutingManager.java	78.62% <50.00%> (-2.53%)	⬇️
...indexsegment/generator/SegmentGeneratorConfig.java	72.44% <50.00%> (+10.18%)	⬆️
...main/java/org/apache/pinot/spi/data/FieldSpec.java	86.25% <50.00%> (-1.25%)	⬇️
...e/operator/dociditerators/SVScanDocIdIterator.java	68.96% <60.00%> (-0.55%)	⬇️
... and 98 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 44b0a91...48033d1. Read the comment docs.