Function to turn single word reuse locations into the dictionary nota… …

…tion

Validate that keys 'as' and 'at' are present in dictionary notation

Actually resolve the shorthands when reading schemas

Unrelated: removed dead code

Precompute the full path of a nesting, considering nesting as another… …

… name

First pass at self-nesting

Missed a generated file

Code formatting

Changelog

Fix asciidoc link when nesting as another name

Change the wording of using a field set at the root of events (instea… …

…d of 'top level').

Remove commented code

WIP

Start massive refactor of schema_reader... …

Bunch of stuff moving to new script schema_processor.py

Make the 'reusable' field set properties explicit

Move a bunch of code to schema_processor.py

Give schemas/README.md some much needed attention

new schema loader WIP

About to turn the schema loader into a shotgun... …

Anything can be merged over other fields. Here's not the place to enforce
good ECS citizenry.

The schema loader now merges EVERYTHING, taking special care of arrays

Import the changes from PR elastic#821

Move nesting structure comment to top of file

Adjust tests for safe loading of schemas for new script name

Start future schema cleaner w/ basic visitor function to navigate the… …

… nested tree

Raise clear error on missing schema name

Add 'mock' testing dependency

Make 'title' a schema_detail as well

Keep improving schema/readme

Create a helper for list subtraction... …

My brain can't deal with Python list comprehensions

schema level cleanups looking pretty good

Extract the check for mandatory schema attributes to its own function

Raise on multiline short descriptions

Fill in field name for intermediary fields as well

Refine docs around setting description and short in the YAMLs

Start the field cleaning code, reuse the short description check

First checks for fields

Implement the simple defaults for fields

better multi-fields datatype support than before

Precalculate the 'path' to a field as we build the deeply nested stru… …

…cture

Implement flat/dashed names for all fields. Close to feature parity w… …

…ith schema_reader

Add intermediary=True to fields implicitly created to contain others. …

They shouldn't be held to the same standards of completeness (e.g. presence of a description) as the other fields.

Correct my engrish. intermediary => intermediate

Precalculate the leaf field name

In loader.py it's too early to determine namespacing under fs name

Comment out all the 'path' stuff. Must be calculated later

Tentative final state for cleaner.py, much simplified

Comments at the top

Move validation & normalizing of field reuse attributes to new script

Remove 'path' stuff from cleaner tests. Will happen in finalizer

Add another test for the already normalized self reuse

Why does this throw my brain for a loop

Both foreign reuse and self-reuse work now :-D

Move visitor to its own module

First reasonable stab at finalizing the fields.

New way to capture what's 'nested_here', make process.parent a reuse … …

…tests

Start capturing original_fieldset for reused fields as well, streamli… …

…ne tests

Clean the allowed values as well

Fix problem with original_fieldset and write test for calculate_final… …

…_values

Function docstrings in finalizer.py

Remove inaccurate comment. finalizer.py works on the structure in place

Reasonable implementation of filtering subsets

First pass at moving the flattening to the intermediate files generatoar

Calculate flat_name for multi_fields

Move helper func is_intermediate(field) to the helpers

WIP new generator using all the new schema scripts

WIP on intermediate_files.py generator for ECS YAML files

Initial stab at fixing an issue around chained nestings: original_fie… …

…ldset is incorrect

Remove old debugging code, replace with comment as a hint

Fix bug where explicit legit object fields were incorrectly flagged a… …

…s intermediate fields

Don't publish the deeply nested ecs.yml for now

Remove old versions of the code

Introduce new optional attribute about reuse, for order of performing… …

… reuses

Perform reuse in prescribed order. Temp copy by reference no longer n… …

…ecessary.

Rewrite intermediate files generator using visitor pattern

Comment out a bunch of things for initial PR review... …

These are improvements we'll want to have, but they'll come as a separate PR.

Fix issue introduced in recent changes around reuse order. Add test.

All generated files except asciidocs, as of latest changes

Revert back to passing normal nested fields to asciidoc generator.

introduce the new attribute 'reused_here' immediately, after all... …

It removes one reason why we had to pass the deeply nested field structure to asciidoc in the first place.

Append short description to 'reused_here'

Use the new 'reused_here' attribute to populate the field reuse section

Fix code listing allowed_values to use nested representation

Fix last remaining place that assumed deeply nested structure

asciidoc as of now. one major problem left

Fix subtle bug where self-nestings would incorrectly mark parent fiel… …

…dset as reused

Remove subset tests that have been replaced... …

Update new tests to capture one more edge case that was tested in the deleted test.

Use new scripts to load ECS fields in test_ecs_spec.py

Code formatting

Make test output verbose

Reset process.yml with the duplicated fields for initial review

Add nesting of code_sig to process.parent again, for initial review

Also re-nest hash at process.parent

Undo a wording improvement to simplify rewrite PR review

Temporarily re-add object_type for initial review

Merge branch 'master' into nest-as-rampage …

Trivial conflict in changelog.

Interesting conflict with elastic#851. The new scripts/schema/loader.py file
has most of the new code around loading ECS files from git refs. In this
merge commit I'm also slightly adjusting comments and wording of output
messages around this.

Remove old PR ID from relevant changelog entries. This will be a clea… …

…n new PR.

Code comment formatting

Update an outdated docstring

Add failing test to sanity check, to catch mangled tracing fields

Improve an unrelated test

Remove commented out code

Finish the comment for subset_filter.py

Add __init__.py in unit tests so they're 'discovered' properly 🤦🏻‍♂️

Fix load_yaml_file bug

Pin the version of mock