Entity api schema template #208
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Added file entity-api-spec-TEMPLATE.yaml Added python file schema-template-transformer.py ~Derek Furst 6/21/2021
Added file entity-api-spec-TEMPLATE.yaml Added python file schema-template-transformer.py ~Derek Furst 6/21/2021 added pycharm to gitignore and updated requirements with nested lookup. Added template yaml file for general schema information. Added test package and a test python script. divided up most of schema template transformer into functions. ~Derek Furst 6/28/21
Added file entity-api-spec-TEMPLATE.yaml Added python file schema-template-transformer.py ~Derek Furst 6/21/2021 added pycharm to gitignore and updated requirements with nested lookup. Added template yaml file for general schema information. Added test package and a test python script. divided up most of schema template transformer into functions. ~Derek Furst 6/28/21
…l api schema template with all reused information. Placed all templates into template folder called yaml-templates ~Derek Furst 6/29/2021
…from the command line. Added line to accept argv.
…can now take in as a command line argument the path to another file. It will look for tags starting with "X-replace" and, depending on the tag, replace sections of the given file with sections of other files at given urls which are provided along with the x-replace tag. It acts recursively, and if it encounters a file that also has x-replace tags, it will perform those replacements as well. I added in my yaml-templates folder a "base-schema" template that includes some schemas I've used for testing. Its unlikely all of these schemas would be used, but it was useful for testing.
…general_schema_template_transformer.py and passes it that argument. At this time, the shell script outputs nothing, as the python script itself handles the output. May change to allow the shell script itself to handle the final output. Changed some formatting with the python script just to put comments above the functions instead of next to them.
…pec.yaml is treated as a temporary file. the shell script does 'cat' on this file and pipes stdout to a new file my-spec.yaml. Also, all file paths now calculate the current file path first rather than relying on relative pathing.
…by itself, it outputs to stdout. The build-schema.sh takes that output and pipes it into a new file my-schema.yaml I added a try block around my call to input from file. Rather than simply checking if argument length is greater than one, it will an exception either if no argument is passed or if the argument is to a file that doesn't exist.
…build-schema.sh rather than the python script general_schema_template_transformer.py. This way, we can avoid running the script at all if errors are found. If errors still are encountered at input and output, exception handling exists in the python script. I removed a ton of previously commented out lines to get this ready for final review, and changed variable names to be less redundant and more descriptive. I had an integer count loops through a recursive variable and that has now been replaced. Lastly, I removed a potential source of a future bug. Since adding a third return type for my recursive function, I had forgotten to change an earlier comparator that was either "equal" to a value or "not equal". Now that there's a third possible output, there was a case where something incorrect could be appended to a dictionary. This has been corrected.
…rectory reorganization.
…plate-files reorganized schema template work undir its own directory
…tity-api into entity-api-schema-template
yuanzhou
requested changes
Jul 21, 2021
Member
yuanzhou
left a comment
yuanzhou
left a comment
There was a problem hiding this comment.
@DerekFurstPitt I pulled all the recent new changes from test-release into your branch and appended that commit to your branch too. The unit test failed:
test.test_schema_template_transformer (nose2.loader.ModuleImportFailure) ... ERROR
======================================================================
ERROR: test.test_schema_template_transformer (nose2.loader.ModuleImportFailure)
----------------------------------------------------------------------
ImportError: Failed to import test module: test.test_schema_template_transformer
Traceback (most recent call last):
File "/home/zhy19/HuBMAP/entity-api/venv/lib64/python3.7/site-packages/nose2/plugins/loader/discovery.py", line 201, in _find_tests_in_file
module = util.module_from_name(module_name)
File "/home/zhy19/HuBMAP/entity-api/venv/lib64/python3.7/site-packages/nose2/util.py", line 77, in module_from_name
__import__(name)
File "/home/zhy19/HuBMAP/entity-api/test/test_schema_template_transformer.py", line 2, in <module>
import schema_template_transformer
ModuleNotFoundError: No module named 'schema_template_transformer'
----------------------------------------------------------------------
Ran 1 test in 0.000s
FAILED (errors=1)
Can you fix it?
Member
|
@DerekFurstPitt tweaks I made:
|
yuanzhou
approved these changes
Jul 22, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Added shell script and python script that will take as input a template yaml file (which is in a new directory). This script will look through the yaml file for certain tags. When a tag is reached, it will replace the tag with information at a given url or file path.