This repository has been archived by the owner on Oct 4, 2023. It is now read-only.
Fix import of -e nested templates with preexisting children #573
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nhammond
added a commit
that referenced
this pull request
Feb 23, 2019
* roll back untested log cleanup feature * fix bug that masked errors in failed docker pull attempts by taskmonitor * delete cmd for runs, templates, files * run deletion handles datanodes, children * object deletion from cmdline client * cleanup runs with invalid user_inputs * file export handles metadata * file import handles metadata * track linked files in database * refactor FileManager into Export/ImportManager * unit tests for loomengine_utils.connection * wildcards for google storage, unit tests for file_utils * loomengine_utils unittests * template bulk export * template cleans up DataNodes on delete * migrate template import to utils * ensure DataNode children are sorted by index * import templates with dependencies * bulk-import templates * bulk-import runs * attach nested objects on run import * finish removing django-mptt * optimize deserialization * update template and run export * update loomengine_utils unit tests * push outputs before setting run status to finished * bugfix GOOGLE_STORAGE_BUCKET setting * fix docker-compose tests * lookup template_id for channel validation * fix bug preventing creation of multiple steps by id * rerun mechanism is working * "loom run restart" and "--force-rerun" * bundled inputs * ip_forward true on gcp instances * fix migration for m2m task to task-attempt * fix Task.is_responsive method * added .travis.yml * add build-loom-packages to .travis.yaml * remove .travis.yml * add mptt to server requirements to support old migrations * add set pipefail in bin/run-tests.sh * add Jenkinsfile * update Jenkinsfile * remove "source" from Jenkinsfile * fix loomengine_utils.test.test_file_utils.TestLocalFilePattern.testWildcard * build docker image * change Jenkinsfile from declarative to script * typo in Jenkinsfile * change Jenkinsfile to declarative syntax * missing quotes in Jenkinsfile * inspect env * set noTags False in Jenkinsfile * fix parentheses in Jenkinsfile * set version as GIT_COMMIT first 10 chars * revert Jenkinsfile to default checkout * set-version handles missing doc * remove versions from source control * add git push to Jenkinsfile * remove VERSION from Dockerfile * fix version file handling * enable docker build with no build-args * Jenkinsfile runs integration test stage on certain branches * Jenkinsfile fix tag in docker run * Jenkinsfile fix tag in docker push * fix regex in Jenkinsfile * Jenkinsfile fix regex * fix Jenkinsfile regex * fix regex in Jenkinsfile * fix Jenkinsfile regex * fix regex in Jenkinsfile * Jenkinsfile add buildingTag to integration test stage * Jenkinsfile conditional LOOM_VERSION either tag or commit * Jenkinsfile rewrite LOOM_VERSION logic * convert Jenkinsfile to scripted * debug Jenkinsfile * revert to declarative Jenkinsfile format * add deployment and integration tests * fix Jenkinsfile typo * fix Jenkinsfile typo * Jenkinsfile installs loom client locally * rename postBuild-->post in Jenkinsfile * Jenkinsfile fix path for set-version.sh * fix typos in Jenkinsfile * remove dependency on gsutil and gcloud from client * Jenkinsfile authenticate loom user * Jenkinsfile fix pw_generator * Jenkinsfile get Loom admin credentials from jenkins * Jenkinsfile set username and password on loom server start * add GOOGLE_APPLICATION_CREDENTIALS to Jenkinsfile environment * Jenkinsfile typo * Jenkinsfile increase timeout on integration tests * use ssh-keys, not sshKeys, to not override GCE project keys * fix typo in gcloud_start_server playbook * add refresh_inventory to get instance metadata * add missing ssh_private_key_path var in playbooks * revert changes that removed gcloud from ansible * remove "ansible-generated" note from ssh keys * Tag Docker image with the current branch * typo in jenkinsfile * Jenkinsfile wrap conditional steps in script block * Jenkinsfile remove redundant tag in docker build * fix invalid LOOM_SERVER_NAME in Jenkinsfile * Jenkinsfile configure email * Jenkinsfile add loom server cleanup * Jenkinsfile fix typo * reenable task_attempt cleanup * Jenkinsfile typos * Jenkinsfile typo * Jenkinsfile fix email formatting * Jenkinsfile add confirm-server-name to loom server delete * fix task_attempt cleanup, refactor async * fix unittests * Jenkinsfile release to PyPi if tag exists * Jenkinsfile add smoketest to pypi release * fix Jenkinsfile syntax * Jenkinsfile replace disallowed chars in LOOM_SERVER_NAME * Fix Jenkinsfile syntax * debug Jenkinsfile * debug Jenkinsfile * validate version during build * allow alphanumeric version for dev builds * typo in Jenkinsfile * fixed export of file array inputs in worker * add debug env output to Jenkinsfile * fix Jenkinsfile syntax * Jenkinsfile make LOOM_SERVER_NAME all lowercase * fix retries * cleanup tmp working directory after TaskAttempt * delete token on loom server disconnect * fix retries (#513) * change WORKING_DIR settings in task_monitor inputs/outputs * Make "loom template import missing.file" raise an error * raise ValidationError for duplicate step name * improve client message for google.cloud.exceptions.InternalServerError * improve message for "loom server start" --extra-settings with invalid format * fix import error * fix import errors * never retry on google.cloud.exceptions.Forbidden * check for duplicate user_inputs in run validation * CSRF_COOKIE_SECURE and SESSION_COOKIE_SECURE * require LOOM_HTTPS_PORT_ENABLED if LOOM_LOGIN_REQUIRED * eliminate worker scratch disks * fix retries for resolving EPEL repo * handle preexisting TaskAttempts on Run import * respect --force-rerun on retries * Task serializer does not require data_path * added cleanup to RunSerializer * cascade deletion to run/template children unless protected * LOOM_DISABLE_DELETE prevents object deletion * do not mount .ssh dir on server * add debug info * increase time for ssh keys to propagate to GCP instance * 5s timeout on status requests from client * fix timeout arg in unittests * Client errors with pretty message, no traceback * validate tags and labels as alphanumeric * fix import error * create bootdisk and instance in one step to avoid leaving disk on failure * Extend interactive test timeout * remove "disk_type" setting, not supported by ansible gce module * create settings bundle and copy it to server * show server configuration errors in terminal * Client prompts to login if 403 error * update unittests * introduce timeout to kill long-running tasks * no traceback on client init errors * include both TaskAttempt containers in cleanup * refactor client tests * fix cleanup and error msg for runs with missing inputs * update unit test * refactor integration test * lock down kombu version * change how template import/export handles steps * add playbooks to MANIFEST.in for client package * fix recursive-include syntax * handle duplicate TaskAttempts on run import * rename test settings * migrate prefetch methods to models * preserve relative file path on export/import * fix unittests * refactor serializers inheritance * fix get_data_nodes on task_attempts * FilterMixin can lookup multiple references at once * add filter options back to FileResource * partial work on RunSerializer * partial work on RunSerializer - connect i/o * partial work on Runserializer - import * partial work on RunSerializer - bulk_create DataNodes * partial work on RunSerializer - bulk_create DataObjects * refactor async tasks * completed RunSerializer refactor * update unit tests * remove "sleep" from delayed TaskAttempt retries * Allow delayed save when creating DataNodes * InputCalculator works per run, no more target channel * update requirements * require requests>=2.5.0 in loom_utils * add pycrypto to dependencies * display Loom version and template md5 in browser (#547) * Added documentation on output parser and glob strings (#548) * clarify error for duplicate file names in a task (#549) * "loom server stop" uses internal ip when needed (#550) * Prerelease 0.7.0 (#551) * update release notes for 0.7.0 * fix PyPi test, doc version * update jenkins configuration * correct typo in Jenkinsfile * client and worker pass pycodestyle checks (#553) * Issue 554 (#557) * remove comma from SQL CASE statements * set persistent_boot_disk=True, fixes ignored disk size setting * add elasticsearch and curator to loom server dependencies * fix sql syntax in case statements * Prerelease 0.7.1 (#558) * update releasenotes for 0.7.1, Jenkins runs full integration tests for tagged releases * correct jenkins stage name for integration tests * Add doc build test * allow integration tests to run with self-signed cert * bugfix matching DataNodes to TaskOutputs for bulk_create * Task.create_unsaved_task_from_input_set returns correct types if no new task * fix bug in output routing for parallel steps * Use full git hash as version bc setuptools normalization edits short hash * add workspace cleanup step (#565) * add size and index special functions to jinja dict in templates (#567) * remove debug stop * Fix import of -e nested templates with preexisting children (#573) * 574 duplicate task attempts (#575) * Kill child task_attempts before retry * remove redundant kill_children command * migrate Dockerfile to ubuntu 18.04 (#580) * use force_rerun settings in bulk_create_tasks (#581) * 0.7.2 release notes * --no-cache on docker build * move Docker loom install to /opt * moved playbook paths to /opt/loom * change install path in Jenkinsfile * lengthen retries for pulling from pypi * no run contents in list view due to performance, moved to detail * remove links to kibana logs * refactor portal's data cache to eliminate stale data * hide steps in run detail if none exist * show nested steps in template detail * migrate to python3 * changes for python3 migration * roll back python3 changes * raise warning, not error, if version not PEP440 * Restrict django<2 for python2 compatibility * Prerelease 0.7.2 (#584) * 0.7.2 release notes * --no-cache on docker build * move Docker loom install to /opt * moved playbook paths to /opt/loom * change install path in Jenkinsfile * lengthen retries for pulling from pypi * no run contents in list view due to performance, moved to detail * remove links to kibana logs * refactor portal's data cache to eliminate stale data * hide steps in run detail if none exist * show nested steps in template detail * migrate to python3 * changes for python3 migration * roll back python3 changes * raise warning, not error, if version not PEP440 * Restrict django<2 for python2 compatibility * restore links to logs in sidebar and run detail * update release notes
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #572