Skip to content
This repository has been archived by the owner on Jul 14, 2022. It is now read-only.

(PXP-7446): Feat/expression tools #45

Merged
merged 49 commits into from
Feb 23, 2021
Merged

(PXP-7446): Feat/expression tools #45

merged 49 commits into from
Feb 23, 2021

Conversation

cterrazas2
Copy link
Contributor

Jira Ticket: PXP-7446

New Features

  • Brought back ExpressionTools with EBS, this means ExpressionTools for any given workflow is now executed in its own task pod. Note, EBS was implemented in Mariner release 1.1.0

Deployment changes

  • ExpressionTools now dispatches as a K8s job outside of the MarinerEngine

cterrazas2 and others added 30 commits January 28, 2021 21:05
@cterrazas2 @m0nhawk
Testing expression work flow. Null inputs not working.
b71fa3d
@m0nhawk
debug: some extra debug logs
c70fb7b
@m0nhawk
debug: see the types
1970619
@m0nhawk
debug: test
3a639a2
@cterrazas2
get to expression tool js vm parsing and make tool dir
c226031
@cterrazas2
add FileMode to make tool working dir
fdbe516
@cterrazas2
change to MkdirAll for any necessary parent paths
439c2ee
@cterrazas2
some debugging for initworkdir
9fff9fc
@cterrazas2
js debugging
d788fa0
@cterrazas2
js debugging
0bb53b8
@cterrazas2
output and tool debugging
9c2579b
@cterrazas2
Add inputs to js vm
273454a
@cterrazas2
input and output debugging
d3ed910
@cterrazas2
log s3 glob
926b49a
@cterrazas2
log s3 glob
87d0061
@cterrazas2
log s3 glob
81eaa73
@cterrazas2
just log s3 path
d1537be
@cterrazas2
add error handling to laodInput and s3 bucket logs
289b28b
@cterrazas2
output pattern log for s3
98d17de
@cterrazas2
match any sequence after s3Pattern to key
55540ea
@cterrazas2
modify default check for inputs for tool
add0534
@cterrazas2
fix go error
3d67c97
@cterrazas2
move s3Pattern declaration out of loop
cb1a69d
@cterrazas2
debugging s3 input paths
ce5bfab
@cterrazas2
strip s3 path
72ca71c
@cterrazas2
fix s3 path and create tool dir if not specified in cwl
0fe9964
@cterrazas2
fix s3 strip path
d77d8e7
@cterrazas2
test local path for input
f4a1432
@cterrazas2
troubleshoot sidecar for s3 fetches
336a8a3
@cterrazas2
test s3 downloader path
169dea4
@cterrazas2 cterrazas2 merged commit 1cb8dcf into master Feb 23, 2021
@cterrazas2 cterrazas2 deleted the feat/expression-tools branch February 23, 2021 19:14
@PlanXCyborg PlanXCyborg mentioned this pull request Jul 29, 2021
Open
@PlanXCyborg PlanXCyborg mentioned this pull request Aug 16, 2021
Closed
@PlanXCyborg PlanXCyborg mentioned this pull request Oct 19, 2021
Merged
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@m0nhawk m0nhawk m0nhawk approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@cterrazas2 @m0nhawk