CDAP-4409 Script Context #5551
CDAP-4409 Script Context #5551
Conversation
|
|
||
| For example, in a validator transform, you can get the validator object and then call its functions as part of your script:: | ||
|
|
||
| var coreValidator = context.getValidator(\"coreValidator\"); |
There was a problem hiding this comment.
the backslash is necessary?
There was a problem hiding this comment.
yes and no...this is copied and is out-of-context. It's actually JavaScript embedded in JSON, so it does actually need the backslashes when it is in the JSON. But, it's mis-leading in this case. I will remove them, and point out that this is JavaScript.
* release/3.4: (126 commits) CDAP-5461 revert temporary parquet plugin renames changes some background-color and box-shadow properties CDAP-5579 rename NamespacedArtifactId to ArtifactId reduces top margin of dashboard view CDAP-5228 Added HBase compat module for CDH 5.7 adds left box shadow to lone breadcrumb changes alignment and layout of breadcrumbs set size and weigher for plugin service cache CDAP-5277 added caching of plugin instantiators CDAP-5458 Store WorkflowToken in separate row instead of storing it as a part of Workflow run record. add chunking error handler so UI does not crash update express code to handle pipe stream error disable download button when the explore query is not active anymore stream explore queries result directly to client CDAP-5267 Update how PartitionFilter gets serialized in the runtime arguments. Fixes using PartitionFilter.ALWAYS_MATCH as the partition filter. CDAP-4644 Implementation to add MapReduce and Spark logs in the Workflow. Enable LogSaverTest (CDAP-4408) Remove LocalLocationFactory and LocalLocation (CDAP-5092) Support Scala application (CDAP-5378) Added getWorkflowInfo to Spark contexts ...
| lines.append(MISSING_FILE_TEMPLATE % input_file) | ||
| return_code = 2 | ||
|
|
||
| output = open(output_file,'w') |
There was a problem hiding this comment.
line 141 is redundant. Not sure if opening the same file for writing would cause any weird issues.
There was a problem hiding this comment.
Fixed. Good catch.
* release/3.4: Fix for changed file: a line was removed, so reduce line counts by one, and update the MD5 hash. Fixes per comments. Edited Security (Beta) command descriptions. Fixes per comments. Correct error code handling. Add CLI doc build to "Quick Build" (either "web-only" or "github-only"). This makes these build longer, but it means that these builds fail only if the CLI docs can't be included. Otherwise, it would be complicated to skip the error for the Quick Builds but make the error fatal for the full builds. Fix handling of errors. Update the CLI commands, update the build process to automate and build the correct rst automatically. Update CLI Docs. Removed extra file opening in Python script cdap-docs/tools/docs-extract-table.py
| pieces1 = [piece.strip() for piece in line1.split(PIPE)][1:-1] | ||
| pieces2 = [piece.strip() for piece in line2.split(PIPE)][1:-1] | ||
|
|
||
| if len(pieces1) != len(pieces2): |
There was a problem hiding this comment.
Added an exception.
|
LGTM |
| with open(output_file,'w') as output: | ||
| output.write(TABLE_HEADER) | ||
| for line in lines: | ||
| output.write(line+'\n') |
There was a problem hiding this comment.
you want to close this output stream after writing.
There was a problem hiding this comment.
the with open(...
syntax in python automatically closes the file after the indentation is over.
|
lgtm |
|
would be good to squash before merging. this PR had 8 commits.. |
|
JJ had already merged release/3.4 into this branch, which makes it difficult to squash. |
|
if he hadn't had any conflicts when merging release/3.4 into his branch, then while rebasing we can move his commits to the top and squash. that shouldn't be too bad. anyway, just a suggestion for future when merging. |
* release/3.4: (90 commits) Adds docs for plugin functions in custom widget json section [CDAP-4695] Fix stream delete in Windows SDK: close stream writer before renaming directory add dataset properties for externalDataset CDAP-5728 Removing hydrator-ext prefix from external datasets name CDAP-5664 Add valid dataset name check so that plugins can use it to verify if their referenceName property is a valid dataset id CDAP-4409 Script Context (#5551) FileSet Datasets need to remove files they added upon tx rollback. fix pipeline name validation to allow hyphen fix instances and schedule cloning adds vendor prefixes for calc CDAP-5656 bugfix avoid double counting by including workflow name, run-id and node in MR, Spark context, also corresponding changes in metrics backend Make org.spark-project and fastxml.scala classes load with the SparkRunnerClassLoader to make sure there is no class leakage even in unit-test Removes un-necessary code from output-schema widget for hydrator++ plugins fix USER scope in postactions removes hydrator feature routes.js Changing IntegrationTestBase to use ApplicationId instead of Id.Application Removes router.js file from old hydrator feature updates hydratorplusplus routes; conslidates styling fix export bug on transform connections fix clone issue for spark engine ... Conflicts: cdap-docs/cdap-apps/build.sh picked remote.
Fix for https://issues.cask.co/browse/CDAP-4409 and https://issues.cask.co/browse/CDAP-4194
Passes Quick Build 3
Updated to match current changes in the Hydrator repo.
Builds the Validator function table from the Markdown file downloaded from the Hydrator repo.
Pages of interest: