Task/rdmp 151 Custom Load Metadata Locations #1784
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
…sk/RDMP-151-archive-to-secondary-location
…sk/RDMP-151-archive-to-secondary-location
…nto task/RDMP-151-archive-to-secondary-location
JFriel
commented
Jun 7, 2024
JFriel
commented
Jun 7, 2024
JFriel
commented
Jun 7, 2024
JFriel
commented
Jun 7, 2024
rdteviotdale
approved these changes
Jun 7, 2024
…nto task/RDMP-151-archive-to-secondary-location
JFriel
added a commit
that referenced
this pull request
Jul 2, 2024
* add extraction additions * interim * add test * interim * working dedupe * improved checking * add timestamp option * fix extra looping * add check * start on tests * tidy up code * update link * tidy up * Rename executeFullExtractionToDatabaseMSSql.md to ExecuteFullExtractionToDatabaseMSSql.md * fix typo * add docs * update * update documentation * attempt fix docs * update docs * tidy up code * better tests * add real test * tidy up * interim * grab existiing entity * no new data * add basic tests * attempt to fix test * interim * interim commit * working clash * add test * fix test * improved clash checker * tidy up * update test * fix up test * update from codeql * tidy up code * fix bad merge * fix typo * skip over for now * revert change * Task/RDMP-180 Add instance settings table (#1820) * working settings interface * add documentation * add missing files * update namespace * add icon * update from run * make key unique * add tests * update tests * update for tests * fix unique name issue * tidy up * tidy up from review * works * nested deprications * recursive deprication * tidy up * add newline * Task/rdmp 174 dqe improvements (#1849) * working scallable graph * add changelog * add axis override * interim * working increments * working ui refresh * update changelog * tidy up code * add missing file * tidy up * Task/rdmp 155 migrate catalogue tables (#1805) * start of UI * interim * working switch * improved ui * fix build * rename duped file * imterim * add checks * start of tests * local tests working * add tests * improved ui * tidy up * add single item use * broken test * updated tests * tidy up imports * add some documentation * fix documentation issue * update test * update from codeql * codeql update * fix typo * fix build * fix bad merge * Task/rdmp 151 Custom Load Metadata Locations (#1784) * intermediate commit * split of load directory paths * working with ui * add migration * add tests * fix version * add docs * fix image * tidy up * update patch id * interim * working build * fix linux pathing * start up update patch * update patch * update tests * update patch * fix typo * update docs * codeql updates * use paths * upate tests * update codeql * update test * fix stub test * tidy up * use correct number * Bugfix/rdmp 193 is test type (#1860) * Fix initial INSERTs of logging table * Only MSSQL needs integer-as-bool hack, sane DBs can use TRUE/FALSE keywords --------- Co-authored-by: James A Sutherland <> * fix patch * fix metadata * fix typo * add bugfix for deleting depricated extended properties * Task/RDMP172 Add Ability to Remove GroupBy In Aggregate Builder (#1815) * start og groupby toggle working * ui toggles * add patch * fix versioning issue * add changelog * tidy up * attempt test fix * fix changelog * tidy up code * codeql * update patch number * ad dmissing file * update patch * updtae patch * add go * update patch * Task/rdmp 187 cohort versioning (#1847) * working settings interface * add documentation * add missing files * update namespace * add icon * update from run * make key unique * add tests * update tests * update for tests * fix unique name issue * tidy up * add db change * init * fix merge * working build * interim * initial save working * add correct versioning * working ui * interim * better clone * tidy up code * working filtered list * tidy up ui * remove old code * add basic test * tidy up * add tests * interim ui tidyup * tidy up * tidy up * add skip * fix tests * add control * open new window * tidy up code * tidy up * tidy up * fix build * list cohort versions * add docs * updates from uat * tidy up * tidy up * Task/RDMP-153 Improve RDMP Plugin Experience (#1786) * add keyboard shortcut * add .rdmp plugin handling * fix picker * working with RC * add todo note * tidy up * tidy up * add changelog * update build to pull .rdmp files * fix changelog * fix changelog * Task/rdmp 191 extract to db improvements (#1862) * WORKING TRIGGER * working update * fix build * better trigger * sort of working * fix clash * add null check * add todo note * add come notes * update changelog * attempt to fix tests * only add column once * tidy up code * move pk dropping * put pk in correct place * reorder * add check for trigger * better check for PK * start of index * working indexes * tidy up code * index tests * interim test * working triggers * allow nulls * fix patch * fix typo * tidy up * comment out test * re-add test * dispose of pipeline * tidy up * add more tests * update test * improved pk matching * fuix up tests * updates from review * tidy up tests * fix graph * update test plan * make loadmetasplit backwards compatable * improve lmd ui * closing catalogue * Update TestPlan.md * fix spelling * remove with values --------- Co-authored-by: Brian <76164974+bpeacock001@users.noreply.github.com> Co-authored-by: rdteviotdale <r.d.teviotdale@dundee.ac.uk>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Allows you to specify the location for each directory used by Data Loads.
The idea behind this is to allow DLS to store archives on a secondary disk due to storage concerns
To Test:
Create a data load
When setting the LocationOfFlatFiles, choose the advanced option and select some folders to use
Run the load and the folders should be populated with the data load files