Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Task/rdmp 191 extract to db improvements #1862

Merged
merged 43 commits into from
Jun 27, 2024
Merged

Task/rdmp 191 extract to db improvements #1862

merged 43 commits into from
Jun 27, 2024

Conversation

JFriel
Copy link
Collaborator

@JFriel JFriel commented Jun 21, 2024

Proposed Change

Adds to ability to use the RDMP trigger system to store history of overwritten export table records
Adds the ability to create indexes on export tables

Type of change

What types of changes does your code introduce? Tick all that apply.

  • Bugfix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation-Only Update
  • Other (if none of the other choices apply)

Checklist

By opening this PR, I confirm that I have:

  • Ensured that the PR branch is in sync with the target branch (i.e. it is automatically merge-able)
  • Created or updated any tests if relevant
  • Have validated this change against the Test Plan
  • Requested a review by one of the repository maintainers
  • Have written new documentation or updated existing documentation to detail any new or updated functionality and how to use it
  • Have added an entry into the changelog

JFriel added 30 commits May 30, 2024 13:12
@JFriel
WORKING TRIGGER
5b18302
@JFriel
working update
5e3728e
@JFriel
fix build
d4f7f19
@JFriel
better trigger
75172c6
@JFriel
sort of working
6bfc630
@JFriel
fix clash
d131e9a
@JFriel
add null check
e2fbc68
@JFriel
add todo note
8a94edf
@JFriel
Merge branch 'release/8.2.0' of https://github.com/HicServices/RDMP i…
555f08f 
…nto task/RDMP-191-extract-to-db-improvements
@JFriel
add come notes
368ba64
@JFriel
update changelog
bebdcf7
@JFriel
attempt to fix tests
a524b86
@JFriel
only add column once
db94da9
@JFriel
tidy up code
3d5e86b
@JFriel
move pk dropping
735cc9f
@JFriel
put pk in correct place
319c359
@JFriel
reorder
f71439f
@JFriel
add check for trigger
b28f03d
@JFriel
better check for PK
93c4bfb
@JFriel
start of index
c03b100
@JFriel
working indexes
e171631
@JFriel
tidy up code
18e5168
@JFriel
index tests
c5edaec
@JFriel
interim test
054fc28
@JFriel
working triggers
b19cb68
@JFriel
allow nulls
6e223b7
@JFriel
Merge branch 'release/8.2.0' of https://github.com/HicServices/RDMP i…
92c17db 
…nto task/RDMP-191-extract-to-db-improvements
@JFriel
Merge branch 'release/8.2.0' of https://github.com/HicServices/RDMP i…
d2e4997 
…nto task/RDMP-191-extract-to-db-improvements
@JFriel
fix patch
4f7b3b2
@JFriel
fix typo
bb9d3ea
@JFriel JFriel marked this pull request as ready for review June 27, 2024 09:02
JBaird00183
JBaird00183 approved these changes Jun 27, 2024
View reviewed changes
Copy link
Contributor

@JBaird00183 JBaird00183 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Demonstrated and happy that James is making some tweaks before releasing

@JFriel JFriel merged commit 2202774 into release/8.2.0 Jun 27, 2024
5 checks passed
@JFriel JFriel deleted the task/RDMP-191-extract-to-db-improvements branch June 27, 2024 09:07
JFriel added a commit that referenced this pull request Jul 2, 2024
@JFriel @bpeacock001 @rdteviotdale
Release/8.2.0 (#1867)
4ee56cc 
* add extraction additions

* interim

* add test

* interim

* working dedupe

* improved checking

* add timestamp option

* fix extra looping

* add check

* start on tests

* tidy up code

* update link

* tidy up

* Rename executeFullExtractionToDatabaseMSSql.md to ExecuteFullExtractionToDatabaseMSSql.md

* fix typo

* add docs

* update

* update documentation

* attempt fix docs

* update docs

* tidy up code

* better tests

* add real test

* tidy up

* interim

* grab existiing entity

* no new data

* add basic tests

* attempt to fix test

* interim

* interim commit

* working clash

* add test

* fix test

* improved clash checker

* tidy up

* update test

* fix up test

* update from codeql

* tidy up code

* fix bad merge

* fix typo

* skip over for now

* revert change

* Task/RDMP-180 Add instance settings table (#1820)

* working settings interface

* add documentation

* add missing files

* update namespace

* add icon

* update from run

* make key unique

* add tests

* update tests

* update for tests

* fix unique name issue

* tidy up

* tidy up from review

* works

* nested deprications

* recursive deprication

* tidy up

* add newline

* Task/rdmp 174 dqe improvements (#1849)

* working scallable graph

* add changelog

* add axis override

* interim

* working increments

* working ui refresh

* update changelog

* tidy up code

* add missing file

* tidy up

* Task/rdmp 155 migrate catalogue tables (#1805)

* start of UI

* interim

* working switch

* improved ui

* fix build

* rename duped file

* imterim

* add checks

* start of tests

* local tests  working

* add tests

* improved ui

* tidy up

* add single item use

* broken test

* updated tests

* tidy up imports

* add some documentation

* fix documentation issue

* update test

* update from codeql

* codeql update

* fix typo

* fix build

* fix bad merge

* Task/rdmp 151 Custom Load Metadata Locations (#1784)

* intermediate commit

* split of load directory paths

* working with ui

* add migration

* add tests

* fix version

* add docs

* fix image

* tidy up

* update patch id

* interim

* working build

* fix linux pathing

* start up update patch

* update patch

* update tests

* update patch

* fix typo

* update docs

* codeql updates

* use paths

* upate tests

* update codeql

* update test

* fix stub test

* tidy up

* use correct number

* Bugfix/rdmp 193 is test type (#1860)

* Fix initial INSERTs of logging table
* Only MSSQL needs integer-as-bool hack, sane DBs can use TRUE/FALSE keywords

---------

Co-authored-by: James A Sutherland <>

* fix patch

* fix metadata

* fix typo

* add bugfix for deleting depricated extended properties

* Task/RDMP172 Add Ability to Remove GroupBy In Aggregate Builder (#1815)

* start og groupby toggle working

* ui toggles

* add patch

* fix versioning issue

* add changelog

* tidy up

* attempt test fix

* fix changelog

* tidy up code

* codeql

* update patch number

* ad dmissing file

* update patch

* updtae patch

* add go

* update patch

* Task/rdmp 187 cohort versioning (#1847)

* working settings interface

* add documentation

* add missing files

* update namespace

* add icon

* update from run

* make key unique

* add tests

* update tests

* update for tests

* fix unique name issue

* tidy up

* add db change

* init

* fix merge

* working build

* interim

* initial save working

* add correct versioning

* working ui

* interim

* better clone

* tidy up code

* working filtered list

* tidy up ui

* remove old code

* add basic test

* tidy up

* add tests

* interim ui tidyup

* tidy up

* tidy up

* add skip

* fix tests

* add control

* open new window

* tidy up code

* tidy up

* tidy up

* fix build

* list cohort versions

* add docs

* updates from uat

* tidy up

* tidy up

* Task/RDMP-153 Improve RDMP Plugin Experience (#1786)

* add keyboard shortcut

* add .rdmp plugin handling

* fix picker

* working with RC

* add todo note

* tidy up

* tidy up

* add changelog

* update build to pull .rdmp files

* fix changelog

* fix changelog

* Task/rdmp 191 extract to db improvements (#1862)

* WORKING TRIGGER

* working update

* fix build

* better trigger

* sort of working

* fix clash

* add null check

* add todo note

* add come notes

* update changelog

* attempt to fix tests

* only add column once

* tidy up code

* move pk dropping

* put pk in correct place

* reorder

* add check for trigger

* better check for PK

* start of index

* working indexes

* tidy up code

* index tests

* interim test

* working triggers

* allow nulls

* fix patch

* fix typo

* tidy up

* comment out test

* re-add test

* dispose of pipeline

* tidy up

* add more tests

* update test

* improved pk matching

* fuix up tests

* updates from review

* tidy up tests

* fix graph

* update test plan

* make loadmetasplit backwards compatable

* improve lmd ui

* closing catalogue

* Update TestPlan.md

* fix spelling

* remove with values

---------

Co-authored-by: Brian <76164974+bpeacock001@users.noreply.github.com>
Co-authored-by: rdteviotdale <r.d.teviotdale@dundee.ac.uk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JBaird00183 JBaird00183 JBaird00183 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@JFriel @jas88 @JBaird00183