Invalidating artifact/metadata store if there is a change in one of them

[bcdurak]

Describe changes

I implemented a mechanism that puts a stop to any operation if the corresponding artifact store/metadata store pair is not properly associated.

It works as follows:

• The ZenStore now features an association table.
• Once you register a stack, the uuids of the artifact- and metadata stores are saved in pairs in this table.
• Stacks get validated through a validate method when you run a pipeline, register/update stacks, etc. This method now also checks whether the stack in question has the correct entry in this table for its artifact- metadata store pair.
• It is possible to bypass this by using a flag -r when creating/updating stack over the CLI, which will reset the previous associations of these components and establish a new one.

TODO:

• There are some failing tests. I am working on it at the moment.

Pre-requisites

Please ensure you have done the following:

• I have read the CONTRIBUTING.md document.
• If my change requires a change to docs, I have updated the documentation accordingly.
• If I have added an integration, I have updated the integrations table.
• I have added tests to cover my changes.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Other (add details above)

[github-actions]

@htahir1, @schustmi, @safoinme
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

[github-actions]

@htahir1, @schustmi, @safoinme
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

[github-actions]

@htahir1, @schustmi, @safoinme
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

[github-actions]

@htahir1, @schustmi, @safoinme
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

[github-actions]

@htahir1, @schustmi, @safoinme
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

[github-actions]

@htahir1, @schustmi, @safoinme
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

[github-actions]

@htahir1, @schustmi, @safoinme
1.25 (one and twenty-five hundredths) business days have passed since the review started. Pretty please review the PR.

[bcdurak]

@AlexejPenner @fa9r I have fixed the issue regarding the UUIDs and the SQL ZenStore, added new tests, and updated the docs accordingly.

[fa9r]

Great changes, love the new argument name and descriptions, very clear now!

[AlexejPenner]

LGTM - just out of curiosity, how does this interact with existing stacks? Is this a breaking change?

[schustmi]

Functionality-wise your implementation looks great, the only thing I'm a little concerned about is that the functionality of resetting the store associations doesn't really belong in the validation.

I'd see it more as a method on the BaseZenStore, e.g.

class BaseZenStore:
    def decouple_stores(artifact_store_uuid, metadata_store_uuid):
        ...
        
class Repository:
    def register_stack(stack, decouple):
        if decouple:
            self.zen_store.decouple_stores(...)
        self.zen_store.register_stack(...)

I know this is quite late in the review and I just randomly took a look, feel free to ignore it in this PR but I think it makes sense to split it up longterm.