Support DataONE Member Node Configuration when running in k8s

[artntek]

Part of Epic #1623
related to #1665

Requested/identified by Val@ESSDIVE
Admin UI: DataONE Configuration - allow submitting settings to CN without saving locally, so functionality is not restricted by read-only properties in k8s

[artntek]

Notes

See: https://knb.ecoinformatics.org/knb/docs/dataone.html#configuring-metacat-as-a-member-node

Jing:
Metacat will compile a node document (xml) based on the form which the operator fills out and sends the node document to cn.
So the best way is to look at the code. I believe it is in MNodeService class and the method name is getCapacities.
If we don’t modify those properties locally. I am wondering if it creates inconsistence between node documents in cn and in member node itself.

[artntek]

ToDo

1. allow updating the settings via values.yaml (already works)
2. Don't rely on operator hitting "Register" in the UI; instead:
   1. on startup, check if metacat is running in a container.
   2. If yes, then run a validation on the properties values
   3. if that passes, automatically do the registration call to the CN
3. Figure out what to do about messaging, if validation fails

Don't change the code for the "DataONE Configuration" admin page - leave it as is, so it cannot be submitted in k8s. Later, improve the UX

[artntek]

SUPERSEDED BY VERSION 2

Proposed logic for the registration and update processes, after discussions with @mbjones and @taojing2002:

Version 1

flowchart TD

SUP1((("`SUPERSEDED BY VERSION 2`")))

k8s(["Kuberneters Startup, or\nLegacy Admin Page Submit"])
--> START{"`**Previously 
        Registered as a 
        DataONE Member 
        Node (MN)?**`"}
    START -->|YES| IDCr
    START -->|NO| IDC

subgraph PREREG ["Existing Registration"]

SUP2((("`SUPERSEDED BY VERSION 2`")))
    IDCr{"`**nodeId changed?**
                    (*yaml:nodeId !=
                    DB nodeId_history latest?*)`"}
    IDCr -->|YES| AUTOr{"`*K8s ONLY:*
                    Registration 
                    Change Requested?
                    (*yaml flag == 
                    today's date?*)`"}
        AUTOr -->|YES| CERTr{"`New
                    NodeId 
                    Matches Client 
                    Cert?`"}
            CERTr -->|YES| UPDCNr["`UPDATE @ CN
                                    *Push MNconfig to CN.
                                    (nodeId CHANGED)*`"]
                UPDCNr --> SUCCESSr{"`Update
                                    successful?`"}
                SUCCESSr -->|YES| UPDBr["`UPDATE DB
                                        *(authoritativeMNId
                                        & nodeId history)*`"]
                SUCCESSr -->|NO| ERRORr
            CERTr -->|NO| ERRORr["`LOG ERROR`"]
        AUTOr -->|NO| ERRORr 
    IDCr -->|NO\nbut other data\nmay have changed| UPCNr["`UPDATE @ CN
                        *(Push MN config to CN.
                        (nodeId UNCHANGED)*`"]
    ERRORr & UPCNr & UPDBr --> DONEr(["Done"])
end

subgraph UNREG ["NOT Previously Registered"]
SUP3((("`SUPERSEDED BY VERSION 2`")))
    IDC{"`**nodeId changed?**
        (*yaml:nodeId !=
        DB nodeId_history latest?*)`"}
    IDC -->|YES| AUTO{"`*K8s ONLY:*
                    Registration 
                    Change Requested?
                    (*yaml flag == 
                    today's date?*)`"}
    IDC -->|NO| DONE
    AUTO -->|YES| CERT{"`New
                    NodeId 
                    Matches Client 
                    Cert?`"}
        CERT -->|YES| REGCN["`REGISTER @ CN`"]
            REGCN --> SUCCESS{"`Register
                                successful?`"}
        CERT -->|NO| ERROR["`LOG ERROR`"]
        SUCCESS -->|YES| UPDB
        SUCCESS -->|NO| ERROR["`LOG ERROR`"]
    AUTO -->|NO:\n\nlocal change\nonly| UPDB["`UPDATE DB
                        *(authoritativeMNId
                        & nodeId history)*`"]
    ERROR & UPDB--> DONE(["Done"])
end

Loading

[artntek]

Proposed logic for the registration and update processes (Version 2):

Change Log:

1. Both flows

   • Update successful? decision node merged with upstream update @ CN node to simplify

2. "Existing Registration" flow

   • Added FAIL outcome to update @ CN [..] (nodeId UNCHANGED) node (results in LOG ERROR)
   • No other changes

3. "NOT Previously Registered" case

   • Eliminate the nodeId changed? node. The user may now change the nodeId and set up the client cert locally, without requesting a registration. They can then subsequently submit a registration at a later time, by updating only the .Values.metacat.dataone.autoRegisterMemberNode yaml flag. (In Version 1 user had to change the nodeId at the same time as setting the autoRegisterMemberNode yaml flag to request registration.)

flowchart TD

k8s(["Kuberneters Startup, or\nLegacy Admin Page Submit"])
--> START{"`**Previously 
        Registered as a 
        DataONE Member 
        Node (MN)?**`"}
    START -->|YES| IDCr
    START -->|NO| AUTO

subgraph PREREG ["Existing Registration"]
    IDCr{"`**nodeId changed?**
                    (*yaml:nodeId !=
                    DB nodeId_history latest?*)`"}
    IDCr -->|YES| AUTOr{"`*K8s ONLY:*
                    Registration 
                    Change Requested?
                    (*yaml flag == 
                    today's date?*)`"}
        AUTOr -->|YES| CERTr{"`New
                    NodeId 
                    Matches Client 
                    Cert?`"}
            CERTr -->|YES| UPDCNr["`UPDATE @ CN
                                    *Push MNconfig to CN.
                                    (nodeId CHANGED)*`"]
                UPDCNr -->|SUCCESS| UPDBr["`UPDATE DB
                                        *(authoritativeMNId
                                        & nodeId history)*`"]
                UPDCNr -->|FAIL| ERRORr
            CERTr -->|NO| ERRORr["`LOG ERROR`"]
        AUTOr -->|NO| ERRORr 
    IDCr -->|NO\nbut other data\nmay have changed| UPCNr["`UPDATE @ CN
                        *(Push MN config to CN.
                        (nodeId UNCHANGED)*`"]
                UPCNr -->|SUCCESS| DONEr
                UPCNr -->|FAIL| ERRORr
    ERRORr & UPDBr --> DONEr(["Done"])
end

subgraph UNREG ["NOT Previously Registered"]
    AUTO{"`*K8s ONLY:*
            Registration 
            Change Requested?
            (*yaml flag == 
            today's date?*)`"}
    AUTO -->|YES| CERT{"`NodeId 
                    Matches Client 
                    Cert?`"}
        CERT -->|YES| REGCN["`REGISTER 
                               @ CN`"]
        CERT -->|NO| ERROR["`LOG ERROR`"]
        REGCN -->|SUCCESS| IDC{"`nodeId changed?
                        (yaml:nodeId !=
                        DB nodeId_history
                        latest?)`"}
            IDC -->|YES| UPDB
            IDC -->|NO| DONE
        REGCN -->|FAIL| ERROR["`LOG ERROR`"]
    AUTO -->|NO:\n\nlocal change\nonly| UPDB["`UPDATE DB
                        *(authoritativeMNId
                        & nodeId history)*`"]
    ERROR & UPDB--> DONE(["Done"])
end

Loading

[artntek]

resolved by PR #1682