Beta test operational deployment

[jordanpadams]

Some possible discussion points / documentation updates:

• Per operate/reg-custom.html, When you load data, unknown (undefined) fields are ignored. - does Solr still throw some sort of warning message so the user knows there was an issue with the metadata.
• Use of /tmp - Most PDS node users will literally use exactly the path we say. /tmp is dangerous because it can get wiped out and often does not have enough disk space to handle large data ingestions. we should maybe add something from the outset that says "put all your data in some data directory you know is stable and has X storage capabilities", create that directory (do we want to create an environment variable or just let the nodes figure it out?), and then use it throughout the docs.
• Generating Solr fields from PDS4 data dictionaries and Updating Solr schema from PDS4 data dictionaries these look excellent. I think we may need to clarify the docs elsewhere where if you try to ingest data, and it fails because fields do not exist, it may mean you are trying to ingest some data where the registry schema has not been updated with the data dictionaries.
• From prod/zookeeper-vm.html for Install Java, let's just link to the Java Installation page we already have.
• Production Deployment - the details here are fantastic, and I think totally necessary. but we should think about how we can simplify this in some way. could we bootstrap this? or because of the differing OSes, maybe we just have 2 top-level procedures, 1 for single node deployment, another for multi-node, with a procedure for setting each of these up or just link to the docs you have that pretty much call out these examples? it would just be great if we could direct users to one place to do some baseline deployment, and then we can link off to all the config details.

[jordanpadams]

August Schedule Status: Delayed due to new task #37. No affect on I&T delivery.

[jordanpadams]

@tloubrieu-jpl Tasks:

• Rename pds-dev-el7 to pds-dev3
• Follow instructions for setting up cluster and installing ES

@tdddblog Tasks:

• Download all OREX and InSIght labels to /path/data/home/pds4

[tloubrieu-jpl]

@tdddblog has made progress on download. We'll need all the data.

[jordanpadams]

October Schedule Status: Slipped to start of Build 11.1 This does not impact build 11.0 as it can be done in-sync with I&T

[jordanpadams]

October Schedule Status ⏳: On track

[jordanpadams]

@tloubrieu-jpl where do we think we are at here? do we at least think we are at a point where we can distribute the registry to the DNs to begin testing?

[tloubrieu-jpl]

@jordanpadams I was able to close the 2 remaining tickets on this epics.

I tested the attribute search on the dev cluster with 1 millions records and the response came back in less that 2 seconds. This is not a full coverage tests but besides that I have been testing the new registry for the development of the API and everything worked fine. So I am confident we can distribute the registry for testing.

[jordanpadams]

thanks @tloubrieu-jpl

[ghost]

1