Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pds registry app 263 docs docker compose tl #42

Conversation

tloubrieu-jpl
Copy link
Member

🗒️ Summary

make the documentation a bit more user oriented by shifting the focus away from docker-compose when it is not useful to the user.

If you are good with that and if you don't catch typo and errors we can merge in your branch and then your branc in main.

**************

Once the above setup is complete, the Registry Tools are used to process and ingest product
metadata into the Registry. Details on how to run these tools is found in the :doc:`User
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor typo: "Details on how to run these tools is" should be "Details on how to run these tools are".

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


The high level architecture of the PDS Registry Service is shown below:

.. image:: _static/images/registry_service.png
Copy link
Contributor

@ramesh-maddegoda ramesh-maddegoda Apr 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This image (_static/images/registry_service.png) is not visible/available.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I missed to add it to git. Done now.

@@ -2,6 +2,10 @@
Admin Tasks
===========

Admin tasks describes processes for the intialization/deletion of the registry and reference dictionnary updates.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor typo: "Admin tasks describes" should be "Admin tasks describe"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@@ -2,6 +2,10 @@
Admin Tasks
===========

Admin tasks describes processes for the intialization/deletion of the registry and reference dictionnary updates.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minort typo: dictionnary

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

On-Prem vs. Cloud
*****************

In a typical on-premise deployment, the PDS Registry Application is composed, server-side of
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel it is better to reword this (from line 8 to 11).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to simplify the sentence, these are not my words so I am not necessarily comfortable in doing that but I agree this sentence is a bit complicated, although it sounds like good english.

Systems Manager Parameter Store.

Amazon's content delivery network, Cloudfront, is placed in front of
the Elastic Load Balancer (ELB). Cloudfront provides high-speed, low-latency
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel it is better to reword "Cloudfront provides high-speed, low-latency access while also ensuring unintention or malicious bulk downloading of data is avoided".

Suggestion (may be not the best suggestion) : "Cloudfront provides high-speed, low-latency access, while also ensuring the avoidance of unintentional or malicious bulk downloading of data".

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about "Cloudfront provides high-speed, low-latency
access while also preventing from unintentional or malicious bulk download
of data" ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your suggestion is good!


This documentation is intended for the users and adminitrators who contributes to the PDS Registry Service. The users of the PDS Registry Service through the web API should refer to the `PDS Search API user's guide <https://nasa-pds.github.io/pds-api/>`_
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor typo: "This documentation is intended for the users and adminitrators who contributes to the PDS Registry Service" should be "This documentation is intended for the users and administrators who contribute to the PDS Registry Service"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

querying of the PDS Registry Service on the cloud from the perspective of a Discipline
Node (DN).

Initial set-up
Copy link
Contributor

@ramesh-maddegoda ramesh-maddegoda Apr 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Title Case Suggestion: Better to have set-up start with an upper case S (as Set-up)to make this heading consistent with the Title Case used in other headings in this page.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Therefore, the deployment procedures described here can be used to deploy server-side and/or client-side registry components, without having to install the individual components manually.

.. warning::
these deployment are not intended for production given that 1. docker compose works on a single host and 2. permanent storage is left to docker to manage.
Copy link
Contributor

@ramesh-maddegoda ramesh-maddegoda Apr 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor suggestion: It is better to start this warning with an upper case T (These) to make it consistent with the .. note: below.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, done

pds-batch-loader.sh <harvest job confg file> Executes the Standalone Harvest client-side tool. The `pds-core-registry` server-side profile
This tool is recommended for small data sets of must be up and running
up to 10,000 PDS4 labels.
pds-service-loader.sh <harvest job confg file> Executes the Scalable Harvest client-side tool. The `pds-service-loader` server-side profile
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems I have made a mistake when preparing this table previously.

The Prerequisites of the pds-service-loader.sh should be,

The pds-loader-services server-side profile must be up and running

instead of

The pds-service-loader server-side profile must be up and running.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I introduced that mistake, anyway it is corrected now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, that was my mistake.


cd ..

Deployment profiles
Copy link
Contributor

@ramesh-maddegoda ramesh-maddegoda Apr 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Title Case Suggestion: It is better to start the work profiles with an upper case P (Profiles) in this heading to be consistent with the Title Case used in most of the heading in this page (E.g.: Clean-up the Deployment, Download and Configure the PDS Registry Application)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

====================== ==================================================== ==============================================


Client-side operations
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Title Case Suggestion: Client-side Operations

It is better to use the Title Case in this heading to be consistent with the Title Cases used on most of the headings in this page.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

================================================ ==================================================== ==============================================


Common deployment scenarii
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Title Case Suggestion: Common Deployment Scenarii

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

docker-registry-api-1 | :: Spring Boot :: (v2.3.1.RELEASE)
docker-registry-api-1 |

Run Standalone Harvest to load some PDS4 products
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Title Case Suggestion: Run Standalone Harvest to Load Some PDS4 Products

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


1) Open a new terminal and change the current working directory to the ``<REGISTRY_ROOT>/docker/scripts`` directory.

2) Define an harvest job configuration as specified in :doc:`Harvest Job Configuration<../user/harvest_job_configuration>`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor typo: "an harvest" should be "a harvest"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


3) Run the harvest command:

./pds-batch-loader.sh <harvest job confg file>
Copy link
Contributor

@ramesh-maddegoda ramesh-maddegoda Apr 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to use a code block here for,

./pds-batch-loader.sh  <harvest job confg file>

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


1) Open a new terminal and change the current working directory to the ``<REGISTRY_ROOT>/docker/scripts`` directory.

2) Define an harvest job configuration as specified in :doc:`Harvest Job Configuration<../user/harvest_job_configuration>`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The following link did not appear in the generated html.

:doc:Harvest Job Configuration<../user/harvest_job_configuration>

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I missed to add the file on git. This done now


1) Open a new terminal and change the current working directory to the ``<REGISTRY_ROOT>/docker/scripts/`` directory.

2) Define an harvest job configuration as specified in :doc:`Harvest Job Configuration<../user/harvest_job_configuration>`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The following link did not appear in the generated html.

:doc:Harvest Job Configuration<../user/harvest_job_configuration>

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I missed to add the file on git. This done now


1) Open a new terminal and change the current working directory to the ``<REGISTRY_ROOT>/docker/scripts/`` directory.

2) Define an harvest job configuration as specified in :doc:`Harvest Job Configuration<../user/harvest_job_configuration>`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor typo: "an harvest" should be "a harvest"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


3) Run the harvest command:

./pds-service-loader.sh <harvest job confg file>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to use a code block here for,

./pds-service-loader.sh  <harvest job confg file>

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean done.

docker compose --profile=pds-core-registry down


Core registry with Scalable Harvest
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Title Case Suggestion: Core Registry with Scalable Harvest

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


docker compose --profile=pds-loader-services up

Run Scalable Harvest Service client to load some PDS4 products
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Title Case Suggestion: Run Scalable Harvest Service Client to Load Some PDS4 Products

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Some applications, like Harvest or Registry Manager will not run with **Java 1.8**.


Test if Java is already installed
Copy link
Contributor

@ramesh-maddegoda ramesh-maddegoda Apr 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like I missed this one when restructuring the previous document.

Title Case Suggestion: Test If Java Is Already Installed

This was a tough one. I used the https://titlecaseconverter.com/ get the correct suggestion :-)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to learn that, to stays 'to' but 'if' becomes 'If' as well as 'Is'. It is a whole field of expertise that I had no idea of !

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, done

Along with your Opensearch user credentials you will receive the URL which points to your Opensearch
instance - we will refer to this as the Opensearch URL.

Changing Your Opensearch password
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Title Case Suggestion: Changing Your Opensearch Password

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, I first thought you missed it so I corrected and then I saw your comment !

Copy link
Contributor

@ramesh-maddegoda ramesh-maddegoda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@tloubrieu-jpl tloubrieu-jpl merged commit 18306be into pds_registry_app_263_docs_docker_compose Apr 28, 2022
tloubrieu-jpl added a commit that referenced this pull request Apr 28, 2022
…38)

* Add instructions to use docker compose to deploy registry components

Refer to the task nasa-pds-engineering-node/pds-registry-app#263

* Pds registry app 263 docs docker compose tl (#42)

* Incorporate cloud section consisting of architecture overview and quickstart. REsTful API documentation endpoint is TBD

* Add API documentation URL

* update integration tests for v1.0.0

* trying to integrate the cloud service deployment and the registry application documentation

* Revise image to utiize updated architecture diagram.

* add some more general introductions

* Add instructions to use docker compose to deploy registry components

Refer to the task nasa-pds-engineering-node/pds-registry-app#263

* wip: review docker compose doc, trying to take the user point of view

* wip

* make the doc a bit more user oriented, use .sh script to trigger data loading instead of docker compose naked commands

* update typos and all sort of wording improvments after review

Co-authored-by: Jimmie Young <jdyoung@jpl.nasa.gov>
Co-authored-by: Thomas Loubrieu <loubrieu@jpl.nasa.gov>
Co-authored-by: Jimmie Young <jimmie.d.young@gmail.com>
Co-authored-by: Ramesh Maddegoda <94033485+ramesh-maddegoda@users.noreply.github.com>

Co-authored-by: thomas loubrieu <60993872+tloubrieu-jpl@users.noreply.github.com>
Co-authored-by: Jimmie Young <jdyoung@jpl.nasa.gov>
Co-authored-by: Thomas Loubrieu <loubrieu@jpl.nasa.gov>
Co-authored-by: Jimmie Young <jimmie.d.young@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants