Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

As a system, I can support up to 25 simultaneous writes from Harvest #226

Closed
jordanpadams opened this issue Sep 7, 2023 · 3 comments
Closed
Assignees
Labels
B14.0 p.must-have requirement the current issue is a requirement

Comments

@jordanpadams
Copy link
Member

jordanpadams commented Sep 7, 2023

Checked for duplicates

Yes - I've already checked

πŸ§‘β€πŸ”¬ User Persona(s)

System

πŸ’ͺ Motivation

...so that I can support better scalability

πŸ“– Additional Details

No response

Acceptance Criteria

Given a running Registry / OpenSearch
When I perform multiple simultaneous harvest executions
Then I expect OpenSearch to support those writes without error

βš™οΈ Engineering Details

This requirement is intentionally vague. We would like to know what # of simultaneous runs of harvest. Right now, I took at stab at 25, but getting an understanding of where this breaks would be great.

@jordanpadams jordanpadams added needs:triage requirement the current issue is a requirement labels Sep 7, 2023
@jordanpadams jordanpadams self-assigned this Sep 7, 2023
@jordanpadams jordanpadams changed the title As a system, I want to support TBD simultaneous writes from Harvest As a system, I can support up to 25 simultaneous writes from Harvest Sep 8, 2023
@miguelp1986 miguelp1986 self-assigned this Oct 5, 2023
@miguelp1986
Copy link
Contributor

@tloubrieu-jpl I'm having problems having my Harvest app, which I've built from the latest release files, talk to my local, Dockerized OpenSearch instance.
When running a test command, I get a certificate error:

harvest -c /usr/local/harvest-3.8.0/conf/examples/directories_copy.xml                                                                   ─╯
[SUMMARY] Reading configuration from /usr/local/harvest-3.8.0/conf/examples/directories_copy.xml
[SUMMARY] Output directory: /tmp/harvest/out
[SUMMARY] Elasticsearch URL: https://localhost:9200/, index: registry
[INFO] Connecting to Elasticsearch
[ERROR] Host name 'localhost' does not match the certificate subject provided by the peer (CN=elasticsearch, OU=PDS, O=ORG, L=LA, ST=CALIFORNIA, C=CA)

I'm following along with https://nasa-pds.github.io/registry/user/load1.html#standalone-harvest-quick-start.

Attached is my Harvest config file:
directories_copy copy.xml.txt

@miguelp1986
Copy link
Contributor

I fixed my issue. I added registry-api and elasticsearch as domains alongside localhost in my /etc/hosts file. Now Postman works with Registry-Api tests without having to change the baseUrl to localhost and my Harvest command line tool works with OpeanSearch when I have it pointing to https://elasticsearch:9200 in the config file.

@tloubrieu-jpl
Copy link
Member

Great ! Thanks @miguelp1986

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
B14.0 p.must-have requirement the current issue is a requirement
Projects
None yet
Development

No branches or pull requests

3 participants