Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Onestop-clients update scripts #1507

Open
5 tasks done
erinreeves opened this issue May 20, 2021 · 3 comments
Open
5 tasks done

Onestop-clients update scripts #1507

erinreeves opened this issue May 20, 2021 · 3 comments
Assignees

Comments

@erinreeves
Copy link
Contributor

erinreeves commented May 20, 2021

Summary

  • Update script constructors where used to take config location and now takes a dict
  • Changed constructor parameters of metadata_type to classname_metadata_type so could have two metadata_types in same config if needed by user.
As a user
I want to be able to execute the scripts, in the scripts directory
So that I can perform the different supported actions onestop-clients has to offer.

Additional Tasks: Due to being unable to run the full OneStop stack locally I had to execute the scripts on bastion(cedar-devs). This required building an image of onestop-python-client, onestop-s3-handler, and onestop-sme to hub.docker so I could install that on bastion and have it pull the image down and deploy it to an EKS node (which is not able to see local files on bastion).

  • Build image of onestop-python-client, onestop-s3-handler, and onestop-sme to hub.docker
  • Able to use helm to create sme pod
  • Scripts execute on bastion
@erinreeves erinreeves self-assigned this May 20, 2021
@erinreeves erinreeves changed the title Onestop-clients fixed scripts Onestop-clients update scripts May 20, 2021
@erinreeves
Copy link
Contributor Author

Need to revisit image names

@erinreeves
Copy link
Contributor Author

2021-06-05 03:30:20.364 INFO 1 --- [-StreamThread-1] org.apache.kafka.streams.KafkaStreams : stream-client [psi-registry-ad13de8e-a5f2-4c4a-95e4-daa0a3746902] State transition from RUNNING to ERROR
2021-06-05 03:30:20.366 ERROR 1 --- [-StreamThread-1] o.c.o.kafka.common.util.KafkaHelpers : Streams app entered a bad state: ERROR
2021-06-05 03:30:20.367 ERROR 1 --- [-StreamThread-1] org.apache.kafka.streams.KafkaStreams : stream-client [psi-registry-ad13de8e-a5f2-4c4a-95e4-daa0a3746902] All stream threads have died. The instance will be in error state and should be closed.
2021-06-05 03:30:20.367 INFO 1 --- [-StreamThread-1] o.a.k.s.p.internals.StreamThread : stream-thread [psi-registry-ad13de8e-a5f2-4c4a-95e4-daa0a3746902-StreamThread-1] Shutdown complete

@erinreeves
Copy link
Contributor Author

Pull request cedardevs/onestop-clients#58

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant