New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get Pachyderm Working with OpenShift #336

Closed
munchee13 opened this Issue Apr 21, 2016 · 21 comments

Comments

Projects
None yet
10 participants
@munchee13
Copy link

munchee13 commented Apr 21, 2016

Per our discussions, it looks like it may be a privilege issue or something similar. I have attached the steps to install the OpenShift vagrant image and then how to deploy pachyderm. You can troubleshoot through normal Kube commands as well. Everything is created and running, except the pachd pods do not start.

https://gist.github.com/munchee13/8cf64f2c1797d1d60891b28a193767f6

@sjezewski

This comment has been minimized.

Copy link
Collaborator

sjezewski commented May 5, 2016

K8s has support in theory. Need to test this though. Required for talk at Red Hat Summit

@derekchiang derekchiang added this to the v1.1 milestone May 5, 2016

@JoeyZwicker

This comment has been minimized.

Copy link
Member

JoeyZwicker commented Jun 10, 2016

@munchee13 Hey Chris, we're working on testing out Pachyderm on OpenShift, but we're getting a bit stuck because of our unfamiliarity with OpenShift and how those primitives translate to Kubernetes'. Any chance you'd be able to help us out with the OpenShift side of things? @jdoliner can chime in with more specific questions.

@jdoliner

This comment has been minimized.

Copy link
Member

jdoliner commented Jun 10, 2016

@munchee13 first snag I seem to have hit. I setup openshift using this guide:

https://docs.openshift.org/latest/getting_started/administrators.html#running-in-a-docker-container

and seem to be hitting and issue where none of the nodes become available and thus pods can't be scheduled. Any ideas on how to fix that?

@munchee13

This comment has been minimized.

Copy link

munchee13 commented Jun 11, 2016

Did you post the inquiry to the community list? That is likely the best
place to get some help.

Cheers,
Chris

From: Joe Doliner notifications@github.com notifications@github.com
Reply: pachyderm/pachyderm
reply@reply.github.com
reply@reply.github.com
Date: June 10, 2016 at 5:49:00 PM
To: pachyderm/pachyderm pachyderm@noreply.github.com
pachyderm@noreply.github.com
CC: munchee13 cmorgan@redhat.com cmorgan@redhat.com, Mention
mention@noreply.github.com mention@noreply.github.com
Subject: Re: [pachyderm/pachyderm] Get Pachyderm Working with OpenShift
(#336)

@munchee13 https://github.com/munchee13 first snag I seem to have hit. I

setup openshift using this guide:

https://docs.openshift.org/latest/getting_started/administrators.html#running-in-a-docker-container

and seem to be hitting and issue where none of the nodes become available
and thus pods can't be scheduled. Any ideas on how to fix that?


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#336 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/ABxhZRvj5L58B6xiYm1lSgUrlA781LVzks5qKdvMgaJpZM4IMoFk
.

@JoeyZwicker

This comment has been minimized.

Copy link
Member

JoeyZwicker commented Jun 11, 2016

Chris also suggested:
Maybe try with just the all-in-one VM instead: http://openshift.org/vm

@JoeyZwicker

This comment has been minimized.

Copy link
Member

JoeyZwicker commented Jun 13, 2016

@jdoliner

This comment has been minimized.

Copy link
Member

jdoliner commented Jun 13, 2016

Alright so far I tried setting openshift origin up inside a container using https://docs.openshift.org/latest/getting_started/administrators.html#running-in-a-docker-container

I managed to get this working, the issue with nodes not becoming available was caused by a full disk, restarting the machine fixed it.

The next snag I hit was with privileged containers. That seemed to go away when I did oc edit scc and enabled privileged containers for all the profiles. This seemed to work.. but only sometimes. Sometimes it accepted containers with privileged security contexts and sometimes not.

Lastly I hit a snag with ports. Apparently openshift, unlike kubernetes, won't let you bind to low ports inside containers.

@sambooo

This comment has been minimized.

Copy link
Contributor

sambooo commented Jun 21, 2016

I've been able to get pachyderm running on openshift origin by doing the following:

  • Init the vagrant box: vagrant init thesteve0/openshift-origin
  • Forward port 30650 in Vagrantfile: config.vm.network "forwarded_port", guest: 30650, host: 30650
  • Start the vm: vagrant up
  • Login as admin: oc login, username and password are both admin
  • Enable privileged containers: oc edit scc and set allowPrivilegedContainer: true everywhere
  • Download the manifest: curl -O http://pachyderm.io/manifest.json
  • Search for hostPath in manifest.json and remove those three lines plus the previous comma.
  • Submit the manifest: oc create -f manifest.json

Now pachctl should work, you can test it with pachctl version which should tell you your client and server versions.

@sambooo

This comment has been minimized.

Copy link
Contributor

sambooo commented Jun 21, 2016

An important note: Errors regarding timeouts on etcd/rethinkdb are probably unrelated to this issue. They were creating noise that made it hard to pin this down, and those errors are harmless unless they continue for a large number of restarts.

@jdoliner

This comment has been minimized.

Copy link
Member

jdoliner commented Jun 28, 2016

Only thing left to do is document steps necessary in SETUP.md

@jdoliner jdoliner self-assigned this Jun 28, 2016

@derekchiang derekchiang self-assigned this Jul 6, 2016

@derekchiang

This comment has been minimized.

Copy link
Collaborator

derekchiang commented Jul 6, 2016

Added instructions to SETUP.md. Credits to @sambooo.

@derekchiang derekchiang closed this Jul 6, 2016

@haps-basset

This comment has been minimized.

Copy link

haps-basset commented Aug 31, 2017

Hello
I need your help. I am trying to evaluate capability of pachyderm in our operational context. I would like to know how to integrate the pachyderm with openshift baseline. The installation procedure is extremely laconic thus any support in this matter highly appreciated.
pachctl deploy custom --persistent-disk --object-store [flags]
Would you be able to explain what is it persistent disk with regards to OS definitions? Is it a president disk or volume? What is it an object store etc? I am must admit I am not OS admin just basic user and in this case I need to know what I will gain from investment in pachyderm.
Simple HOW-TO for dummies is my dream in this stage of engagement with this concept e.g.

  1. What needs to be done before; shell I create an user pachyderm?
  2. How to create in OS environment persistent disk?
  3. How to create in OS environment object store?
  4. What are those persistent disk args object store args arguments?
    I have spoken with local OS experts and admins and they are also confused in term of terminology thus all suggestions where not workable or rejected by pachctl.
    Furthermore, even pachyderm support was not able to explain what needs to be done so I am trying to find good soul willing to help me solve this mistery.
    Haps.

@adrianog

This comment has been minimized.

Copy link

adrianog commented Feb 8, 2018

Hi @haps-basset

Did you get to the bottom of this? Has anything changed in Openshift (or Pachyderm) since you last tried this?
I notice an Openshift deployment page with a recent date here https://goo.gl/eg888a: is that relevant for your problems?

@haps-basset

This comment has been minimized.

Copy link

haps-basset commented Mar 6, 2018

No I did not and we have decided to drop further study and evaluation of this product.
We are interesting in usage some product which should support our research then testing or verifying its suitability.
Published by link point to the same page as on yours main web page thus is just useless.

@dwhitena

This comment has been minimized.

Copy link
Collaborator

dwhitena commented Mar 6, 2018

Sorry for the delay and confusion here @haps-basset. Looks like this issue slipped through the cracks a bit, but we are more than happy to help you work through this. If you do try again, please paste you result here and/or let us know what questions come up in our public Slack channel: http://slack.pachyderm.io/.

Regarding your Openshift questions:

  1. Regarding users, you should be able to deploy Pachyderm from anywhere that you can access your cluster (i.e., via oc).

  2. Regarding the persistent disk, this would be a "persistent volume" is OS terms. More information here: https://docs.openshift.com/enterprise/3.1/dev_guide/persistent_volumes.html

  3. Regarding the object store, this could be any S3 compliant object store solution like Minio, Rook, Ceph, Swift, etc. or any one of the cloud provider's object store solutions like S3, GCS, or Azure blob storage.

  4. Regarding the object store args, this will depend on what object store you are using. Can you give a little more information about which object store you are using?

Note, the custom deploy is meant out-of-the-box to support clusters deployed in the cloud, but with a non-cloud-provider object store (like Minio). As such, you will likely have to do what is mentioned here in terms of running the deploy with --dry-run and then modifying your manifest according if any adjustments need to be made for your particular k8s/OS deployment.

Let us know which parts of this don't make sense, and we can definitely provide more details and help.

@haps-basset

This comment has been minimized.

Copy link

haps-basset commented Mar 11, 2018

In our case we are only consider usage of private PaaS thus any suggestion to use non-private solution is just useless for us. In order to deploy private PaaS, we had considered OpenShift. Due to the fact that deployment of Pachyderm on OpenShift was unsuccessful we had decided to drop further study of this product. Bear in mind, deployment OpenShift is not cheap. To clarify, we thought about service support from your company as well but due to problems with evaluation risk with utilisation of the system was too high and the program was stopped.

@dwhitena

This comment has been minimized.

Copy link
Collaborator

dwhitena commented Mar 12, 2018

Hi @haps-basset, thanks for the update, and sorry for the issues with deployment. I am running through some OpenShift deployment stuff over the next few weeks, and I'm hoping to update the docs accordingly for future users. Please reach out any time if we can be of assistance with future deploys.

@jarcher

This comment has been minimized.

Copy link

jarcher commented May 16, 2018

Have you guys tested on 3.7 and 3.9 yet and been able to update your doc accordingly?

@jdoliner

This comment has been minimized.

Copy link
Member

jdoliner commented May 16, 2018

@jarcher we haven't, are you interested in using Pachyderm on those versions?

@adrianog

This comment has been minimized.

Copy link

adrianog commented May 19, 2018

Hi

Can we use native Containerized Red Hat Gluster Storage for the object store? (RE: "this could be any S3 compliant object store solution like Minio, Rook, Ceph, Swift, etc. or any one of the cloud provider's object store solutions like S3, GCS, or Azure blob storage.").

@JoeyZwicker

This comment has been minimized.

Copy link
Member

JoeyZwicker commented May 20, 2018

@adrianog Yes, we've seen successful Gluster deployments in the past. It's not something we've tested much ourselves, but let us know if you run into any problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment