Add BigCouch support #117

Closed
wants to merge 3 commits into
from

Projects

None yet

5 participants

@vkuznet
dmwm member

Please take changes for BigCouch back-end support in WMAgent. Upon previous request (PR#116) this PR contains changes as single commit. With this PR you may discard #116.

@vkuznet vkuznet referenced this pull request Mar 1, 2014
Closed

Wmagent bigcouch #116

@geneguvo
dmwm member

Hi Valentin,

Did you have a chance to test it using the new set of scripts for wmagent that Alan sent you? I see you including its wmagent/deploy changes, but I don't see anywhere the wmagent_reorg file. Perhaps you forgot to include it?

@vkuznet
dmwm member
@geneguvo
dmwm member

The fact the manage_reorg file is there doesn't hurt anything. Also, you are already including the corresponding changes on wmagent/deploy...

@geneguvo
dmwm member

Also, it is backward compatible with the current instructions AFAIR, otherwise it would not have worked for you: that is, the default variant defaults to what is in use currently and therefore the current wmagent instructions shall still work.

@vkuznet
dmwm member
@vkuznet
dmwm member
@geneguvo
dmwm member

Then I don't understand you.... look at the wmagent/deploy file you committed (https://github.com/vkuznet/deployment/blob/5f63d242a9dc6697b1d5507ad2cb20944430e204/wmagent/deploy). It has pieces of code for manage_reorg (and the variants). If the purpose was to separate wmagent reorganization from the bigcouch support, why did you include these changes?

If the wmagent/deploy file you committed is what you used, you certainly tested it with the current wmagent deploy instructions :-)

@vkuznet
dmwm member
@geneguvo
dmwm member

Hi Valentin,

I did significant changes to bigcouch.spec and the related deployment/bigcouch scripts after my review. Therefore, I think you need to regenerate this pull request for the wmagent to catch up with the latest changes. Could you do that ?

@vkuznet
dmwm member
@geneguvo
dmwm member

Hi Valentin,

No, I haven't tested on a cluster environment. It was difficult enough for making all the bits fall in the right place already for single-node. Let's do one step each time.

About the ports, I did the setup so that both CouchDB and BigCouch can co-exist on the same machine. I've actually deployed both on cmsweb-dev. CouchDB keeps using port 5984, while BigCouch will use 5985 (main port) and 5986 (the back door, or admin port). That is, I used the port range we already had allocated for couch. Besides that, the Distributed erlang will listen to connections on the port range 9100-9105.

What you need to do is to drop the couch files from deployment/wmagent/bigcouch. It will then bring bigcouch with the offsite configuration that is what wmagents were supposed to use. You don't need to copy config files over the bigcouch RPM area. All the bigcouch configuration is under deployment/bigcouch/.

Cheers,
Diego.

@vkuznet
dmwm member
@amaltaro

Valentin, the bigcouch RPM was made available today in comp.pre only:
https://hypernews.cern.ch/HyperNews/CMS/get/webInterfaces/1210.html
I believe that's fine for your tests, no?. Thanks

@vkuznet
dmwm member
@vkuznet
dmwm member
@amaltaro

If we add this dependency on wmagent.spec, then we are going to deploy every time both couch and bigcouch in the nodes, I don't think we want it. Couldn't you put this dependency on and build only private RPMs? At least now we have a bigcouch.spec to be used. If you can't, let me know and I'll make them available in my repo.

@vkuznet
dmwm member
@geneguvo
dmwm member

Hi Valentin,
You can use the "comp" meta-package to find out all CMS COMP dependencies, including bigcouch and wmagent. Besides, I think wmagent used to have it's own meta-package, "wmagent-dev" if I remember correctly. You could add bigcouch as a dependency there if you feel too annoying to have to build the full "comp" whenever you need to change some spec file. In general, building "comp" or "wmagent-dev" won't make much difference unless you changing things inner in the software stack.

Concerning the configuration files, you should not need ports.config because I put the corresponding configuration directly on vm.args. Besides vm.args and local.ini, the only config files that have an effect are those containing the secrets: the hmackey.ini and erlang.cookie file (see https://github.com/dmwm/deployment/blob/master/bigcouch/deploy#L56). If you want distributed erlang to work, all the nodes must have the same "erlang.cookie". The hmackey.ini only makes sense for applications running behind the cmsweb frontends, but that's not the case of wmagents.

Cheers,
Diego.

@vkuznet
dmwm member

I am still confused with WMAgent/BigCouch deployment procedure. Originally I used deployment/Deploy script to install wmagent, see instructions here twiki:https://github.com/dmwm/WMCore/wiki/All-in-one-test. The configuration for BigCouch,CouchDB and MySQL were part of deployment/wmagent.

Now I was requested to drop bigcouch configuration from deployment/wmagent but I end-up with the following puzzle:

  • The Deploy script will deploy only cms packages, the WMAgent is cms package
  • If I add BigCouch as dependency to wmagent or wmagent-dev, it will be installed in comp/arch/external area. But the package will contain bare code, no configuration files.

My question is who should bring BigCouch configuration files from deployment/bigcouch area into release one (i.e. $root/current/config/bigcouch) and how/where this should be done.

The next question is how bigcouch will appear in $root/current/apps area? The Deploy script only make appropriate link for cms packages not externals and since wmagent installation does not use deployment/admin/InstallDev script (which can make those links as well) I don't know how bigcouch will appear under $root/current/apps area.

Finally, current deployment/wmagent/ area contains configuration files for mysql and couchdb which are copied into $root/current/config/{couchdb,mysql} areas, but Diego explicitly requested to drop bigcouch configuration from deployment/wmagent since now it resides in deployment/bigcouch area. This creates a precedent since deployment/wmagent will contain couchdb and mysql but bigcouch will reside elsewhere.

To sum-up, I think we're mixing different stuff here. The CouchDB and BigCouch are in deployment, but MySQL/ORACLE are not. The WMAgent needs configuration for all of them and right now it keeps then under its belt (they resides under deployment/wmagent area). If you ask to remove bigcouch config from deployment/wmagent, then I think CouchDB local.init configuration should follow the case. If we want to treat CouchDB/BigCouch as services, then we either need to modify Deploy script to allow their deployment independently from wmagent or move them from external into cms land (this will allow them to be installed via Deploy which will bring their configs in place). Bottom line, I'm puzzled with entire procedure and before making any action I rather prefer to get clear idea about all components.

@geneguvo
dmwm member

Hi Valentin,

Most of this confusion would have been avoided if you had started with the new wmagent deployment script... but let's see if I can help you anyway.

The deployment/bigcouch scripts is the one to bring the bigcouch configuration. To get it in place, you need to put "deploy bigcouch offsite" on your wmagent/deploy script (in the deploy_wmagent_deps section). If it is not yet there on the default variant, you must put it as it is on the other mysql/oracle variants.

You can use pure Deploy script if you want (no InstallDev) to deploy any service. It doesn't matter if this is a CMS or external RPM. We never had a distinction and I don't know where this assumption came from.

Yes, since you are not using the new wmagent deploy scripts, your setup will have couchdb, mysql coming together in the wmagent configuration area, but bigcouch on a separate area.

If you coming to the O&C week, it is probably easier we meet and accomplish this together.

Cheers,
Diego.

@vkuznet
dmwm member

Hi Diego, sorry I'm not going to C&O week, family issues. Therefore we need to work remotely.
My PR was based on new deployment script (at least I thought so and it was based on what Alan gave me from his private repo) and it does have deploy bigcouch offsite, see https://github.com/dmwm/deployment/pull/117/files#diff-250c1cdf491c9cb407e078aa445506f5R9

I did run pure Deploy script, but as I wrote it does not allow to use external packages, see https://github.com/dmwm/deployment/blob/master/Deploy#L53
and if I run it as

./Deploy -R bigcouch@<some tag>

it will not pick up bigcouch package since it is not in cms namespace. May be I'm missing something, please refer me to concrete example how to install external package via Deploy script.

So, I still need more info from you. Please have a look at PR files and clarify if wmagent/deploy script is new/old (if old please specify which one to pick up); if we'll use Deploy procedure I need to know details how to install external package; and I still need to clarify which prefix to use in deploy script to pick-up bigcouch from separate area.

@geneguvo
dmwm member

Hi Valentin,

You have the new wmagent deploy scripts (except that you missing the manage_reorg file as we discussed before) but you still using the old "all-in-one" deployment instructions, which uses the "default" variant and therefore does not exercise the bits I added.

There is no restriction on being a CMS or external package. See for instance what we do to deploy mongodb since ever. It is an external, but gets deployed exactly as any other (CMS) service.

What you missing is that the "release" RPM you use on the Deploy -R option must depend on all the RPM packages needed, either directly or indirectly. This means that either you use "-R comp@" or use "-R wmagent-dev@" but add bigcouch RPM as a dependency on wmagent-dev.spec file.

Now you must make a choice: if you want to go with the "all-in-one" instructions, you need to add "deploy bigcouch offsite" to the default variant too, and hack things around on wmagent to start/stop/push things correctly to bigcouch (as it does for couchdb and mysql); if you want to go with the new deploy procedure, then you could deploy it with:

(VER=HG1404x REPO="-r comp=comp.pre" A=/data/cfg/admin; cd /data;  $A/InstallDev -R comp@$VER -S -s image -v $VER $REPO -p "wmagent/mysql")

(A=/data/cfg/admin; cd /data; $A/InstallDev -S -s start) # or stop, status...

Note the "-S" option to InstallDev commands. This is needed to run it on the single-user mode. If you prefer, you could still use the pure Deploy (prep, sw, post) commands to deploy it too.

Also note I'm deploying the mysql variant, not the default one. If you opt for the "oracle" variant instead, I think you must request the DB on devdb11 and deploy the schema too.

@ticoann

Thanks Valentin and Diego for all the work.
I think we should go with the standard deployment (cmsweb) install of current all-in-one.
However, we first need to try and instruct all the parties which need to deploy the agents (ops, crab group, etc) and make sure whether all the task they do can be done. (since the manage script hasn't modified in terms of commands used, I guess it shouldn't be a problem)

What you missing is that the "release" RPM you use on the Deploy -R option must depend on all the RPM packages needed, either directly or indirectly. This means that either you use "-R comp@" or use "-R wmagent-dev@" but add bigcouch RPM as a dependency on wmagent-dev.spec file.

Diego, So we can't do wmagent@0.9.94c, but do we have to use wmagent-dev.spec (due to bigcouch dependency) to apply different version?

@geneguvo
dmwm member

Hi Seangchan,
If you instead want to use wmagent@ver, you can. However, you add the dependency there instead of wmagent-dev. In practice, it doesn't make any difference. It will look a little bit weird, though, since you don't have code (library) dependencies on it, but a service dependency.

Cheers,
Diego.

@ticoann

Hi Diego, It is not big deal. Normally, I don't think WMAgent can follow the same deployment schedule as cmsweb deployment. So I thought it would be cosmetically better to use wmagent.spec than wmagent-dev.spec.
Can wmagent be deployed along side with bigcouch? something like following?
Which might not be the better solution, though.

A=/data/cfg/admin; cd /data;  $A/InstallDev -R comp@$VER -S -s image -v $VER $REPO -p bigcouch wmagent@version
@geneguvo
dmwm member

This has absolutely nothing to do with the cmsweb deployment schedule, this is software packaging and distribution.

You can use any package as the "release" reference package. It could be wmagent, wmagent-dev, wmtools, anything... you normally create a convenient meta-package and make it depend on all the services you'd like to deploy from the command line, including wmagent and bigcouch. However, you could instead just use the already existent "comp" meta-package if you don't want to create another one. Or put everything under the roof of the "wmagent" package, but for a newcomer trying to understand the wmagent business it would look like wmagent is using bigcouch as a library, which is not the case.

Concerning the question on deploying both services together.... yes, for sure you can. What you cannot do is deploy two variants of the same service at the same time (does not seem to make any sense). Agents must use the bigcouch/offsite variant, so either you remove bigcouch completely from your command line or, if you specify it, you specify "bigcouch/offsite" so that it is made consistent to what you depending on from inside the wmagent/deploy script.

And yes, you can specify the @version on the command line, but you don't need to if you just specify the right meta-package version.

For instance, you'd deploy with:
"""
A=/data/cfg/admin; cd /data; $A/InstallDev -R comp@HGXXXX-compX -S -s image -v $VER $REPO -p wmagent
"""
Where HGXXXX-compX is whatever version your private build of "comp" produced.

Or, depend on bigcouch from wmagent.spec and deploy it with:
"""
A=/data/cfg/admin; cd /data; $A/InstallDev -R wmagent@versionX -S -s image -v $VERX $REPO -p wmagent
"""

Where versionX is whatever you obtained when doing you new build of the wmagent package.

Did it help?

@geneguvo
dmwm member

I had a look into it again and I don't have anything else to add. It is fine from my side.

I don't test wmagent myself, though, because currently it still has its own different set of installation instructions. However, Valentin said on private messages it worked for him, and provided the instructions used. I'm therefore going to merge this PR. Please update the all-in-one instructions if needed, otherwise it would break with normal wmagent operations.

@ticoann

@geneguvo, Hi Diego, please hold off the the merge. I would like to discuss this ops and let them try the deployment first.

@geneguvo
dmwm member

Sure. No problem. I'll wait until you give the green light.

@ticoann

Thanks, Diego.

@BrunoCoimbra

Hey @vkuznet is this PR still valid?

@vkuznet
dmwm member

We don't need this anymore, closing the ticket.

@vkuznet vkuznet closed this Aug 19, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment