[DNM] multimds suite sync with fs

[batrick]

This PR does two things:

• Simplifies fs sub-suites by using symlinks to common configurations wherever possible (making unique aspects of a sub-suite stand out and increasing code reuse).
• Synchronizes the multimds suite with the fs suite (again, lots of symlinks). This adds several sub-suites/tasks that were missing in multimds.

I'm currently running a test on this branch here: http://pulpito.ceph.com/pdonnell-2016-08-10_03:03:20-multimds-master---basic-mira

[batrick]

Here's a run of the fs suite I did over the weekend:

http://pulpito.ceph.com/pdonnell-2016-08-19_22:12:32-fs-master-testing-basic-mira/

Found 7 failures... I'm looking into if those are new problems, caused by the reorg, or legit issues found due to the reorg.

[batrick]

Mostly the same failures as this nightly on master:

http://pulpito.ceph.com/teuthology-2016-08-13_17:15:02-fs-master---basic-smithi/

[gregsfortytwo]

Why commit "suites/fs: unify/reuse common cluster layout"? I mean it makes the symlinks shorter but most of our clusters are common across suites and this one was a modified duplicate of one of the others in its original folder; moving it down makes them harder to find for global changes.

[gregsfortytwo]

I'm not really comfortable referencing down into parallel trees -- people generally aren't prepared for that and we'll miss it in future reviews. What's the benefit of sharing these besides removing a single task line that actually does vary across some tests?

[batrick]

In this particular case, it's just making the sub-suites consistent. Some sub-suites had the ceph-fuse task specified within a task (e.g. suites/fs/traceless/tasks/cfuse_workunit_suites_dbench.yaml) and others at the top-level (suites/fs/snaps/mount/ceph-fuse.yaml).

[gregsfortytwo]

Okay, so overall you're trying to remove duplicates between the multimds and other suites and that makes sense. But I'm worried we're going to find that people change the standard fs suites in ways that bust up multi-mds without realizing it. If nothing else, we should add a doc explaining what's copied and why, and in the future we can pull stuff out into a common folder as needed. Ideally I'd like to see a tree of what stuff we expect to share and what we expect to be different, but I'm not sure if we have enough models right now to do it right and it may not be worth the time to start off with.

[gregsfortytwo]

Oh, and are there any test runs of this on suites/fs and suites/multimds? :)

[gregsfortytwo]

ping @batrick
@jcsp, do you have any thoughts on how we organize this?

[batrick]

Why commit "suites/fs: unify/reuse common cluster layout"? I mean it makes the symlinks shorter but most of our clusters are common across suites and this one was a modified duplicate of one of the others in its original folder; moving it down makes them harder to find for global changes.

The intent is just to make it clear if a sub-suite is doing something different from basic.

moving it down makes them harder to find for global changes.

I didn't get this part. Which global changes?

[batrick]

Okay, so overall you're trying to remove duplicates between the multimds and other suites and that makes sense. But I'm worried we're going to find that people change the standard fs suites in ways that bust up multi-mds without realizing it. If nothing else, we should add a doc explaining what's copied and why, and in the future we can pull stuff out into a common folder as needed. Ideally I'd like to see a tree of what stuff we expect to share and what we expect to be different, but I'm not sure if we have enough models right now to do it right and it may not be worth the time to start off with.

Well, most of the work is in identifying the differences which I've already done. I don't mind moving the common stuff to a shared directory so it's more obvious.

[batrick]

Oh, and are there any test runs of this on suites/fs and suites/multimds? :)

fs: http://pulpito.ceph.com/pdonnell-2016-08-19_22:12:32-fs-master-testing-basic-mira/

multimds: http://pulpito.ceph.com/pdonnell-2016-08-11_20:40:52-multimds-master-testing-basic-mira/

[jcsp]

There's a failure in test_python.sh in that fs suite run. Looks like that test was run without the ceph--fuse task previously, and it's failing when run with it -- probably need to specialise that out again

[jcsp]

Weirdly this is not showing up as having a merge conflict with the btrfs->xfs change that I just made in master, so I guess it needs updating by hand?

[jcsp]

Please can you remove the multimds/recovery piece from here, the wip-kclient-python branch will do that (with the associated fixes that are required for those tests to actually work)

Oops, I was thinking of kclient/recovery. However, I think transplanting into multimds/recovery probably isn't useful: these python tests all run with an explicit number of MDSs (some of them already use multiple daemons in the fs suite), so running them again in the multimds suite probably isn't helpful.

[jcsp]

From my point of view the advantage of the symlinks (as opposed to duplicating yaml fragments) is not so much the de-duplication as it is to make obvious which bits are like the rest of the tests, and which bits differ in a particular suite.

I am interested in the idea of adding a fs_common or similar folder (peer to fs, kcephfs, multimds) containing the common bits, to avoid quite so many places reaching into fs/basic (will be painful if we ever needed to modify fs/basic).

(Background: I'm not exactly in love with the whole YAML tree mechanism for specifying test permutations to begin with, so while this still isn't a super-clean thing overall, I do think it's probably an improvement and makes the tests easier to read by highlighting which bits differ)