[Test] e2e test fails if a Config DB is pre-initialized on the test bed

[msterin]

The subject says it all. The test fail. They should try to remove the config DB first (and ignore rm failures) and then proceed

//CC @pshahzeb @shuklanirdesh82

[shaominchen]

I'd suggest to add some bootstrapping logic before running any tests:

1. Verify the required environment variables are properly initialized
2. Clean up the Config DB if it already exists
3. Maybe more...

This way the test should fail gracefully instead of reporting some very confusing error messages.

[shaominchen]

I will take care of this.

[govint]

Do we also know what is the error, is it a bug vs. a test code issue. Can a customer hit such a problem with a config DB? Just asking as we don't have the logs for the error.

[msterin]

it is a test issue. in customer environment it's perfectly fine to have the DB and the CLI is supposed to handle it all

[shaominchen]

@govint Copy the error from PR #1464:

FAIL: basic_test.go:159: BasicTestSuite.TestVmGroupVolumeIsolation

basic_test.go:164:
c.Assert(err, IsNil, Commentf(out))
... value *exec.ExitError = &exec.ExitError{ProcessState:(*os.ProcessState)(0xc4200ae6a0), Stderr:[]uint8(nil)} ("exit status 2")
... Warning: this feature is EXPERIMENTAL
Config DB is already initialized. Use 'rm --local' or 'rm --unlink' to reset
Additional information: {'DB_Mode': 'SingleNode (local DB exists)', 'DB_LocalPath': '/etc/vmware/vmdkops/auth-db', 'DB_SharedLocation': 'N/A'}

[shaominchen]

A quick fix would be to improve the current ConfigInit util function: check if the DB is already initialized - if it's already initialized, just remove it.

I can send a quick fix for above solution. However, to make the whole end-to-end test more robust, I think we should do what I suggested above. @govint Do you have any concern for my suggestion above, especially, cleaning up the Config DB if it already exists before each test group?

[shaominchen]

I've sent out the quick fix (#1516) for the current specific issue, and filed a new issue (#1517) to track the efforts of improving the robustness of e2e test infrastructure.