roachtest: add CREATE INDEX on tpcc 1000 cluster

[vivekmenezes]

The roachtest creates an index against a table with
30M rows on a tpcc cluster with a 1000 warehouses.

The test takes around 3 hours to run.

Release note: None

[cockroach-teamcity]

This change is 

[cockroach-teamcity]

This change is 

[petermattis]

Reviewable status: complete! 0 of 0 LGTMs obtained

---

pkg/cmd/roachtest/create_index.go, line 28 at r1 (raw file):

	numNodes := 5
	r.Add(testSpec{
		Name:    fmt.Sprintf("createindex/tpcc/warehouses=%d/nodes=%d", warehouses, numNodes),

Drive-by comment: the naming of roachtests is becoming inconsistent. We have backupTPCC vs cancel/tpcc vs cdc/w=1000 (which runs TPC-C). We have schemachange and now createindex/... which is also a schema change test. I don't have a proposal here. Perhaps this isn't a big problem. Mainly I'm bringing this up in the hopes of nerd sniping @andreimatei.

[vivekmenezes]

bors r+

[vivekmenezes]

bors r+

[craig]

Build succeeded

• GitHub CI (Cockroach)

[craig]

Build succeeded

• GitHub CI (Cockroach)

[petermattis]

Reviewable status: complete! 0 of 0 LGTMs obtained

---

pkg/cmd/roachtest/create_index.go, line 28 at r1 (raw file):

Previously, petermattis (Peter Mattis) wrote…

Drive-by comment: the naming of roachtests is becoming inconsistent. We have backupTPCC vs cancel/tpcc vs cdc/w=1000 (which runs TPC-C). We have schemachange and now createindex/... which is also a schema change test. I don't have a proposal here. Perhaps this isn't a big problem. Mainly I'm bringing this up in the hopes of nerd sniping @andreimatei.

Ping. I don't want this to get lost.

[andreimatei]

Well I'm not familiar with most of these tests... I'm not sure what you want me to do :)

[andreimatei]

Well I'm not familiar with most of these tests... I'm not sure what you want me to do :)

[petermattis]

Well I'm not familiar with most of these tests... I'm not sure what you want me to do :)

Propose a naming scheme so that we avoid having a dumping ground. Don't you think all of the schema change tests shows be grouped somehow?

[petermattis]

Well I'm not familiar with most of these tests... I'm not sure what you want me to do :)

Propose a naming scheme so that we avoid having a dumping ground. Don't you think all of the schema change tests shows be grouped somehow?

[andreimatei]

I do think naming conventions are good for grouping schema change tests in commands like roachtest list. And since I'm getting rid of subtests, I think that names made up of /-separated components is great.
In the case of the test introduced in this patch (createindex/tpcc/warehouses=%d/nodes=%d), I don't really know if this should be considered a schema change test or a tpcc + nemesis test (basically, do we want to emphasize the schema change aspect or the tpcc aspect?).
But I can't really make this determination; for example I don't know what the failing criteria for this test is. I see that the index creation part fails if the create index statements returns an error. I don't know when the TPCC part fails - I see a --tolerate-errors being passed for unexplained reasons, so I assume it doesn't fail much :P.
So, if it's a schema change test, I think it'd prolly be good if the test name started with either schemachange.
If it's a "tpcc test", I think we would have probably benefited from introducing a framework for composing workloads (tpcc) with nemeses (schema change), and expressed the test purely declaratively - and then you'd get an auto-generated name too.

But, like, beyond bike shedding over particulars, I don't feel prepared to philosophize over generalities yet. If you put me in charge of this, I'll respectfully punt.

[andreimatei]

I do think naming conventions are good for grouping schema change tests in commands like roachtest list. And since I'm getting rid of subtests, I think that names made up of /-separated components is great.
In the case of the test introduced in this patch (createindex/tpcc/warehouses=%d/nodes=%d), I don't really know if this should be considered a schema change test or a tpcc + nemesis test (basically, do we want to emphasize the schema change aspect or the tpcc aspect?).
But I can't really make this determination; for example I don't know what the failing criteria for this test is. I see that the index creation part fails if the create index statements returns an error. I don't know when the TPCC part fails - I see a --tolerate-errors being passed for unexplained reasons, so I assume it doesn't fail much :P.
So, if it's a schema change test, I think it'd prolly be good if the test name started with either schemachange.
If it's a "tpcc test", I think we would have probably benefited from introducing a framework for composing workloads (tpcc) with nemeses (schema change), and expressed the test purely declaratively - and then you'd get an auto-generated name too.

But, like, beyond bike shedding over particulars, I don't feel prepared to philosophize over generalities yet. If you put me in charge of this, I'll respectfully punt.

[petermattis]

Given that the author was exercising schema changes, I think the test name should be grouped with other schema change tests. For example, schemachange/createindex/tpcc-%d/nodes=%d.

@vivekmenezes You've been silent on this convo. Can you chime in here?

[petermattis]

Given that the author was exercising schema changes, I think the test name should be grouped with other schema change tests. For example, schemachange/createindex/tpcc-%d/nodes=%d.

@vivekmenezes You've been silent on this convo. Can you chime in here?

[vivekmenezes]

@petermattis using the schemachange prefix makes sense at this stage so we at least have some organization at the top level.

[vivekmenezes]

@petermattis using the schemachange prefix makes sense at this stage so we at least have some organization at the top level.