New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

roachtest, workload: large re-write of interleaved-partition test and workload #31311

Open
wants to merge 1 commit into
base: master
from

Conversation

Projects
None yet
2 participants
@BramGruneir
Member

BramGruneir commented Oct 12, 2018

The interleaved partition workload and test were not performing the correct
operations and there was a good amount of leftover cruft from experiments that
were run. This commit fixes most of those problems.

A non-exhaustive selection of the updates:

  • All inserts occurred in the east so everything was stored with the
    leaseholders being locked to the east dc. These should be split evenly between
    east and west and now are again.
  • Both the retrieve and update operations did not run against actual session
    ids. So all of their operations where essentially noops.
  • The workloads are now run from different machines to not interfere with
    cockroach.
  • I greatly reduced the number of options being passed into the test. There was
    no need for most of them.
  • Removed the non-partitioned index on sessions.created.
  • Updated the session ids to show both the node in which they were added as
    well as the worker number. This proves we have a nice spread, as expected.

There's still some more to do here, such as put the full retrieve and update
operations inside a transaction. And try to reduce contention by adding in a
partitioned index for deletes.

All of this work was prompted by #28921, which shows that we had collisions.
Now that the workload is running correctly, if this happens again, we will have
more insight into what's going on.

Release note: None

@cockroach-teamcity

This comment has been minimized.

Show comment
Hide comment
@cockroach-teamcity

cockroach-teamcity Oct 12, 2018

Member

This change is Reviewable

Member

cockroach-teamcity commented Oct 12, 2018

This change is Reviewable

@BramGruneir BramGruneir requested a review from petermattis Oct 12, 2018

@petermattis petermattis requested a review from nvanbenschoten Oct 12, 2018

roachtest, workload: large re-write of interleaved-partition test and…
… workload

The interleaved partition workload and test were not performing the correct
operations and there was a good amount of leftover cruft from experiments that
were run. This commit fixes most of those problems.

A non-exhaustive selection of the updates:

- All inserts occurred in the east so everything was stored with the
leaseholders being locked to the east dc.  These should be split evenly between
east and west and now are again.
- Both the retrieve and update operations did not run against actual session
ids.  So all of their operations where essentially noops.
- The workloads are now run from different machines to not interfere with
cockroach.
- I greatly reduced the number of options being passed into the test. There was
no need for most of them.
- Removed the non-partitioned index on sessions.created.
- Updated the session ids to show both the node in which they were added as
well as the worker number. This proves we have a nice spread, as expected.

There's still some more to do here, such as put the full retrieve and update
operations inside a transaction.  And try to reduce contention by adding in a
partitioned index for deletes.

All of this work was prompted by #28921, which shows that we had collisions.
Now that the workload is running correctly, if this happens again, we will have
more insight into what's going on.

Release note: None

@BramGruneir BramGruneir requested a review from m-schneider Oct 16, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment