Make owned series service in ingester work with partitions ring. #7508
Conversation
|
|
||
| oss := i.ownedSeriesService | ||
| // | ||
| // TODO: When using partitions ring, we need to update all tenants after partition is registered into the ring. That is done by partitionLifecycler's starting method. |
There was a problem hiding this comment.
I will send new PR for this.
|
|
||
| user string |
There was a problem hiding this comment.
moved ownedServiceTestUser to TestOwnedSeriesService test, because TestOwnedSeriesServiceWithPartitionsRing needs different user, so that we can test some scenarios for moving series between partitions.
| require.NoError(t, err) | ||
| require.False(t, changed) | ||
| }) | ||
| func TestOwnedSeriesServiceWithPartitionsRing(t *testing.T) { |
There was a problem hiding this comment.
This is copy of TestOwnedSeriesService test adapted to use partitions. I did not try to unify the tests, to avoid making this even more complicated.
| } | ||
| } | ||
|
|
||
| func TestOwnedSeriesIngesterRingStrategyRingChanged(t *testing.T) { |
There was a problem hiding this comment.
This is renamed TestOwnedSeriesRingChanged test.
| }) | ||
| } | ||
|
|
||
| func TestOwnedSeriesPartitionsRingStrategyRingChanged(t *testing.T) { |
There was a problem hiding this comment.
This is TestOwnedSeriesIngesterRingStrategyRingChanged adapted for partitions. Some test cases are different.
| require.NoError(t, err) | ||
| require.False(t, changed) | ||
| }) | ||
| func TestOwnedSeriesServiceWithPartitionsRing(t *testing.T) { |
| return prepareIngesterWithBlockStorageAndOverridesAndPartitionRing(t, ingesterCfg, overrides, ingestersRing, nil, dataDir, bucketDir, registerer) | ||
| } | ||
|
|
||
| func prepareIngesterWithBlockStorageAndOverridesAndPartitionRing(t testing.TB, ingesterCfg Config, overrides *validation.Overrides, ingestersRing ring.ReadRing, partitionsRing *ring.PartitionRingWatcher, dataDir string, bucketDir string, registerer prometheus.Registerer) (*Ingester, error) { |
There was a problem hiding this comment.
Have you considered using createTestIngesterWithIngestStorage(). If there's no hard blocker I would prefer to use an ingester which have been fully setup to use the ingest storage (e.g. fake Kafka, etc...). I think it will also simplify your setup in TestOwnedSeriesServiceWithPartitionsRing.
There was a problem hiding this comment.
Maybe it's not easy to restart the ingester in the "new user trigger from WAL replay" test case, but I would give it a try anyway.
There was a problem hiding this comment.
Gave that a try. Not sure if it's simpler, but I've realized that I should push series via Kafka, and not Push method.
pstibrany
force-pushed
the
partitions-owned-series-on-top-of-owned-series-limiting
branch
from
7fb201d
to
adf2ec4
Compare
Signed-off-by: Peter Štibraný <pstibrany@gmail.com>
Signed-off-by: Peter Štibraný <pstibrany@gmail.com>
Signed-off-by: Peter Štibraný <pstibrany@gmail.com>
Signed-off-by: Peter Štibraný <pstibrany@gmail.com>
Signed-off-by: Peter Štibraný <pstibrany@gmail.com>
Signed-off-by: Peter Štibraný <pstibrany@gmail.com>
pstibrany
force-pushed
the
partitions-owned-series-on-top-of-owned-series-limiting
branch
from
adf2ec4
to
3c9a4ff
Compare
pstibrany
deleted the
partitions-owned-series-on-top-of-owned-series-limiting
branch
What this PR does
This PR modifies "owned series" service to make it work with partitions ring (if ingest storage is configured) as well as ingester ring.
Checklist
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX].about-versioning.mdupdated with experimental features.