Shard API (Mark shard(s) as read only) #1860
Conversation
Codecov Report
@@ Coverage Diff @@
## master #1860 +/- ##
==========================================
+ Coverage 66.08% 66.23% +0.14%
==========================================
Files 411 413 +2
Lines 30931 31333 +402
==========================================
+ Hits 20442 20753 +311
- Misses 8691 8768 +77
- Partials 1798 1812 +14
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
adapters/repos/db/lsmkv/bucket.go
Outdated
| @@ -45,6 +46,9 @@ type Bucket struct { | |||
| legacyMapSortingBeforeCompaction bool | |||
|
|
|||
| stopFlushCycle chan struct{} | |||
|
|
|||
| status storagestate.Status | |||
| statusLock *sync.Mutex | |||
There was a problem hiding this comment.
is there a reason why statusLock is a pointer?
There was a problem hiding this comment.
I took the example from the attached internal issue video, which declared this field as a pointer. I don't have a reason aside from that 🙂 can change to value if preferred
| @@ -41,6 +42,9 @@ type SegmentGroup struct { | |||
| // for backward-compatibility with states where the disk state for maps was | |||
| // not guaranteed to be sorted yet | |||
| mapRequiresSorting bool | |||
|
|
|||
| status storagestate.Status | |||
| statusLock *sync.Mutex | |||
There was a problem hiding this comment.
here also is there a reason why statusLock is a pointer?
test/integration/run.sh
Outdated
| go test -count 1 -coverpkg=./adapters/repos/... -coverprofile=coverage-integration.txt -race -tags=$tags "$@" ./adapters/repos/... | ||
| go test -count 1 -race -tags=$tags "$@" ./usecases/classification/... | ||
| go test -v -count 1 -coverpkg=./adapters/repos/... -coverprofile=coverage-integration.txt -race -tags=$tags "$@" ./adapters/repos/... | ||
| go test -v -count 1 -race -tags=$tags "$@" ./usecases/classification/... |
There was a problem hiding this comment.
I think we should remove -v or better maybe introduce a param for example --verbose that would add this -v if needed
Changes
Introduce shard API, with endpoints:
GET /v1/schema/{className}/shardsPUT /v1/schema/{className}/shards/{shardName}These endpoints facilitate marking a shard as
READONLY, which is useful in situations where the shard must be immutable; taking a disk snapshot for example.Under the hood, if a shard is marked
READONLY, segment compaction and memtable flushing are halted until the shard is reset toREADY. In this case, Weaviate will emit logs indicating that a compaction would have occurred, but was skipped due to the current shard status.Fixes #1839