Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat msp storage switch to es #2861

Merged
merged 10 commits into from
Nov 5, 2021

Conversation

li2329800
Copy link
Contributor

@li2329800 li2329800 commented Nov 4, 2021

What type of this PR

Add one of the following kinds:
/kind feature
/kind bugfix

What this PR does / why we need it:

Which issue(s) this PR fixes:

Specified Reviewers:

/assign @liuhaoyang @recallsong

ChangeLog

Language Changelog
🇺🇸 English 1.Fix the bug that span lost in msp platform, detail2849, 2820
2. Support ES as beckend storage in msp platform
🇨🇳 中文 1.合并msp平台span丢失问题的修复,具体见这两个pr2849, 2820
2.实现了msp平台后端存储对于ES的支持

Need cherry-pick to release versions?

Need cherry-pick to release/1.4

@codecov
Copy link

codecov bot commented Nov 4, 2021

Codecov Report

Merging #2861 (1f0a424) into master (edad9a1) will increase coverage by 0.17%.
The diff coverage is 21.88%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2861      +/-   ##
==========================================
+ Coverage   17.09%   17.27%   +0.17%     
==========================================
  Files        1292     1301       +9     
  Lines      131580   132388     +808     
==========================================
+ Hits        22498    22873     +375     
- Misses     106759   107151     +392     
- Partials     2323     2364      +41     
Impacted Files Coverage Δ
...sp/apm/exception/query/error_event_list_storage.go 0.00% <0.00%> (ø)
...ules/msp/apm/exception/query/error_list_storage.go 0.00% <0.00%> (ø)
modules/msp/apm/trace/query/mock_storage.go 0.00% <0.00%> (ø)
...les/msp/apm/trace/storage/cassandra_v1/provider.go 17.24% <ø> (ø)
.../msp/apm/exception/query/mock_entity_query_grpc.go 15.00% <15.00%> (ø)
modules/msp/apm/exception/query/provider.go 20.00% <16.66%> (ø)
modules/msp/apm/trace/query/trace.service.go 30.49% <24.65%> (ø)
...dules/msp/apm/exception/query/exception.service.go 25.10% <25.10%> (ø)
modules/msp/apm/trace/query/span_list_storage.go 35.89% <35.89%> (ø)
...s/msp/apm/exception/query/mock_event_query_grpc.go 44.11% <44.11%> (ø)
... and 22 more

indices := p.Loader.Indices(ctx, sel.StartTime, sel.EndTime, loader.KeyPath{
Recursive: true,
})
fmt.Println(indices)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove it

"github.com/erda-project/erda/modules/msp/apm/exception/erda-error/storage"
)

// Data .
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Error

// Data .
type Error = interface{}

// ListIterator .
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ErrorListIterator

}

// NewListIterator .
func NewErrorListIterator(list ...Error) storekit.Iterator {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NewErrorListIterator

}

func (it *scrollIterator) fetch(dir iteratorDir) error {
fmt.Println(it.indices)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove it

@@ -90,7 +90,7 @@ func (p *provider) Close() error {
func init() {
servicehub.Register("trace-storage", &servicehub.Spec{
Services: []string{"trace-storage"},
Dependencies: []string{"kafka", "cassandra", "kafka.topic.initializer"},
Dependencies: []string{"kafka", "cassandra"},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why delete kafka.topic.initializer

@sfwn
Copy link
Member

sfwn commented Nov 5, 2021

Adjust your changelog into table please.

@recallsong
Copy link
Contributor

/approve

@erda-bot erda-bot merged commit ff66051 into erda-project:master Nov 5, 2021
@li2329800 li2329800 added the msp microsevice platform label Nov 5, 2021
@li2329800
Copy link
Contributor Author

/cherry-pick release/1.4

erda-bot pushed a commit to erda-bot/erda that referenced this pull request Nov 5, 2021
* feature: msp storage switch to es

* bugifx: msp fix queryspanbytraceid bug

* feature-msp-storage-support-es&bugfix-msp-span-lost

* fix ci go-import order problem

* fix queryspancountbytraceid bug

* fix code review problem from recallsong

* fix ci check-imports problem

* fix ci fmt problem

* fix ci fmt problem again, sadly

* change span es index shards default setting to 1
erda-bot added a commit that referenced this pull request Nov 5, 2021
* feature: msp storage switch to es

* bugifx: msp fix queryspanbytraceid bug

* feature-msp-storage-support-es&bugfix-msp-span-lost

* fix ci go-import order problem

* fix queryspancountbytraceid bug

* fix code review problem from recallsong

* fix ci check-imports problem

* fix ci fmt problem

* fix ci fmt problem again, sadly

* change span es index shards default setting to 1

Co-authored-by: li2329800 <9552289+li2329800@users.noreply.github.com>
li2329800 added a commit to li2329800/erda that referenced this pull request Nov 8, 2021
li2329800 added a commit to li2329800/erda that referenced this pull request Nov 8, 2021
li2329800 added a commit to li2329800/erda that referenced this pull request Nov 9, 2021
liuhaoyang pushed a commit that referenced this pull request Nov 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved msp microsevice platform
Development

Successfully merging this pull request may close these issues.

None yet

4 participants