Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Elasticsearch: Fix resource calls for paths that include : #82327

Merged
merged 4 commits into from
Feb 13, 2024
Merged

Elasticsearch: Fix resource calls for paths that include : #82327

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
0438c37
Elasticsearch: Fix resource calls for paths that include :
ivanahuckova Feb 9, 2024
1bdd02c
Add tests
ivanahuckova Feb 12, 2024
bbacb56
Add test case and comment
ivanahuckova Feb 13, 2024
78571e5
Remove redundant comment
ivanahuckova Feb 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
24 changes: 12 additions & 12 deletions pkg/tsdb/elasticsearch/elasticsearch.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -205,21 +205,11 @@ func (s *Service) CallResource(ctx context.Context, req *backend.CallResourceReq
return err
}

esUrl, err := url.Parse(ds.URL)
if err != nil {
logger.Error("Failed to parse data source URL", "error", err, "url", ds.URL)
return err
}

resourcePath, err := url.Parse(req.Path)
esUrl, err := createElasticsearchURL(req, ds)
if err != nil {
logger.Error("Failed to parse data source path", "error", err, "url", req.Path)
return err
logger.Error("Failed to create request url", "error", err, "url", ds.URL, "path", req.Path)
}

// We take the path and the query-string only
esUrl.RawQuery = resourcePath.RawQuery
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, are we sure we never need or want to support query parameters in resources calls? Because I think some APIs in ES use GET query parameters, so I think we should also add this to createElasticsearchURL. WDYT?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resource path should not have any RawQuery based on following check req.Path != "" && !strings.HasSuffix(req.Path, "/_mapping") && req.Path != "_msearch" && req.Path != "_mapping". So it is not needed as there are currently no queries supported. Maybe I can add a comment explaining it - so when someone allows additional path/resource calls, they are aware of needing to implement raw query.

I also noticed that I haven't add test for why we are doing it - so I will also add test for path with : to ensure that it is covered.

But to summarise we can't use url.Parse(req.Path) anymore as it throws error for path that includes : , so I am worried, that if we wanted to go 1 step ahead and implement raw queries (even if it is not needed), we might introduce some issue related to splitting on ?. Or do you have any other ideas.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bbacb56

esUrl.Path = path.Join(esUrl.Path, resourcePath.Path)
request, err := http.NewRequestWithContext(ctx, req.Method, esUrl.String(), bytes.NewBuffer(req.Body))
if err != nil {
logger.Error("Failed to create request", "error", err, "url", esUrl.String())
Expand Down Expand Up @@ -269,3 +259,13 @@ func (s *Service) CallResource(ctx context.Context, req *backend.CallResourceReq
Body: body,
})
}

func createElasticsearchURL(req *backend.CallResourceRequest, ds *es.DatasourceInfo) (*url.URL, error) {
esUrl, err := url.Parse(ds.URL)
if err != nil {
return nil, fmt.Errorf("failed to parse data source URL: %s, error: %w", ds.URL, err)
}

esUrl.Path = path.Join(esUrl.Path, req.Path)
return esUrl, nil
}
28 changes: 28 additions & 0 deletions pkg/tsdb/elasticsearch/elasticsearch_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import (
"github.com/stretchr/testify/require"

"github.com/grafana/grafana/pkg/infra/httpclient"
es "github.com/grafana/grafana/pkg/tsdb/elasticsearch/client"
)

type datasourceInfo struct {
Expand Down Expand Up @@ -71,3 +72,30 @@ func TestNewInstanceSettings(t *testing.T) {
})
})
}

func TestCreateElasticsearchURL(t *testing.T) {
tt := []struct {
name string
settings es.DatasourceInfo
req backend.CallResourceRequest
expected string
}{
{name: "with /_msearch path and valid url", settings: es.DatasourceInfo{URL: "http://localhost:9200"}, req: backend.CallResourceRequest{Path: "_msearch"}, expected: "http://localhost:9200/_msearch"},
{name: "with _msearch path and valid url", settings: es.DatasourceInfo{URL: "http://localhost:9200"}, req: backend.CallResourceRequest{Path: "_msearch"}, expected: "http://localhost:9200/_msearch"},
{name: "with _msearch path and valid url with /", settings: es.DatasourceInfo{URL: "http://localhost:9200/"}, req: backend.CallResourceRequest{Path: "_msearch"}, expected: "http://localhost:9200/_msearch"},
{name: "with _mapping path and valid url", settings: es.DatasourceInfo{URL: "http://localhost:9200"}, req: backend.CallResourceRequest{Path: "/_mapping"}, expected: "http://localhost:9200/_mapping"},
{name: "with /_mapping path and valid url", settings: es.DatasourceInfo{URL: "http://localhost:9200"}, req: backend.CallResourceRequest{Path: "/_mapping"}, expected: "http://localhost:9200/_mapping"},
{name: "with /_mapping path and valid url with /", settings: es.DatasourceInfo{URL: "http://localhost:9200/"}, req: backend.CallResourceRequest{Path: "/_mapping"}, expected: "http://localhost:9200/_mapping"},
{name: "with abc/_mapping path and valid url", settings: es.DatasourceInfo{URL: "http://localhost:9200"}, req: backend.CallResourceRequest{Path: "abc/_mapping"}, expected: "http://localhost:9200/abc/_mapping"},
{name: "with /abc/_mapping path and valid url", settings: es.DatasourceInfo{URL: "http://localhost:9200"}, req: backend.CallResourceRequest{Path: "abc/_mapping"}, expected: "http://localhost:9200/abc/_mapping"},
{name: "with /abc/_mapping path and valid url", settings: es.DatasourceInfo{URL: "http://localhost:9200/"}, req: backend.CallResourceRequest{Path: "abc/_mapping"}, expected: "http://localhost:9200/abc/_mapping"},
}

for _, test := range tt {
t.Run(test.name, func(t *testing.T) {
url, err := createElasticsearchURL(&test.req, &test.settings)
require.NoError(t, err)
require.Equal(t, test.expected, url.String())
})
}
}