Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prometheus-sd add namespace and service api #10662

Open
Joey777210 opened this issue Jun 18, 2023 · 15 comments
Open

Prometheus-sd add namespace and service api #10662

Joey777210 opened this issue Jun 18, 2023 · 15 comments
Labels
area/Document Category issues or prs related to document. good first issue Good for newcomers plugin
Milestone

Comments

@Joey777210
Copy link
Contributor

Is your feature request related to a problem? Please describe.
If use Nacos for Prometheus http sd. Prometheus will get all services in Nacos. But in many scene, Nacos is use by a lot system.
So prometheus sd support Namespace and service dimention insulate are important.

Describe the solution you'd like
Add two api.

Describe alternatives you've considered
Add two interfaces, and specify the Namespace and the service

@KomachiSion
Copy link
Collaborator

Is prometheus sd protocol support specify the Namespace and the service?

@Joey777210
Copy link
Contributor Author

Joey777210 commented Jun 19, 2023

Actually, prometheus do not support Nacos as service discovery directly.
Currently, we can only use Prometheus Http SD config for Nacos, see this issue: prometheus/prometheus#7539 . And prometheus don't really care what api path you give, so we can config namespace and service through path variable.

Prometheus http sd: https://prometheus.io/docs/prometheus/latest/http_sd/

@985492783
Copy link
Contributor

我的想法是做一个@PrometheusParam的注解,然后value可以是${namespace}|${group}的占位符,用AspectJ去做切面替换占位符,然后以参数形式注入到不同的prometheus监控中,这样别的接口也可以使用,扩展起来方便,是否可行 @KomachiSion

@KomachiSion
Copy link
Collaborator

我的想法是做一个@PrometheusParam的注解,然后value可以是${namespace}|${group}的占位符,用AspectJ去做切面替换占位符,然后以参数形式注入到不同的prometheus监控中,这样别的接口也可以使用,扩展起来方便,是否可行 @KomachiSion

没必要这么复杂, 如果prometheus sd 不关心具体的uri是什么,那完全可以提供全量的,命名空间级别,服务级别的接口精确控制, 甚至应该限制使用命名空间级别,服务级别,减少全量接口的使用。

@KomachiSion
Copy link
Collaborator

Actually, prometheus do not support Nacos as service discovery directly. Currently, we can only use Prometheus Http SD config for Nacos, see this issue: prometheus/prometheus#7539 . And prometheus don't really care what api path you give, so we can config namespace and service through path variable.

Prometheus http sd: https://prometheus.io/docs/prometheus/latest/http_sd/

I don't know much about the prometheus sd protocol. From the documentation, I can only see that target hosts are required to be configured, so I always thought it was a fixed uri. Not sure how to configure the uri.

If it is the uri that can be customized, then specifying the namespace and serviceName should be the recommended practice

@985492783
Copy link
Contributor

我的想法是做一个@PrometheusParam的注解,然后value可以是${namespace}|${group}的占位符,用AspectJ去做切面替换占位符,然后以参数形式注入到不同的prometheus监控中,这样别的接口也可以使用,扩展起来方便,是否可行 @KomachiSion

没必要这么复杂, 如果prometheus sd 不关心具体的uri是什么,那完全可以提供全量的,命名空间级别,服务级别的接口精确控制, 甚至应该限制使用命名空间级别,服务级别,减少全量接口的使用。

get

@Joey777210
Copy link
Contributor Author

Actually, prometheus do not support Nacos as service discovery directly. Currently, we can only use Prometheus Http SD config for Nacos, see this issue: prometheus/prometheus#7539 . And prometheus don't really care what api path you give, so we can config namespace and service through path variable.
Prometheus http sd: https://prometheus.io/docs/prometheus/latest/http_sd/

I don't know much about the prometheus sd protocol. From the documentation, I can only see that target hosts are required to be configured, so I always thought it was a fixed uri. Not sure how to configure the uri.

If it is the uri that can be customized, then specifying the namespace and serviceName should be the recommended practice

sry, posted wrong doc, see this: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#http_sd_config

@KomachiSion
Copy link
Collaborator

Actually, prometheus do not support Nacos as service discovery directly. Currently, we can only use Prometheus Http SD config for Nacos, see this issue: prometheus/prometheus#7539 . And prometheus don't really care what api path you give, so we can config namespace and service through path variable.
Prometheus http sd: https://prometheus.io/docs/prometheus/latest/http_sd/

I don't know much about the prometheus sd protocol. From the documentation, I can only see that target hosts are required to be configured, so I always thought it was a fixed uri. Not sure how to configure the uri.
If it is the uri that can be customized, then specifying the namespace and serviceName should be the recommended practice

sry, posted wrong doc, see this: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#http_sd_config

So when users config __meta_url for each target, prometheus will call the specifed url replace the fixed one?

@Joey777210
Copy link
Contributor Author

Actually, prometheus do not support Nacos as service discovery directly. Currently, we can only use Prometheus Http SD config for Nacos, see this issue: prometheus/prometheus#7539 . And prometheus don't really care what api path you give, so we can config namespace and service through path variable.
Prometheus http sd: https://prometheus.io/docs/prometheus/latest/http_sd/

I don't know much about the prometheus sd protocol. From the documentation, I can only see that target hosts are required to be configured, so I always thought it was a fixed uri. Not sure how to configure the uri.
If it is the uri that can be customized, then specifying the namespace and serviceName should be the recommended practice

sry, posted wrong doc, see this: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#http_sd_config

So when users config __meta_url for each target, prometheus will call the specifed url replace the fixed one?

I think users should config url.

@KomachiSion
Copy link
Collaborator

I don't found the config url, can you provide a demo configuration?

What's more, the usage of prometheus sd document is leak. Would you mind try to add the docuement of usage after the issue completed?

@Joey777210
Copy link
Contributor Author

Joey777210 commented Jun 29, 2023

I don't found the config url, can you provide a demo configuration?

What's more, the usage of prometheus sd document is leak. Would you mind try to add the docuement of usage after the issue completed?

Here is a demo

- job_name: 'http_sd_example'
  metrics_path: /metrics
  scheme: http
  http_sd_configs:
  - url: https://example.com/targets.json
    timeout: 5s

And for document,for sure I will write one.

@KomachiSion
Copy link
Collaborator

OK, I understand, I will review the PR and comment in PR some request changes. If no, PR will be merged.

After this, Please help community add an usage document in nacos-group/

@KomachiSion KomachiSion added kind/enhancement Category issues or prs related to enhancement. plugin labels Jun 30, 2023
@KomachiSion KomachiSion added this to the 2.3.0 milestone Jun 30, 2023
KomachiSion pushed a commit that referenced this issue Jul 17, 2023
* add apis

* add tests

* fix checkstyle

* param namespace to namespaceId

* namespace to namespaceId

* fix test case

* fix test case
@KomachiSion
Copy link
Collaborator

PR merged, Please help to submit usage document.

@Joey777210
Copy link
Contributor Author

@KomachiSion I will write a document after solving this security issue #10752

realJackSun added a commit that referenced this issue Aug 10, 2023
…lop branch (#10942)

* [ISSUE #10734] Implement http request param check filter and http param extractors (#10758)

* For #10734,Implement grpc server interceptor and grpc param extractors

* For #10734,add unit test for grpc server interceptor and grpc param extractors

* For #10734,alter the test case

* For #10734,delete the ConnectionSetupRequestParamExtractor

* For #10734,add the naming http request param check filter and implement the naming http request param extractors

* For #10734,add unit test for naming http request param extractors

* For #10734,Implement grpc server interceptor and grpc param extractors

* For #10734,add unit test for grpc server interceptor and grpc param extractors

* For #10734,delete the ConnectionSetupRequestParamExtractor

* For #10734,add the naming http request param check filter and implement the naming http request param extractors

* For #10734,add unit test for naming http request param extractors

* For #10734,add the config http request param check filter and implement the config http request param extractors and unit test

* For #10734,add the console http request param check filter and implement the console http request param extractors and unit test

* For #10734,fix code style

* For #10734,alter the logic of exception handle in filter

* For #10734,fix code style

* dump change check task submit (#10755)

* dump change check task submit

* delete config nid convert error fix

* fix test case

* checkstyle

* Refactor grpc tls (#10759)

* Move Tls negotiator to GrpcSdkServer.

* use protocol negotiator builder replace directly create.

* use SPI load negotiator and set tls as default negotiator.

* Remove tlsconfig in BaseRpcServer.

* Add some ut.

* For checkstyle.

* fix word spelling in `AuthenticationManagerDelegator` (#10777)

* fix(#10427): When the execution of handleServerRequest() encounters an exception, record the log and throw an exception, then quickly response to the server errResponse (#10770)

* fix(#10585): selectInstances and selectOneHealthyInstance methods, if the parameter subscribe is true. Subscription is required when clientProxy.isSubscribe() is false. (#10805)

* disable check port input when use registered port (#10799)

* Add the handle to overload connection (#10783)

* add the handle to overload connection

* fast return

* [ISSUE #10662]Prometheus-sd add namespace and service api (#10663)

* add apis

* add tests

* fix checkstyle

* param namespace to namespaceId

* namespace to namespaceId

* fix test case

* fix test case

* Service instance should display related color when healthy or unhealthy (#10811)

* fix a couple of invaild props (#10810)

* feat(#10539): When the operation configuration fails, log. (#10804)

* [ISSUE #10744]feat:Add HealthControllerV2 and HealthControllerV2Test (#10786)

* [ISSUE #10744]feat:Add HealthControllerV2 and HealthControllerV2Test

* fix:V2 api return Result

* [ISSUE #10734] Refactor the ParamChecker and ParamExtractor (#10775)

* For #10734,refactor the paramextractor and ParamChecker

* For #10734,alter the rules of ParamCheck

* For #10734,alter the rules of ParamCheck

* For #10734,fix bug

* For #10734,fix bug and alter the ParamCheckRules.java

* For #10734,fix code style

* For #10734,fix the param check rules

* For #10734,implement the server param check config

* For #10734,optimize the logic

* For #10734,optimize the logic

* For #10734,optimize the logic

* Refactor Prometheus Module (#10827)

* Refactor Prometheus Module

* Complete Test Case

* format

* format

* For #10734,fix the param check rule (#10826)

* [ISSUE #10792]When nacos client use endpoint, the registration center should support configuring context-path and cluster-name like the configuration center (#10793)

* Reactor code in datasource-plugin (#10791)

* Reactor code in datasource-plugin

* Fix Abstract Mapper Test Case

* Add Empty Check

* Fix Checkstyle

* fix checkstyle

* fix check style

* fix check style

* Fix CheckStyle

* Fix SQL Blank

* bugfix for PersistentClientOperationServiceImpl log (#10825)

* For #10734,fix the param check rule (#10858)

* fix(#10831): When using the deregisterInstance method to remove one of multiple instances registered by batchRegisterInstance, all instances registered by batchRegisterInstance will be removed. (#10836)

* UnsupportedFeatureError (#10860)

* fix(distro): fix issue#10880. (#10881)

* feat(#5608 && #10223): When the custom instance id is empty, the id will be automatically generated. (#10812)

* fix: test-code branch (#10904)

* add nacos ci

* delete client version of nacos ci

* fix: test-code branch

* console-ui 新增 toml 语言支持,修复之前 properties 主题未生效的问题 (#10896)

* console-ui 新增 toml 语言支持,修复之前 properties 主题未生效的问题

* console-ui 新增 toml 语言支持,修复之前 properties 主题未生效的问题

* 老规矩,要编译一波

* feat(10891): Provide a configuration item for the maximum number of push retries, instead of directly hardcoding it to 50 times in the code. (#10895)

* [ISSUE #10824] Remove udp port param for v1-client (#10914)

* Remove UDP Param

* Fix gRPC client

* fix test case

* fix test case

* fix test case

* fix test case

* Fix login failed when close auth.

---------

Co-authored-by: Sunrisea <49605583+Sunrisea@users.noreply.github.com>
Co-authored-by: nov.lzf <liuzunfei@gmail.com>
Co-authored-by: 杨翊 SionYang <xiweng.yy@alibaba-inc.com>
Co-authored-by: ZhangShenao <15201440436@163.com>
Co-authored-by: blake.qiu <46370663+Bo-Qiu@users.noreply.github.com>
Co-authored-by: Joey777210 <53630996+Joey777210@users.noreply.github.com>
Co-authored-by: chenyiqin <83362909+Daydreamer-ia@users.noreply.github.com>
Co-authored-by: maoling <11016631+maoling@users.noreply.github.com>
Co-authored-by: DiligenceLai <96920907+DiligenceLai@users.noreply.github.com>
Co-authored-by: huhongjie2014 <139250842+huhongjie2014@users.noreply.github.com>
Co-authored-by: lu-xiaoshuang <121755080+lu-xiaoshuang@users.noreply.github.com>
Co-authored-by: zt9788 <zt9788@users.noreply.github.com>
Co-authored-by: 阿魁 <670569467@qq.com>
Co-authored-by: wuyfee <30968107+Wuyunfan-BUPT@users.noreply.github.com>
Co-authored-by: Darren Luo <jn296@qq.com>
Co-authored-by: xxc <xchaos8@126.com>
@KomachiSion KomachiSion modified the milestones: 2.3.0, 2.3.1 Dec 5, 2023
@KomachiSion KomachiSion modified the milestones: 2.3.1, 2.4.0 Feb 27, 2024
@KomachiSion KomachiSion added the area/Document Category issues or prs related to document. label Feb 27, 2024
@KomachiSion
Copy link
Collaborator

This issue only waiting the relative usage document, Any community contributors prefer to contribute?

@KomachiSion KomachiSion removed kind/enhancement Category issues or prs related to enhancement. kind/discussion Category issues related to discussion labels Feb 27, 2024
@KomachiSion KomachiSion added the good first issue Good for newcomers label Feb 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/Document Category issues or prs related to document. good first issue Good for newcomers plugin
Projects
None yet
Development

No branches or pull requests

3 participants