feat: greptimedb data bridge

[killme2008]

Fixes #10647

Summary

🤖 Generated by Copilot at e850a15

This pull request adds a new bridge type greptimedb to the emqx_bridge application, which enables emqx to send messages to a greptimedb database via gRPC. It also adds a new application emqx_bridge_greptimedb, which implements the bridge logic, the configuration schema, the api reference, and the unit tests for the greptimedb bridge type. It also adds the docker-compose files, the rebar.config file, and the i18n files to support the new bridge type.

PR Checklist

Please convert it to a draft if any of the following conditions are not met. Reviewers may skip over until all the items are checked:

• Added tests for the changes
• Changed lines covered in coverage report
• Change log has been added to changes/(ce|ee)/(feat|perf|fix)-<PR-id>.en.md files
• For internal contributor: there is a jira ticket to track this change
• If there should be document changes, a PR to emqx-docs.git is sent, or a jira ticket is created to follow up
• Schema changes are backward compatible

Checklist for CI (.github/workflows) changes

• If changed package build workflow, pass this action (manual trigger)
• Change log has been added to changes/ dir for user-facing artifacts update

[lafirest]

A cool work

[id]

@killme2008 thanks a lot for your contribution!

Could you have a look at CI checks failures and try to fix them? Also please let us know if you don't know how to fix something.

[killme2008]

@killme2008 thanks a lot for your contribution!

Could you have a look at CI checks failures and try to fix them? Also please let us know if you don't know how to fix something.

Thank you. I am fixing these failures and adding more tests. So it's still a draft, I'll request your review when it's ready.

[killme2008]

@id Hi, the code is ready for review. Would you please approve the Continuous Integration (CI) to run? Then I will investigate the failures in the CI checks and address them accordingly. Thank you.

[id]

@killme2008 CI is running

[killme2008]

@id @lafirest Hi, dear EMQX developers, this PR is ready for code review. Please take the time to review it. Thank you

[killme2008]

All of the new tests have successfully passed:

TEST INFO: 2 test(s), 29 case(s) in 2 suite(s)

Testing lib.emqx_bridge_greptimedb.emqx_bridge_greptimedb_SUITE: Starting test, 28 test cases
Config override: log level is set to 'warning'
Listener http:dashboard on :18083 started.
%%% emqx_bridge_greptimedb_SUITE ==> with_batch.sync_query.grpcv1_tcp.t_authentication_error_on_send_message: OK
%%% emqx_bridge_greptimedb_SUITE ==> with_batch.sync_query.grpcv1_tcp.t_bad_timestamp: OK
%%% emqx_bridge_greptimedb_SUITE ==> with_batch.sync_query.grpcv1_tcp.t_boolean_variants: OK
%%% emqx_bridge_greptimedb_SUITE ==> with_batch.sync_query.grpcv1_tcp.t_const_timestamp: OK
%%% emqx_bridge_greptimedb_SUITE ==> with_batch.sync_query.grpcv1_tcp.t_create_disconnected: OK
%%% emqx_bridge_greptimedb_SUITE ==> with_batch.sync_query.grpcv1_tcp.t_get_status: OK
%%% emqx_bridge_greptimedb_SUITE ==> with_batch.sync_query.grpcv1_tcp.t_missing_field: OK
%%% emqx_bridge_greptimedb_SUITE ==> with_batch.sync_query.grpcv1_tcp.t_start_already_started: OK
%%% emqx_bridge_greptimedb_SUITE ==> with_batch.sync_query.grpcv1_tcp.t_start_error: OK
%%% emqx_bridge_greptimedb_SUITE ==> with_batch.sync_query.grpcv1_tcp.t_start_exception: OK
%%% emqx_bridge_greptimedb_SUITE ==> with_batch.sync_query.grpcv1_tcp.t_start_ok: OK
%%% emqx_bridge_greptimedb_SUITE ==> with_batch.sync_query.grpcv1_tcp.t_start_ok_no_subject_tags_write_syntax: OK
%%% emqx_bridge_greptimedb_SUITE ==> with_batch.sync_query.grpcv1_tcp.t_start_ok_timestamp_write_syntax: OK
%%% emqx_bridge_greptimedb_SUITE ==> with_batch.sync_query.grpcv1_tcp.t_write_failure: OK
%%% emqx_bridge_greptimedb_SUITE ==> without_batch.sync_query.grpcv1_tcp.t_authentication_error_on_send_message: OK
%%% emqx_bridge_greptimedb_SUITE ==> without_batch.sync_query.grpcv1_tcp.t_bad_timestamp: OK
%%% emqx_bridge_greptimedb_SUITE ==> without_batch.sync_query.grpcv1_tcp.t_boolean_variants: OK
%%% emqx_bridge_greptimedb_SUITE ==> without_batch.sync_query.grpcv1_tcp.t_const_timestamp: OK
%%% emqx_bridge_greptimedb_SUITE ==> without_batch.sync_query.grpcv1_tcp.t_create_disconnected: OK
%%% emqx_bridge_greptimedb_SUITE ==> without_batch.sync_query.grpcv1_tcp.t_get_status: OK
%%% emqx_bridge_greptimedb_SUITE ==> without_batch.sync_query.grpcv1_tcp.t_missing_field: OK
%%% emqx_bridge_greptimedb_SUITE ==> without_batch.sync_query.grpcv1_tcp.t_start_already_started: OK
%%% emqx_bridge_greptimedb_SUITE ==> without_batch.sync_query.grpcv1_tcp.t_start_error: OK
%%% emqx_bridge_greptimedb_SUITE ==> without_batch.sync_query.grpcv1_tcp.t_start_exception: OK
%%% emqx_bridge_greptimedb_SUITE ==> without_batch.sync_query.grpcv1_tcp.t_start_ok: OK
%%% emqx_bridge_greptimedb_SUITE ==> without_batch.sync_query.grpcv1_tcp.t_start_ok_no_subject_tags_write_syntax: OK
%%% emqx_bridge_greptimedb_SUITE ==> without_batch.sync_query.grpcv1_tcp.t_start_ok_timestamp_write_syntax: OK
%%% emqx_bridge_greptimedb_SUITE ==> without_batch.sync_query.grpcv1_tcp.t_write_failure: OK
Stop listener http:dashboard on :18083 successfully.
Testing lib.emqx_bridge_greptimedb.emqx_bridge_greptimedb_SUITE: TEST COMPLETE, 28 ok, 0 failed of 28 test cases

Testing lib.emqx_bridge_greptimedb.emqx_bridge_greptimedb_connector_SUITE: Starting test, 1 test cases
%%% emqx_bridge_greptimedb_connector_SUITE ==> t_lifecycle: OK
Testing lib.emqx_bridge_greptimedb.emqx_bridge_greptimedb_connector_SUITE: TEST COMPLETE, 1 ok, 0 failed of 1 test cases

Updating /Users/dennis/programming/erlang/emqx/_build/test/logs/index.html ... done
Updating /Users/dennis/programming/erlang/emqx/_build/test/logs/all_runs.html ... done
[os_mon] memory supervisor port (memsup): Erlang has closed
[os_mon] cpu supervisor port (cpu_sup): Erlang has closed

[killme2008]

@lafirest I resolved almost all of the comments, and some comments not sure are reserved. PTAL, thank u.

[id]

@killme2008 one of the test cases is failing in the new bridge test suite even after re-run could you take a look?
Also, could you rebase on the latest master?

[killme2008]

@id I am currently investigating the issue. It is strange that I am unable to reproduce it on my local machine. It seems that the greptimedb client is unable to establish a connection to the database in the CI docker container. I appreciate your reminder and will rebase the master later. Thank you.

[coveralls]

Pull Request Test Coverage Report for Build 5640738765

• 265 of 288 (92.01%) changed or added relevant lines in 3 files are covered.
• 18 unchanged lines in 8 files lost coverage.
• Overall coverage increased (+0.1%) to 81.581%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
apps/emqx_bridge_greptimedb/src/emqx_bridge_greptimedb.erl	87	92	94.57%
apps/emqx_bridge_greptimedb/src/emqx_bridge_greptimedb_connector.erl	173	191	90.58%

Files with Coverage Reduction	New Missed Lines	%
apps/emqx_ft/src/emqx_ft_storage_exporter_fs.erl	1	80.43%
apps/emqx_management/src/emqx_mgmt_util.erl	1	13.79%
apps/emqx/src/emqx_banned.erl	1	89.55%
apps/emqx/src/emqx_cm.erl	1	89.47%
apps/emqx_utils/src/emqx_utils.erl	1	82.63%
apps/emqx_dashboard/src/emqx_dashboard_monitor.erl	2	64.83%
apps/emqx_resource/src/emqx_resource_buffer_worker.erl	3	93.4%
apps/emqx/src/emqx_reason_codes.erl	8	93.38%

Totals
Change from base Build 5625410312:	0.1%
Covered Lines:	31253
Relevant Lines:	38309

---

💛 - Coveralls

[killme2008]

Thanks to @lafirest help and after some investigations, the CI failure is fixed. @id

[id]

@killme2008 looks like you still need to include this commit into your PR? 8439ce0 (#11334)

[killme2008]

@id Good catch! thank u! Looks like i missed this commit when rebasing with master.

[killme2008]

@id I misunderstood. @lafirest Already create another PR #11334

I closed this PR and let's discuss in that PR.