Skip to content

Conversation

@morningman
Copy link
Contributor

@morningman morningman commented Jul 28, 2022

Proposed changes

Issue Number: close #xxx

Problem Summary

This PR refreshes Doris' current metrics with additions, deletions, and renaming. and added documentation.
The changes are as follows:

  1. Delete useless modules

    1. BE
      _file_cache,_index_stream_lru_cache 和 _etl_thread_pool
  2. Added Metrics

    1. FE:

      doris_fe_txn_status{type="xxx"} 
      
  3. Deleted Metrics

    1. FE:
      doris_fe_query_begin
      doris_fe_load_add
      doris_fe_partition_all
      doris_fe_load_finished
      jvm_direct_buffer_pool
      
    2. BE
      doris_be_push_request_write_bytes_per_second
      doris_be_streaming_load_bytes
      doris_be_segment_rows_by_short_key
      doris_be_segment_rows_read_by_zone_map
      doris_be_http_requests_total
      doris_be_http_request_send_bytes
      doris_be_txn_exec_plan_total
      
  4. Renamed Metrics

    1. FE
      pending_load_task_scheduler_pool -> pending-load-task-scheduler-pool
      loading_load_task_scheduler_pool -> loading-load-task-scheduler-pool
      Load submitter -> load-submitter
      export_pending_job_pool -> export-pending-job-pool
      export_exporting_job_pool -> export-exporting-job-pool
      load_pending_thread_num_normal_priority_pool -> load-pending-thread-num-normal-priority-pool
      load_etl_thread_num_normal_priority_pool -> load-etl-thread-num-normal-priority-pool
      Metric-Timer-Pool -> metric-timer-pool
      doris-mysql-nio-pool -> mysql-nio-pool
      Connect-Scheduler-Check-Timer -> connect-scheduler-check-timer
      cache_mode_sql -> cache_added{type="sql"}
      query_mode_partition -> cache_added{type="partition"}
      cache_hit_sql -> cache_hit(type="sql")
      cache_hit_partition -> cache_hit(type="partition")
      edit_log_write -> edit_log{type="write"}
      edit_log_read -> edit_logP{type="read"}
      txn_reject -> txn_counter{type="reject"}
      txn_begin -> txn_counter{type="begin"}
      txn_success -> txn_counter{type="success"}
      txn_failed -> txn_counter{type="failed"}
      
    2. BE
      doris_be_capacity -> doris_be_cache_capacity
      doris_be_usage -> doris_be_cache_usage
      doris_be_usage_ratio -> doris_be_cache_usage_ratio
      doris_be_lookup_count -> doris_be_cache_lookup_count
      doris_be_hit_count -> doris_be_cache_hit_count
      doris_be_hit_ratio -> doris_be_cache_hit_ratio
      txn_begin_request_total -> stream_load_txn_begin_request_total
      txn_commit_request_total -> stream_load_txn_commit_request_total
      txn_rollback_request_total -> stream_load_txn_rollback_request_total
      segment_total_read_times -> segment_read_total
      segment_total_row_num -> segment_row_total
      
  5. Others

    1. The FE side distinguishes between process monitoring and machine monitoring. Process monitoring uses the doris_fe_ prefix. Machine monitoring uses the system_ prefix.
    2. For metrics in json format on the FE side, a prefix (doris_fe or system) is added before the name, which is consistent with the text mode.

Checklist(Required)

  1. Type of your changes:
    • Improvement
    • Fix
    • Feature-WIP
    • Feature
    • Doc
    • Refator
    • Others:
  2. Does it affect the original behavior:
    • Yes: Some of metrics' name has been changed
    • No
    • I don't know
  3. Has unit tests been added:
    • Yes
    • No
    • No Need
  4. Has document been added or modified:
    • Yes
    • No
    • No Need
  5. Does it need to update dependencies:
    • Yes
    • No
  6. Are there any changes that cannot be rolled back:
    • Yes
    • No

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@morningman morningman added area/metrics Issues or PRs related to the metrics kind/refactor Issues or PRs to refactor code doc-1.1.x labels Jul 28, 2022
@github-actions github-actions bot added area/load Issues or PRs related to all kinds of load area/planner Issues or PRs related to the query planner area/routine load kind/docs Categorizes issue or PR as related to documentation. area/vectorization labels Jul 28, 2022
@morningman morningman changed the title [doc][refactor](metrics) Reorganize FE metrics and add document [doc][refactor](metrics) Reorganize FE and BE metrics and add document Jul 29, 2022
Copy link
Contributor

@yiguolei yiguolei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yiguolei yiguolei merged commit abbf75d into apache:master Aug 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/load Issues or PRs related to all kinds of load area/metrics Issues or PRs related to the metrics area/planner Issues or PRs related to the query planner area/routine load area/vectorization doc-1.1.x kind/docs Categorizes issue or PR as related to documentation. kind/refactor Issues or PRs to refactor code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants