Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

An occasional problem with the generate component #2244

Closed
gaoshenga opened this issue Nov 15, 2023 · 3 comments
Closed

An occasional problem with the generate component #2244

gaoshenga opened this issue Nov 15, 2023 · 3 comments

Comments

@gaoshenga
Copy link

input:
  label: interval
  generate:
    interval: 5s
    mapping: root = {}
pipeline:
  processors:

Report the following errors when using the generate component to generate data periodically.

{"@service":"benthos","label":"interval","level":"debug","msg":"Waiting for pending acks to resolve before shutting down.","path":"root.input","time":"2023-11-12T13:23:32+08:00"}
{"@service":"benthos","label":"interval","level":"debug","msg":"Pending acks resolved.","path":"root.input","time":"2023-11-12T13:23:32+08:00"}

Could you help me with this problem?

@mfamador
Copy link
Contributor

Is that your complete config?

From what I can see it works:

❯ benthos -c <(echo "input:
  label: interval
  generate:
    interval: 5s
    mapping: root = {}")
INFO Running main config from specified file       @service=benthos benthos_version=4.23.0 path=/dev/fd/13
INFO Listening for HTTP requests at: http://0.0.0.0:4195  @service=benthos
INFO Launching a benthos instance, use CTRL+C to close  @service=benthos
{}
{}
{}

@gaoshenga
Copy link
Author

gaoshenga commented Nov 15, 2023

This is my complete configuration

input:
  label: interval
  generate:
    interval: 5s
    mapping: root = {}
pipeline:
  processors:
    - branch:
        processors:
          - sql_raw:
              driver: mysql
              dsn: root:1234@ea123@tcp(172.0.0.22:3306)/adb
              query: SELECT * from vw_sc_out where id = 1;
        result_map: root.mysql = this
    - label: fieldselect_1
      mapping: |
        root = this
        root.fieldselect = this.mysql
    - label: mappingrelation_1
      mapping: |
        root.mapping =
        {"0_日":"100:MBCJSCKB_1.day_LJ.PV","0_月":"100:MBCJSCKB_1.mon_LJ.PV"}

        root.clapdata = this.fieldselect.collapse()
    - label: mappingrelation_2
      mapping: |
        root.clapdata = this.clapdata.map_each_key(key -> key.replace_all(".",
        "_"))

        root.mapping = this.mapping
    - label: outputmapping
      mapping: >
        root.data = [{"nodeId": this.mapping.0_日, "value":
        this.clapdata.0_日},{"nodeId": this.mapping.0_月, "value":
        this.clapdata.0_月}]
output:
  broker:
    outputs:
      - http_client:
          headers:
            Content-Type: application/json
          rate_limit: ""
          timeout: 5s
          url: http://172.0.0.1:6888/write
          verb: POST
        label: http
logger:
  level: ALL
  format: json
  add_timestamp: true
  file:
    path: /usr/json/job.json
    rotate: true

@mihaitodor
Copy link
Collaborator

mihaitodor commented Nov 16, 2023

Those are just debug messages. What issue are you actually seeing? If the sql_raw processor fails, then you might want to stick a log processor with message: ${! error() } and another one with message: ${! content() } after it to see what you're getting from it.

PS: Converting to a discussion as per ##2026.

@redpanda-data redpanda-data locked and limited conversation to collaborators Nov 16, 2023
@mihaitodor mihaitodor converted this issue into discussion #2247 Nov 16, 2023

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants