Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG]Meta authentication failure caused the request to write data to fail #1184

Closed
Benxiaohai001 opened this issue May 15, 2023 · 4 comments
Closed

[BUG]Meta authentication failure caused the request to write data to fail #1184

Benxiaohai001 opened this issue May 15, 2023 · 4 comments
Assignees
@Benxiaohai001
Labels
type/bug Something isn't working
Milestone
2.3.2

Comments

@Benxiaohai001
Copy link
Member

Describe the bug

Meta authentication failure caused the request to write data to fail
version:afb13f5da9182f68c8b5c42d375d4ba7182f01c3

To Reproduce

Jemter set up a test plan, and wrote data through the /api/v1/sql?db=public interface; 200 concurrent loops were performed 10 times, and it was found that 1% of the writes failed. Failed request return information：{"error_code":"010016","error_message":"Auth error: Request meta cluster error: NetworkError: Curl("Send failed since rewinding of the data stream failed"): Send failed since rewinding of the data stream failed"}

<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="5.0" jmeter="5.4.3">
  <hashTree>
    <TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="insert" enabled="true">
      <stringProp name="TestPlan.comments"></stringProp>
      <boolProp name="TestPlan.functional_mode">false</boolProp>
      <boolProp name="TestPlan.tearDown_on_shutdown">true</boolProp>
      <boolProp name="TestPlan.serialize_threadgroups">false</boolProp>
      <elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
        <collectionProp name="Arguments.arguments"/>
      </elementProp>
      <stringProp name="TestPlan.user_define_classpath"></stringProp>
    </TestPlan>
    <hashTree>
      <ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="Thread Group" enabled="true">
        <stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
        <elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
          <boolProp name="LoopController.continue_forever">false</boolProp>
          <stringProp name="LoopController.loops">10</stringProp>
        </elementProp>
        <stringProp name="ThreadGroup.num_threads">200</stringProp>
        <stringProp name="ThreadGroup.ramp_time">0</stringProp>
        <boolProp name="ThreadGroup.scheduler">false</boolProp>
        <stringProp name="ThreadGroup.duration"></stringProp>
        <stringProp name="ThreadGroup.delay"></stringProp>
        <boolProp name="ThreadGroup.same_user_on_next_iteration">true</boolProp>
        <boolProp name="ThreadGroup.delayedStart">true</boolProp>
      </ThreadGroup>
      <hashTree>
        <ConfigTestElement guiclass="HttpDefaultsGui" testclass="ConfigTestElement" testname="HTTP Request Defaults" enabled="true">
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
            <collectionProp name="Arguments.arguments"/>
          </elementProp>
          <stringProp name="HTTPSampler.domain">127.0.0.1</stringProp>
          <stringProp name="HTTPSampler.port">8902</stringProp>
          <stringProp name="HTTPSampler.protocol">http</stringProp>
          <stringProp name="HTTPSampler.contentEncoding"></stringProp>
          <stringProp name="HTTPSampler.path">/api/v1/sql?db=public</stringProp>
          <stringProp name="HTTPSampler.concurrentPool">6</stringProp>
          <stringProp name="HTTPSampler.connect_timeout"></stringProp>
          <stringProp name="HTTPSampler.response_timeout"></stringProp>
        </ConfigTestElement>
        <hashTree/>
        <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="HTTP Request" enabled="true">
          <boolProp name="HTTPSampler.postBodyRaw">true</boolProp>
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments">
            <collectionProp name="Arguments.arguments">
              <elementProp name="" elementType="HTTPArgument">
                <boolProp name="HTTPArgument.always_encode">false</boolProp>
                <stringProp name="Argument.value">INSERT INTO air (TIME, station, visibility, temperature, presssure) VALUES (${timestamp1}, &apos;XiaoMaiDao1&apos;, 56, 69, 411),(${timestamp2}, &apos;XiaoMaiDao2&apos;, 56, 69, 411),(${timestamp3}, &apos;XiaoMaiDao4&apos;, 56, 69, 411),(${timestamp4}, &apos;XiaoMaiDao1&apos;, 56, 69, 711);</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
            </collectionProp>
          </elementProp>
          <stringProp name="HTTPSampler.domain"></stringProp>
          <stringProp name="HTTPSampler.port"></stringProp>
          <stringProp name="HTTPSampler.protocol"></stringProp>
          <stringProp name="HTTPSampler.contentEncoding"></stringProp>
          <stringProp name="HTTPSampler.path"></stringProp>
          <stringProp name="HTTPSampler.method">POST</stringProp>
          <boolProp name="HTTPSampler.follow_redirects">true</boolProp>
          <boolProp name="HTTPSampler.auto_redirects">false</boolProp>
          <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
          <boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp>
          <stringProp name="HTTPSampler.embedded_url_re"></stringProp>
          <stringProp name="HTTPSampler.connect_timeout"></stringProp>
          <stringProp name="HTTPSampler.response_timeout"></stringProp>
        </HTTPSamplerProxy>
        <hashTree>
          <ResultCollector guiclass="ViewResultsFullVisualizer" testclass="ResultCollector" testname="View Results Tree" enabled="true">
            <boolProp name="ResultCollector.error_logging">false</boolProp>
            <objProp>
              <name>saveConfig</name>
              <value class="SampleSaveConfiguration">
                <time>true</time>
                <latency>true</latency>
                <timestamp>true</timestamp>
                <success>true</success>
                <label>true</label>
                <code>true</code>
                <message>true</message>
                <threadName>true</threadName>
                <dataType>true</dataType>
                <encoding>false</encoding>
                <assertions>true</assertions>
                <subresults>true</subresults>
                <responseData>false</responseData>
                <samplerData>false</samplerData>
                <xml>false</xml>
                <fieldNames>true</fieldNames>
                <responseHeaders>false</responseHeaders>
                <requestHeaders>false</requestHeaders>
                <responseDataOnError>false</responseDataOnError>
                <saveAssertionResultsFailureMessage>true</saveAssertionResultsFailureMessage>
                <assertionsResultsToSave>0</assertionsResultsToSave>
                <bytes>true</bytes>
                <sentBytes>true</sentBytes>
                <url>true</url>
                <threadCounts>true</threadCounts>
                <idleTime>true</idleTime>
                <connectTime>true</connectTime>
              </value>
            </objProp>
            <stringProp name="filename"></stringProp>
          </ResultCollector>
          <hashTree/>
          <ResultCollector guiclass="TableVisualizer" testclass="ResultCollector" testname="View Results in Table" enabled="true">
            <boolProp name="ResultCollector.error_logging">false</boolProp>
            <objProp>
              <name>saveConfig</name>
              <value class="SampleSaveConfiguration">
                <time>true</time>
                <latency>true</latency>
                <timestamp>true</timestamp>
                <success>true</success>
                <label>true</label>
                <code>true</code>
                <message>true</message>
                <threadName>true</threadName>
                <dataType>true</dataType>
                <encoding>false</encoding>
                <assertions>true</assertions>
                <subresults>true</subresults>
                <responseData>false</responseData>
                <samplerData>false</samplerData>
                <xml>false</xml>
                <fieldNames>true</fieldNames>
                <responseHeaders>false</responseHeaders>
                <requestHeaders>false</requestHeaders>
                <responseDataOnError>false</responseDataOnError>
                <saveAssertionResultsFailureMessage>true</saveAssertionResultsFailureMessage>
                <assertionsResultsToSave>0</assertionsResultsToSave>
                <bytes>true</bytes>
                <sentBytes>true</sentBytes>
                <url>true</url>
                <threadCounts>true</threadCounts>
                <idleTime>true</idleTime>
                <connectTime>true</connectTime>
              </value>
            </objProp>
            <stringProp name="filename"></stringProp>
          </ResultCollector>
          <hashTree/>
          <ResultCollector guiclass="StatGraphVisualizer" testclass="ResultCollector" testname="Aggregate Graph" enabled="true">
            <boolProp name="ResultCollector.error_logging">false</boolProp>
            <objProp>
              <name>saveConfig</name>
              <value class="SampleSaveConfiguration">
                <time>true</time>
                <latency>true</latency>
                <timestamp>true</timestamp>
                <success>true</success>
                <label>true</label>
                <code>true</code>
                <message>true</message>
                <threadName>true</threadName>
                <dataType>true</dataType>
                <encoding>false</encoding>
                <assertions>true</assertions>
                <subresults>true</subresults>
                <responseData>false</responseData>
                <samplerData>false</samplerData>
                <xml>false</xml>
                <fieldNames>true</fieldNames>
                <responseHeaders>false</responseHeaders>
                <requestHeaders>false</requestHeaders>
                <responseDataOnError>false</responseDataOnError>
                <saveAssertionResultsFailureMessage>true</saveAssertionResultsFailureMessage>
                <assertionsResultsToSave>0</assertionsResultsToSave>
                <bytes>true</bytes>
                <sentBytes>true</sentBytes>
                <url>true</url>
                <threadCounts>true</threadCounts>
                <idleTime>true</idleTime>
                <connectTime>true</connectTime>
              </value>
            </objProp>
            <stringProp name="filename"></stringProp>
          </ResultCollector>
          <hashTree/>
          <ResultCollector guiclass="StatVisualizer" testclass="ResultCollector" testname="Aggregate Report" enabled="true">
            <boolProp name="ResultCollector.error_logging">false</boolProp>
            <objProp>
              <name>saveConfig</name>
              <value class="SampleSaveConfiguration">
                <time>true</time>
                <latency>true</latency>
                <timestamp>true</timestamp>
                <success>true</success>
                <label>true</label>
                <code>true</code>
                <message>true</message>
                <threadName>true</threadName>
                <dataType>true</dataType>
                <encoding>false</encoding>
                <assertions>true</assertions>
                <subresults>true</subresults>
                <responseData>false</responseData>
                <samplerData>false</samplerData>
                <xml>false</xml>
                <fieldNames>true</fieldNames>
                <responseHeaders>false</responseHeaders>
                <requestHeaders>false</requestHeaders>
                <responseDataOnError>false</responseDataOnError>
                <saveAssertionResultsFailureMessage>true</saveAssertionResultsFailureMessage>
                <assertionsResultsToSave>0</assertionsResultsToSave>
                <bytes>true</bytes>
                <sentBytes>true</sentBytes>
                <url>true</url>
                <threadCounts>true</threadCounts>
                <idleTime>true</idleTime>
                <connectTime>true</connectTime>
              </value>
            </objProp>
            <stringProp name="filename"></stringProp>
          </ResultCollector>
          <hashTree/>
        </hashTree>
        <JDBCSampler guiclass="TestBeanGUI" testclass="JDBCSampler" testname="JDBC Request" enabled="true">
          <stringProp name="dataSource"></stringProp>
          <stringProp name="query"></stringProp>
          <stringProp name="queryArguments"></stringProp>
          <stringProp name="queryArgumentsTypes"></stringProp>
          <stringProp name="queryTimeout"></stringProp>
          <stringProp name="queryType">Select Statement</stringProp>
          <stringProp name="resultSetHandler">Store as String</stringProp>
          <stringProp name="resultSetMaxRows"></stringProp>
          <stringProp name="resultVariable"></stringProp>
          <stringProp name="variableNames"></stringProp>
        </JDBCSampler>
        <hashTree/>
        <Arguments guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
          <collectionProp name="Arguments.arguments">
            <elementProp name="last" elementType="Argument">
              <stringProp name="Argument.name">last</stringProp>
              <stringProp name="Argument.value">${timestamp}</stringProp>
              <stringProp name="Argument.metadata">=</stringProp>
            </elementProp>
            <elementProp name="base" elementType="Argument">
              <stringProp name="Argument.name">base</stringProp>
              <stringProp name="Argument.value">1667456411000000000</stringProp>
              <stringProp name="Argument.metadata">=</stringProp>
            </elementProp>
          </collectionProp>
        </Arguments>
        <hashTree/>
        <CounterConfig guiclass="CounterConfigGui" testclass="CounterConfig" testname="Counter" enabled="true">
          <stringProp name="CounterConfig.start">1667456411000000000</stringProp>
          <stringProp name="CounterConfig.end"></stringProp>
          <stringProp name="CounterConfig.incr">1</stringProp>
          <stringProp name="CounterConfig.name">timestamp1</stringProp>
          <stringProp name="CounterConfig.format"></stringProp>
          <boolProp name="CounterConfig.per_user">false</boolProp>
        </CounterConfig>
        <hashTree/>
        <CounterConfig guiclass="CounterConfigGui" testclass="CounterConfig" testname="Counter2" enabled="true">
          <stringProp name="CounterConfig.start">1667456412000000000</stringProp>
          <stringProp name="CounterConfig.end"></stringProp>
          <stringProp name="CounterConfig.incr">1</stringProp>
          <stringProp name="CounterConfig.name">timestamp2</stringProp>
          <stringProp name="CounterConfig.format"></stringProp>
          <boolProp name="CounterConfig.per_user">false</boolProp>
        </CounterConfig>
        <hashTree/>
        <CounterConfig guiclass="CounterConfigGui" testclass="CounterConfig" testname="Counter3" enabled="true">
          <stringProp name="CounterConfig.start">1667456413000000000</stringProp>
          <stringProp name="CounterConfig.end"></stringProp>
          <stringProp name="CounterConfig.incr">1</stringProp>
          <stringProp name="CounterConfig.name">timestamp3</stringProp>
          <stringProp name="CounterConfig.format"></stringProp>
          <boolProp name="CounterConfig.per_user">false</boolProp>
        </CounterConfig>
        <hashTree/>
        <CounterConfig guiclass="CounterConfigGui" testclass="CounterConfig" testname="Counter4" enabled="true">
          <stringProp name="CounterConfig.start">1667456414000000000</stringProp>
          <stringProp name="CounterConfig.end"></stringProp>
          <stringProp name="CounterConfig.incr">1</stringProp>
          <stringProp name="CounterConfig.name">timestamp4</stringProp>
          <stringProp name="CounterConfig.format"></stringProp>
          <boolProp name="CounterConfig.per_user">false</boolProp>
        </CounterConfig>
        <hashTree/>
        <CounterConfig guiclass="CounterConfigGui" testclass="CounterConfig" testname="Counter5" enabled="true">
          <stringProp name="CounterConfig.start">1667456415000000000</stringProp>
          <stringProp name="CounterConfig.end"></stringProp>
          <stringProp name="CounterConfig.incr">1</stringProp>
          <stringProp name="CounterConfig.name">timestamp5</stringProp>
          <stringProp name="CounterConfig.format"></stringProp>
          <boolProp name="CounterConfig.per_user">false</boolProp>
        </CounterConfig>
        <hashTree/>
      </hashTree>
      <HeaderManager guiclass="HeaderPanel" testclass="HeaderManager" testname="HTTP Header Manager" enabled="true">
        <collectionProp name="HeaderManager.headers">
          <elementProp name="" elementType="Header">
            <stringProp name="Header.name">Content-Type</stringProp>
            <stringProp name="Header.value">application/json</stringProp>
          </elementProp>
        </collectionProp>
      </HeaderManager>
      <hashTree/>
      <AuthManager guiclass="AuthPanel" testclass="AuthManager" testname="HTTP Authorization Manager" enabled="true">
        <collectionProp name="AuthManager.auth_list">
          <elementProp name="" elementType="Authorization">
            <stringProp name="Authorization.url">http://127.0.0.1</stringProp>
            <stringProp name="Authorization.username">root</stringProp>
            <stringProp name="Authorization.password"></stringProp>
            <stringProp name="Authorization.domain"></stringProp>
            <stringProp name="Authorization.realm"></stringProp>
            <stringProp name="Authorization.mechanism">BASIC_DIGEST</stringProp>
          </elementProp>
        </collectionProp>
        <boolProp name="AuthManager.controlledByThreadGroup">false</boolProp>
      </AuthManager>
      <hashTree/>
    </hashTree>
  </hashTree>
</jmeterTestPlan>

Expected behavior

No response

Additional context

{"error_code":"010016","error_message":"Auth error: Request meta cluster error: NetworkError: Curl("Send failed since rewinding of the data stream failed"): Send failed since rewinding of the data stream failed"}
@Benxiaohai001 Benxiaohai001 added the type/bug Something isn't working label May 15, 2023
@roseboy-liu roseboy-liu mentioned this issue May 16, 2023
Closed
@roseboy-liu roseboy-liu self-assigned this May 22, 2023
@roseboy-liu roseboy-liu mentioned this issue Jun 1, 2023
Merged
@Benxiaohai001
Copy link
Member Author

ce version :cnosdb 2.3.0, revision ad5725e
cpu 16 mem 32
When writing line protocol data, the following error is reported (occasionally), is the root cause associated with this issue the same?

cat /data/cnosdb_iot_123_2022.gz| gunzip | load_cnosdb --urls http://127.0.0.1:8902 --workers 50
Error writing: [DebugInfo: worker #43, dest url: http://127.0.0.1:8902] Invalid write response (status 422): {"error_code":"010016","error_message":"Auth error: Request meta cluster error: NetworkError: reqwest::error::Error: error sending request for url (http://127.0.0.1:8901/read): connection closed before message completed source: connection closed before message completed"}

@roseboy-liu
Copy link
Contributor

hyperium/hyper#2136. The issue talk about the question.

@roseboy-liu
Copy link
Contributor

roseboy-liu commented Jun 13, 2023 •
edited

I can not reproduce the problem, after merge this pull request. use the same jemter config file.
image

@roseboy-liu roseboy-liu added this to the 2.3.2 milestone Jul 10, 2023
@Benxiaohai001
Copy link
Member Author

version: aa70bf1
After multiple executions, no problems were found, and close the issue.
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Benxiaohai001 Benxiaohai001

Labels
type/bug Something isn't working
Projects
None yet
Milestone
2.3.2
Development

Successfully merging a pull request may close this issue.

fix issue #1152, change http client to reqwest roseboy-liu/cnosdb
2 participants
@Benxiaohai001 @roseboy-liu