Caliper doesn't heed the localhost: false setting in my network configuration file (Fabric Gateway)

[Quasso]

Which Caliper version are you using?

v0.4.2

Which Node.JS version are you using?

v10

Which operating system are you using?

MacOS Big Sur 11.6.3 (20G415)

Please provide some context for your error. For example, when did the error occur? What were you trying to achieve, and how?

Some context

I have a Fabric network running pods/containers via Kubernetes, and I brought the caliper-benchmarks repository into my repo containing all the network configuration using Git Subtree.

All of my configuration JSON files were already successfully implemented and in use for a custom NestJS API that allows for enrolling of admin users & submitting of transactions.

In trying to connect to my existing Fabric network using the Gateway connector with discovery enabled (via the fabric-node-sdk which the rest of my project uses, where needed), I ran into a problem after the initial Gateway/Admin user connection was established.

When did the error occur?

I had my core --caliper-networkconfig YAML config file set up to use the setting (object notation):

• caliper.fabric.gateway.localhost: false

Essentially, at first I gathered based on the documentation that the setting above was all that was necessary in order to ensure that the Kube DNS was used, since my instance of Caliper is created as a pod inside my cluster.

What was I trying to achieve?

To run a simple test workload and publish some transactions on-chain.

What was the observed incorrect behavior?

Caliper was using localhost resolution instead of the hostname values (peer0.org1 etc) inside the configuration files (it was confusing because most of the log used the hostname, just one part mentioned localhost and I skimmed over that at first, until speaking in RocketChat).

Please provide the error logs and their surroundings.

2022-01-28T15:40:43.433Z - error: [ServiceEndpoint]: waitForReady - Failed to connect to remote gRPC server **orderer.org1:7050** url:**grpcs://localhost:7050** timeout:3000
2022-01-28T15:40:43.433Z - error: [DiscoveryService]: _buildOrderer[allorgs] - Unable to connect to the discovered orderer orderer.org1:7050 due to Error: Failed to connect before the deadline on Committer- name: **orderer.org1:7050**, **url:grpcs://localhost:7050**, connected:false, connectAttempted:true


(**=Bold for emphasis/context)

Please provide your benchmark configuration file content, if possible.

test:
  workers:
    type: local
    number: 10
  rounds:
    - label: Create an asset on the Chain.
      txNumber: 30
      rateControl:
          type: fixed-load
          opts:
            tps: 30
      workload:
        module: ./subtree/<redacted>/createAsset.js
        arguments:
          assets: 30
          contractId: <correct_contract_id>

Please provide your network configuration file content, if possible.

name: Caliper test
version: "2.0.0" # their standard, NOT configurable (use info instead for versioning)
# mutual-tls: true
# above is documented, but has no discernable impact. Crucial to set sutOptions -> mutualTls: true

caliper:
  blockchain: fabric
  sutOptions:
    mutualTls: true
  fabric:
        # Sets the sleep behaviors after different init actions
        sleepafter:
            # The time in milliseconds to sleep after creating the channels (if created channels at all)
            createchannel: 5000
            # The time in milliseconds to sleep after joining the channels (if joined channels at all)
            joinchannel: 5000
            # The time in milliseconds to sleep after instantiated the contracts (if instantiated contracts at all)
            instantiatecontract: 5000
        # Enables extra verification steps during the transaction proposal phase
        verify:
            # Indicates whether to verify the identity of the endorsers and their signatures for each proposal response after endorsing a transaction.
            # Note, that this is a CPU intensive step, use it with caution
            proposalresponse: false
            # Indicates whether to verify that the read-write sets returned by the endorsers match
            readwritesets: true
        # Contains worker-side timeouts related to the initialization part of the adapter
        timeout:
            # Timeout in milliseconds for the endorsement part of a contract instantiation
            contractinstantiate: 300000
            # Timeout in milliseconds for receiving the event about the result of a contract instantiation
            contractinstantiateevent: 100000
            # The default timeout in milliseconds to use for invoking or querying transactions (applied for the entire life-cycle)
            invokeorquery: 60000
        # Determines how automatic load balancing is applied if the worker callback module doesn’t provide explicit targets.
        # Use the value 'worker' to perform worker-based load balancing, meaning that each worker process will have fix target peers and target orderer.
        # Use the value 'tx' to perform transaction-based load balancing, meaning that the peer and orderer targets change for every submitted transaction or query.
        loadbalancing: tx
        # Indicates whether to temporarily set the GOPATH environment variable to the Caliper root directory
        overwritegopath: true
        # Determines the reported commit time of a transaction based on the given percentage of event sources
        latencythreshold: 1.0
        # Indicates whether to count queries as workload, i.e., whether the generated report should include them
        countqueryasload: true
        # Settings for the Fabric Gateway API
        gateway:
            # Indicates whether to use the Fabric Gateway API
            enabled: true
            # Indicates whether to use the localhost default within the Fabric Gateway API
            localhost: false
            # Indicates whether to use the Fabric discovery mechanism (via Gateway API)
            discovery: true
            # Which event strategy to use
            eventstrategy: msp_any
            # Which query strategy to use
            querystrategy: msp_round_robin
channels:
  - channelName: allorgs
    contracts:
    - id: <correct_contract_id>

organizations:
  - mspid: Org1MSP
    identities:
      certificates:
      - name: 'Admin'
        admin: true
        clientPrivateKey:
          path: '../crypto-config/peerOrganizations/org1/users/Admin@org1/msp/keystore/priv_sk'
        clientSignedCert:
          path: '../crypto-config/peerOrganizations/org1/users/Admin@org1/msp/signcerts/Admin@org1-cert.pem'
    connectionProfile:
      path: '../ccp/connection-org1.json'
      discover: true

Please provide your workload module content, if possible.

N/A

Please provide any additional information you deem relevant to the error.

I was able to find the solution to the problem with the help of @davidkel. It essentially requires passing the CLI flag --caliper-fabric-gateway-localhost false to work with hostnames. I am very happy with submitting a PR to improve the main area of documentation a bit to outline this as a potential issue and offer the suggestion of using the flag.

[davidkel]

This is about the difference between a network configuration file and a caliper configuration file and the first confusion was that this is a single file but unfortunately it isn't and so we need to ensure in the docs that this is clear. The other part is the ability to modify the default caliper config entries either by using a custom caliper configuration file or command line options. Within fabric we could be explicit in all the relevant configuration entries with examples on how to use both via CLI and via a custom caliper config file

[Quasso]

Okay makes sense & sounds good to me, I am happy to do my best to take care of this based on what you've said. I will hook up a PR in due course!

[davidkel]

Fabric has set 2.2 now as unmaintained and 2.5 is the only TLS version. The legacy node sdk has been deprecated in 2.5 now. It's probably not worth investing in using these SDKs anymore and only investing in the peer gateway in caliper for 2.5 and beyond versions of fabric