Skip to content
This repository has been archived by the owner on Feb 12, 2024. It is now read-only.

interface-ipfs-core: timeout tests pass without implementation #3161

Closed
koivunej opened this issue Jul 9, 2020 · 5 comments · Fixed by #3178
Closed

interface-ipfs-core: timeout tests pass without implementation #3161

koivunej opened this issue Jul 9, 2020 · 5 comments · Fixed by #3178
Labels
kind/enhancement A net-new feature or improvement to an existing feature status/ready Ready to be worked

Comments

@koivunej
Copy link
Contributor

koivunej commented Jul 9, 2020

  • Version:

From package-lock.json:

interface-ipfs-core
"interface-ipfs-core": {
      "version": "0.137.0",
      "resolved": "https://registry.npmjs.org/interface-ipfs-core/-/interface-ipfs-core-0.137.0.tgz",
      "integrity": "sha512-//opRMeMUVnkxgkUa+ssZQh2XjhOw2stGxJsGI+ZN0d0TPsWRL3uk069eHWsQGODWo95B4663z4Y0dzxgTQQfQ==",

And:

ipfs-http-client
"ipfs-http-client": {
          "version": "44.0.3",
          "resolved": "https://registry.npmjs.org/ipfs-http-client/-/ipfs-http-client-44.0.3.tgz",
          "integrity": "sha512-uO/UkMryuKMqIzNkyWv056RpGZZX7UPGwIE49Y0ZAAqAmm08pxIiC1B93NP4jBefW/aNFjazcyhAewMqg1YeNw==",
          "dev": true,
          "requires": {
            "abort-controller": "^3.0.0",
            "bignumber.js": "^9.0.0",
            "bs58": "^4.0.1",
            "buffer": "^5.4.2",
            "cids": "^0.8.0",
            "debug": "^4.1.0",
            "form-data": "^3.0.0",
            "ipfs-block": "^0.8.1",
            "ipfs-core-utils": "^0.2.2",
            "ipfs-utils": "^2.2.2",
            "ipld-dag-cbor": "^0.15.1",
            "ipld-dag-pb": "^0.18.3",
            "ipld-raw": "^4.0.1",
            "iso-url": "^0.4.7",
            "it-tar": "^1.2.1",
            "it-to-buffer": "^1.0.0",
            "it-to-stream": "^0.1.1",
            "merge-options": "^2.0.0",
            "multiaddr": "^7.2.1",
            "multiaddr-to-uri": "^5.1.0",
            "multibase": "^0.7.0",
            "multicodec": "^1.0.0",
            "multihashes": "^0.4.14",
            "nanoid": "^3.0.2",
            "node-fetch": "^2.6.0",
            "parse-duration": "^0.1.2",
            "stream-to-it": "^0.2.0"
          }
        }
  • Platform: Linux joonas-t495 5.3.0-62-generic Add json loader #56-Ubuntu SMP Tue Jun 23 11:20:52 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
$ node --version
v13.14.0
  • Subsystem: interface-ipfs-core

Severity:

Description:

Run the tests before implementing the any timeout=1ms query parameter handling on rust-ipfs.

Steps to reproduce the error:

  1. install rustup
  2. update stable to 1.44.1
  3. clone rust-ipfs, rs-ipfs/rust-ipfs@8437373 is good commit for reproducing
  4. cargo build -p http
  5. clone ipfs-rust-conformance, rs-ipfs/ipfs-rust-conformance@1c20b41 is good commit for reproducing
  6. symlink the built ipfs-http binary next to rust.sh under ipfs-rust-conformance as http
  7. run tests: IPFS_RUST_EXEC="$(pwd)/rust.sh" npm test -- --grep 'should respect timeout option when'

All tests should pass even if the functionality has not been implemented. There are probably issues with following the above instructions, so no harm in not trying. Good to print them out for once at least :)

I suspect that ipfs-http-client library internally handles the timeout while also rendering it to the query parameter. Perhaps the http client level timeout should be disabled when doing these timeout tests? If there is such way in the ipfs-http-client, I couldn't find it in https://github.com/ipfs/js-ipfs/blob/master/packages/ipfs-http-client/README.md#global-timeouts.

@koivunej koivunej added the need/triage Needs initial labeling and prioritization label Jul 9, 2020
@welcome
Copy link

welcome bot commented Jul 9, 2020

Thank you for submitting your first issue to this repository! A maintainer will be here shortly to triage and review.
In the meantime, please double-check that you have provided all the necessary information to make this process easy! Any information that can help save additional round trips is useful! We currently aim to give initial feedback within two business days. If this does not happen, feel free to leave a comment.
Please keep an eye on how this issue will be labeled, as labels give an overview of priorities, assignments and additional actions requested by the maintainers:

  • "Priority" labels will show how urgent this is for the team.
  • "Status" labels will show if this is ready to be worked on, blocked, or in progress.
  • "Need" labels will indicate if additional input or analysis is required.

Finally, remember to use https://discuss.ipfs.io if you just need general support.

@achingbrain
Copy link
Member

I think this is a hangover from before timeouts were implemented in js-ipfs, so the client also has a timeout.

Perhaps the http client level timeout should be disabled when doing these timeout tests?

Sounds good, we can add an option to the http client.

@achingbrain achingbrain added kind/enhancement A net-new feature or improvement to an existing feature and removed need/triage Needs initial labeling and prioritization labels Jul 9, 2020
@jacobheun jacobheun added the status/ready Ready to be worked label Jul 9, 2020
@achingbrain
Copy link
Member

achingbrain commented Jul 17, 2020

This might be better done if we just remove the default timeout.

@koivunej could you try removing the || 60000 * 20 from your local copy? If that works we can probably do without it since it predates server-side timeouts for js-ipfs.

I'm trying to follow your instructions but I'm getting an error error: package ID specification 'http' matched no packages during the cargo build -p http step. I notice the ipfs-rust-conformance module has been archived so I guess the instructions won't repro any more.

achingbrain added a commit that referenced this issue Jul 17, 2020
Do not specify a default timeout - since requests are now abortable,
killing the connection (e.g. ctrl+c) will tear down long-lived
resources on the node.

fixes: #3161
@koivunej
Copy link
Contributor Author

koivunej commented Jul 20, 2020

@achingbrain I applied this:

@@ -110,7 +110,7 @@
     /** @type {ClientOptions} */
     const opts = normalizeInput(options)
     super({
-      timeout: parseTimeout(opts.timeout) || 60000 * 20,
+      timeout: parseTimeout(opts.timeout),
       headers: opts.headers,
       base: normalizeInput(opts.url).toString(),
       handleError: errorHandler,

With this patch and how we currently run the interface-http-core tests now in rust-ipfs/conformance (we simplified from two repos to just one), I cannot see any difference in outcome before or after removing the timeout implementation from dag/resolve and block/get while passing the tests.

Tried also adding an "assertion" that opts.timeout is never truthy after normalizeInput(options), and it's never truthy.

I got a single random test failure when running npm test -- --grep timeout with the "should respect timeout option when getting a DAG node" as follows:

  1) .dag.get
       should respect timeout option when getting a DAG node:
     Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (.../rust-ipfs/conformance/test/index.js)
      at listOnTimeout (internal/timers.js:549:17)
      at processTimers (internal/timers.js:492:7)

When running the test in loop with:

$ while true; do IPFS_RUST_EXEC="$(pwd)/rust.sh" npm test -- --grep 'should respect timeout option when getting a DAG node'; done

It passes every time in 300-400ms:

> ipfs-rust-conformance@1.0.0 test .../rust-ipfs/conformance
> npx mocha "--grep" "should respect timeout option when getting a DAG node"


  .dag.get
    ✓ should respect timeout option when getting a DAG node


  1 passing (389ms)

If it passes normally in 300-400ms and the timeout is 2000ms, I would expect testing the build without any ?timeout handling to consistently hit the 2s timeout, which it certainly would do in a manual test with curl:

$ curl '127.0.0.1:41233/api/v0/dag/resolve?arg=/ipfs/QmUm3BmzTKE1ncMiKJ5EKA5uVMamiJiy3mGxopGAA12hvX&timeout=1ms'

This hangs forever as expected with the timeout parameter processing commented out.

@koivunej
Copy link
Contributor Author

koivunej commented Jul 20, 2020

Fixed repro instructions (I had used wrong name for ipfs-http of course :):

  1. install rustup
  2. update stable to 1.44.1 (or later)
  3. clone rust-ipfs, rs-ipfs/rust-ipfs@2576a6d (latest master) is good commit for reproducing
  4. apply patch below
  5. cargo build -p ipfs-http
  6. cd conformance && ./setup.sh
    • this does npm install and patches refs-local.js test to not do pinning
  7. run tests: IPFS_RUST_EXEC="$(pwd)/rust.sh" npm test -- --grep 'should respect timeout option when'

Patch to disable ?timeout= handling on dag/resolve and block/get:

diff --git a/http/src/v0/block.rs b/http/src/v0/block.rs
index f2ac799..22d5819 100644
--- a/http/src/v0/block.rs
+++ b/http/src/v0/block.rs
@@ -32,10 +32,8 @@ async fn get_query<T: IpfsTypes>(
     let cid: Cid = query.arg.parse().map_err(StringError::from)?;
     let data = ipfs
         .get_block(&cid)
-        .maybe_timeout(query.timeout.map(StringSerialized::into_inner))
         .await
         .map_err(StringError::from)?
-        .map_err(StringError::from)?
         .into_vec();

     let response = Response::builder().body(data);
diff --git a/http/src/v0/dag.rs b/http/src/v0/dag.rs
index c93eb78..84ad57e 100644
--- a/http/src/v0/dag.rs
+++ b/http/src/v0/dag.rs
@@ -129,9 +129,7 @@ async fn inner_resolve<T: IpfsTypes>(
     let path = IpfsPath::try_from(opts.arg.as_str()).map_err(StringError::from)?;

     let (current, _, remaining) = walk_path(&ipfs, path)
-        .maybe_timeout(opts.timeout.map(StringSerialized::into_inner))
         .await
-        .map_err(StringError::from)?
         .map_err(StringError::from)?;

     let remaining = {

achingbrain added a commit that referenced this issue Aug 27, 2021
Do not set up a client-side timeout, instead rely on the server-side version.

The global timeout is still respected, users can use an AbortSignal to do
timeouts on a per-call basis if desired.

fixes: #3161
achingbrain added a commit that referenced this issue Sep 1, 2021
Do not set up a client-side timeout, instead rely on the server-side version.

The global timeout is still respected, users can use an AbortSignal to do
timeouts on a per-call basis if desired.

Also brings refs API in line with go-IPFS - timeouts are recorded in the `.err`
prop of the output instead of the whole API call throwing a `TimeoutError`

fixes: #3161
SgtPooki referenced this issue in ipfs/js-kubo-rpc-client Aug 18, 2022
Do not set up a client-side timeout, instead rely on the server-side version.

The global timeout is still respected, users can use an AbortSignal to do
timeouts on a per-call basis if desired.

Also brings refs API in line with go-IPFS - timeouts are recorded in the `.err`
prop of the output instead of the whole API call throwing a `TimeoutError`

fixes: #3161
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/enhancement A net-new feature or improvement to an existing feature status/ready Ready to be worked
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants