Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use fetch instead of axios #314

Open
wants to merge 32 commits into
base: main
Choose a base branch
from
Open

Use fetch instead of axios #314

wants to merge 32 commits into from

Conversation

glynnbird
Copy link
Contributor

@glynnbird glynnbird commented Dec 16, 2022

Overview

Replaces axios HTTP library with fetch, powered-by undici. The upshot of this is that we reduce the number of dependencies to 1 and possibly 0 in the future.

Note this is for merging after April 2023 when Node 14 becomes end-of-life but discussion as to whether Nano should go in this direction is welcome in this PR.

Comments and advice welcome.

fetch

Some history: originally Nano was built on top of the request library which was later deprecated. At this point I refactorted it to use axios instead. This PR eliminates axios and other axios-related dependencies and instead uses the new kid on the block: the fetch API

The fetch feature has found widespread adoption in web browsers as a means of handling outbound HTTP requests. It has found its way into Node.js as a global function and is marked as an experimental feature in Node 18/19 and will likely be mainstream in Node 20.

Node.js's fetch capability is powered by the undici package which in turn uses Node's low-level network libraries instead of being based on the higher-level http/https built-in modules. It purports to be significantly faster (according to its own benchmarks) than traffic routed through http/https modules, as is the case with other HTTP libraries like axios & request.

Automated testing

Replacing axios with fetch means also ditching the nock library for mocking HTTP requests and responses, because nock works by intercepting requests originating from the http/https layer, which is bypassed by undici. Fortunately, undici provides its own Mocking tooling.

The outcome

This branch's needs no runtime dependencies. Current dependencies:

  "dependencies": {
    "http-cookie-agent": "^4.0.2",
    "@types/tough-cookie": "^4.0.2",
    "axios": "^1.1.3",
    "qs": "^6.11.0",
    "tough-cookie": "^4.1.2",
    "node-abort-controller": "^3.0.1"
  },
  "devDependencies": {
    "@types/node": "^18.11.9",
    "jest": "^29.2.2",
    "nock": "^13.2.9",
    "standard": "^17.0.0",
    "typescript": "^4.8.4"
  }

Post-PR dependencies:

  "dependencies": {
  },
  "devDependencies": {
    "undici": "^5.24.0",
    "@types/node": "^20.6.2",
    "typescript": "^5.2.2"
  }

Backwards compatibility

None of Nano's API has changed except when a user is supplying non-default connection handling parameters. Gone is requestDefaults which dates back to the "request" days and instead an optional agentOptions can be provided which is documented in the README and in TypeScript.

const agentOptions = {
  bodyTimeout: 30000,
  headersTimeout: 30000,
  keepAliveMaxTimeout: 600000,
  keepAliveTimeout: 30000,
  keepAliveTimeoutThreshold: 1000,
  maxHeaderSize: 16384,
  maxResponseSize: -1,
  pipelining: 6,
  connect: { 
    timeout: 10000
  },
  strictContentLength: true,
  connections: null,
  maxRedirections: 0
}
const undici = require('undici')
const undiciOptions = new undici.Agent(agentOptions)
const nano = Nano({ url: 'http://127.0.0.1:5984', undiciOptions })

Node versioning

As Node 16 is EOL, we can release v11 of Nano and make it the Node 18+ version. The v10 series of Nano would still be supported for the time being for older versions of node.

Testing recommendations

The test suite has been rewritten to use the built-in Node.js test runner (one fewer dependency!) and uses the undici.MockAgent to simulate responses for each of Nano's API calls, just as Nock did previously.

Run with:

npm run test

GitHub issue number

Fixes #307

Related Pull Requests

n/a

Checklist

  • Code is written and works correctly;
  • Changes are covered by tests;
  • Documentation reflects the changes;

@glynnbird
Copy link
Contributor Author

glynnbird commented Dec 21, 2022

My simple benchmarking of small HTTP requests measures that Nano "11" would be 0 to 10% faster than Nano 10.1.

Testing method: create simple Node Express app to deliver canned responses to nano requests. Time Nano 10.1 & Nano 11 in doing different nano calls over many cycles and calculate the average response time (ms).

Nano 10.1 Nano "11" % speed up
/_all_dbs 0.59 0.5309 10.0
/db/id 0.573 0.543 5.2
/db/_all_docs?limit=100&include_docs=true 0.793 0.779 1.8
/db/_all_docs?limit=10000&include_docs=true 23.425 23.388 0.2

^ my reading of these findings is that the smaller the response payload, the more the efficiency gains of using undici for outbound requests shows up. For larger response bodies, the benefit is swallowed up by the physics of transferring packets over the network.

@glynnbird glynnbird marked this pull request as draft January 26, 2023 14:30
@glynnbird glynnbird changed the title Refactor to use fetch instead of axios Use fetch instead of axios May 3, 2023
@2colours
Copy link

@glynnbird Hi, any update or idea about how to progress this? It's not just one less dependency but axios itself has questionable code quality - along with the workarounds that currently make this package work with axios.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

fetch instead of axios
2 participants