Support basic auth when forking

[davidmurdoch]

Infura uses HTTP Basic Auth now: https://infura.io/docs/gettingStarted/authentication

We'll need to add support for this.

Initial thoughts are to use url-based basic auth, via wss://:YOUR-PROJECT-SECRET@example.com/v3/YOUR-PROJECT-ID

[nonsense]

This seems to be an issue when using Chainstack as well, as they use HTTP Basic Auth.

[davidmurdoch]

@aintabb, The basic auth check/parsing would happen here: https://github.com/trufflesuite/ganache-core/blob/685b1d9600f01d64faa96f46b0bd2197748d5682/lib/forking/forked_blockchain.js#L39-L61

To test, you'll need a free Infura account from: https://infura.io/. Then create a new Ethereum project and set it so it requires a Project Secret. The Project Secret is the part that goes into the URL: wss://:YOUR-PROJECT-SECRET@example.com/v3/YOUR-PROJECT-ID. The RegEx needs to be updated to allow for the optional project secret while still parsing out the "fork_block_number", which is delimited by an "@" sign, just like BASIC auth; so that's the tricky bit.

Tests need to be written which would go here: https://github.com/trufflesuite/ganache-core/blob/develop/test/smoke/forking/infura/simple.js (you should NOT commit your Infura url or project secret here! I can test this locally).

To get started developing, checkout the develop branch, run npm install (I using recommend node 12), then to test run npm test.

Let me know if you are up for the task or not!

Thanks!

[aintabb]

@davidmurdoch Thanks for the detailed explanation. I am gonna start to working on it.

[davidmurdoch]

Awesome. I've assigned you just so no one else starts on it. Let me know if you no longer want to work on it and I'll remove the assignment. Thanks!!!

[aintabb]

@davidmurdoch Sure, you're welcome!

[aintabb]

Hi @davidmurdoch ,
Is there way to debug this project without running tests? I am guessing it does need a specific test allocated to him since this is core library.

[davidmurdoch]

I use VS Code's integrated debugger via the following configuration:

.vscode/launch.json:

{
  // Use IntelliSense to learn about possible attributes.
  // Hover to view descriptions of existing attributes.
  // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
  "version": "0.2.0",
  "configurations": [
    {
      "type": "node",
      "request": "launch",
      "name": "Mocha All",
      "program": "${workspaceFolder}/node_modules/mocha/bin/_mocha",
      "args": [
        "--timeout",
        "999999",
        "--colors",
        "${workspaceFolder}/test/{__FILENAME_YOU_WANT_TO_TEST_GOES_HERE__}.js",
        "--recursive",
        "--globals",
        "_scratch,sanitizedData"
      ],
      "console": "integratedTerminal",
      "internalConsoleOptions": "neverOpen"
    }
    ]
  }

Don't forget to replace {__FILENAME_YOU_WANT_TO_TEST_GOES_HERE__} with the test filename you want to replace.

You can then set breakpoints in the margin in VS Code:

And when you run the Mocha All debug config:

VSCode will pause once the application reaches your breakpoints. You can then inspect the runtime variables and stack, as well as step through/over code.

See https://code.visualstudio.com/docs/nodejs/nodejs-debugging for detailed info.

And this looks like a decent short video walk-through of using and configuring the debugger (disclaimer: I haven't watched the whole thing): https://www.youtube.com/watch?v=2oFKNL7vYV8

[aintabb]

Thank you, I appreciate that.

[aintabb]

@davidmurdoch During "Basic Auth", do the urls still get block height value delimited by @? If so, I don't there will be an issue during parsing.
I created mock urls by checking example values during debugging. "Fork" and "fork_block_url"s still being parsed when there was a "Basic Auth" type url.

[davidmurdoch]

@aintabb You should still be able to parse wss://:YOUR-PROJECT-SECRET@example.com/v3/YOUR-PROJECT-ID@YOUR-FORK-BLOCKNUMBER. It might require switching from using regular expression to something more robust, like Node's url module.

To get the the "project secret": https://nodejs.org/api/url.html#url_url_password
To get the fork block number you could use pathname + (search ? ?${search} : "") (https://nodejs.org/api/url.html#url_url_password, https://nodejs.org/api/url.html#url_url_search), and then parse out the fork block number from that (similar to the way we do it now).

That said... I'm now thinking we don't even need to grab the password/secret itself, since the basic auth format (i.e., https://:YOUR-PROJECT-SECRET@mainnet.infura.io/v3/YOUR-PROJECT-ID) is exactly what Infura expects from us; we will just need to strip the fork block number.

Hopefully that made sense? Let me know if you have more questions!

[aintabb]

@davidmurdoch Thanks for the response and the explanation. Still when we receive basic auth format url; let's say https://:YOUR-PROJECT-SECRET@mainnet.infura.io/v3/YOUR-PROJECT-ID@FORK_BLOCK_NUMBER and run the regex; /@([0-9]+)$/, we still get fork and fork_block_number values without an issue.
Just try in your console; Here is the regex -> /@([0-9]+)$/
and the code block which is responsible for parsing ->

[davidmurdoch]

@aintabb, that sounds right; you'll have to tweak things a bit to get it working.

Just FYI, I'm on holiday now (in the US) and likely won't be able to get back to my desktop to help out until until Monday.

[aintabb]

@davidmurdoch No worries! I am in US too. Enjoy your holiday!