RPC: Add getrpcwhitelist method #19117

luke-jr · 2020-05-31T00:27:12Z

This PR implements a new RPC, getrpcwhitelist, that returns whitelisted methods for the calling user.

Example:
./src/bitcoin-cli -regtest getrpcwhitelist
{
  "methods": [
    "getbalance",
    "getrpcwhitelist",
    "getwalletinfo"
  ]
}
A new functional test, rpc_getrpcwhitelist, and a release-notes entry are included.

Fixes #18822

Uses a JSON Object for methods for future extensions.

Not clear what it should do for users that don't have a whitelist (ie, can access all methods). Currently errors I think.

MarcoFalke · 2020-05-31T00:37:59Z

ACK after documenting the circular dep in the linter:

A new circular dependency in the form of "httprpc -> rpc/server -> httprpc" appears to have been introduced.

DrahtBot · 2020-05-31T05:37:21Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

test: Add test for rpcwhitelistdefault #17805 (test: Add test for rpcwhitelistdefault by emilengler)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

…nsion to sub-call permissions

luke-jr · 2020-05-31T17:39:53Z

@MarcoFalke Done

promag · 2020-05-31T21:57:01Z

Uses a JSON Object for methods for future extensions

Any idea in mind?

MarcoFalke · 2020-05-31T22:15:26Z

Any idea in mind?

The developer notes recommend it.

promag · 2020-06-01T22:59:06Z

@MarcoFalke I mean nothing comes to my mind that could be assigned to each whitelist'ed method.

I think we could name this getrpccredentials and it could return { "whitelist": ["..."] } or {} for when the user doesn't have a whitelist. IMO this is more extensible.

MarcoFalke · 2020-06-01T23:10:30Z

Nothing comes to my mind either, but the key-value pair make the return value a bit more self-documenting. Also, there should be no downside just following the dev notes.

promag

I think we could name this getrpccredentials

I have pushed e8b18af with the above suggestion.

promag · 2020-06-02T10:32:09Z

src/httprpc.cpp

+
+const std::set<std::string>& GetWhitelistedRpcs(const std::string& user_name)
+{
+    return g_rpc_whitelist.at(user_name);


This fails when user is not in the map.

Not clear what it should do for users that don't have a whitelist (ie, can access all methods). Currently errors I think.

I believe it should return the list of all available RPC methods. It's consistent and it'd enable feature detection. See #19087 for more discussion about feature detection.

If the rpc whitelist is empty, one could call help to get all methods. But maybe just returning the whole set here seems appropriate. How to deal with hidden RPCs, though?

I think it should return empty - no whitelist is defined for the user.

Hmm, should it be designed for simplicity of bitcoind or practicality of clients? I suppose this feature will be most useful for auto-detecting available features, but maybe I'm missing something?

I guess that makes sense, but I think hidden methods should not be included by default.

Do I understand correctly that the hidden methods are used for testing only?

jonatack · 2020-06-03T18:58:48Z

Concept ACK. Will review.

Kixunil · 2020-06-05T08:06:09Z

I do have an idea for extending the whitelist (I actually wanted to suggest it before I learned about whitelist being extendable!): each RPC call could be versioned using semver (two numbers should be enough) so that then the clients can very precisely detect compatibility. New options are added to various RPC methods over time, so it'd be great for the client to know what the node supports. I'd even suggest starting right away and give each method version "1.0".

The advantage over just versioning whole RPC is higher flexibility - one breaking method doesn't have to affect compatibility with a client that doesn't use it. It increases maintenance burden, but hopefully not by too much? Bumping a number whenever RPC method is changed doesn't seem too bad.

promag · 2020-06-05T14:23:24Z

@Kixunil this PR is about returning the whitelist of the current user, not about feature detection nor about interface versioning - IMO server version is enough for the client - so please lets discuss it in the right place.

Kixunil · 2020-06-05T15:41:37Z

@promag fair enough, thanks for clarifying!

luke-jr · 2020-06-05T16:27:51Z

@promag "Server versioning" is not a good method for feature discovery... Knots for example has many RPCs before they're available in Core.

And if there are multiple ways to do something, now you'd have 2 steps: first, figure out if the version supports the "better" way, then check if the "better" way is not excluded by a whitelist.

MarcoFalke · 2020-06-05T16:35:06Z

Knots for example has many RPCs before they're available in Core.

Then, the help RPC should already return them, no?

luke-jr · 2020-06-05T17:43:01Z

Yes, my response was to @promag's proposal that server versioning is a good way to do it. It isn't.

JeremyRubin · 2020-06-05T23:51:15Z

@promag returning the whitelist is kinda fundamentally about feature detection, and that leads to an interesting note...

Whitelists can contain RPCs that don't exist, so it's possible to misuse the output here if you assume the whitelist is valid rpcs.

We probably do want to figure out a robust feature detection call and filter against the whitelists...

promag · 2020-06-06T00:08:34Z

@JeremyRubin IIUC this RPC exposes configuration data and, like you say, might reference inexistent RPC, especially if one goes back and forth different versions with same configuration.

We probably do want to figure out a robust feature detection call and filter against the whitelists...

Right, that's why I suggested to discuss feature discovery elsewhere.

JeremyRubin · 2020-06-06T00:51:01Z

I think we shouldn't make an RPC for whitelists potentially, we should just add a feature discovery API (which is why it's relevant to discuss here)

promag · 2020-06-06T01:07:22Z

I wouldn't consider them exclusive but in that case it makes sense to discuss here.

Kixunil · 2020-06-07T07:18:08Z

What's the use case for purely sending the whitelist anyway? How it could be useful without other things like feature detection?

promag · 2020-06-11T10:56:37Z

What's the use case for purely sending the whitelist anyway? How it could be useful without other things like feature detection?

@Kixunil please see #12248 (comment) for related discussion.

Looks like we should take a different approach then? (to avoid whitelist usage and combine with feature discovery)

Kixunil · 2020-06-11T17:52:33Z

@promag so do I understand correctly the reason why write just whitelist is because it's simple?

luke-jr · 2020-08-20T23:41:55Z

Seems like a generic feature discovery method is a better approach, so closing this

JeremyRubin · 2020-08-21T00:06:25Z

@luke-jr do you have a reference on this? I like feature discovery!

When we discussed in core dev meeting everyone was nacking any generic feature discovery feature.

fanquake added the RPC/REST/ZMQ label May 31, 2020

luke-jr force-pushed the rpc_getrpcwhitelist branch from 0be24a4 to e4f0dab Compare May 31, 2020

This was referenced May 31, 2020

Bugfix: QA: Read response data for non-JSON responses in authproxy #19118

Closed

Bugfix: QA: TestNodeCLI: Behave the same as authproxy for HTTP Forbidden #19119

Closed

QA: Support making RPC calls with different authentication #19120

Closed

This was referenced May 31, 2020

test: Add test for rpcwhitelistdefault #17805

Closed

Log RPC parameters (arguments) if -debug=rpcparams #16365

Draft

brakmic and others added 2 commits May 31, 2020

RPC: Add getrpcwhitelist method

985f0d8

RPC: getrpcwhitelist: Return methods as a JSON Object for future expa…

94fad2e

…nsion to sub-call permissions

luke-jr force-pushed the rpc_getrpcwhitelist branch from e4f0dab to 94fad2e Compare May 31, 2020

bitcoin locked as spam and limited conversation to collaborators May 31, 2020

bitcoin unlocked this conversation May 31, 2020

bitcoin deleted a comment from kantornnam Jun 1, 2020

JeremyRubin mentioned this pull request Jun 1, 2020

Inconsistent RPC error handling #19087

Open

promag reviewed Jun 2, 2020

View changes

JeremyRubin mentioned this pull request Jun 11, 2020

RPC Whitelist Files #12248

Open

DrahtBot mentioned this pull request Jul 16, 2020

rpc: remove deprecated CRPCCommand constructor #18531

Merged

luke-jr closed this Aug 20, 2020

bitcoin locked as resolved and limited conversation to collaborators Feb 15, 2022

RPC: Add getrpcwhitelist method #19117

RPC: Add getrpcwhitelist method #19117

luke-jr commented May 31, 2020

MarcoFalke commented May 31, 2020

DrahtBot commented May 31, 2020 •

edited

luke-jr commented May 31, 2020

promag commented May 31, 2020

MarcoFalke commented May 31, 2020

promag commented Jun 1, 2020 •

edited

MarcoFalke commented Jun 1, 2020

promag left a comment •

edited

promag Jun 2, 2020

luke-jr Jun 2, 2020

Kixunil Jun 5, 2020

MarcoFalke Jun 5, 2020

promag Jun 5, 2020

Kixunil Jun 5, 2020

MarcoFalke Jun 5, 2020 •

edited

Kixunil Jun 5, 2020

jonatack commented Jun 3, 2020

Kixunil commented Jun 5, 2020

promag commented Jun 5, 2020

Kixunil commented Jun 5, 2020

luke-jr commented Jun 5, 2020

MarcoFalke commented Jun 5, 2020

luke-jr commented Jun 5, 2020

JeremyRubin commented Jun 5, 2020

promag commented Jun 6, 2020

JeremyRubin commented Jun 6, 2020

promag commented Jun 6, 2020

Kixunil commented Jun 7, 2020

promag commented Jun 11, 2020

Kixunil commented Jun 11, 2020

luke-jr commented Aug 20, 2020

JeremyRubin commented Aug 21, 2020 •

edited

RPC: Add getrpcwhitelist method #19117

RPC: Add getrpcwhitelist method #19117

Conversation

luke-jr commented May 31, 2020

MarcoFalke commented May 31, 2020

DrahtBot commented May 31, 2020 • edited

Conflicts

luke-jr commented May 31, 2020

promag commented May 31, 2020

MarcoFalke commented May 31, 2020

promag commented Jun 1, 2020 • edited

MarcoFalke commented Jun 1, 2020

promag left a comment • edited

promag Jun 2, 2020

Choose a reason for hiding this comment

luke-jr Jun 2, 2020

Choose a reason for hiding this comment

Kixunil Jun 5, 2020

Choose a reason for hiding this comment

MarcoFalke Jun 5, 2020

Choose a reason for hiding this comment

promag Jun 5, 2020

Choose a reason for hiding this comment

Kixunil Jun 5, 2020

Choose a reason for hiding this comment

MarcoFalke Jun 5, 2020 • edited

Choose a reason for hiding this comment

Kixunil Jun 5, 2020

Choose a reason for hiding this comment

jonatack commented Jun 3, 2020

Kixunil commented Jun 5, 2020

promag commented Jun 5, 2020

Kixunil commented Jun 5, 2020

luke-jr commented Jun 5, 2020

MarcoFalke commented Jun 5, 2020

luke-jr commented Jun 5, 2020

JeremyRubin commented Jun 5, 2020

promag commented Jun 6, 2020

JeremyRubin commented Jun 6, 2020

promag commented Jun 6, 2020

Kixunil commented Jun 7, 2020

promag commented Jun 11, 2020

Kixunil commented Jun 11, 2020

luke-jr commented Aug 20, 2020

JeremyRubin commented Aug 21, 2020 • edited

DrahtBot commented May 31, 2020 •

edited

promag commented Jun 1, 2020 •

edited

promag left a comment •

edited

MarcoFalke Jun 5, 2020 •

edited

JeremyRubin commented Aug 21, 2020 •

edited