Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wallet: Replace %w by wallet name in -walletnotify script #13339

Merged
merged 2 commits into from Feb 17, 2020

Conversation

@promag
Copy link
Member

@promag promag commented May 29, 2018

Fixes #13237.

@@ -988,6 +988,7 @@ bool CWallet::AddToWallet(const CWalletTx& wtxIn, bool fFlushOnClose)
if (!strCmd.empty())
{
boost::replace_all(strCmd, "%s", wtxIn.GetHash().GetHex());
boost::replace_all(strCmd, "%w", GetName());
Copy link
Member

@laanwj laanwj May 29, 2018

As the wallet name can contain any character, we need shell-escaping here.

@promag
Copy link
Member Author

@promag promag commented May 29, 2018

@laanwj see 4fa29f8, what would be the solution in that case? only escape '? IIUC there is no function to do that as it depends on the shell.

@jamesob
Copy link
Member

@jamesob jamesob commented May 29, 2018

Concept ACK - seems like a nice consideration

@laanwj
Copy link
Member

@laanwj laanwj commented May 29, 2018

@laanwj see 4fa29f8, what would be the solution in that case? only escape '? IIUC there is no function to do that as it depends on the shell.

The best way would be to use an invocation (use one of the exec* functions) that passes arguments without involving the shell. That's quite a big change, though, and existing usages of the arguments might assume a shell is involved.

Lacking that, you can escape for bash and most other shells by surrounding with ' and replacing single-quotes with '"'"'.

It's extremely important to be careful about this, I have seen terrible things happen due to unproperly escaped shell input, and I will NACK this unless this is addressed.

@promag
Copy link
Member Author

@promag promag commented May 29, 2018

@laanwj exec* yes that would be breaking change. Agree on the NACK reasons. I think I'll go with https://unix.stackexchange.com/a/30936. I'll change the test to cover more cases.

@laanwj
Copy link
Member

@laanwj laanwj commented May 29, 2018

@laanwj exec* yes that would be breaking change. Agree on the NACK reasons. I think I'll go with https://unix.stackexchange.com/a/30936. I'll change the test to cover more cases.

Agree. Right - things to test would be $VAR and ${VAR} in the name, $(command) as well as various quote characters.

@promag promag force-pushed the 2018-05-walletnotify branch from 4fa29f8 to 764ee97 May 29, 2018
@promag
Copy link
Member Author

@promag promag commented May 29, 2018

Agree. Right - things to test would be $VAR and ${VAR} in the name, $(command) as well as various quote characters.

I don't understand what you mean. You suggest the wallet name can have $VAR?

@laanwj
Copy link
Member

@laanwj laanwj commented May 30, 2018

I don't understand what you mean. You suggest the wallet name can have $VAR?

Yes, those sequences could be part of the wallet name. They are the typical ways to fool shell escaping and do arbitrary command execution, so I suggested them for testing whether shell escaping is working as expected.

@jonasschnelli
Copy link
Member

@jonasschnelli jonasschnelli commented May 30, 2018

Not sure if we want to further extend walletnotify.
A more future-proof way is probably the recently created RPC approach #13262.
It would still be possible to ship a little python script that emulates the walletnotify "API".

@promag
Copy link
Member Author

@promag promag commented May 30, 2018

IMO we should consider adding this to fully support multi wallets without taking into account #13262. For existing systems that are based on -walletnotify changing to listsincetx would be more problematic. Beside that, currently listsincetx requires a connection for each wallet. I do consider listsincetx better than -walletnotify though.

@laanwj
Copy link
Member

@laanwj laanwj commented May 30, 2018

I think it's fine to add this to walletnotify, too. Agree that another notification system would be better, but this is useful to users in the short term. It's is a small change to the code.

@MarcoFalke
Copy link
Member

@MarcoFalke MarcoFalke commented May 30, 2018

Needs rebase due to merge of #13341

@promag promag force-pushed the 2018-05-walletnotify branch from 764ee97 to cef0327 May 30, 2018
@@ -988,6 +988,9 @@ bool CWallet::AddToWallet(const CWalletTx& wtxIn, bool fFlushOnClose)
if (!strCmd.empty())
{
boost::replace_all(strCmd, "%s", wtxIn.GetHash().GetHex());
std::string name = GetName();
boost::replace_all(name, "'", "'\''");
Copy link
Member

@laanwj laanwj May 31, 2018

Please factor this out to a function ShellEscape which does boost::replace_all(name, "'", "'\''");; return "'" + name + "'". We might need that in other places at some point, and it makes it an easier to review unit.
(also it's possible that for WIN32/cmd this needs something else?)

Copy link
Member Author

@promag promag May 31, 2018

Will do.

@@ -24,7 +24,8 @@ def setup_network(self):
"-blocknotify=echo %%s >> %s" % self.block_filename],
["-blockversion=211",
"-rescan",
"-walletnotify=echo %%s >> %s" % self.tx_filename]]
"-wallet=t- 1",
"-walletnotify=echo %%w\:%%s >> %s" % self.tx_filename]]
Copy link
Member

@luke-jr luke-jr Jul 7, 2018

This would work even without %w properly escaped.

Copy link
Member

@MarcoFalke MarcoFalke Jul 8, 2018

Could just use '...%w\:%s >> {}'.format(self.tx_filename) to avoid this confusion.

@laanwj
Copy link
Member

@laanwj laanwj commented Sep 12, 2018

Needs rebase and comments addressed.

@laanwj laanwj added this to the 0.18.0 milestone Oct 11, 2018
@meshcollider
Copy link
Member

@meshcollider meshcollider commented Nov 12, 2018

LGTM cef0327 but yeah needs rebase + comments addressed 👍

@practicalswift
Copy link
Contributor

@practicalswift practicalswift commented Nov 13, 2018

@promag This no longer compiles:

error: no member named 'replace_all' in namespace 'boost'

Perhaps a good opportunity to get rid of boost::replace_all? :-)

@promag promag force-pushed the 2018-05-walletnotify branch from 0292b09 to f43c5e2 Nov 13, 2018
@promag promag force-pushed the 2018-05-walletnotify branch from f43c5e2 to c8843e4 Nov 13, 2018
src/util/system.cpp Show resolved Hide resolved
@promag promag mentioned this pull request Oct 22, 2019
1 task
@laanwj
Copy link
Member

@laanwj laanwj commented Dec 13, 2019

ACK 8267184

@promag
Copy link
Member Author

@promag promag commented Jan 6, 2020

BTW #17878 adds support for wallet ZMQ notifications which include the wallet name.

@@ -1156,6 +1156,11 @@ bool CWallet::AddToWallet(const CWalletTx& wtxIn, bool fFlushOnClose)
if (!strCmd.empty())
{
boost::replace_all(strCmd, "%s", wtxIn.GetHash().GetHex());
#ifdef WIN32
boost::replace_all(strCmd, "%w", GetName());
Copy link
Contributor

@ryanofsky ryanofsky Jan 8, 2020

This line is unsafe since it can corrupt wallet names and trigger arbitrary shell commands to run from an RPC. Suggested fix from previous thread #13339 (comment) is 26e0017

@promag promag force-pushed the 2018-05-walletnotify branch 3 times, most recently from e19d9c1 to 1c335d5 Jan 9, 2020
@promag
Copy link
Member Author

@promag promag commented Jan 9, 2020

Thanks @ryanofsky, rebased, squashed and added you as co author. Updated test to take into account behavior difference in windows.

Copy link
Contributor

@ryanofsky ryanofsky left a comment

Code review ACK a89a170


class NotificationsTest(BitcoinTestFramework):
def set_test_params(self):
self.num_nodes = 2
self.setup_clean_chain = True

def setup_network(self):
self.wallet = ''.join(chr(i) for i in range(FILE_CHAR_START, FILE_CHAR_END) if chr(i) not in FILE_CHAR_BLACKLIST)
Copy link
Contributor

@ryanofsky ryanofsky Jan 9, 2020

This looks potentially dangerous...

Maybe prefix it with "wallet" so you can tab-complete if you ever need to touch it...

I like luke's suggestion to prefix with something. Maybe a prefix like "feature_notifications" to give a hint where the file comes from, and to help with tab completion

@@ -13,13 +13,19 @@
connect_nodes,
)

# Linux allow all characters other than \x00
Copy link
Contributor

@ryanofsky ryanofsky Jan 9, 2020

Even with various locales? How is Unicode handled?

Should be no issue here. This remains in unicode and utf8 formats until it hits bitcoin filesystem code which uses utf8 directly on non-windows systems, and converts to wide characters for windows apis.

# Windows disallow control characters (0-31) and /\?%:|"<>
FILE_CHAR_START = 32 if os.name == 'nt' else 1
FILE_CHAR_END = 128
FILE_CHAR_BLACKLIST = '/\\?%*:|"<>' if os.name == 'nt' else '/'
Copy link
Contributor

@ryanofsky ryanofsky Jan 9, 2020

What about macOS, BSD, Amiga, etc?

Would be nice to autodetect this in the test.

It shouldn't be too hard to autodetect by just writing to files and seeing what succeeds, but in the worst case this test just doesn't work on an obscure platform and the fix is trivial, so I don't think it's is a big deal

test/functional/feature_notifications.py Outdated Show resolved Hide resolved
@jb55
Copy link
Contributor

@jb55 jb55 commented Jan 10, 2020

Linux Tested ACK 1c335d5

src/wallet/init.cpp Outdated Show resolved Hide resolved
Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
@promag promag force-pushed the 2018-05-walletnotify branch from 1c335d5 to 56d2307 Jan 15, 2020
@fanquake fanquake added this to In progress in Multiwallet support Jan 21, 2020
@promag
Copy link
Member Author

@promag promag commented Feb 17, 2020

Replaced with @ryanofsky amended commit.

@laanwj
Copy link
Member

@laanwj laanwj commented Feb 17, 2020

ACK 4e9efac

laanwj added a commit that referenced this issue Feb 17, 2020
4e9efac test: Check wallet name in -walletnotify script (João Barbosa)
9a5b5ee wallet: Replace %w by wallet name in -walletnotify script (João Barbosa)

Pull request description:

  Fixes #13237.

ACKs for top commit:
  laanwj:
    ACK 4e9efac

Tree-SHA512: 189dd1c785485f2e974d7c12531851b2a977778b3b954aa95efd527322ba3345924cfd587fb9c90b0fa979202af0ab2d90e53d125fe266a36c94f757e4176203
@laanwj laanwj merged commit 4e9efac into bitcoin:master Feb 17, 2020
2 checks passed
@promag promag deleted the 2018-05-walletnotify branch Feb 17, 2020
sidhujag added a commit to syscoin/syscoin that referenced this issue Feb 18, 2020
…fy script

4e9efac test: Check wallet name in -walletnotify script (João Barbosa)
9a5b5ee wallet: Replace %w by wallet name in -walletnotify script (João Barbosa)

Pull request description:

  Fixes bitcoin#13237.

ACKs for top commit:
  laanwj:
    ACK 4e9efac

Tree-SHA512: 189dd1c785485f2e974d7c12531851b2a977778b3b954aa95efd527322ba3345924cfd587fb9c90b0fa979202af0ab2d90e53d125fe266a36c94f757e4176203
@fanquake fanquake moved this from In progress to Done in Multiwallet support Mar 9, 2020
jasonbcox pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this issue Oct 26, 2020
…cript

Summary:
test: Check wallet name in -walletnotify script (João Barbosa)
wallet: Replace %w by wallet name in -walletnotify script (João Barbosa)

Pull request description:

  Fixes #13237.

https://github.com/bitcoin/bitcoin/pull/13339/files

---

Backport of Core [[bitcoin/bitcoin#13339 | PR13339]]

Test Plan:
  ninja all check check-functional

Reviewers: #bitcoin_abc, PiRK

Reviewed By: PiRK

Differential Revision: https://reviews.bitcoinabc.org/D8129
sidhujag added a commit to syscoin-core/syscoin that referenced this issue Nov 10, 2020
…fy script

4e9efac test: Check wallet name in -walletnotify script (João Barbosa)
9a5b5ee wallet: Replace %w by wallet name in -walletnotify script (João Barbosa)

Pull request description:

  Fixes bitcoin#13237.

ACKs for top commit:
  laanwj:
    ACK 4e9efac

Tree-SHA512: 189dd1c785485f2e974d7c12531851b2a977778b3b954aa95efd527322ba3345924cfd587fb9c90b0fa979202af0ab2d90e53d125fe266a36c94f757e4176203
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment