Support for The Update Framework (TUF) #719

Open
wants to merge 28 commits into
from

Conversation

Projects
None yet
8 participants

This PR contains an implementation of a client for The Update Framework (TUF) which provides a complementary/supplemental security system for the existing metadata used by RubyGems.

We've documented our plans here:

https://docs.google.com/document/d/1o68vVLAgqyK--u0sfJx8ORlDSjc3oyMtD8dj7kNuOa0/

Similar work is being done in Python to integrate TUF with PyPI:

https://github.com/theupdateframework/pep-on-pypi-with-tuf

TUF provides a security model for software update systems which uses a "least authority" approach with multiple roles with limited power and a combination of online and offline keys:

TUF Trust Model

We (by which I mean @xaviershay ;) have also implemented server support for RubyGems.org which we'll be submitting in a separate PR.

There are a number of roles in TUF and the existing work doesn't implement all of them. Here's a checklist of what's been accomplished and what's needed:

  • Gem::TUF::PublicKey
  • Gem::TUF::PrivateKey/KeyStore?
  • Gem::TUF::Signer/Verifier
  • Gem::TUF::Root
  • Gem::TUF::Targets
  • Gem::TUF::Targets::Claimed (offline delegation to offline developer keys)
  • Gem::TUF::Targets::RecentlyClaimed (online delegation to offline developer keys)
  • Gem::TUF::Targets::Unclaimed (grandfather in old gems)
  • Gem::TUF::Releases
  • Gem::TUF::Timestamp
  • Gem::RemoteFetcher support for TUF
  • Ruby 1.8 support for JSON
  • Move CanonicalJSON to the Gem::TUF namespace

should be a check here to not add duplicate entries for the same key id (i.e. make this method idempotent). Probably want to raise if the digests are different.

Yeah, definitely

Tony Arcieri and others added some commits Nov 20, 2013

Needs to be under the Gem namespace

Tony Arcieri and others added some commits Nov 20, 2013

Tony Arcieri Remove Gem::TUF::Role in lieu of keyid/key hashes
The Gem::TUF::Root#verify method encapsulates verification, so having a
separate object that encapsulates verification data isn't helpful.

Instead, we can pass around keyid => key hashes for each role. This
removes all the code needed to calculate keyids from keys, which was
wrong anyway.
0c67dbb
Tony Arcieri Derp, forgot to run tests before committing :( 3fc512b
Charlie Sanders and Neal Harris release object, release.txt and bad_release.txt 3300a9a
Charlie Sanders and Neal Harris Adds Gem::TUF::Release to verify release.txt 2ccfc8b
Tony Arcieri Have Gem::TUF::Verifier check expiration dates 4ff37ec
Tony Arcieri Fix for style 554406f
Charlie Sanders and Neal Harris changed release.txt sig; added some root_update tests e1b1733
Charlie Sanders and Neal Harris test refactor 3949228
@xaviershay xaviershay Support processing of gems that have already been loaded into an IO o…
…bject.

In particular, this feature is used by rubygems.org to extract metadata
from uploaded gems that have not yet been written to storage.
ca30745
@nealharris nealharris script to generate keys and metadata for each role b183543
@nealharris nealharris whitespace ea1d826
@nealharris nealharris add dummy target file fa11e3d
Tony Arcieri Initial Gem::TUF::PublicKey class
This extracts the concerns around TUF public keys (i.e.
[canonical] JSON serialization and keyid calculation) into a
Gem::TUF::PublicKey class
dfb7aa0
@nealharris nealharris fixed delegations 5d965a2
@nealharris nealharris cleanup c17f9c2
@nealharris nealharris put key_id method back 23743fa
@nealharris nealharris put back tony's work 9f1c23a
@tarcieri tarcieri Fix expiration dates
Moved to a 1 year offset
7f7d258

@drbrain drbrain commented on the diff Nov 26, 2013

lib/rubygems/tuf/release.rb
@@ -0,0 +1,24 @@
+require 'json'
+
+class Gem::TUF::Release
+ attr_reader :targets
+
+ def initialize root, release_txt
+ parsed = JSON.parse release_txt
+ @release = root.verify(:release, parsed)
+ @targets = @release["meta"]["targets.txt"]
+ end
+
+ def should_update_root? current_root_txt
+ @release["meta"]["root.txt"]["hashes"].each do |type, expected_digest|
+ current_digest = case type
@drbrain

drbrain Nov 26, 2013

Owner

Why not use Digest.const_get type.upcase here, along with excluding weak hashes. This will allow older clients to work if a stronger digest is chosen, so long as ruby supports it.

@drbrain drbrain commented on the diff Nov 26, 2013

lib/rubygems/util/canonical_json.rb
@@ -0,0 +1,27 @@
+# let's shoot for http://tools.ietf.org/html/draft-staykov-hu-json-canonical-form-00
+module CanonicalJSON
+ # shamelessly copied from
+ # https://github.com/tent/tent-canonical-json-ruby/blob/master/lib/tent-canonical-json.rb
@drbrain

drbrain Nov 26, 2013

Owner

If this is license-compatible or otherwise approved can this be explicitly stated?

@drbrain drbrain commented on the diff Nov 26, 2013

test/rubygems/test_gem_package.rb
@@ -778,6 +778,15 @@ def test_spec
assert_equal @spec, package.spec
end
+ def test_spec_from_io
+ # This functionality is used by rubygems.org to extract spec data from an
+ # uploaded gem before it is written to storage.
@drbrain

drbrain Nov 26, 2013

Owner

I think this comment should go on the implementation

Owner

drbrain commented Nov 26, 2013

I would like to see # :nodoc: added to internal TUF methods that users shouldn't touch. It'll make my job as maintainer easier. Also, some more method documentation would be a nice-to-have.

@paracycle paracycle referenced this pull request in sferik/twitter Mar 10, 2014

@sferik sferik Stop signing gem [ci skip] 0291149

@drbrain drbrain modified the milestone: Future, 2.3 May 28, 2014

copiousfreetime removed the triage label Jan 30, 2016

Contributor

copiousfreetime commented Jan 30, 2016

Is TUF still on the roadmap? @drbrain @indirect

Owner

indirect commented Jan 31, 2016

Yes, it's on the list after we launch the new index and get RubyGems a bit more maintained.

lynnco removed this from the Future milestone Jun 9, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment