Checksum requirement and enhancements #112

Closed
ferventcoder opened this Issue Feb 22, 2015 · 6 comments

Projects

None yet

4 participants

@ferventcoder
Member
ferventcoder commented Feb 22, 2015 edited

Give the user more options when dealing with downloads

  • Allow user to pass their own checksum (and checksum type)
  • --ignore-checksums (done with #33)
  • Provide pending requirement for checksums for packages missing them (allows community to let maintainers know they need to update their packages).
  • Do not allow installs from downloaded binaries that are not checksummed from the package by default.
  • Create feature allowEmptyChecksums set to false by default.
  • Allow passing switch --allow-empty-checksums, --require-checksums at runtime to override default behavior.
  • Create feature allowEmptyChecksumsSecure set to true by default (for HTTPS).

Choco v1 will not allow package installs without checksums by default. For folks to be insecure, they will need to explicitly ignore checksumming or be more secure by providing their own. This has been moved up to 0.10.0.

Bumping up the priority on this after a recent incident - see https://twitter.com/ferventcoder/status/760892254542102528 and http://us8.campaign-archive1.com/?u=86a6d80146a0da7f2223712e4&id=f2fe8dbe6b for details.

NOTE: We decided to relax the checksum missing failure to only HTTP for now, but will require it for HTTPS as well when #895 releases.

Running into this issue?

Option 1 - Pass the checksums through

You can pass the checksums through to the install / upgrade:

choco install somepackage --download-checksum cheksumvalue --download-checksum-type checksumtypehere

NOTE: Does not work in the current Boxstarter version as it is using chocolatey.lib and and older version of Chocolatey.

See https://chocolatey.org/docs/commands-install / https://chocolatey.org/docs/commands-upgrade:

    --checksum, --downloadchecksum, --download-checksum=VALUE
     Download Checksum - a user provided checksum for downloaded resources 
       for the package. Overrides the package checksum (if it has one).  
       Defaults to empty. Available in 0.10.0+.

     --checksum64, --checksumx64, --downloadchecksumx64, --download-checksum-x64=VALUE
     Download Checksum 64bit - a user provided checksum for 64bit downloaded 
       resources for the package. Overrides the package 64-bit checksum (if it 
       has one). Defaults to same as Download Checksum. Available in 0.10.0+.

     --checksumtype, --checksum-type, --downloadchecksumtype, --download-checksum-type=VALUE
     Download Checksum Type - a user provided checksum type. Overrides the 
       package checksum type (if it has one). Used in conjunction with Download 
       Checksum. Available values are 'md5', 'sha1', 'sha256' or 'sha512'. 
       Defaults to 'md5'. Available in 0.10.0+.

     --checksumtype64, --checksumtypex64, --checksum-type-x64, --downloadchecksumtypex64, --download-checksum-type-x64=VALUE
     Download Checksum Type 64bit - a user provided checksum for 64bit 
       downloaded resources for the package. Overrides the package 64-bit 
       checksum (if it has one). Used in conjunction with Download Checksum 
       64bit. Available values are 'md5', 'sha1', 'sha256' or 'sha512'. 
       Defaults to same as Download Checksum Type. Available in 0.10.0+.

Option 2 - Fix the package to download over HTTPS if possible, and use checksums

Update the package accordingly so it complies with the new security enhancements.

Option 3 - If the package is internal, embed the resources in the package (or put on an internal share)

Many organizations using Chocolatey internally will create packages that use internal or embedded resources. They also go through a process called recompiling to download an existing package and recompile it to use internal resources - https://chocolatey.org/docs/how-to-recompile-packages

Option 4 - Accept the risk

Now we are into the non-recommended options. So if you want to use this package and you are fine with the risks over a package downloading over HTTP without verifying the integrity of the binaries then you can use the following:

choco install somepackage --allow-empty-checksums

If you are using a tool that is using an older version of Chocolatey, like Boxstarter or ChocolateyGUI, but you've upgraded your local Chocolatey to v0.10.0, you may want to use:

choco install somepackage --ignore-checksums

See https://chocolatey.org/docs/commands-install / https://chocolatey.org/docs/commands-upgrade:

     --ignorechecksum, --ignore-checksum, --ignorechecksums, --ignore-checksums
     IgnoreChecksums - Ignore checksums provided by the package. Overrides 
       the default feature 'checksumFiles' set to 'True'. Available in 0.9.9.9+.

     --allowemptychecksum, --allowemptychecksums, --allow-empty-checksums
     Allow Empty Checksums - Allow packages to have empty/missing checksums 
       for downloaded resources from non-secure locations (HTTP, FTP). Use this 
       switch is not recommended if using sources that download resources from 
       the internet. Overrides the default feature 'allowEmptyChecksums' set to 
       'False'. Available in 0.10.0+.

     --allowemptychecksumsecure, --allowemptychecksumssecure, --allow-empty-checksums-secure
     Allow Empty Checksums Secure - Allow packages to have empty checksums 
       for downloaded resources from secure locations (HTTPS). Overrides the 
       default feature 'allowEmptyChecksumsSecure' set to 'True'. Available in 
       0.10.0+.

     --requirechecksum, --requirechecksums, --require-checksums
     Require Checksums - Requires packages to have checksums for downloaded 
       resources (both non-secure and secure). Overrides the default feature 
       'allowEmptyChecksums' set to 'False' and 'allowEmptyChecksumsSecure' set 
       to 'True'. Available in 0.10.0+.

Option 5 - Accept the risk for everything

You can turn off the behavior by running an exec that will turn on the feature allowEmptyChecksums. This is a nuclear option and subverts the protections brought in by this feature. You can also turn off the feature checksumFiles to turn off checksumming entirely.

Option 6 - Downgrade Chocolatey

Downgrading to an older version doesn't remove the risk that is there in downloading resources from the internet. 0.10.0 is more upfront about those issues so you can make an informed, aware decision on what you are installing.

@ferventcoder ferventcoder added this to the 0.9.10 milestone Feb 22, 2015
@dtgm
Contributor
dtgm commented Feb 22, 2015

Is the feature choco feature disable -n=checksumFiles intended to be the equivalent of always passing --ignore-checksum?

@ferventcoder
Member

@dtgm yes and that is #33 - note that right now that feature is ignored. It was only left in configuration because previous choco had it.

@ferventcoder ferventcoder modified the milestone: 0.9.10, 0.9.10.1 Oct 3, 2015
@ferventcoder ferventcoder modified the milestone: 0.9.10.4, 0.9.10.5 Aug 3, 2016
@ferventcoder ferventcoder self-assigned this Aug 3, 2016
@ferventcoder ferventcoder changed the title from Checksum enhancements to Checksum requirement and enhancements Aug 9, 2016
@ferventcoder ferventcoder added a commit that referenced this issue Aug 10, 2016
@ferventcoder ferventcoder (GH-112) Allow user passed checksums
Allow users to pass their own checksums for packages. This enables
consumers to do their own 3rd party verification of remote resources
that are downloaded.
990b699
@ferventcoder ferventcoder added a commit that referenced this issue Aug 10, 2016
@ferventcoder ferventcoder (GH-112) Checksum remote files unless checksum
If the origination of the file is somewhere local, do not require
checksumming the file unless a checksum is passed.
685d8c2
@ferventcoder ferventcoder added a commit that referenced this issue Aug 10, 2016
@ferventcoder ferventcoder (GH-112) Allow Empty Checksums
With the move to requiring checksums for remote resources, allow a user
to turn off the requirement for checksums when packages download remote
resources.
f109679
@ferventcoder ferventcoder added a commit that referenced this issue Aug 10, 2016
@ferventcoder ferventcoder (GH-112) Require checksums
Checksums are now required when downloading remote resources. This
provides safety when using Chocolatey with packages that may not be
safe. Any remote resource requires a checksum, as both HTTP and HTTPS
sources could be compromised and the downloaded binaries need to be
validated for integrity against what was originally intended by the
packager.

A user can choose to provide their own or override the default safety
mechanism with `--allow-empty-checksums`, but this is a user decision
and not solely a package maintainer decision anymore.
7357857
@ferventcoder ferventcoder added a commit that referenced this issue Aug 10, 2016
@ferventcoder ferventcoder (GH-112) Update parameter documentation
Note that checksums are now required.
533e6ae
@ferventcoder ferventcoder added a commit that referenced this issue Aug 10, 2016
@ferventcoder ferventcoder (GH-112) Checksums / Install Template
Provide insight that checksums are now required and how a maintainer
can go about gathering that information. Also suggest urls use https
instead of just http.
a829e94
@ferventcoder ferventcoder added a commit that referenced this issue Aug 10, 2016
@ferventcoder ferventcoder (GH-112) update options for new version
0.9.10.4 is now 0.10.0
25af57d
@ferventcoder ferventcoder added a commit that referenced this issue Aug 10, 2016
@ferventcoder ferventcoder (GH-112) Relax HTTPS checksum failure
If downloaded from https location, relax checksum missing failure
until 0.10.1.
a2af077
@ferventcoder ferventcoder added a commit that referenced this issue Aug 10, 2016
@ferventcoder ferventcoder Merge branch 'stable'
* stable: (23 commits)
  (GH-897) Fix DISM /all for newer Windows versions
  (maint) bring back in nuspec into solution folders
  (GH-598) Add output directory for choco pack
  (version) bump to 0.10.0
  (doc) update CHANGELOG/nuspec
  (GH-840) Do not set User Environment for SYSTEM
  (GH-840) Set TMP to cacheLocation as well
  (GH-830) Ensure process PSModulePath items
  (GH-112) Relax HTTPS checksum failure
  (maint) move noop run
  (GH-833) Don't set profile under certain conditions
  (doc) update generated docs
  (GH-112) update options for new version
  (GH-112) Checksums / Install Template
  (GH-112) Update parameter documentation
  (GH-112) Require checksums
  (GH-112) Allow Empty Checksums
  (GH-112) Checksum remote files unless checksum
  (GH-112) Allow user passed checksums
  (GH-112) rename checksum files for proper casing
  ...
459d60d
@ferventcoder ferventcoder added a commit that referenced this issue Aug 11, 2016
@ferventcoder ferventcoder (GH-112) Empty Checksums for Secure Locations
Secure locations, such as HTTPS, have quite a bit of protection
surrounding them. By default allow missing/empty checksums when
the locations are secure. Allow the feature to be shut off either by
turning off the feature `allowEmptyChecksumsSecure` or using the switch
`--require-checksums`.
48c33dd
@ferventcoder ferventcoder added a commit that referenced this issue Aug 11, 2016
@ferventcoder ferventcoder (GH-112) clear existing environment vars
Ensure there is no outside influence for environment variables that are
set by choco.
2cbb40f
@ferventcoder ferventcoder added a commit that referenced this issue Aug 11, 2016
@ferventcoder ferventcoder (GH-112) Get-ChecksumValid - update debug params
When debugging parameters are passed, ensure that originalUrl is also
passed for debugging purposes.
fd9de3f
@ferventcoder ferventcoder added a commit that referenced this issue Aug 11, 2016
@ferventcoder ferventcoder Merge branch 'stable'
* stable:
  (GH-112) update tab expansion
  (doc) update generated docs
  (GH-112) Get-ChecksumValid - update debug params
  (GH-833) Don't set tab completion if profile null
  (doc) update CHANGELOG/nuspec
  (GH-866) Fix - packages.config fails on licensed config
  (GH-112) clear existing environment vars
  (doc) add checksum verification notes
  (GH-112) Empty Checksums for Secure Locations
  (maint) remove magic strings for environment vars
  (maint) move environment vars to env section
d5e1850
@adcreare adcreare added a commit to adcreare/ChocolateyPackages that referenced this issue Aug 16, 2016
@adcreare adcreare Updated urls to use HTTPS over HTTP
- changed this due to changes made in the choco client chocolatey/choco#112
- and issues this has caused to existing build processes chocolatey/choco#910
6c8cff7
@activescott

For "option 2" above, I suggest the briefest of explanations of how to do that with a link to the appropriate documentation. Presumably it is:

Supply a checksum value to the checksum and/or checksum64 parameters of the HelpersInstallChocolateyZipPackage.

@activescott

Also, I'm using Install-ChocolateyZipPackage to install a local zip file that is embed in the pkg. Are local files exempt from the upcoming checksum requirement or will I need to use Get-ChocolateyUnzip (apparently I didn't see that or it wasn't there when we created lessmsi's package)?

...or should I just supply a checksum to Install-ChocolateyZipPackage?

@activescott activescott referenced this issue in activescott/lessmsi Sep 4, 2016
Closed

choco fix for upcoming checksum requirement #73

@ferventcoder
Member

@activescott local files won't need it. Get-ChocolateyUnzip has always been used by Install-ChocolateyZipPackage so you just likely missed it. Plus the documentation surrounding functions has recently had a huge makeover, so it's much better now, especially in discovery of other methods.

@aaronk1
aaronk1 commented Sep 15, 2016

@ferventcoder Awesome post. Especially appreciate the boxstarter note. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment