Implemented "ignore-expiry" flag as in "cabal --ignore-expiry

[l984]

Note: Documentation fixes for https://docs.haskellstack.org/en/stable/ should target the "stable" branch, not master.

Please include the following checklist in your PR:

• [X ] Any changes that could be relevant to users have been recorded in the ChangeLog.md
• [X ] The documentation has been updated, if necessary.

Please also shortly describe how you tested your change. Bonus points for added tests!

This change simply exposes the same functionality which cabal already implements and exposes using "--ignore-expiry" flag.

[dbaynard]

Hi @l984, thanks for the PR.

We usually ask for people to raise issues before requesting a PR, so would you kindly provide answers to the following questions:

• What behaviour does this PR change?
• What is the current behaviour?
• What is the new behaviour?

In addition, you've ticked the box saying 'The documentation has been updated if necessary' but you haven't updated any documentation, and it is necessary. If this PR is to be merged it will need that.

There are also no tests; it would be good to see an integration test for this.

Thanks!

[l984]

Hi and thank you for the quick response!

Original motivation

My original idea was to implement local stack/hackage partial mirror
so that I can (re-)test automatic haskell stack installation without direct
access to Internet and without a need to (re-)download tons of data.

However, stack/hackage package management system apparently implements
some expiration logic which not only prevents from using an outdated
mirror/repository but does not even expose any way to disable expiration
logic.

This PR introduces a new optional "ignore-expiry" boolean flag to the
~/.stack/config.yaml file just for this very purpose.

How the problem looks without the PR

Without this PR when using stack with outdated mirror stack complains
on timestamps. After brief examination I saw that the problem was
related to the file ~/.stack/indices/Hackage/timestamp.json which puts
limits on expiration time. In my case I can see this:

... "expires":"2019-03-09T18:16:28Z" ...

Honestly I do not really understand the motives behind these
restrictions about timestamps. They do not act like a mere "http cache
expiry" but like a real "kill switch" by making repository totally
useless after expiration.

The PR solves this problem.

What does this PR do

The default behaviour remains the same. However, the new optional
enable-expiry flag in the ~/.stack/config.yaml file allows to disable
checking for expired timestamps.

References to the documentaion about timestamp/expiry logic

Documentation about timestamp and expiration time is probably
related to the keywords and sections about Timestamp role,
timestamp.json and Update the root metadata file in The Update
Framework Specification.

Expiry logic is implemented by the checkForUpdates and documentation
for this function explicitly refers to the TUF spec.

Finally, here is cabal documentation about ignore-expiry flag.

Free software

Without any possibility to disable expiration checks the user of
stack/hackage can not use long-living mirrors and has to fully rely on
regular updates on the hackage server. If I understand correctly, any
disruption in hackage update process will result to users not being
able to install haskell stack.

From my point of view this approach takes away control from the user
of the software and gives full control to the publisher. This is very
much in the spirit of Google Play, Android Updates and not in the
spirit of free software.

More things to be done about this PR

Documentation

The PR already has an update to the documentation - see the change in
the doc/yaml_configuration.md file. I could document it a bit more
but the whole secion package-indices is quite brief so I thought
that ignore-expiry should have the same coverage as other fields in
this section.

Tests

It would be difficult to make proper testcases - I would certainly
need help with this. Thank you in advance.

Thank you!

[dbaynard]

Hi @l984, thank you for answering the questions.

Going through your responses, I realised that it would be helpful to have a quick repro (rather than having to wait for the cache to expire). Can you reproduce this by editing your timestamp file?

This PR introduces a new optional ignore-expiry boolean flag to the ~/.stack/config.yaml file just for this very purpose.

This seems reasonable. The scoping looks appropriate; too. I'm slightly concerned that it might result in non-reproducible builds (as a user may make changes to their local hackage, necessary for a package to compile, and other users will not be able to compile that package) but I don't think that is changed by this PR.

References to the documentaion about timestamp/expiry logic

Super helpful — thanks.

Shall we add The hackage security readme, too, which explains the detail (timestamp ensures that the hackage snapshot as a whole is up to date).

Can you edit the timestamp and have the hackage snapshot work, in current stack/cabal-installl?

Honestly I do not really understand the motives behind these restrictions about timestamps.

I don't understand the restrictions. I think the the link I've put just above might help — by my reading, it's for cache invalidation (including preventing 'freeze attacks').

If I understand correctly, any disruption in hackage update process will result to users not being able to install haskell stack.

If by 'hackage update process' you mean any disruption to you updating your copy, then you're not quite correct. Stack is aware of hackage mirrors, which is useful on the uncommon occasions that the main hackage mirror is down (it used to use the fpcomplete mirror by default — I believe that's been changed, now).

The update may fail because of local connectivity problems. In that case, will stack (or cabal-install) use the cached version, or refuse because it has expired?

If by 'hackage update process' you mean that hackage itself is no longer updated, then yes there will be problems, but I would not expect them to differ from those with other similar public services.

The PR already has an update to the documentation - see the change in the doc/yaml_configuration.md file. I could document it a bit more…

Yes, please.

It would be difficult to make proper testcases - I would certainly need help with this.

If updating the timestamp manually fixes the issue (and indeed triggers it) that's a way to proceed.