[#239][#249] Further filepath refactor

[aeqz]

Description

Problem: after refactoring the FilePath usages in the codebase to have a canonical representation of them, we noticed that further improvements could be applied, such as clarifying whether the path is system dependent and avoiding absolute file system paths.

Solution: we now use POSIX relative paths during the analysis, and system dependant ones for reading file contents in the scan phase.

Related issue(s)

Fixes #239
Fixes #249
Relates #219

✅ Checklist for your Pull Request

Related changes

• Tests

  • If I added new functionality, I added tests covering it.
  • If I fixed a bug, I added a regression test to prevent the bug from
    silently reappearing again.

• Documentation

  • I checked whether I should update the docs and did so if necessary:
    • README
    • Haddock

• Public contracts

  • Any modifications of public contracts comply with the Evolution
    of Public Contracts policy.
  • I added an entry to the changelog if my changes are visible to the users
    and
  • provided a migration guide for breaking changes if possible

Stylistic guide

• My commits comply with the policy used in Serokell.
• My code complies with the style guide.

[Martoon-00]

	1 Warning
⚠️	Please, only use letters, digits and dashes in the branch name. Found: ["#"]

Generated by 🚫 Danger

[aeqz]

How should I properly write the PR and commit issue id referring to two different issues?

I have tackled both of them at the same time because, when trying to refine when POSIX links are used and when platform dependent ones, I realised that we just need relative POSIX links during the verification phase (either relative to the repository root or to other repository file) and system dependent ones during the scan phase, which do not require further processing.

[Martoon-00]

I have tackled both of them at the same time because

Yeah, sounds fair.

Usually we write e.g. [#123][#456] Text in PR title or commit subject if it concerns multiple tickets.

[Martoon-00]

I reviewed superficially for now, and quite like how it looks like.

And otherall to me this looks simpler and more robust now.

@YuriRomanowski I would also appreciate your review, if you find time; meanwhile I'll try to spend more time on your PRs.

[aeqz]

This seems quite arbitrary but, as now RelPosixLinks correspond to the original ones, with no normalisation applied, the output was displaying paths like "././file.md" (and this would be the same even by using </> from System.FilePath, I just changed it to avoid String <-> Text conversions):

ghci> import System.FilePath
ghci> "." </> "./file.md"
"././file.md"

[aeqz]

Okay, I guess it is now better than before. Although if I got from the beginning that everything is consistent in this regard, I would probably abort my suggestion to change the existing code.

I will undo that change. As you say, it was already consistent.

Now I wonder if counting "other" references as local progress could be better initialised by using lenses:

vrLocal = initProgress $
  (length $ references ^.. folded . to rInfo . _RIFile) +
  (length $ references ^.. folded . to rInfo . _RIExternal . _ELOther)

This is not valid, but something similar would be cool instead:

vrLocal = initProgress $ length $ 
  references ^.. folded . to rInfo . (_RIFile <|> _RIExternal . _ELOther)

[Martoon-00]

I will undo that change. As you say, it was already consistent.

Thanks 🙏

This is not valid, but something similar would be cool instead:

Oh, I cannot miss this call for perfection!

AFAIR internally Traversals are about lists, not Maybes, so you should try <> instead of <|>.

Plus _RIFile and _RIExternal . _ELOther may return different types, so to lead everything to one type I would use united lens.

This works for me:

> let vals = [ Left "a", Right (Left 0), Right (Right 1) ] :: [Either String (Either Word Int)]
> vals ^.. L.folded . (L._Left . L.united <> L._Right . L._Right . L.united)
[(),()]

[aeqz]

AFAIR internally Traversals are about lists, not Maybes, so you should try <> instead of <|>

Nice! I was also trying to unify types, but to Either instead of (), and also thinking that this sum type getters are "partial", so that's why my intuition was going more like "try with this getter that goes to Maybe (Left a) or, if it fails, this other one that goes to Maybe (Right b). But, as you pointed out, ^.. is about lists, not maybes.

Your proposal is simpler and worked perfectly 👌

[Martoon-00]

One more comment

[aeqz]

Mmm, this seems like a justified case to start using OS-dependent bats tests 🤔 Like, set up some env variable in CI config, and rely on it in the test?

I would be worried about some points anyways:

• We should set the corresponding OS env variable for running bats tests in our device.
• Perhaps some Windows user or developer could have some trouble when cloning the repository if it contains such test files. I currently don't have a Windows device for trying it.

[Martoon-00]

Mmm, this seems like a justified case to start using OS-dependent bats tests thinking Like, set up some env variable in CI config, and rely on it in the test?

I would be worried about some points anyways:

* We should set the corresponding OS env variable for running bats tests in our device.

* Perhaps some Windows user or developer could have some trouble when cloning the repository if it contains such test files. I currently don't have a Windows device for trying it.

That's true, I forgot that tests are run not only by CIs.

Hmm, if I remember correctly, we already had tests that imply different output under Linux and Windows.

That test is written as assert_diff ... || assert_diff ... which is a bit sad, but if in our case at Windows we expect quite custom "File does not exist", I think that should work. What would you say?

[aeqz]

That test is written as assert_diff ... || assert_diff ... which is a bit sad, but if in our case at Windows we expect quite custom "File does not exist", I think that should work. What would you say?

I tried to add my example and the Windows CI failed: the repo cannot be cloned if it contains the a\a.md file. Then I tried to create this file on the fly during the test and it cannot be created on Windows, so I made the test to pass if this file cannot be created.

[Martoon-00]

The last request: do we have tests on paths case-sensitivity? That aBc.md file would not be considered as referred by abc.md link?

I think that would be aa finishing touch for this refactoring.

[aeqz]

The last request: do we have tests on paths case-sensitivity?

We have tests for anchor case-sensitivity, but not for path case-sensitivity. I will add them 👍

[Martoon-00]

Great, thank you for this work!

Please squash the fixup commits and let's merge.