Use preview of pipeline caching in azure builds

[jneira]

• Azure has started recently to offer caching services as a preview: https://docs.microsoft.com/en-us/azure/devops/pipelines/caching
• It has some warts affecting linux and macos builds:
  • Cached files lost their attributes inluding being executables so the cache cant be used out-of-the-box (ironically in windows works). I had to pack the directories in tar files to preserve its attrs..
  • But tar files cant be cached directly cause the tar utility sees them as directories although they are not (!!) .. so i had to put the tar files in a directory to keep them safe

When the first issue is corrected, the unix and macos config will be similar to windows one.

With cache builds terminate in about 15 mins!
See linux and windows builds here: https://dev.azure.com/jneira/haskell-ide-engine/_build/results?buildId=177

[jneira]

It should be $(HOME)/.stack but it seems it cant be used in build variables: https://github.com/MicrosoftDocs/vsts-docs/issues/4337

[fendor]

Two possibilities:
If you can execute a command, something like stack path --stack-root should give you the root.
Otherwise, I think you can change the location of ~/.stack which may also be a solution.

[jneira]

variables is defined prior any other command, so its possible values are limited to predefined variables and expressions. We would have to set in in the .bashrc to use stack path.
I used the default one cause in macos write outside the home directory throwed permission errors (if it doesnt i would use /.stack).

[jneira]

Ah, just in case, values defined in variables are set in all script steps, so stack must use the definition.

[samuelpilz]

I suggest adding a line to the bashrc-files that sets the stack-root since we have other env-variables there. The path uses the same trick: The path variable could only be overwritten, not appended when using the variables section on the yaml

[jneira]

Now we could move the variable to bashrc but if we want to use it directly in the CacheBeta task, like it is in the windows script (and hopefully like we will able to use in linux and macos) the variable only can be in the variables block, cause we can't load the bashrc file in any task but bash one, and it limits somewhat its possible use.
Otoh there is an issue to make possible use the home directory as initial var (we only need the home directory to make sure we can write in the given path for macos)

[samuelpilz]

I didn't think of the issue that the task cannot load the bashrc file. Having a variable does indeed seem to be the best solution.

[jneira]

All envs using the cache with build: https://dev.azure.com/jneira/haskell-ide-engine/_build/results?buildId=179

[jneira]

Btw, shouldn't azure builds be as a check in the pr?

[samuelpilz]

Btw, shouldn't azure builds be as a check in the pr?

Unfortunately, no one with the appropriate rights has added azure as check to this repo. And I do not have the rights to add the required hooks.

[jneira]

Btw, shouldn't azure builds be as a check in the pr?

Unfortunately, no one with the appropriate rights has added azure as check to this repo. And I do not have the rights to add the required hooks.

Well, it is needed to leverage the work done about it, couldn't @fendor or @alanz add the azure build as a check to the repo?
(sorry for ping if it is not the case)

[samuelpilz]

I have created a team haskell-ide-engine https://dev.azure.com/haskell-ide-engine/ and I am willing to invite any collaborator of hie. The main project of this team (https://dev.azure.com/haskell-ide-engine/haskell-ide-engine/_build) still needs a pipeline (just a few clicks)

[samuelpilz]

This looks fine.

Btw @jneira, I would like to add you to the the haskell-ide-engine organization on azure. What is the email adress I should use for the invitation?

[jneira]

atreyu.bbb at gmail dot com would be fine, thanks!

[samuelpilz]

Invitation sent.

Is this ready to merge?

@alanz could you give one of us (@power-fungus, @jneira, @fendor) the permission to add the relevant hooks to this repository? Or I can add you to the azure-team as well

[jneira]

Yeah, it is finished

[samuelpilz]

What is the preferred way to get this to master? merge, rebase, squash?