New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

haskell: building bindings failed #1670

Closed
markus2330 opened this Issue Oct 29, 2017 · 11 comments

Comments

Projects
None yet
3 participants
@markus2330
Contributor

markus2330 commented Oct 29, 2017

Warning: The package list for 'hackage.haskell.org' does not exist. Run 'cabal
update' to download it.
Resolving dependencies...
Warning: solver failed to find a solution:
Could not resolve dependencies:
rejecting: libelektra-haskell-0.8.20:!test (constraint from config file,
command line flag, or user target requires opposite flag selection)
trying: libelektra-haskell-0.8.20:*test
unknown package: QuickCheck (dependency of libelektra-haskell-0.8.20:*test)
Dependency tree exhaustively searched.
Trying configure anyway.
Configuring libelektra-haskell-0.8.20...
cabal: Encountered missing dependencies:
QuickCheck -any, hspec -any

Seems like a cabal update && cabal install QuickCheck hspec is missing to make the binding happy. Docu is quite sparse and does not mention this at all.

@markus2330 markus2330 added the build label Oct 29, 2017

@e1528532 e1528532 referenced this issue Oct 29, 2017

Merged

Haskell bindings check dependencies #1671

5 of 5 tasks complete
@e1528532

This comment has been minimized.

Show comment
Hide comment
@e1528532

e1528532 Oct 30, 2017

Contributor

The build seems to be fixed with the PR. I think @sanssecours did some changes to have the haskell bindings treated differently by the build server / have an own build for that. Is that still necessary? If we can just build this in the normal macOS build we can free up valuable macOS instances so the travis builds are faster.

Contributor

e1528532 commented Oct 30, 2017

The build seems to be fixed with the PR. I think @sanssecours did some changes to have the haskell bindings treated differently by the build server / have an own build for that. Is that still necessary? If we can just build this in the normal macOS build we can free up valuable macOS instances so the travis builds are faster.

@sanssecours

This comment has been minimized.

Show comment
Hide comment
@sanssecours

sanssecours Oct 31, 2017

Contributor

I think @sanssecours did some changes to have the haskell bindings treated differently by the build server / have an own build for that. Is that still necessary?

If you want to continue testing the Haskell bindings, then the answer is probably yes. We could add Haskell to the general build instances, but I fear that would mean that Travis would report a lot of timeouts (Mac build jobs can only run for 50 minutes until they are canceled).

Contributor

sanssecours commented Oct 31, 2017

I think @sanssecours did some changes to have the haskell bindings treated differently by the build server / have an own build for that. Is that still necessary?

If you want to continue testing the Haskell bindings, then the answer is probably yes. We could add Haskell to the general build instances, but I fear that would mean that Travis would report a lot of timeouts (Mac build jobs can only run for 50 minutes until they are canceled).

@sanssecours

This comment has been minimized.

Show comment
Hide comment
@sanssecours

sanssecours Oct 31, 2017

Contributor

I added Haskell to the main Mac build jobs in a branch of my repo. Unfortunately we have to wait until Travis works correctly again to see the results.

Contributor

sanssecours commented Oct 31, 2017

I added Haskell to the main Mac build jobs in a branch of my repo. Unfortunately we have to wait until Travis works correctly again to see the results.

@sanssecours sanssecours referenced this issue Oct 31, 2017

Merged

News: Rephrase Parts of the Release Notes #1661

1 of 1 task complete
@markus2330

This comment has been minimized.

Show comment
Hide comment
@markus2330

markus2330 Oct 31, 2017

Contributor

As haskell is now more important (@e1528532 is actively working on it) we could remove some other less important binding/plugin to reduce build time. What above removing java binding+plugin from the build, would that change the build time? (It might be needed by @vanessaNWK soon but not now.) Or we remove the qt-gui from the build. (Also no changes planned in the near future.)

Is it possible to talk to travis like to jenkins (from github)? Then we could start excluded parts explicitly. (This feature is low priority).

Contributor

markus2330 commented Oct 31, 2017

As haskell is now more important (@e1528532 is actively working on it) we could remove some other less important binding/plugin to reduce build time. What above removing java binding+plugin from the build, would that change the build time? (It might be needed by @vanessaNWK soon but not now.) Or we remove the qt-gui from the build. (Also no changes planned in the near future.)

Is it possible to talk to travis like to jenkins (from github)? Then we could start excluded parts explicitly. (This feature is low priority).

@sanssecours

This comment has been minimized.

Show comment
Hide comment
@sanssecours

sanssecours Oct 31, 2017

Contributor

What above removing java binding+plugin from the build, would that change the build time?

I guess so. However installing Java should be fast (binary package), so I do not think that it would change much unless the Java bindings take a long time to build. BTW: We do not build the JNI (Java) plugin anyway, since it is still broken.

Or we remove the qt-gui from the build. (Also no changes planned in the near future.)

The tool can still break if there are any changes to Qt. You already fixed an issue concerning such a problem.

Is it possible to talk to travis like to jenkins (from github)?

Not that I know of.

Contributor

sanssecours commented Oct 31, 2017

What above removing java binding+plugin from the build, would that change the build time?

I guess so. However installing Java should be fast (binary package), so I do not think that it would change much unless the Java bindings take a long time to build. BTW: We do not build the JNI (Java) plugin anyway, since it is still broken.

Or we remove the qt-gui from the build. (Also no changes planned in the near future.)

The tool can still break if there are any changes to Qt. You already fixed an issue concerning such a problem.

Is it possible to talk to travis like to jenkins (from github)?

Not that I know of.

@e1528532

This comment has been minimized.

Show comment
Hide comment
@e1528532

e1528532 Oct 31, 2017

Contributor

Basically the compilation of the haskell bindings is quite fast, only a few seconds. I guess the issue is that downloading the dependencies takes so long (so updating cabal, then installing c2hs, hspec, QuickCheck). https://docs.travis-ci.com/user/speeding-up-the-build/ says there is the possibility to cache dependencies, so that could be useful to speed things up here. I haven't looked at it in further detail but it says

You can cache arbitrary directories, such as Gradle, Maven, Composer and npm cache directories, between builds by listing them in your .travis.yml:

so if we list the cabal directory to be cached, it wouldn't have to reinstall the dependencies every time and can spare the update i think, speeding up the whole process a lot.

Contributor

e1528532 commented Oct 31, 2017

Basically the compilation of the haskell bindings is quite fast, only a few seconds. I guess the issue is that downloading the dependencies takes so long (so updating cabal, then installing c2hs, hspec, QuickCheck). https://docs.travis-ci.com/user/speeding-up-the-build/ says there is the possibility to cache dependencies, so that could be useful to speed things up here. I haven't looked at it in further detail but it says

You can cache arbitrary directories, such as Gradle, Maven, Composer and npm cache directories, between builds by listing them in your .travis.yml:

so if we list the cabal directory to be cached, it wouldn't have to reinstall the dependencies every time and can spare the update i think, speeding up the whole process a lot.

@sanssecours

This comment has been minimized.

Show comment
Hide comment
@sanssecours

sanssecours Oct 31, 2017

Contributor

I added caching in the latest commit. Things do not look good currently though.

Contributor

sanssecours commented Oct 31, 2017

I added caching in the latest commit. Things do not look good currently though.

@e1528532

This comment has been minimized.

Show comment
Hide comment
@e1528532

e1528532 Oct 31, 2017

Contributor

Hmm. I have an idea what could be wrong, i'll have time around 2 to check it.

Contributor

e1528532 commented Oct 31, 2017

Hmm. I have an idea what could be wrong, i'll have time around 2 to check it.

@sanssecours

This comment has been minimized.

Show comment
Hide comment
@sanssecours

sanssecours Nov 1, 2017

Contributor

I rebased my recent work concerning Travis on the master branch. Either I mis-configered the caching or even with caching, building the Haskell binding/plugin in the main job takes too much time.

Contributor

sanssecours commented Nov 1, 2017

I rebased my recent work concerning Travis on the master branch. Either I mis-configered the caching or even with caching, building the Haskell binding/plugin in the main job takes too much time.

@e1528532

This comment has been minimized.

Show comment
Hide comment
@e1528532

e1528532 Nov 3, 2017

Contributor

Hmm yes the behavior of cabal is indeed strange. First of all cabal update will always take some time as the package repository changes quite often so it most always downloads it again. This shouldn't be necessary if its cached after the first invocation.

Another thing i noticed is that while it doesn't reinstall hspec and QuickCheck unless there are newer versions, it always rebuilds alex, c2hs and happy even if its the same version. It looks like it does that because the latters are programs, don't ask me why though. I also tried the flag --avoid-reinstalls but it doesn't change anything either. You can most likely observe the same behavior if you try it on your machine.

So a possible suggestion to avoid these reinstalls and updates is to check the presence of the programs beforehand, and stop reinstall in case they are already cached.
It should be easy to check if alex, c2hs and happy are available as they are programs, so in that case they'd be on the path. Otherwise run cabal update and install them as now. If the cache works this should avoid this reinstallation issue. On my local machine they are cached/located in /Users/<user>/Library/Haskell/bin, so maybe the /Users/<user>/Library/Haskell directory has to be cached as well.
To check if hspec and QuickCheck are available you could use ghc-pkg latest hspec which returns 0 if the package exists, but i think those 2 librarys are not reinstalled with cabal install automatically so its not necessary.

Contributor

e1528532 commented Nov 3, 2017

Hmm yes the behavior of cabal is indeed strange. First of all cabal update will always take some time as the package repository changes quite often so it most always downloads it again. This shouldn't be necessary if its cached after the first invocation.

Another thing i noticed is that while it doesn't reinstall hspec and QuickCheck unless there are newer versions, it always rebuilds alex, c2hs and happy even if its the same version. It looks like it does that because the latters are programs, don't ask me why though. I also tried the flag --avoid-reinstalls but it doesn't change anything either. You can most likely observe the same behavior if you try it on your machine.

So a possible suggestion to avoid these reinstalls and updates is to check the presence of the programs beforehand, and stop reinstall in case they are already cached.
It should be easy to check if alex, c2hs and happy are available as they are programs, so in that case they'd be on the path. Otherwise run cabal update and install them as now. If the cache works this should avoid this reinstallation issue. On my local machine they are cached/located in /Users/<user>/Library/Haskell/bin, so maybe the /Users/<user>/Library/Haskell directory has to be cached as well.
To check if hspec and QuickCheck are available you could use ghc-pkg latest hspec which returns 0 if the package exists, but i think those 2 librarys are not reinstalled with cabal install automatically so its not necessary.

@e1528532

This comment has been minimized.

Show comment
Hide comment
@e1528532

e1528532 Jan 23, 2018

Contributor

this is fixed for now by having a separate haskell build and caching indeed seems to work now after the first successfull build.

Contributor

e1528532 commented Jan 23, 2018

this is fixed for now by having a separate haskell build and caching indeed seems to work now after the first successfull build.

@e1528532 e1528532 closed this Jan 23, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment