Update to LTS-11 #187

ehamberg · 2018-05-14T05:28:43Z

Updates stack.yaml to use LTS-11 (GHC 8.2.2). Not sure if the TODO should be kept.

fkm3

Thanks for updating it!

I'm kicking off the tests.

fkm3 · 2018-05-14T18:20:27Z

stack.yaml

@@ -16,12 +16,9 @@ packages:
 extra-deps:
 - snappy-framing-0.1.1
 - snappy-0.2.0.2
-# TODO: Remove these once the new versions are in lts-9.
+- haskell-src-exts-1.19.1


Can you add a comment explaining why we need an older version? e.g. name the module that needs to be updated

@fkm3 Done!

fkm3 · 2018-05-14T19:28:01Z

I'm getting this error when I try it:

$ stack setup
Downloaded lts-11.8 build plan.    
AesonException "Error in $.packages.cassava.constraints.flags['bytestring--lt-0_10_4']: Invalid flag name: \"bytestring--lt-0_10_4\""

ehamberg · 2018-05-14T19:57:17Z

@fkm3 Looks like this is an issue with some older versions of stack. Can you try to upgrade stack through your package manager or run stack upgrade? From what I can see this was fixed in stack 1.6.1.

fkm3 · 2018-05-14T21:04:20Z

That does fix it locally for me.

The tests use the Dockerfile which gets stack from download.fpcomplete.com/ubuntu. Maybe that repo has an old version of stack. I'll look more closely later today if I get a chance.

Required by tensorflow#187. The version we were using is old enough that it doesn't work with the latest stackage LTS. haskellstack.org says There is also a Ubuntu package for Ubuntu 16.10 and up, but the distribution's Stack version lags behind, ... So, instead of asking them to update it, it's probably better to download the tar of the version we want. Somehow updating stack surfaced a new pedantic warning in GradientTest, so I've fixed that as well (and cleanup some code using snake_case).

Required by tensorflow#187. The version we were using is old enough that it doesn't work with the latest stackage LTS. haskellstack.org says There is also a Ubuntu package for Ubuntu 16.10 and up, but the distribution's Stack version lags behind, ... So, instead of asking them to update it, it's probably better to download the tar of the version we want. Somehow updating stack surfaced a new pedantic warning in GradientTest, so I've fixed that as well (and cleaned up some code using snake_case).

fkm3 · 2018-05-15T03:51:56Z

Sent #189 to fix the stack version.

blackgnezdo · 2018-05-15T04:23:24Z

How bad is it to jump to ghc 8.4 and the nightly stackage snapshot? If we are to experience the pain of the upgrade, might as well reduce the incidence (even at the increased intensity :)

Required by #187. The version we were using is old enough that it doesn't work with the latest stackage LTS. haskellstack.org says There is also a Ubuntu package for Ubuntu 16.10 and up, but the distribution's Stack version lags behind, ... So, instead of asking them to update it, it's probably better to download the tar of the version we want. Somehow updating stack surfaced a new pedantic warning in GradientTest, so I've fixed that as well.

fkm3 · 2018-05-16T03:24:24Z

If you sync this to HEAD, then the CI tests should pass and we can merge it.

@blackgnezdo, won't switching to nightly increase the incident? No opinion from me though :)

blackgnezdo · 2018-05-16T03:44:45Z

I meant pick a dated snapshot of nightly. But I'm also somewhat ambivalent. Just throwing it out there. @ehamberg could you rebase to make sure your change is compatible with HEAD?

ehamberg · 2018-05-16T06:57:18Z

@blackgnezdo Sure. Done.

fkm3 · 2018-05-16T15:29:03Z

You still need to merge baa501b into this PR, otherwise the tests will fail. Thanks!

Required by tensorflow#187. The version we were using is old enough that it doesn't work with the latest stackage LTS. haskellstack.org says There is also a Ubuntu package for Ubuntu 16.10 and up, but the distribution's Stack version lags behind, ... So, instead of asking them to update it, it's probably better to download the tar of the version we want. Somehow updating stack surfaced a new pedantic warning in GradientTest, so I've fixed that as well.

ehamberg · 2018-05-16T17:49:55Z

@fkm3 Ah. Missed that. Rebased on top of the latest master now.

blackgnezdo · 2018-05-16T19:57:16Z

I see some build errors

tensorflow-mnist-0.1.0.0: Test suite ParseTest passed

-- Dumping log file due to warnings: /tfhs/.stack-work/logs/tensorflow-records-conduit-0.1.0.0.log

Configuring tensorflow-records-conduit-0.1.0.0...
Preprocessing library for tensorflow-records-conduit-0.1.0.0..
Building library for tensorflow-records-conduit-0.1.0.0..
[1 of 1] Compiling TensorFlow.Records.Conduit ( src/TensorFlow/Records/Conduit.hs, .stack-work/dist/x86_64-linux/Cabal-2.0.1.0/build/TensorFlow/Records/Conduit.o )

/tfhs/tensorflow-records-conduit/src/TensorFlow/Records/Conduit.hs:40:36: warning: [-Wdeprecations]
In the use of type constructor or class `Conduit'
(imported from Data.Conduit, but defined in conduit-1.3.0.2:Data.Conduit.Internal.Conduit):
Deprecated: "Use ConduitT directly"
|
40 | decodeTFRecords :: MonadThrow m => Conduit B.ByteString m BL.ByteString
| ^^^^^^^

/tfhs/tensorflow-records-conduit/src/TensorFlow/Records/Conduit.hs:44:67: warning: [-Wdeprecations]
In the use of type constructor or class `Producer'
(imported from Data.Conduit, but defined in conduit-1.3.0.2:Data.Conduit.Internal.Conduit):
Deprecated: "Use ConduitT directly"
|
44 | sourceTFRecords :: (MonadResource m, MonadThrow m) => FilePath -> Producer m BL.ByteString
| ^^^^^^^^

/tfhs/tensorflow-records-conduit/src/TensorFlow/Records/Conduit.hs:45:40: warning: [-Wdeprecations]
In the use of `=$='
(imported from Data.Conduit, but defined in conduit-1.3.0.2:Data.Conduit.Internal.Conduit):
Deprecated: "Use .|"
|
45 | sourceTFRecords path = sourceFile path =$= decodeTFRecords
| ^^^

/tfhs/tensorflow-records-conduit/src/TensorFlow/Records/Conduit.hs:48:31: warning: [-Wdeprecations]
In the use of type constructor or class `Conduit'
(imported from Data.Conduit, but defined in conduit-1.3.0.2:Data.Conduit.Internal.Conduit):
Deprecated: "Use ConduitT directly"
|
48 | encodeTFRecords :: Monad m => Conduit BL.ByteString m B.ByteString
| ^^^^^^^

/tfhs/tensorflow-records-conduit/src/TensorFlow/Records/Conduit.hs:52:51: warning: [-Wdeprecations]
In the use of type constructor or class `Consumer'
(imported from Data.Conduit, but defined in conduit-1.3.0.2:Data.Conduit.Internal.Conduit):
Deprecated: "Use ConduitT directly"
|
52 | sinkTFRecords :: (MonadResource m) => FilePath -> Consumer BL.ByteString m ()
| ^^^^^^^^

/tfhs/tensorflow-records-conduit/src/TensorFlow/Records/Conduit.hs:53:38: warning: [-Wdeprecations]
In the use of `=$='
(imported from Data.Conduit, but defined in conduit-1.3.0.2:Data.Conduit.Internal.Conduit):
Deprecated: "Use .|"
|
53 | sinkTFRecords path = encodeTFRecords =$= sinkFile path
| ^^^

: error:
Failing due to -Werror.

-- End of log file: /tfhs/.stack-work/logs/tensorflow-records-conduit-0.1.0.0.log

Log files have been written to: /tfhs/.stack-work/logs/

-- While building custom Setup.hs for package tensorflow-records-conduit-0.1.0.0 using:
/root/.stack/setup-exe-cache/x86_64-linux/Cabal-simple_mPHDZzAJ_2.0.1.0_ghc-8.2.2 --builddir=.stack-work/dist/x86_64-linux/Cabal-2.0.1.0 build lib:tensorflow-records-conduit --ghc-options " -ddump-hi -ddump-to-file"
Process exited with code: ExitFailure 1
Logs have been written to: /tfhs/.stack-work/logs/tensorflow-records-conduit-0.1.0.0.log

Configuring tensorflow-records-conduit-0.1.0.0...
Preprocessing library for tensorflow-records-conduit-0.1.0.0..
Building library for tensorflow-records-conduit-0.1.0.0..
[1 of 1] Compiling TensorFlow.Records.Conduit ( src/TensorFlow/Records/Conduit.hs, .stack-work/dist/x86_64-linux/Cabal-2.0.1.0/build/TensorFlow/Records/Conduit.o )

/tfhs/tensorflow-records-conduit/src/TensorFlow/Records/Conduit.hs:40:36: warning: [-Wdeprecations]
    In the use of type constructor or class `Conduit'
    (imported from Data.Conduit, but defined in conduit-1.3.0.2:Data.Conduit.Internal.Conduit):
    Deprecated: "Use ConduitT directly"
   |
40 | decodeTFRecords :: MonadThrow m => Conduit B.ByteString m BL.ByteString
   |                                    ^^^^^^^

/tfhs/tensorflow-records-conduit/src/TensorFlow/Records/Conduit.hs:44:67: warning: [-Wdeprecations]
    In the use of type constructor or class `Producer'
    (imported from Data.Conduit, but defined in conduit-1.3.0.2:Data.Conduit.Internal.Conduit):
    Deprecated: "Use ConduitT directly"
   |
44 | sourceTFRecords :: (MonadResource m, MonadThrow m) => FilePath -> Producer m BL.ByteString
   |                                                                   ^^^^^^^^

/tfhs/tensorflow-records-conduit/src/TensorFlow/Records/Conduit.hs:45:40: warning: [-Wdeprecations]
    In the use of `=$='
    (imported from Data.Conduit, but defined in conduit-1.3.0.2:Data.Conduit.Internal.Conduit):
    Deprecated: "Use .|"
   |
45 | sourceTFRecords path = sourceFile path =$= decodeTFRecords
   |                                        ^^^

/tfhs/tensorflow-records-conduit/src/TensorFlow/Records/Conduit.hs:48:31: warning: [-Wdeprecations]
    In the use of type constructor or class `Conduit'
    (imported from Data.Conduit, but defined in conduit-1.3.0.2:Data.Conduit.Internal.Conduit):
    Deprecated: "Use ConduitT directly"
   |
48 | encodeTFRecords :: Monad m => Conduit BL.ByteString m B.ByteString
   |                               ^^^^^^^

/tfhs/tensorflow-records-conduit/src/TensorFlow/Records/Conduit.hs:52:51: warning: [-Wdeprecations]
    In the use of type constructor or class `Consumer'
    (imported from Data.Conduit, but defined in conduit-1.3.0.2:Data.Conduit.Internal.Conduit):
    Deprecated: "Use ConduitT directly"
   |
52 | sinkTFRecords :: (MonadResource m) => FilePath -> Consumer BL.ByteString m ()
   |                                                   ^^^^^^^^

/tfhs/tensorflow-records-conduit/src/TensorFlow/Records/Conduit.hs:53:38: warning: [-Wdeprecations]
    In the use of `=$='
    (imported from Data.Conduit, but defined in conduit-1.3.0.2:Data.Conduit.Internal.Conduit):
    Deprecated: "Use .|"
   |
53 | sinkTFRecords path = encodeTFRecords =$= sinkFile path
   |                                      ^^^

<no location info>: error: 
Failing due to -Werror.

[ID: 4581505] Build finished after 975 secs, exit value: 1

Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
Build script failed with exit code: 1[11:12:27] /tmp/workspace/workspace/tensorflow_haskell/github/presubmit/artifacts/bazel_invocation_ids does not exist in the workspace, skipping bazel invocation collection

ehamberg · 2018-05-17T06:37:51Z

@blackgnezdo Looks like it's failing due to deprecation warnings + -Werror. I can have a look at fixing the Conduit warnings tomorrow, probably.

- The `Conduit`, `Producer` and `Consumer` aliases are deprecated and `ConduitT` is used directly instead - `=$=` is deprecated and replaced by `.|`

ehamberg · 2018-05-18T08:11:46Z

@blackgnezdo I believe this is fixed now.

blackgnezdo · 2018-05-18T23:15:37Z

@fkm3, I think we are good to merge this now?

fkm3 · 2018-05-18T23:20:18Z

Thanks!

fkm3 requested changes May 14, 2018

View reviewed changes

fkm3 added the kokoro:run label May 14, 2018

kokoro-team removed the kokoro:run label May 14, 2018

fkm3 mentioned this pull request May 15, 2018

Use newer version of stack in CI #189

Merged

ehamberg force-pushed the lts-11 branch from c6a3e54 to 2e9b3aa Compare May 16, 2018 06:56

ehamberg force-pushed the lts-11 branch from 2e9b3aa to 772e2c1 Compare May 16, 2018 06:59

blackgnezdo added the kokoro:run label May 16, 2018

kokoro-team removed the kokoro:run label May 16, 2018

Update to LTS-11

95f261f

ehamberg force-pushed the lts-11 branch from 772e2c1 to 95f261f Compare May 16, 2018 17:41

fkm3 added the kokoro:run label May 16, 2018

kokoro-team removed the kokoro:run label May 16, 2018

Fix Conduit 1.3 deprecation warnings

cc6d8b0

- The `Conduit`, `Producer` and `Consumer` aliases are deprecated and `ConduitT` is used directly instead - `=$=` is deprecated and replaced by `.|`

blackgnezdo added the kokoro:run label May 18, 2018

kokoro-team removed the kokoro:run label May 18, 2018

fkm3 approved these changes May 18, 2018

View reviewed changes

fkm3 merged commit 88dafe8 into tensorflow:master May 18, 2018

ehamberg deleted the lts-11 branch May 19, 2018 12:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update to LTS-11 #187

Update to LTS-11 #187

ehamberg commented May 14, 2018

fkm3 left a comment

fkm3 May 14, 2018

ehamberg May 14, 2018

fkm3 commented May 14, 2018

ehamberg commented May 14, 2018

fkm3 commented May 14, 2018

fkm3 commented May 15, 2018

blackgnezdo commented May 15, 2018

fkm3 commented May 16, 2018 •

edited

blackgnezdo commented May 16, 2018

ehamberg commented May 16, 2018

fkm3 commented May 16, 2018

ehamberg commented May 16, 2018

blackgnezdo commented May 16, 2018

ehamberg commented May 17, 2018

ehamberg commented May 18, 2018

blackgnezdo commented May 18, 2018

fkm3 commented May 18, 2018

Update to LTS-11 #187

Update to LTS-11 #187

Conversation

ehamberg commented May 14, 2018

fkm3 left a comment

Choose a reason for hiding this comment

fkm3 May 14, 2018

Choose a reason for hiding this comment

ehamberg May 14, 2018

Choose a reason for hiding this comment

fkm3 commented May 14, 2018

ehamberg commented May 14, 2018

fkm3 commented May 14, 2018

fkm3 commented May 15, 2018

blackgnezdo commented May 15, 2018

fkm3 commented May 16, 2018 • edited

blackgnezdo commented May 16, 2018

ehamberg commented May 16, 2018

fkm3 commented May 16, 2018

ehamberg commented May 16, 2018

blackgnezdo commented May 16, 2018

ehamberg commented May 17, 2018

ehamberg commented May 18, 2018

blackgnezdo commented May 18, 2018

fkm3 commented May 18, 2018

fkm3 commented May 16, 2018 •

edited