New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Excessive memory usage while "Resolving dependencies..." #4059

Open
alexbiehl opened this Issue Oct 29, 2016 · 21 comments

Comments

Projects
None yet
4 participants
@alexbiehl
Member

alexbiehl commented Oct 29, 2016

Activity monitoring while cabal new-build in cabal repository root. Unfortunately I only have 8gb of RAM!

bildschirmfoto 2016-10-29 um 19 18 33

@alexbiehl

This comment has been minimized.

Show comment
Hide comment
@alexbiehl

alexbiehl Oct 29, 2016

Member

Trace:

Alexanders-iMac:cabal alexbiehl$ cabal new-build -v3
File monitor 'improved-plan' unchanged.
Number of threads used: 8.
buildTargets': fromList [(UnitId
"Cabal-1.24.0.0",[BuildDefaultComponents,BuildDefaultComponents,BuildDefaultComponents]),(UnitId
"Cabal-1.25.0.0-inplace",[BuildDefaultComponents,BuildDefaultComponents,BuildDefaultComponents]),(UnitId
"Cabal-1.25.0.0-inplace-package-tests",[BuildDefaultComponents,BuildDefaultComponents,BuildDefaultComponents]),(UnitId
"Cabal-1.25.0.0-inplace-unit-tests",[BuildDefaultComponents,BuildDefaultComponents,BuildDefaultComponents]),(UnitId
"HTTP-4000.3.3-83c012783b085c0c232f7f844546a02a7093f8c90291bc211fe270976aca2e12",[BuildDefaultComponents]),(UnitId
"QuickCheck-2.9.2-9489342e2e0a905fde8fe760390109eaf6d198f2165f31e452c4b8483c8d9b97",[BuildDefaultComponents]),(UnitId
"ansi-terminal-0.6.2.3-e74a6ec64c54bae09e3e83e38f4bada2c6f8abd7613da1a86f727ec98c8e2f85",[BuildDefaultComponents]),(UnitId
"ansi-wl-pprint-0.6.7.3-27d2be9cba49b3882a1a60d9a087aa6fe6295b639979778f74174818868f520e",[BuildDefaultComponents]),(UnitId
"async-2.1.0-e26d74c696c3c67ea814fb65cdcc1f9be64eb47d410387f08a8cd9ef0ecc0de6",[BuildDefaultComponents]),(UnitId
"base16-bytestring-0.1.1.6-172d6f6b954a6ba445f81a57e2808feb08885a2bb0924ece25c0ac805ff938d6",[BuildDefaultComponents]),(UnitId
"base64-bytestring-1.0.0.1-db138f8c14e9745c36eadbf18d8b131a1467fb63c4d44ada63f93aaf38a67ba4",[BuildDefaultComponents]),(UnitId
"cabal-install-1.25.0.0-inplace",[BuildDefaultComponents]),(UnitId
"clock-0.7.2-0e54de7d673bf986da1791be0da87585f245ba3bc8e5bcf337bc26e02f91d4a8",[BuildDefaultComponents]),(UnitId
"cryptohash-sha256-0.11.100.1-46012f043d93da78eb72d331df2eefaf6386531f09dc716700c11b5a82d2647f",[BuildDefaultComponents]),(UnitId
"ed25519-0.0.5.0-bb2de528f86166521b399a681462dedb06601569cd1a9f24cca522f91692d540",[BuildDefaultComponents]),(UnitId
"edit-distance-0.2.2.1-988881a5ee47c0f0fa144b85f79f805ba2bb8a83f2e9bacac67ad7842acb4dab",[BuildDefaultComponents]),(UnitId
"exceptions-0.8.3-ed2cabc53de91367d33967cc314bc0e6f53dcfaee1bfa2c8a15bcfb158985973",[BuildDefaultComponents]),(UnitId
"hackage-security-0.5.2.2-inplace",[BuildDefaultComponents]),(UnitId
"happy-1.19.5-c2aa70908dd132856d459fad09f14b15eb6c52d1fd2dca1a19670e7034aa1edf",[BuildDefaultComponents]),(UnitId
"hashable-1.2.4.0-fe3e343ea609f3e13d10e5df6b016ecf4a08f8ca135bac97e6608be0249d69d3",[BuildDefaultComponents]),(UnitId
"haskell-lexer-1.0.1-e4470ba8a3d81fc06ef3df636f68f016d1fb4e50bb4cc550d6a6af7c4b7db497",[BuildDefaultComponents]),(UnitId
"mtl-2.2.1-e511aea57dffccfd5bc7e1cf3607b606a6afaa58535fe85b2e086d0f2f0c0d87",[BuildDefaultComponents]),(UnitId
"network-2.6.3.1-5d0efde49b47f3555df2e717ddce6ba1aae7a861d5a250494f33473aa8c68503",[BuildDefaultComponents]),(UnitId
"network-uri-2.6.1.0-2a33e0023d57859b3f613e84d64695651c5103aa20166751ab8427bf90f7ba3e",[BuildDefaultComponents]),(UnitId
"old-locale-1.0.0.7-b9dc6b834e03c41379cdc21cfbf76a88ebfb755814b872765db2820a2e5ae1bb",[BuildDefaultComponents]),(UnitId
"old-time-1.1.0.3-866f369d19a8b1534f5ea8bf2b9b93eed2050259e6b83c39cee5897093b625e4",[BuildDefaultComponents]),(UnitId
"optparse-applicative-0.13.0.0-15cc278c2101660ceb75dc742741fea5df4b70cc79c6f189ba8e28709fef1cf5",[BuildDefaultComponents]),(UnitId
"parsec-3.1.11-53aedc1e6eb2cffff907231ffbe080d3afc762b09b0a048bcea0b822c8740244",[BuildDefaultComponents]),(UnitId
"pretty-show-1.6.12-3cd8072769961ac6f41f52ed449405d05040a76f8398fcb22e59ccdff8444df6",[BuildDefaultComponents,BuildDefaultComponents]),(UnitId
"pretty-show-1.6.12-e2843aff4611803b74dd1e13ceb075db4657d7d1c3035df9032b2834633696c9",[BuildDefaultComponents,BuildDefaultComponents]),(UnitId
"primitive-0.6.1.0-fb35eb8b14ee70dc76a72edcfb4043b405962a72d0e1b9c7f1b85b4476cc36c4",[BuildDefaultComponents]),(UnitId
"random-1.1-d05d517703a0d849531e91a97b93c4724cd05b9bc2c0e4afe621c1fbd936681c",[BuildDefaultComponents]),(UnitId
"regex-base-0.93.2-f7ee83d25479bc229a068c06d810a88c8ec89565f7c47f44301141fc95bd2730",[BuildDefaultComponents]),(UnitId
"regex-posix-0.95.2-13358eced4ad057489e816c21382b96f4a382df82d243f2623f5cf6667950df9",[BuildDefaultComponents]),(UnitId
"regex-tdfa-1.2.2-5ba8716229fdd37ca192e32e3d24d67e06d59ef679b4c45674211b036d614e74",[BuildDefaultComponents]),(UnitId
"stm-2.4.4.1-bdcb4ff24ac2364a72399b8dca2dc1586e5ead6e152e4d3dc9b2ea19b808d655",[BuildDefaultComponents]),(UnitId
"tagged-0.8.5-998c1f733d6edc5abb535ceb22ab8283422afda18f33613d5dcfdb44ef7a07e0",[BuildDefaultComponents]),(UnitId
"tar-0.5.0.3-073ad06089d547e214e17058cc648d4109cc86f0508dcf1af45a56fcf64b9b02",[BuildDefaultComponents]),(UnitId
"tasty-0.11.0.4-b5d9a4f160514ab134215f742e59500fc369e254ea6158a33307c15de6033e20",[BuildDefaultComponents]),(UnitId
"tasty-hunit-0.9.2-184caf707dc5f4a1dfa0503da869f89bf5edf199555f51f0b014f35e6f9cf9b7",[BuildDefaultComponents]),(UnitId
"tasty-quickcheck-0.8.4-0f21cf888cdd80ed827597a8dd81487b9683197841d7aff5466b011e606bd1b5",[BuildDefaultComponents]),(UnitId
"text-1.2.2.1-4cece97f476d32dec402bb45924094236d4d6f158a60631a17626c7acde16c26",[BuildDefaultComponents]),(UnitId
"tf-random-0.5-ff12bed19cee24ae37f438b5575aded7af6211f4a8f6d29802306c397a79c59e",[BuildDefaultComponents]),(UnitId
"transformers-compat-0.5.1.4-aea9aaef1d0faa5f16c1aa318f0d443f092de13836f158a247ac77d0e855c64a",[BuildDefaultComponents]),(UnitId
"unbounded-delays-0.1.0.9-cd577aaf72b39d2fda2d7f606eb41e673a3458b1b7fc55a2d1531a009550ec0c",[BuildDefaultComponents]),(UnitId
"zlib-0.6.1.1-bce22f9350f7d12551b04181a0be71f99f4bd261b03ee79e5abfb0de0c168802",[BuildDefaultComponents])]
Member

alexbiehl commented Oct 29, 2016

Trace:

Alexanders-iMac:cabal alexbiehl$ cabal new-build -v3
File monitor 'improved-plan' unchanged.
Number of threads used: 8.
buildTargets': fromList [(UnitId
"Cabal-1.24.0.0",[BuildDefaultComponents,BuildDefaultComponents,BuildDefaultComponents]),(UnitId
"Cabal-1.25.0.0-inplace",[BuildDefaultComponents,BuildDefaultComponents,BuildDefaultComponents]),(UnitId
"Cabal-1.25.0.0-inplace-package-tests",[BuildDefaultComponents,BuildDefaultComponents,BuildDefaultComponents]),(UnitId
"Cabal-1.25.0.0-inplace-unit-tests",[BuildDefaultComponents,BuildDefaultComponents,BuildDefaultComponents]),(UnitId
"HTTP-4000.3.3-83c012783b085c0c232f7f844546a02a7093f8c90291bc211fe270976aca2e12",[BuildDefaultComponents]),(UnitId
"QuickCheck-2.9.2-9489342e2e0a905fde8fe760390109eaf6d198f2165f31e452c4b8483c8d9b97",[BuildDefaultComponents]),(UnitId
"ansi-terminal-0.6.2.3-e74a6ec64c54bae09e3e83e38f4bada2c6f8abd7613da1a86f727ec98c8e2f85",[BuildDefaultComponents]),(UnitId
"ansi-wl-pprint-0.6.7.3-27d2be9cba49b3882a1a60d9a087aa6fe6295b639979778f74174818868f520e",[BuildDefaultComponents]),(UnitId
"async-2.1.0-e26d74c696c3c67ea814fb65cdcc1f9be64eb47d410387f08a8cd9ef0ecc0de6",[BuildDefaultComponents]),(UnitId
"base16-bytestring-0.1.1.6-172d6f6b954a6ba445f81a57e2808feb08885a2bb0924ece25c0ac805ff938d6",[BuildDefaultComponents]),(UnitId
"base64-bytestring-1.0.0.1-db138f8c14e9745c36eadbf18d8b131a1467fb63c4d44ada63f93aaf38a67ba4",[BuildDefaultComponents]),(UnitId
"cabal-install-1.25.0.0-inplace",[BuildDefaultComponents]),(UnitId
"clock-0.7.2-0e54de7d673bf986da1791be0da87585f245ba3bc8e5bcf337bc26e02f91d4a8",[BuildDefaultComponents]),(UnitId
"cryptohash-sha256-0.11.100.1-46012f043d93da78eb72d331df2eefaf6386531f09dc716700c11b5a82d2647f",[BuildDefaultComponents]),(UnitId
"ed25519-0.0.5.0-bb2de528f86166521b399a681462dedb06601569cd1a9f24cca522f91692d540",[BuildDefaultComponents]),(UnitId
"edit-distance-0.2.2.1-988881a5ee47c0f0fa144b85f79f805ba2bb8a83f2e9bacac67ad7842acb4dab",[BuildDefaultComponents]),(UnitId
"exceptions-0.8.3-ed2cabc53de91367d33967cc314bc0e6f53dcfaee1bfa2c8a15bcfb158985973",[BuildDefaultComponents]),(UnitId
"hackage-security-0.5.2.2-inplace",[BuildDefaultComponents]),(UnitId
"happy-1.19.5-c2aa70908dd132856d459fad09f14b15eb6c52d1fd2dca1a19670e7034aa1edf",[BuildDefaultComponents]),(UnitId
"hashable-1.2.4.0-fe3e343ea609f3e13d10e5df6b016ecf4a08f8ca135bac97e6608be0249d69d3",[BuildDefaultComponents]),(UnitId
"haskell-lexer-1.0.1-e4470ba8a3d81fc06ef3df636f68f016d1fb4e50bb4cc550d6a6af7c4b7db497",[BuildDefaultComponents]),(UnitId
"mtl-2.2.1-e511aea57dffccfd5bc7e1cf3607b606a6afaa58535fe85b2e086d0f2f0c0d87",[BuildDefaultComponents]),(UnitId
"network-2.6.3.1-5d0efde49b47f3555df2e717ddce6ba1aae7a861d5a250494f33473aa8c68503",[BuildDefaultComponents]),(UnitId
"network-uri-2.6.1.0-2a33e0023d57859b3f613e84d64695651c5103aa20166751ab8427bf90f7ba3e",[BuildDefaultComponents]),(UnitId
"old-locale-1.0.0.7-b9dc6b834e03c41379cdc21cfbf76a88ebfb755814b872765db2820a2e5ae1bb",[BuildDefaultComponents]),(UnitId
"old-time-1.1.0.3-866f369d19a8b1534f5ea8bf2b9b93eed2050259e6b83c39cee5897093b625e4",[BuildDefaultComponents]),(UnitId
"optparse-applicative-0.13.0.0-15cc278c2101660ceb75dc742741fea5df4b70cc79c6f189ba8e28709fef1cf5",[BuildDefaultComponents]),(UnitId
"parsec-3.1.11-53aedc1e6eb2cffff907231ffbe080d3afc762b09b0a048bcea0b822c8740244",[BuildDefaultComponents]),(UnitId
"pretty-show-1.6.12-3cd8072769961ac6f41f52ed449405d05040a76f8398fcb22e59ccdff8444df6",[BuildDefaultComponents,BuildDefaultComponents]),(UnitId
"pretty-show-1.6.12-e2843aff4611803b74dd1e13ceb075db4657d7d1c3035df9032b2834633696c9",[BuildDefaultComponents,BuildDefaultComponents]),(UnitId
"primitive-0.6.1.0-fb35eb8b14ee70dc76a72edcfb4043b405962a72d0e1b9c7f1b85b4476cc36c4",[BuildDefaultComponents]),(UnitId
"random-1.1-d05d517703a0d849531e91a97b93c4724cd05b9bc2c0e4afe621c1fbd936681c",[BuildDefaultComponents]),(UnitId
"regex-base-0.93.2-f7ee83d25479bc229a068c06d810a88c8ec89565f7c47f44301141fc95bd2730",[BuildDefaultComponents]),(UnitId
"regex-posix-0.95.2-13358eced4ad057489e816c21382b96f4a382df82d243f2623f5cf6667950df9",[BuildDefaultComponents]),(UnitId
"regex-tdfa-1.2.2-5ba8716229fdd37ca192e32e3d24d67e06d59ef679b4c45674211b036d614e74",[BuildDefaultComponents]),(UnitId
"stm-2.4.4.1-bdcb4ff24ac2364a72399b8dca2dc1586e5ead6e152e4d3dc9b2ea19b808d655",[BuildDefaultComponents]),(UnitId
"tagged-0.8.5-998c1f733d6edc5abb535ceb22ab8283422afda18f33613d5dcfdb44ef7a07e0",[BuildDefaultComponents]),(UnitId
"tar-0.5.0.3-073ad06089d547e214e17058cc648d4109cc86f0508dcf1af45a56fcf64b9b02",[BuildDefaultComponents]),(UnitId
"tasty-0.11.0.4-b5d9a4f160514ab134215f742e59500fc369e254ea6158a33307c15de6033e20",[BuildDefaultComponents]),(UnitId
"tasty-hunit-0.9.2-184caf707dc5f4a1dfa0503da869f89bf5edf199555f51f0b014f35e6f9cf9b7",[BuildDefaultComponents]),(UnitId
"tasty-quickcheck-0.8.4-0f21cf888cdd80ed827597a8dd81487b9683197841d7aff5466b011e606bd1b5",[BuildDefaultComponents]),(UnitId
"text-1.2.2.1-4cece97f476d32dec402bb45924094236d4d6f158a60631a17626c7acde16c26",[BuildDefaultComponents]),(UnitId
"tf-random-0.5-ff12bed19cee24ae37f438b5575aded7af6211f4a8f6d29802306c397a79c59e",[BuildDefaultComponents]),(UnitId
"transformers-compat-0.5.1.4-aea9aaef1d0faa5f16c1aa318f0d443f092de13836f158a247ac77d0e855c64a",[BuildDefaultComponents]),(UnitId
"unbounded-delays-0.1.0.9-cd577aaf72b39d2fda2d7f606eb41e673a3458b1b7fc55a2d1531a009550ec0c",[BuildDefaultComponents]),(UnitId
"zlib-0.6.1.1-bce22f9350f7d12551b04181a0be71f99f4bd261b03ee79e5abfb0de0c168802",[BuildDefaultComponents])]
@alexbiehl

This comment has been minimized.

Show comment
Hide comment
@alexbiehl

alexbiehl Oct 29, 2016

Member

(Using -j1 didn't change anything)

Member

alexbiehl commented Oct 29, 2016

(Using -j1 didn't change anything)

@hvr

This comment has been minimized.

Show comment
Hide comment
@hvr

hvr Oct 29, 2016

Member

Is this with an empty package cache?

I just tried locally with a clean Git clone (and cabal HEAD):

$ cabal new-build --dry +RTS -s
Resolving dependencies...
In order, the following would be built (use -v for more details):
 - Cabal-1.25.0.0 (lib) (first run)
 - hackage-security-0.5.2.2 (lib) (requires build)
 - cabal-install-1.25.0.0 (exe:cabal) (first run)
   1,443,679,984 bytes allocated in the heap
     596,339,616 bytes copied during GC
      58,169,168 bytes maximum residency (21 sample(s))
       2,315,728 bytes maximum slop
             153 MB total memory in use (0 MB lost due to fragmentation)

                                     Tot time (elapsed)  Avg pause  Max pause
  Gen  0      2727 colls,     0 par    0.344s   0.344s     0.0001s    0.0012s
  Gen  1        21 colls,     0 par    0.421s   0.421s     0.0201s    0.1021s

  TASKS: 4 (1 bound, 3 peak workers (3 total), using -N1)

  SPARKS: 0 (0 converted, 0 overflowed, 0 dud, 0 GC'd, 0 fizzled)

  INIT    time    0.001s  (  0.001s elapsed)
  MUT     time    0.760s  (  1.042s elapsed)
  GC      time    0.766s  (  0.765s elapsed)
  EXIT    time    0.007s  (  0.006s elapsed)
  Total   time    1.534s  (  1.814s elapsed)

  Alloc rate    1,900,601,007 bytes per MUT second

  Productivity  50.0% of total user, 42.3% of total elapsed

gc_alloc_block_sync: 0
whitehole_spin: 0
gen[0].sync: 0
gen[1].sync: 0
Member

hvr commented Oct 29, 2016

Is this with an empty package cache?

I just tried locally with a clean Git clone (and cabal HEAD):

$ cabal new-build --dry +RTS -s
Resolving dependencies...
In order, the following would be built (use -v for more details):
 - Cabal-1.25.0.0 (lib) (first run)
 - hackage-security-0.5.2.2 (lib) (requires build)
 - cabal-install-1.25.0.0 (exe:cabal) (first run)
   1,443,679,984 bytes allocated in the heap
     596,339,616 bytes copied during GC
      58,169,168 bytes maximum residency (21 sample(s))
       2,315,728 bytes maximum slop
             153 MB total memory in use (0 MB lost due to fragmentation)

                                     Tot time (elapsed)  Avg pause  Max pause
  Gen  0      2727 colls,     0 par    0.344s   0.344s     0.0001s    0.0012s
  Gen  1        21 colls,     0 par    0.421s   0.421s     0.0201s    0.1021s

  TASKS: 4 (1 bound, 3 peak workers (3 total), using -N1)

  SPARKS: 0 (0 converted, 0 overflowed, 0 dud, 0 GC'd, 0 fizzled)

  INIT    time    0.001s  (  0.001s elapsed)
  MUT     time    0.760s  (  1.042s elapsed)
  GC      time    0.766s  (  0.765s elapsed)
  EXIT    time    0.007s  (  0.006s elapsed)
  Total   time    1.534s  (  1.814s elapsed)

  Alloc rate    1,900,601,007 bytes per MUT second

  Productivity  50.0% of total user, 42.3% of total elapsed

gc_alloc_block_sync: 0
whitehole_spin: 0
gen[0].sync: 0
gen[1].sync: 0
@alexbiehl

This comment has been minimized.

Show comment
Hide comment
@alexbiehl

alexbiehl Oct 29, 2016

Member

I deleted dist-newstyle and now everything works as expected (and with expected memory)

Member

alexbiehl commented Oct 29, 2016

I deleted dist-newstyle and now everything works as expected (and with expected memory)

@alexbiehl

This comment has been minimized.

Show comment
Hide comment
@alexbiehl

alexbiehl Oct 29, 2016

Member

It is the dist-newstyle/build/cache directory. When I delete dist-newtyle/cache its still exploding in memory. But deleting dist-newstyle/build/cache works.

Member

alexbiehl commented Oct 29, 2016

It is the dist-newstyle/build/cache directory. When I delete dist-newtyle/cache its still exploding in memory. But deleting dist-newstyle/build/cache works.

@ezyang

This comment has been minimized.

Show comment
Hide comment
@ezyang

ezyang Oct 31, 2016

Contributor

Uh, I don't think we ever create a dist-newstyle/build/cache directory? Next time this happens, mind tacking on +RTS -h and sharing the heap profile after you c-c it? (upload it at http://heap.ezyang.com/ or something)

(Without a repro case I'm afraid this will be hard to debug, sorry.)

Contributor

ezyang commented Oct 31, 2016

Uh, I don't think we ever create a dist-newstyle/build/cache directory? Next time this happens, mind tacking on +RTS -h and sharing the heap profile after you c-c it? (upload it at http://heap.ezyang.com/ or something)

(Without a repro case I'm afraid this will be hard to debug, sorry.)

@alexbiehl

This comment has been minimized.

Show comment
Hide comment
@alexbiehl

alexbiehl Oct 31, 2016

Member

Yes, I will when I get home.

Member

alexbiehl commented Oct 31, 2016

Yes, I will when I get home.

@alexbiehl

This comment has been minimized.

Show comment
Hide comment
@alexbiehl

alexbiehl Nov 1, 2016

Member

http://heap.ezyang.com/view/03919c8f2a908bcd03e76846edbe6f6f7149d916

The relevant profiling trace:

   topHandler                                                                               Distribution.Simple.Utils                 52807          1    0.0    0.0    99.9  100.0
    topHandlerWith                                                                          Distribution.Simple.Utils                 52808          1    0.0    0.0    99.9  100.0
     mainWorker.commandSpecs                                                                Main                                      53114          0    0.0    0.0    99.9  100.0
      regularCmd                                                                            Main                                      53115          0    0.0    0.0    99.9  100.0
       commandAddAction                                                                     Distribution.Simple.Command               53116          0    0.0    0.0    99.9  100.0
        fmap                                                                                Distribution.Simple.Command               53117          0    0.0    0.0    99.9  100.0
         commandAddAction.applyDefaultArgs                                                  Distribution.Simple.Command               53118          0    0.0    0.0    99.9  100.0
          buildAction                                                                       Distribution.Client.CmdBuild              53126          1    0.0    0.0    99.9  100.0
           buildAction.verbosity                                                            Distribution.Client.CmdBuild              53856          1    0.0    0.0     0.0    0.0
            fromFlagOrDefault                                                               Distribution.Simple.Setup                 53861          1    0.0    0.0     0.0    0.0
            configVerbosity                                                                 Distribution.Simple.Setup                 53857          1    0.0    0.0     0.0    0.0
           runProjectPreBuildPhase                                                          Distribution.Client.ProjectOrchestration  53287          1    0.0    0.0    99.9  100.0
            rebuildTargetsDryRun                                                            Distribution.Client.ProjectBuilding       54199          1    0.0    0.0    93.8   93.7
             foldMInstallPlanDepOrder                                                       Distribution.Client.ProjectBuilding       54200          1    0.0    0.0    93.8   93.7
              foldMInstallPlanDepOrder.go                                                   Distribution.Client.ProjectBuilding       54443         67    0.0    0.0    93.8   93.7
               foldMInstallPlanDepOrder.go.results'                                         Distribution.Client.ProjectBuilding       54454         66    0.0    0.0     0.0    0.0
                nodeKey                                                                     Distribution.Client.InstallPlan           54455         66    0.0    0.0     0.0    0.0
                 elabUnitId                                                                 Distribution.Client.ProjectPlanning.Types 54500         47    0.0    0.0     0.0    0.0
                 installedUnitId                                                            Distribution.InstalledPackageInfo         54456         19    0.0    0.0     0.0    0.0
               rebuildTargetsDryRun.dryRunPkg                                               Distribution.Client.ProjectBuilding       54444         67    0.0    0.0    93.8   93.7
                rebuildTargetsDryRun.dryRunLocalPkg                                         Distribution.Client.ProjectBuilding       54501          1    0.0    0.0    93.7   93.7
                 checkPackageFileMonitorChanged                                             Distribution.Client.ProjectBuilding       54505          1    0.0    0.0    93.7   93.7
                  checkFileMonitorChanged                                                   Distribution.Client.FileMonitor           54506          1    0.0    0.0    93.7   93.7
                   handleErrorCall                                                          Distribution.Client.FileMonitor           54508          0    0.0    0.0    93.7   93.7
                    readCacheFile                                                           Distribution.Client.FileMonitor           54509          1    0.0    0.0    93.7   93.7
                     readCacheFile.\                                                        Distribution.Client.FileMonitor           54585          1    0.0    0.0    93.7   93.7
                      decodeOrFailIO                                                        Distribution.Compat.Binary                54586          1    0.0    0.0    93.7   93.7
                       get                                                                  Distribution.Client.FileMonitor           54587          0    0.0    0.0    93.7   93.7
                        get                                                                 Distribution.Utils.ShortText              54588          0    0.0    0.0    93.7   93.7
                         get                                                                Distribution.ModuleName                   54589          0   93.7   93.7    93.7   93.7
                          topHandlerWith.rethrowAsyncExceptions                             Distribution.Simple.Utils                 54591          1    0.0    0.0     0.0    0.0
                          stlFromList                                                       Distribution.ModuleName                   54590        264    0.0    0.0     0.0    0.0
Member

alexbiehl commented Nov 1, 2016

http://heap.ezyang.com/view/03919c8f2a908bcd03e76846edbe6f6f7149d916

The relevant profiling trace:

   topHandler                                                                               Distribution.Simple.Utils                 52807          1    0.0    0.0    99.9  100.0
    topHandlerWith                                                                          Distribution.Simple.Utils                 52808          1    0.0    0.0    99.9  100.0
     mainWorker.commandSpecs                                                                Main                                      53114          0    0.0    0.0    99.9  100.0
      regularCmd                                                                            Main                                      53115          0    0.0    0.0    99.9  100.0
       commandAddAction                                                                     Distribution.Simple.Command               53116          0    0.0    0.0    99.9  100.0
        fmap                                                                                Distribution.Simple.Command               53117          0    0.0    0.0    99.9  100.0
         commandAddAction.applyDefaultArgs                                                  Distribution.Simple.Command               53118          0    0.0    0.0    99.9  100.0
          buildAction                                                                       Distribution.Client.CmdBuild              53126          1    0.0    0.0    99.9  100.0
           buildAction.verbosity                                                            Distribution.Client.CmdBuild              53856          1    0.0    0.0     0.0    0.0
            fromFlagOrDefault                                                               Distribution.Simple.Setup                 53861          1    0.0    0.0     0.0    0.0
            configVerbosity                                                                 Distribution.Simple.Setup                 53857          1    0.0    0.0     0.0    0.0
           runProjectPreBuildPhase                                                          Distribution.Client.ProjectOrchestration  53287          1    0.0    0.0    99.9  100.0
            rebuildTargetsDryRun                                                            Distribution.Client.ProjectBuilding       54199          1    0.0    0.0    93.8   93.7
             foldMInstallPlanDepOrder                                                       Distribution.Client.ProjectBuilding       54200          1    0.0    0.0    93.8   93.7
              foldMInstallPlanDepOrder.go                                                   Distribution.Client.ProjectBuilding       54443         67    0.0    0.0    93.8   93.7
               foldMInstallPlanDepOrder.go.results'                                         Distribution.Client.ProjectBuilding       54454         66    0.0    0.0     0.0    0.0
                nodeKey                                                                     Distribution.Client.InstallPlan           54455         66    0.0    0.0     0.0    0.0
                 elabUnitId                                                                 Distribution.Client.ProjectPlanning.Types 54500         47    0.0    0.0     0.0    0.0
                 installedUnitId                                                            Distribution.InstalledPackageInfo         54456         19    0.0    0.0     0.0    0.0
               rebuildTargetsDryRun.dryRunPkg                                               Distribution.Client.ProjectBuilding       54444         67    0.0    0.0    93.8   93.7
                rebuildTargetsDryRun.dryRunLocalPkg                                         Distribution.Client.ProjectBuilding       54501          1    0.0    0.0    93.7   93.7
                 checkPackageFileMonitorChanged                                             Distribution.Client.ProjectBuilding       54505          1    0.0    0.0    93.7   93.7
                  checkFileMonitorChanged                                                   Distribution.Client.FileMonitor           54506          1    0.0    0.0    93.7   93.7
                   handleErrorCall                                                          Distribution.Client.FileMonitor           54508          0    0.0    0.0    93.7   93.7
                    readCacheFile                                                           Distribution.Client.FileMonitor           54509          1    0.0    0.0    93.7   93.7
                     readCacheFile.\                                                        Distribution.Client.FileMonitor           54585          1    0.0    0.0    93.7   93.7
                      decodeOrFailIO                                                        Distribution.Compat.Binary                54586          1    0.0    0.0    93.7   93.7
                       get                                                                  Distribution.Client.FileMonitor           54587          0    0.0    0.0    93.7   93.7
                        get                                                                 Distribution.Utils.ShortText              54588          0    0.0    0.0    93.7   93.7
                         get                                                                Distribution.ModuleName                   54589          0   93.7   93.7    93.7   93.7
                          topHandlerWith.rethrowAsyncExceptions                             Distribution.Simple.Utils                 54591          1    0.0    0.0     0.0    0.0
                          stlFromList                                                       Distribution.ModuleName                   54590        264    0.0    0.0     0.0    0.0
@ezyang

This comment has been minimized.

Show comment
Hide comment
@ezyang

ezyang Nov 1, 2016

Contributor

Thank you, that is really helpful! @dcoutts on IRC suggested that maybe what was happening was we accidentally were trying to serialize a giant blob of binary into a String (explains all the cons cells). UPDATE: Maybe not even to a String; ShortTextLst involves list

Contributor

ezyang commented Nov 1, 2016

Thank you, that is really helpful! @dcoutts on IRC suggested that maybe what was happening was we accidentally were trying to serialize a giant blob of binary into a String (explains all the cons cells). UPDATE: Maybe not even to a String; ShortTextLst involves list

@ezyang

This comment has been minimized.

Show comment
Hide comment
@ezyang

ezyang Nov 1, 2016

Contributor

@alexbiehl Are you by any chance switching between versions of Cabal (e.g., using stock Cabal, and then rebuilidng using HEAD Cabal?)

Contributor

ezyang commented Nov 1, 2016

@alexbiehl Are you by any chance switching between versions of Cabal (e.g., using stock Cabal, and then rebuilidng using HEAD Cabal?)

@ezyang

This comment has been minimized.

Show comment
Hide comment
@ezyang

ezyang Nov 1, 2016

Contributor

If @dcoutts' suggestion is correct, a simple prophylactic measure we can take is to record the version of cabal-install that generated the cache file, and invalidate it if the version doesn't match. This won't help much if you're swapping between different versions of HEAD but will catch a 1.24-HEAD switch.

Contributor

ezyang commented Nov 1, 2016

If @dcoutts' suggestion is correct, a simple prophylactic measure we can take is to record the version of cabal-install that generated the cache file, and invalidate it if the version doesn't match. This won't help much if you're swapping between different versions of HEAD but will catch a 1.24-HEAD switch.

@alexbiehl

This comment has been minimized.

Show comment
Hide comment
@alexbiehl

alexbiehl Nov 1, 2016

Member

@ezyang Yes, that it must be. I am installing HEAD everyday from git. So I could have worked on an outdated dist-newstyle directory; of course its working after I delete it..

Member

alexbiehl commented Nov 1, 2016

@ezyang Yes, that it must be. I am installing HEAD everyday from git. So I could have worked on an outdated dist-newstyle directory; of course its working after I delete it..

@ezyang

This comment has been minimized.

Show comment
Hide comment
@ezyang

ezyang Nov 2, 2016

Contributor

OK, so here are steps to reproduce:

  1. Build a pre 382143a version of cabal
  2. With this cabal, cabal new-build cabal in a new workdir.
  3. Update your boot Cabal to 382143a
  4. With the workdir from (2), cabal new-build cabal

After depsolving, you'll see memory usage skyrocket.

Contributor

ezyang commented Nov 2, 2016

OK, so here are steps to reproduce:

  1. Build a pre 382143a version of cabal
  2. With this cabal, cabal new-build cabal in a new workdir.
  3. Update your boot Cabal to 382143a
  4. With the workdir from (2), cabal new-build cabal

After depsolving, you'll see memory usage skyrocket.

@hvr

This comment has been minimized.

Show comment
Hide comment
@hvr

hvr Nov 2, 2016

Member

@ezyang we could also make use of manually assigned magic/version numbers at the start of the encodings as e.g. in

and manually increment everytime we either suspect or experience such breakage (unless we can figure out a way to autodetect incompatible encodings; otoh encodings such as CBOR are more resilient to the kind of corruption we're most likely seeing here)

Member

hvr commented Nov 2, 2016

@ezyang we could also make use of manually assigned magic/version numbers at the start of the encodings as e.g. in

and manually increment everytime we either suspect or experience such breakage (unless we can figure out a way to autodetect incompatible encodings; otoh encodings such as CBOR are more resilient to the kind of corruption we're most likely seeing here)

ezyang added a commit to ezyang/cabal that referenced this issue Nov 2, 2016

Add version number to file monitor cache.
I'm actually not very happy with this patch, because it
doesn't actually solve any real problems (the next time
we drastically change the binary format, anyone tracking
cabal-install HEAD is going to end up in trouble in
any case, because cabal-install version is not a fine-grained
enough verison, you also need build date.)

Fixes #4059.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>
@ezyang

This comment has been minimized.

Show comment
Hide comment
@ezyang

ezyang Nov 2, 2016

Contributor

I posted a patch but it's not a very robust fix, because it will only really save you when you're skipping between versions of Cabal, and not when you're continuously rebuilding HEAD.

Here's an idea I had though: what if we added a new typeclass BinaryVersion, also Generic derivable, which computed a hash based on the generic structure of the data types in question. This hash is then serialized alongside the binary, and now we have a "version number" that automatically updates when the format changes. This sounds pretty nice!

Contributor

ezyang commented Nov 2, 2016

I posted a patch but it's not a very robust fix, because it will only really save you when you're skipping between versions of Cabal, and not when you're continuously rebuilding HEAD.

Here's an idea I had though: what if we added a new typeclass BinaryVersion, also Generic derivable, which computed a hash based on the generic structure of the data types in question. This hash is then serialized alongside the binary, and now we have a "version number" that automatically updates when the format changes. This sounds pretty nice!

@23Skidoo

This comment has been minimized.

Show comment
Hide comment
@23Skidoo

23Skidoo Nov 2, 2016

Member

If we make cabal-install know its own Git commit ID, can we use that instead? Related: #1506.

Member

23Skidoo commented Nov 2, 2016

If we make cabal-install know its own Git commit ID, can we use that instead? Related: #1506.

@ezyang

This comment has been minimized.

Show comment
Hide comment
@ezyang

ezyang Nov 2, 2016

Contributor

Yes, that would also be sufficient, although it would be a little irritating for all the binary caches to keep invalidating every time you committed.

Contributor

ezyang commented Nov 2, 2016

Yes, that would also be sufficient, although it would be a little irritating for all the binary caches to keep invalidating every time you committed.

@23Skidoo 23Skidoo added the meta: easy label Nov 2, 2016

@hvr

This comment has been minimized.

Show comment
Hide comment
@hvr

hvr Nov 2, 2016

Member

@23Skidoo teaching cabal-install to learn its Git commit id is quite a task to get done properly (you need to take into account cabal sdist), and depending on how this is done, it's most likely in conflict with #3922

Member

hvr commented Nov 2, 2016

@23Skidoo teaching cabal-install to learn its Git commit id is quite a task to get done properly (you need to take into account cabal sdist), and depending on how this is done, it's most likely in conflict with #3922

@23Skidoo

This comment has been minimized.

Show comment
Hide comment
@23Skidoo

23Skidoo Nov 2, 2016

Member

(you need to take into account cabal sdist)

I think we can ignore this case (at least initially), knowing commit ID is most useful when you're building from a working copy.

Member

23Skidoo commented Nov 2, 2016

(you need to take into account cabal sdist)

I think we can ignore this case (at least initially), knowing commit ID is most useful when you're building from a working copy.

ezyang added a commit to ezyang/cabal that referenced this issue Nov 2, 2016

Add fingerprint of Generic representation when serializing.
The idea is we can use Rep to get a full, structural representation
of a type, and the fingerprint it using Typeable.  This gives
us a very concise way of fingerprinting our Binary representation.

This patch is not completely correct; the fingerprint needs
to be overridable when someone writes a custom Binary instance.
But this should be "good enough" in practice; we're not using
these fingerprints to check anything security critical.

TODO: Not sure if I have tagged all the call-sites which could
profit from this.

Fixes #4059.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>
@ezyang

This comment has been minimized.

Show comment
Hide comment
@ezyang

ezyang Nov 2, 2016

Contributor

I posted a lot better patch.

Contributor

ezyang commented Nov 2, 2016

I posted a lot better patch.

ezyang added a commit to ezyang/cabal that referenced this issue Nov 2, 2016

Add fingerprint of Generic representation when serializing.
The idea is we can use Rep to get a full, structural representation
of a type, and the fingerprint it using Typeable.  This gives
us a very concise way of fingerprinting our Binary representation.

This patch is not completely correct; the fingerprint needs
to be overridable when someone writes a custom Binary instance.
But this should be "good enough" in practice; we're not using
these fingerprints to check anything security critical.

TODO: Not sure if I have tagged all the call-sites which could
profit from this.

Fixes #4059.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

ezyang added a commit to ezyang/cabal that referenced this issue Nov 2, 2016

Add fingerprint of Generic representation when serializing.
The idea is we can use Rep to get a full, structural representation
of a type, and the fingerprint it using Typeable.  This gives
us a very concise way of fingerprinting our Binary representation.

This patch is not completely correct; the fingerprint needs
to be overridable when someone writes a custom Binary instance.
But this should be "good enough" in practice; we're not using
these fingerprints to check anything security critical.

TODO: Not sure if I have tagged all the call-sites which could
profit from this.

Fixes #4059.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

ezyang added a commit to ezyang/cabal that referenced this issue Nov 5, 2016

Add fingerprint of Generic representation when serializing.
The idea is we can use Rep to get a full, structural representation
of a type, and the fingerprint it using Typeable.  This gives
us a very concise way of fingerprinting our Binary representation.

This patch is not completely correct; the fingerprint needs
to be overridable when someone writes a custom Binary instance.
But this should be "good enough" in practice; we're not using
these fingerprints to check anything security critical.

TODO: Not sure if I have tagged all the call-sites which could
profit from this.

Fixes #4059.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

@ezyang ezyang closed this in #4078 Nov 5, 2016

ezyang added a commit that referenced this issue Nov 5, 2016

Add fingerprint of Generic representation when serializing.
The idea is we can use Rep to get a full, structural representation
of a type, and the fingerprint it using Typeable.  This gives
us a very concise way of fingerprinting our Binary representation.

This patch is not completely correct; the fingerprint needs
to be overridable when someone writes a custom Binary instance.
But this should be "good enough" in practice; we're not using
these fingerprints to check anything security critical.

TODO: Not sure if I have tagged all the call-sites which could
profit from this.

Fixes #4059.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

@ezyang ezyang added this to the 2.0 milestone Nov 28, 2016

@ezyang ezyang removed the meta: easy label Nov 28, 2016

@ezyang

This comment has been minimized.

Show comment
Hide comment
@ezyang

ezyang Dec 21, 2016

Contributor

I'm reopening this, as we had to revert this patch due to it breaking the build with GHC, see #4184. It's possible that we might be able to still take some form of this patch by patching GHC's build system to reconfigure once it builds the dist-install ghc-cabal, but it's not obvious this is the right thing to do, and this bug is a bit more benign than the other.

Contributor

ezyang commented Dec 21, 2016

I'm reopening this, as we had to revert this patch due to it breaking the build with GHC, see #4184. It's possible that we might be able to still take some form of this patch by patching GHC's build system to reconfigure once it builds the dist-install ghc-cabal, but it's not obvious this is the right thing to do, and this bug is a bit more benign than the other.

@ezyang ezyang reopened this Dec 21, 2016

@23Skidoo 23Skidoo modified the milestones: 2.0, 2.2 Feb 17, 2017

@23Skidoo 23Skidoo removed this from the 2.0 milestone Feb 17, 2017

@23Skidoo 23Skidoo modified the milestones: 2.2, 2.4 Aug 29, 2018

@23Skidoo 23Skidoo modified the milestones: 2.4, 2.4.1 Sep 17, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment