Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LndGrpc.hs (Lightning service) file is too big #97

Open
tim2CF opened this issue Aug 2, 2021 · 1 comment
Open

LndGrpc.hs (Lightning service) file is too big #97

tim2CF opened this issue Aug 2, 2021 · 1 comment
Labels
bug Something isn't working

Comments

@tim2CF
Copy link
Contributor

tim2CF commented Aug 2, 2021

LndGrpc.hs file generated by proto-lens-protoc plugin is too big (3.01 MB). This causes compilation failure on my laptop (8 GB RAM). On machines with bigger RAM capacity it still takes a while to compile this file, but at least it works.

https://github.com/coingaming/lnd-client/blob/e2218e41d971f29adf6d63f80f6feaf032e4cb3e/src/Proto/LndGrpc.hs

Steps to reproduce:

git checkout e2218e41d971f29adf6d63f80f6feaf032e4cb3e
./nix/shell.sh
stack build --fast --file-watch

Any ideas how to fix this? Split .proto source file into multiple source files? Use some special GHC flags? Optimize proto-lens-protoc? Something else?

@tim2CF tim2CF added the bug Something isn't working label Aug 2, 2021
@tim2CF
Copy link
Contributor Author

tim2CF commented Aug 3, 2021

14:55 < timCF> Hello! I have a huge Haskell source code file generated by `proto-lens-protoc` protobuf plugin from very big Google Protobuf document. The size of resulting Haskell file is 
               something around 3MB. And I did found out that GHC 8.6.5 fails to compile this file on relatively weak laptops (8GB RAM) with default settings. But it compiles quite ok on 
               machines with bigger RAM capacity. What can you suggest to 
14:55 < timCF> workaround this issue? Maybe there are some magical GHC flags to reduce RAM consumption? Or I should just invest into GHC upgrade in my project?
14:55 -!- bitdex [~bitdex@gateway/tor-sasl/bitdex] has quit [Quit: = ""]
14:55 < dminuoso> Invest into more RAM? :-p
14:56 < merijn> That, yes :p
14:56 < timCF> Haha :)
14:56 < merijn> Or patch proto-lens-protoc to produce more GHC friendly output
14:56 < merijn> There's work on optimising GHCs memory usage/compile times, but that's hard work and will take a while to actually improve things (and only in newer GHCs)
14:57 < merijn> So "buy more RAM" and "change the generated code" are the only really short-term options you have
14:57 -!- peterhil [~peterhil@dsl-hkibng32-54fb52-57.dhcp.inet.fi] has quit [Ping timeout: 272 seconds]
14:58 < timCF> Do you remember significant improvements on RAM consumption with GHC 9 upgrade? I just didn't tried latest versions of compiler. Is it better?
14:58 < merijn> I can't remember, because I'm still using 8.10 :p
14:58 -!- Atum_ [IRC@user/atum/x-2392232] has quit [Ping timeout: 258 seconds]
14:59 < timCF> Well, things like compilers usually are getting better over time :)
14:59 < hpc> i like the technical correctness of saying "i can't remember" about something that hasn't happened yet
14:59 < dminuoso> GHC hasn't really received any updates recently that majorly change how many resources it consumes.
14:59 < dminuoso> Aside from bug fixes of course
14:59 < merijn> hpc: Technically correct is the best kind of correct :p

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant