Skip to content

Loading…

Split up xmlprotocol.scala #146

Open
devinus opened this Issue · 12 comments

4 participants

@devinus

Unfortunately, I have to start by saying that much to my chagrin I cannot give an example of the WSDL and accompanying XSD files that generates the 65KLOC xmlprotocol.scala, as it's proprietary and I'm bound by non-disclosure agreements between my company and the provider.

However, I can give as much detail and my own observations:

The file I'm sure is clean Scala code and would probably eventually compile given enough time and memory made available to the sbt process. However, my machine only has 4g. At first I was receiving OutOfMemory exceptions during the compilation process on that file that forced me to use -Xmx2g to make more heap space available, then I was receiving errors about the process not having enough memory per stack, so I increased the stack size to -Xss1m as well. Still, the file would simply take too long to compile.

Also, a quick search turns up that the file contains about 3k instances of the implicit keyword.

I was just wondering if there was a way to split up the XMLProtocol trait into logically smaller pieces. Is there any way to split a trait up amongst multiple files?

@eed3si9n
Owner

Others have reported similar issue with larger size code (like #131), and if the problem is coming from implicit resolution, splitting them up ultimately doesn't change the situation. One possibility may be to stop the internal use of implicits by explicitly passing the protocols to the argument. This would reduce the flexibility, but would at least work, if it works.

@devinus

How much work would it be to change it to explicit resolution? I'm here to use and test any experimental branch if you ever set it up.

@eed3si9n
Owner

Not sure, but I'll try hacking around soon.

@devinus

Okay. I'll be ready to test it. I hate making it less flexible, but at least we'll be able to see if it is indeed all the implicit resolution.

@eed3si9n
Owner

It's implemented in try/explicit branch. I have a wsdl that stackoverflows normally, and it's stackoverflowing..

@devinus

@eed3si9n As in, it's still stackoverflowing?

@eed3si9n
Owner

Yea. So if you want you can give it a shot, by publishing it locally, but don't keeps your hope too high.

@devinus

I'm giving it a go right now...

@devinus

Unfortunately, I don't think it was implicit resolution anymore. I'm almost positive it's just that the Scala compiler cannot handle a 65KLOC file. I'm not sure any compiler handles a 65KLOC file that well.

Perhaps, and just brainstorming, but could the XMLProtocol trait be split amongst files? E.g.

trait A {}
trait B {}
trait C {}
trait ABC extends A with B with C {}
@eed3si9n
Owner

ok. I'll keep this open as a potential enhancement in some future major release.

@hoff2

I'm exploring this, as I have a couple of ~80kloc xmlprotocol.scala files I can't compile, but it seems that about 80% of them (line-wise) is all the *TypeFormat classes, which are each quite small on their own, it's just that there are 4000+ of them. Putting all those in their own trait/file probably wouldn't help -- that file would still be 78kloc or so :(

@adamdecaf

So I found 32ae67d from @eed3si9n and merged it into current master. There's just one test that's failing that I'm trying to fix before I'd PR it against scalaxb master, but if anyone wants to help that'd be awesome. Here's the merge so far Banno#2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.