-
Notifications
You must be signed in to change notification settings - Fork 135
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Can't upload large jar file. #5
Comments
@samuel281 Hi there, So you tried setting At Ooyala, we actually don't upload large jars. What we do is that we separate our jars into two layers, the business logic which is really small, and a "platform" module which we append to the classpath. Since the platform doesn't change very often, it is acceptable for now. There is actually a configuration in the job server to get around this, so that you can add a "side jar" aside from the main jar, and don't have to mess with classpaths, but it doesn't work at the moment. I might fix it if I have time, I'm working on a related note. |
thanks! |
I have tested, and changing this configuration value via conf file does not work. But it works when setting it via system property.
|
Have you tried changing it in src/main/resources/application.conf? Maybe On Fri, Mar 28, 2014 at 1:51 PM, pmarnik notifications@github.com wrote:
Evan Chan http://www.ooyala.com/ |
Yes, this was first attempt after realizing that this value isn't read from local.conf. Other values such as http port has been successfully read from local.conf. |
I can add that changing the value in |
@maasg could you provide an example, please? This is what I tried but it's not working
|
I need to add that the correct setting to change is:
See: http://spray.io/documentation/1.1-M8/spray-can/configuration/ |
Puh that was fast ;-p Here is my working config ;-)
|
@wienczny try this: spray.can {
server {
...
parsing {
max-content-length = 100m
}
} |
@wienczny I was busy adding that comment when you posted yours :-) . I realized that I had to play with the settings to get it right. Just needed to remember what exactly was it ... |
Hey guys, There is a better way to solve the large file problem.
Obviously, 2) is a much bigger pain. On Tue, Jun 17, 2014 at 2:51 AM, Gerard Maas notifications@github.com
The fruit of silence is prayer; |
Code & unit test for detecting hang Spark workers
Code & unit test for detecting hang Spark workers
True that updating 60 # check the reference.conf in spray-can/src/main/resources for all defined settings
61 spray.can.server {
62 # uncomment the next line for making this an HTTPS example
63 # ssl-encryption = on
64 idle-timeout = 60 s
65 request-timeout = 40 s
66 pipelining-limit = 2 # for maximum performance (prevents StopReading / ResumeReading messages to the IOBridge)
67 # Needed for HTTP/1.0 requests with missing Host headers
68 default-host-header = "spray.io:8765"
~ 69
+ 70 parsing.max-content-length = 200m
+ 71
+ 72 } As I could upload [2014-12-23 23:57:19,297] INFO spark.jobserver.JarManager [] [akka://JobServer/user/jar-manager] - Storing jar for app smartad, 12117543 bytes |
Hi, I'm currently using this new spark-jobserver.
Almost everything just works really well but I have a small issue.
Sometimes I need to pack a jar with dependencies and it makes jar much larger.
I don't want to restart spark jobserver to change its classpath every time I make a new jar that depend on newly added libraries.
But the request content length limit is too low (it say 8388608 bytes) so I'm afraid that I won't be able to upload that kind of large jars...
I tried to change spray can configuration but it didn't work.
Is there any way to change this limitation?
The text was updated successfully, but these errors were encountered: