compiling test with 2.9.1 throws stackoverflowexception (trying to run tests) #14

Closed
ppurang opened this Issue Apr 23, 2012 · 6 comments

Comments

Projects
None yet
2 participants
@ppurang

ppurang commented Apr 23, 2012

Trying to play with forking and scalac options, max heap and xss, didn't help.

What helped was to use scala 2.9.2. All tests compiled except the "ScaliakBucketSpecs".

Asking myself a question about why no one has run into this; the only answers I can come up with is a) no one's running the tests (very unlikely) or b) under incremental compilation this problem doesn't occur.

One way to avoid these issues would be to migrate to 2.9.2 and to divide the ScaliakBucketSpecs into two or three separate specs as two might not be enough.

@jrwest

This comment has been minimized.

Show comment Hide comment
@jrwest

jrwest Apr 23, 2012

Contributor

I am currently traveling so i will have to get back to you tomorrow with more information but tests are certainly run regularly by our build system from a fresh compile on 2.9.1 every commit (we do a clean before every test run). When I get back to my machine I will look at my scalac/SBT settings as well as the ones on our build server and share them with you. I do plan to split out ScaliakBucketSpecs into several suites. Also I will be publishing a snapshot version to maven central shortly.

Sent from my iPhone

On Apr 23, 2012, at 11:58 PM, Piyush Purangreply@reply.github.com wrote:

Trying to play with forking and scalac options, max heap and xss, didn't help.

What helped was to use scala 2.9.2. All tests compiled except the "ScaliakBucketSpecs".

Asking myself a question about why no one has run into this; the only answers I can come up with is a) no one's running the tests (very unlikely) or b) under incremental compilation this problem doesn't occur.

One way to avoid these issues would be to migrate to 2.9.2 and to divide the ScaliakBucketSpecs into two or three separate specs as two might not be enough.


Reply to this email directly or view it on GitHub:
#14

Contributor

jrwest commented Apr 23, 2012

I am currently traveling so i will have to get back to you tomorrow with more information but tests are certainly run regularly by our build system from a fresh compile on 2.9.1 every commit (we do a clean before every test run). When I get back to my machine I will look at my scalac/SBT settings as well as the ones on our build server and share them with you. I do plan to split out ScaliakBucketSpecs into several suites. Also I will be publishing a snapshot version to maven central shortly.

Sent from my iPhone

On Apr 23, 2012, at 11:58 PM, Piyush Purangreply@reply.github.com wrote:

Trying to play with forking and scalac options, max heap and xss, didn't help.

What helped was to use scala 2.9.2. All tests compiled except the "ScaliakBucketSpecs".

Asking myself a question about why no one has run into this; the only answers I can come up with is a) no one's running the tests (very unlikely) or b) under incremental compilation this problem doesn't occur.

One way to avoid these issues would be to migrate to 2.9.2 and to divide the ScaliakBucketSpecs into two or three separate specs as two might not be enough.


Reply to this email directly or view it on GitHub:
#14

@jrwest

This comment has been minimized.

Show comment Hide comment
@jrwest

jrwest Apr 24, 2012

Contributor

just got back to my machine and ran an sbt clean test without issue. Here is a copy of the sbt script i use system-wide unless there is a local script provided by a project:

#!/bin/bash

SCALA="-XX:MaxPermSize=256m -Xms1G -Xmx1G -Xss4M"
# More Optional Scala Opts
# -XX:+UseTLAB -XX:+AggressiveOpts -XX:+UseFastAccessorMethods"

# OPTIONAL GC STRATEGY
# GC_STRATEGY="-XX:+UseParallelGC -XX:+UseParallelOldGC"

java $SCALA  -jar `dirname $0`/sbt-launch.jar "$@"

# old version 
# java -Xmx512M -jar `dirname $0`/sbt-launch.jar "$@"
Contributor

jrwest commented Apr 24, 2012

just got back to my machine and ran an sbt clean test without issue. Here is a copy of the sbt script i use system-wide unless there is a local script provided by a project:

#!/bin/bash

SCALA="-XX:MaxPermSize=256m -Xms1G -Xmx1G -Xss4M"
# More Optional Scala Opts
# -XX:+UseTLAB -XX:+AggressiveOpts -XX:+UseFastAccessorMethods"

# OPTIONAL GC STRATEGY
# GC_STRATEGY="-XX:+UseParallelGC -XX:+UseParallelOldGC"

java $SCALA  -jar `dirname $0`/sbt-launch.jar "$@"

# old version 
# java -Xmx512M -jar `dirname $0`/sbt-launch.jar "$@"
@ppurang

This comment has been minimized.

Show comment Hide comment
@ppurang

ppurang Apr 24, 2012

Thanks , the -Xss4M was the key.

ppurang commented Apr 24, 2012

Thanks , the -Xss4M was the key.

@ppurang ppurang closed this Apr 24, 2012

@jrwest

This comment has been minimized.

Show comment Hide comment
@jrwest

jrwest Apr 27, 2012

Contributor

FYI scaliak is now built publically on travis.ci (http://travis-ci.org/#!/stackmob/scaliak). Tests are run after every commit. I did have to bump -Xss all the way up to 8M, however. Its on my todo list to split up ScaliakBucketSpecs for this reason as well as the fact that the spec covers a little more than it should.

Contributor

jrwest commented Apr 27, 2012

FYI scaliak is now built publically on travis.ci (http://travis-ci.org/#!/stackmob/scaliak). Tests are run after every commit. I did have to bump -Xss all the way up to 8M, however. Its on my todo list to split up ScaliakBucketSpecs for this reason as well as the fact that the spec covers a little more than it should.

@ppurang

This comment has been minimized.

Show comment Hide comment
@ppurang

ppurang Apr 27, 2012

How about moving on to 2.9.2? and do a cross build for 2.9.1. With 2.9.2 even -Xss1M was enough for all tests except the ScaliakBucketSpecs.

For further refactoring I would suggest using the riak wiki topics http://wiki.basho.com/Developers.html as a way to organize the tests. That is

List Buckets
List Keys
Get Bucket
Set Bucket
Fetch Object
Store Object
Delete Object
Link Walking
MapReduce
Secondary Indexes
etc.

ppurang commented Apr 27, 2012

How about moving on to 2.9.2? and do a cross build for 2.9.1. With 2.9.2 even -Xss1M was enough for all tests except the ScaliakBucketSpecs.

For further refactoring I would suggest using the riak wiki topics http://wiki.basho.com/Developers.html as a way to organize the tests. That is

List Buckets
List Keys
Get Bucket
Set Bucket
Fetch Object
Store Object
Delete Object
Link Walking
MapReduce
Secondary Indexes
etc.

@jrwest

This comment has been minimized.

Show comment Hide comment
@jrwest

jrwest Apr 27, 2012

Contributor

We haven't yet moved to 2.9.2 in production yet, so thats why it has not been done. We will be soon so that will probably happen around the same time. However, Since its pretty easy to cross build I will see if I can get it done sooner (or pull requests are always greatly appreciated ;)).

Thanks for the suggestions on test organization. I think thats probably a good starting point. I will consider it when the tests get refactored. One other issue I have with the tests it ScaliakBucketSpecs tests ScaliakBucket, ReadObject and WriteObject. This is another way i will be drawing lines between the tests.

Contributor

jrwest commented Apr 27, 2012

We haven't yet moved to 2.9.2 in production yet, so thats why it has not been done. We will be soon so that will probably happen around the same time. However, Since its pretty easy to cross build I will see if I can get it done sooner (or pull requests are always greatly appreciated ;)).

Thanks for the suggestions on test organization. I think thats probably a good starting point. I will consider it when the tests get refactored. One other issue I have with the tests it ScaliakBucketSpecs tests ScaliakBucket, ReadObject and WriteObject. This is another way i will be drawing lines between the tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment