io_tune core dump on t2.micro EC2 instances

[tzach]

Installation details
Scylla version (or git commit hash): 1.3
OS (RHEL/CentOS/Ubuntu/AWS AMI): Scylla AMI, t2. micro

The following is a capture of trying to run 1.3 AMI on t2.micro instance type.

    t2.micro is not supported instance type!
To continue startup ScyllaDB on this instance, run 'scylla_io_setup' then 'systemctl start scylla-server'.
To run ScyllaDB on supported instance type, run AMI in m3/c3/i2 types.
[centos@ip-172-31-18-176 ~]$ scylla_io_setup
terminate called after throwing an instance of 'std::system_error'
  what():  open: Permission denied
/usr/sbin/scylla_io_setup: line 87:  1994 Aborted                 (core dumped) iotune --evaluation-directory $DATA_DIR --format envfile --options-file /etc/scylla.d/io.conf $CPUSET
/var/lib/scylla did not pass validation tests, it may not be on XFS and/or has limited disk space.
This is a non-supported setup, and performance is expected to be very bad.
For better performance, placing your data on XFS-formatted directories is required.
 To override this error, see the developer_mode configuration option.

[tzach]

@glommer can you take a look?

[glommer]

Sure.

It sounds like a permission error. Likely at /var/lib/scylla. Maybe some
issue with the AMI setup ?

On Sep 4, 2016 3:19 AM, "Tzach Livyatan" notifications@github.com wrote:

@glommer https://github.com/glommer can you take a look?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1645 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAUNvQoACd46RP1_2xY06LA1jvXvr2Wuks5qmnESgaJpZM4J0edL
.

[slivne]

@tzach @glommer is correct the issue is permissions

you are running scylla_io_setup from the centos user that does not have write privleges into /var/lib/scylla

we can provide the following instructions run 'sudo scylla_io_setup' instead and that should work

(at least it did for me) and ended up with something else

[centos@ip-172-30-0-240 ~]$ sudo scylla_io_setup 
Generating evaluation file sized 10GB... timed out before we could write the entire file. Will continue but accuracy may suffer.
Timed out: timed out before we could write 1GB worth of data. Not enough to continue
/var/lib/scylla did not pass validation tests, it may not be on XFS and/or has limited disk space.
This is a non-supported setup, and performance is expected to be very bad.
For better performance, placing your data on XFS-formatted directories is required.
 To override this error, see the developer_mode configuration option.

So the user can use developer-mode to overcome this.

It is far from being perfect but does provide the correct information (e.g. the user should enable developer-mode)

Is that enough ?

[tzach]

The doc already requires sudo scylla_setup
Is the result of running without sudo is core dump?
This should be fixed.

[slivne]

The doc does require it - and you did not do what the doc requires - you ran scylla_io_setup

[centos@ip-172-31-18-176 ~]$ scylla_io_setup

directly and not sudo scylla_io_setup

we can instruct the user todo that not only in the doc but also when we fail in boot

To continue startup ScyllaDB on this instance, run 'sudo scylla_io_setup' ...

• wth regards to not coredumping if there are not enough permissions - we can do that as well - but overall that does not seem that urgent - in my view the issue is that the instructions that user get on he ami are not inlined with what he should do (what the doc says).

[tzach]

Note that I was following the instructions on the AMI:

To continue startup ScyllaDB on this instance, run 'scylla_io_setup' then 'systemctl start scylla-server'.
To run ScyllaDB on supported instance type, run AMI in m3/c3/i2 types.

this must be fixed.

[slivne]

@glommer what is left is not to coredump in case there are no permissions and to provide the info and exit with an error.

[tzach]

@slivne can I test with 1.4.1 AMI?

[slivne]

yes

On Wed, Nov 9, 2016 at 5:26 PM, Tzach Livyatan notifications@github.com
wrote:

@slivne https://github.com/slivne can I test with 1.4.1 AMI?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1645 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADThCPyKV7hz6fKqlPWjxh72IWTeYB5_ks5q8eYpgaJpZM4J0edL
.

[tzach]

Well, the original problem was solved, but the entire process is not pleasant.
I will open a new issue on it.

[tzach]

Well, the original problem was solved, but the entire process is not pleasant.
I will open a new issue on it.

#1829