Rust framework

[jravetch]

Starting with a clean branch to address #104 . I've updated the PR with commits from the previous PR #172. Rust is installed as expected, including the env variables. I've run the pep8 tests and they all pass. I'm a little confused and may need some help with the small, medium and large tests, though.

[didrocks]

Ok! So, the code itself looks good and there is nothing to modify, let's work on the tests now!

• small tests: this should be straightforward, you don't touch anything impact small (unit) tests, you can still run them if you want to confirm: ./runtests small

Now, medium and large tests. First, ensure you read README.md to understand what those tests are doing.
Let's focus on large tests first. The idea here is to do a real installation on your machine and validate it. Ensure that you removed your rust framework firsrt, all large tests expects that :)
Then, copy your tests/large/test_rust.py from your previous PR.

And run it:
./runtests tests/large/test_rust.py
-> the test should pass if I'm reading it correctly :)

What is missing is to ensure that the second path ("cargo/bin/") is added to the path and that LD_LIBRARY_PATH is set. For this, there are some helper in the test framework.

• For the first, I would just add (below self.assertTrue(self.is_in_path(self.exec_path))):
  self.assertTrue(self.is_in_path(<binary name of something in <cargo/bin>))
• testing the env variable content should be really similar to https://github.com/ubuntu/ubuntu-make/blob/master/tests/large/test_android.py#L438.
  That way, you prepared as well for medium tests (as the wrapper like self.command_as_list()) automatically wraps around ssh commands to connect to the container and such.

You can find the definition of is_in_path() and other helpers such as command_as_list() in tests/large/init.py (LargeFrameworkTests class that your Rust test class inherit) and tests/medium/init.py for the medium tests (that's why we make them inheriting from ContainerTests class).

Feel free to ask me if anything isn't clear! :)

[jravetch]

@didrocks Thanks for the help with the tests. Some issues I have:

1. When testing the LD_LIBRARY_PATH I get the following error:

[nose_cov] DEBUG: nose-cov options
[nose_cov] DEBUG: nose-cov configure
F
======================================================================
FAIL: Install Rust from scratch test case
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jravetch/Projects/ubuntu-make/tests/large/test_rust.py", line 76, in test_default_rust_install
    self.installed_path)
AssertionError: '' != '/home/jravetch/.local/share/umake/rust/rust-lang'
+ /home/jravetch/.local/share/umake/rust/rust-lang

2. If I comment that test out, I get the another error saying rust is not installed and cannot compile:

[nose.plugins.manager] DEBUG: DefaultPluginManager load plugin nose_json = nose_json.plugin:JsonReportPlugin
[nose.plugins.manager] DEBUG: DefaultPluginManager load plugin cov = nose_cov:Cov
[nose_cov] DEBUG: nose-cov options
[nose_cov] DEBUG: nose-cov configure
bash: rustc: command not found
E
======================================================================
ERROR: Install Rust from scratch test case
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jravetch/Projects/ubuntu-make/tests/large/test_rust.py", line 79, in test_default_rust_install
    output = subprocess.check_output(self.command_as_list(compile_command)).decode()
  File "/usr/lib/python3.4/subprocess.py", line 620, in check_output
    raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command '['bash', '-l', '-c', 'rustc /tmp/jravetch/tmpe3c4g3h4/hello.rs']' returned non-zero exit status 127

----------------------------------------------------------------------

3. For the medium tests, I see the following:

I first noticed that I was missing a dependency, sshpass, which I installed. I now see:

E
======================================================================
ERROR: Install Rust from scratch test case
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jravetch/Projects/ubuntu-make/env/lib/python3.4/site-packages/pexpect/expect.py", line 97, in expect_loop
    incoming = spawn.read_nonblocking(spawn.maxread, timeout)
  File "/home/jravetch/Projects/ubuntu-make/env/lib/python3.4/site-packages/pexpect/pty_spawn.py", line 452, in read_nonblocking
    raise TIMEOUT('Timeout exceeded.')
pexpect.exceptions.TIMEOUT: Timeout exceeded.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jravetch/Projects/ubuntu-make/tests/large/test_rust.py", line 62, in test_default_rust_install
    self.expect_and_no_warn("Choose installation path: {}".format(self.installed_path))
  File "/home/jravetch/Projects/ubuntu-make/tests/large/__init__.py", line 133, in expect_and_no_warn
    self.child.expect(expect_query, timeout=timeout)
  File "/home/jravetch/Projects/ubuntu-make/env/lib/python3.4/site-packages/pexpect/spawnbase.py", line 315, in expect
    timeout, searchwindowsize, async)
  File "/home/jravetch/Projects/ubuntu-make/env/lib/python3.4/site-packages/pexpect/spawnbase.py", line 339, in expect_list
    return exp.expect_loop(timeout)
  File "/home/jravetch/Projects/ubuntu-make/env/lib/python3.4/site-packages/pexpect/expect.py", line 104, in expect_loop
    return self.timeout(e)
  File "/home/jravetch/Projects/ubuntu-make/env/lib/python3.4/site-packages/pexpect/expect.py", line 68, in timeout
    raise TIMEOUT(msg)
pexpect.exceptions.TIMEOUT: Timeout exceeded.
<pexpect.pty_spawn.spawn object at 0x7f150579edd8>
command: /usr/bin/sshpass
args: ['/usr/bin/sshpass', '-p', 'user', 'ssh', '-o', 'UserKnownHostsFile=/dev/null', '-o', 'StrictHostKeyChecking=no', '-t', '-q', 'user@172.17.0.2', '/home/jravetch/Projects/ubuntu-make/tests/tools/run_in_umake_dir', '/umake', './bin/umake rust']

Not sure what's going wrong here.

[didrocks]

Please keep the assert_exec_exists before self.is_in.path (we test the existance before testing it's in path) ;)

[didrocks]

it's rather 2014-2015 (as I wrote on the previous comment)

[didrocks]

No need for another property (we only use it once). So just do the join at the place we assertTrue (as in my example I gave :p)

self.assertTrue(self.is_in_path(os.path.join(self.installed_path, "cargo", "bin", "cargo")))

[jravetch]

@didrocks Thanks for the comments. I pushed an update to address those.

[didrocks]

Ah, I know why you are having that issue on the large tests! The issue is that we expect that your shell of choice is bash. However, you are using zsh, and so we add the env variable to your .zprofile only.

Consequently, the test does the right thing: it tries to launch bash, see that the env variable isn't there (empty).
We could use $SHELL instead of harcoding bash to reuse the shell you are using, I don't know zsh quite well, do you mind giving a look if -l -c is available there as well or some similar option?

So I bet if you run:
$ bash
$ ./runtests…

that large test will pass.

On the second error, ti's quite hard to debug without being able to ssh into your docker container to see why it's timing out even on the go framework (it's normal on the rust one, as you didn't provide any fake server). If you prefer, I'm happy to handle the medium tests myself while you are checking why your system didn't work for mediums ones using the go framework.

[jravetch]

@didrocks So I ran the rust large tests in bash, same error. I then tried running the go large tests in zsh and they all passed. So there's something wrong with how the path is being set for rust.

For the docker container, I'm running docker 1.9, when I started the go medium tests, it pulled down your image didrocks/docker-umake-manual. Is there something else I need to setup before running the medium tests?

[didrocks]

@jravetch: ok, I know what's wrong in the way we detect current shell, let me fix it in trunk (then, launching tests under a bash env would work for you as well).
I'm installing zsh to see if something can be done to mimic bash -l -c…
[EDIT] -> hum no, I'm doing the right thing (checking your $SHELL variable to know what profile file to edit…). When you are running bash, try echo $SHELL is giving you "/bin/bash", right? So I guess your .profile is edited…
[EDIT2]: -> The SHELL variable is actually the preferred shell, not the current one. So my bad. This matches for most of the cases, but I need to determine the current shell in a way (or maybe force in the tests bash?
So, if you set:
SHELL=/bin/bash ./runtests…
This should work for you :)

On the docker container, once docker and sshpass is installed (and you saw that it's pulling the right docker image, nothing special.
Do you know a little bit about docker? I would advise to:

• add a return in tests/medium/init.py just after:
  def tearDown(self):
  That way, the container won't be destroyed once the test ends.
• run the test_go.py test, you should have a container hanging now
• docker ps -> get the first column (it's the ID)
• docker run -t -i CONTAINER_ID bash

-> you should be inside the container, with everything ready.

• go to /home//umake directory
• run bin/umake go
  -> see what happens and why the install doesn't work…

[didrocks]

So, to fix the first item, I did push 20cb8b9. This way, we ensure everyone running tests have a similar environment to avoid those headaches. :)
zsh is covered in unit tests anyway.

That should fix the large test for you, keep me posted!

[jravetch]

@didrocks Large tests for rust now pass! Thanks again for getting this to work with zsh. Yeah I've been using docker for some other projects. I added the return right after the def tearDown(self):

Go medium test ran and failed, but the container stayed around.

So for the docker container, the command to run bash is:

docker exec -t -i CONTAINER_ID bash

Which connects me to the running container.

When connected, I go to: /home/jravetch/Projects/ubuntu-make and run bin/umake go. I see the following:

root@golang:/home/jravetch/Projects/ubuntu-make# bin/umake go    
Choose installation path: /root/.local/share/umake/go/go-lang
Downloading and installing requirements                                                                                                                                        |
100% |#########################################################################################################################################################################|
Installing Go Lang
ERROR: Unhandled exception                                                                                                                                                     |
Traceback (most recent call last):
  File "/umake/umake/tools.py", line 154, in wrapper
    function(*args, **kwargs)
  File "/umake/umake/frameworks/baseinstaller.py", line 393, in decompress_and_install_done
    self.post_install()
  File "/umake/umake/frameworks/go.py", line 75, in post_install
    "GOROOT": {"value": self.install_path}})
  File "/umake/umake/tools.py", line 386, in add_env_to_user
    current_shell = os.getenv('SHELL').lower()
AttributeError: 'NoneType' object has no attribute 'lower'
|#################################################################################

[didrocks]

Haha, progress! I don't know if it's the same issue that you are seeing in the real test case, but still, I guess I can cover (and default to bash) in tools.py in case the SHELL variable isn't set + add a test for that, will do it shortly.

I don't ensure that's the reason why the medium tests don't pass for you as the environment you are trying to install in is different from the one you are logging into, but worth a shot (I don't reproduce this issue though, I have TERM exported, I wonder what's the difference between your install and mine, the jenkins infra, and such…) apart from zsh.

[didrocks]

Ok, just pushed to master f1635c0 and refactorize to support empty env variable (we force to bash in that case).

The following commit is just the unit tests for it.

Please try first logging in to confirm that fixes it for you, and then, try to rerun your test and see if this was the issue you were getting in the tests.

[jravetch]

@didrocks Thanks for that commit, it does solve running bin/umake go in the container. However, I still see the same timeout exception when running the go test.

E
======================================================================
ERROR: Install Go from scratch test case
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jravetch/Projects/ubuntu-make/env/lib/python3.4/site-packages/pexpect/expect.py", line 97, in expect_loop
    incoming = spawn.read_nonblocking(spawn.maxread, timeout)
  File "/home/jravetch/Projects/ubuntu-make/env/lib/python3.4/site-packages/pexpect/pty_spawn.py", line 452, in read_nonblocking
    raise TIMEOUT('Timeout exceeded.')
pexpect.exceptions.TIMEOUT: Timeout exceeded.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jravetch/Projects/ubuntu-make/tests/large/test_go.py", line 60, in test_default_go_install
    self.expect_and_no_warn("Choose installation path: {}".format(self.installed_path))
  File "/home/jravetch/Projects/ubuntu-make/tests/large/__init__.py", line 137, in expect_and_no_warn
    self.child.expect(expect_query, timeout=timeout)
  File "/home/jravetch/Projects/ubuntu-make/env/lib/python3.4/site-packages/pexpect/spawnbase.py", line 315, in expect
    timeout, searchwindowsize, async)
  File "/home/jravetch/Projects/ubuntu-make/env/lib/python3.4/site-packages/pexpect/spawnbase.py", line 339, in expect_list
    return exp.expect_loop(timeout)
  File "/home/jravetch/Projects/ubuntu-make/env/lib/python3.4/site-packages/pexpect/expect.py", line 104, in expect_loop
    return self.timeout(e)
  File "/home/jravetch/Projects/ubuntu-make/env/lib/python3.4/site-packages/pexpect/expect.py", line 68, in timeout
    raise TIMEOUT(msg)
pexpect.exceptions.TIMEOUT: Timeout exceeded.
<pexpect.pty_spawn.spawn object at 0x7f7bbe102f98>
command: /usr/bin/sshpass
args: ['/usr/bin/sshpass', '-p', 'user', 'ssh', '-o', 'UserKnownHostsFile=/dev/null', '-o', 'StrictHostKeyChecking=no', '-t', '-q', 'user@172.17.0.3', '/home/jravetch/Projects/ubuntu-make/tests/tools/run_in_umake_dir', '/umake', './bin/umake go']
searcher: None
buffer (last 100 chars): ''
before (last 100 chars): ''
after: <class 'pexpect.exceptions.TIMEOUT'>
match: None
match_index: None
exitstatus: None
flag_eof: False
pid: 17410
child_fd: 7
closed: False
timeout: 30
delimiter: <class 'pexpect.exceptions.EOF'>
logfile: None
logfile_read: None
logfile_send: None
maxread: 2000
ignorecase: False
searchwindowsize: None
delaybeforesend: 0.05
delayafterclose: 0.1
delayafterterminate: 0.1

[didrocks]

Ok, I think this is going to be hard to debug remotely. Keep the return before tearDown to keep the container up. Then run:
/usr/bin/sshpass -p user ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -t -q user@172.17.0.3 /home/jravetch/Projects/ubuntu-make/tests/tools/run_in_umake_dir /umake './bin/umake go'

Of course, the IP will change based on the container, but look at the failure output, it's listed :)
Then, telling me what happens (as it seems it's failing right away on "Choose installation path")
Thanks for hanging tight!

[jravetch]

@didrocks So I ran:

jravetch@ohyeah ~ $ /usr/bin/sshpass -p user ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -t -q user@172.17.0.2 /home/jravetch/Projects/ubuntu-make/tests/tools/run_in_umake_dir /umake './bin/umake go'

And it just hangs there, with nothing happening.

[didrocks]

Interesting, I wonder if there is anything in your local ssh settings preventing this, and so you don't even connect to the container.

Maybe the way to really confirm it's the environment for you is that you create a vm and try running the tests inside it?

Meanwhile, to move this PR, if you agree, I'm happy to finish the work on medium tests, wdyt?

[jravetch]

@didrocks Yeah I'll work on this offline. Please do go ahead with finishing up the medium tests. Thanks again for your help and patience.

[didrocks]

Ok, poking you once done (probably working on this today), so that you can pull and I'll do a final review then.

Keep me posted about your medium tests discovering, I'm intrigued :)

[jravetch]

@didrocks Aha, I figured it out! sshpass needs to run with sudo. Are you guys running with sudo or as root? Is there a way to run without sudo?

[didrocks]

@jravetch interesting, no I'm not running it with sudo and other don't either. I wonder if it's another thing related to zsh and so, playing in the noninteractive side of the shell…
I guess, when you run sudo, you are back to an account using "bash -l -i".

So, do you want to give a try to the medium tests then or prefer me to?

[jravetch]

@didrocks Probably want to dive a bit deeper to figure out what's going on. Also it's getting late here. I did try running in bash and still got the timeout. So yeah if you don't mind finishing up the medium tests for rust, that would be great.

[didrocks]

Will do! Have a good night :)

[jravetch]

@didrocks I think you're right re zsh vs bash. I ran the medium tests in a sudo su session. Now it's complaining about the locale not being set.

F
======================================================================
FAIL: Install Go from scratch test case
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jravetch/Projects/ubuntu-make/tests/large/test_go.py", line 72, in test_default_go_install
    self.assertEqual(output, "hello, world\r\n")
AssertionError: 'bash: warning: setlocale: LC_ALL: cannot [264 chars]\r\n' != 'hello, world\r\n'
- bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
- /bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
- bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
- bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
  hello, world

This stackoverflow issue seems to address it:

http://stackoverflow.com/questions/23084608/shell-function-issue-remove-the-single-quote-from-shell-statement

[didrocks]

Interesting… If you can push the investigation and find a sane way for it to run with your setup, I'm happy to push in the env, so that even (kidding ;)) zsh users can run the tests easily!

[didrocks]

Pushed some modifications, medium tests assets and certificates + more medium tests to emulate errors cases.

If you want to run them, do not forget as well to docker pull the latest image (as it contains the new certificate to mock the rust website).

Do you mind pulling in your branch so that I can do a final review? Thanks!

[jravetch]

@didrocks Thanks for your help on the medium tests. I pulled in your commits from your rust branch into this one. Please review and let me know if we need to cleanup anything before merging.

[didrocks]

I just spotted an additional unused variable in the tests. All the rest looks good! I'm going to merge this manually, remove the additional var and we should be done! As the merge involves multiple persons, we won't rebase this one.

Thanks again for your patience and great work on this! Do not hesitate to keep me posted about the medium tests if you find any good solution for your kind of setup. ;)

I think I'll cut a release either on Tuesday or Thursday as time permits. Thanks again!