Reorganize tests with fixtures

[BoPeng]

Right now our tests uses unittest, and have things such as

class TestActions(unittest.TestCase):

    def setUp(self):
        env.reset()
        self.temp_files = []

    def tearDown(self):
        for f in self.temp_files:
            if file_target(f).exists():
                file_target(f).unlink()

These functions are called for all tests in the group and does not allow individual tests to have more or less setup/teardown procedures.

Fixture in pytest is a more flexible solution for tests and we should try to use it. For example, we should provide fixtures such as

def test_something(input_factory):
   input_factory('a.txt', 'b.txt')

and the advantage is that these files will be automatically removed after the tests are done.

[BoPeng]

Tried on [test_actions.py=(https://github.com/vatlab/sos/blob/master/test/test_actions.py), it is a lot of work but the tests are lot cleaner and clearer.

4210b93

The fixtures that were implemented were

• script_factory

  file_name = script_factory(script)

  writes a SoS script that can be used by the test that will be removed later.

• config_factory

  file_name = config_factory(script)

  writes a configuration file that can be used by the test that will be removed later.

• temp_factory

  temp_factory(file, or_files, dir=dir_or_dirs)

  This fixture will create file or files, or dir, or dirs when it is called, and remove them after the test is completed. Files can be specified as multiple parameters or lists (sequences). The directory will be removed and re-created if it already exists.

  This is used to provide temporary input files to the workflow.

  Use temp_factory (example)

  temp_factory('a.txt', 'b.txt')
  execute_workflow(
      r"""
      [0]
      input: 'a.txt', 'b.txt'
      warn_if(len(_input) == 1)
      """,
      options={'run_mode': 'dryrun'})

• clear_now_and_after

  clear_now_and_after(file, or_files, or_dir, or_dirs)

  Will remove specified files or directories when the fixture is called, and make sure they are also removed after the tests are done.

  This is used to make sure that the tests will not be affected by existing files.

  clear_now_and_after('test_stop_if_0.txt', 'test_stop_if_1.txt')
  
  execute_workflow(r'''
      [10]
      rep = range(2)
      input: for_each='rep'
      output: f'test_stop_if_{_rep}.txt'
      _output.touch()
      stop_if(_rep == 1, no_output=True)
      [20]
      assert(step_input.contains('test_stop_if_0.txt'))
      assert(not step_input.contains('test_stop_if_1.txt'))
      ''')
  
  assert os.path.isfile('test_stop_if_0.txt')
  assert not (os.path.isfile('test_stop_if_1.txt'))

Examples

• Test on variables after test (use of env.sos_dict): this

  execute_workflow(r"""
      [0: shared='ret']
      ret = get_output('echo blah')
      """)
  # use strip because there would be \r\n under windows
  assert env.sos_dict['ret'].strip() == 'blah'

• Test for exception: example

  with pytest.raises(Exception):
      execute_workflow(r"""
          [0]
          get_output('catmouse')
      """)

[BoPeng]

@joannfeng All tests now pass, including windows. I have also wrote a little script to renamed the tests.