New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test cases may fail if config files already exist #2190

Open
markus2330 opened this Issue Aug 18, 2018 · 4 comments

Comments

Projects
None yet
2 participants
@markus2330
Contributor

markus2330 commented Aug 18, 2018

For example:

  • testshell_markdown_ccode and testshell_markdown_tcl fail if ~/.config/config.tcl is a non-TCL file
  • testshell_markdown_tutorial_validation fails if ~/.config/tutorial.dump contains user/tests/tutorial/spec/should_not_be_here

Furthermore, the situation, that our tests simply start modifying already existing config files, is not ideal.

Two ways to resolve this:

  • the shell recorder moves and restores config files if they were already present (but not created by earlier lines of the shell recorder test)
  • the shell recorder prefixes the names of the config files
@sanssecours

This comment has been minimized.

Show comment
Hide comment
@sanssecours

sanssecours Aug 18, 2018

Contributor

I think the most simple solution would be to reserve a certain prefix for MSR config file names. For example, we could just require that every config file name we use in (Markdown) Shell Recorder tests has to start with the text tests_.

Contributor

sanssecours commented Aug 18, 2018

I think the most simple solution would be to reserve a certain prefix for MSR config file names. For example, we could just require that every config file name we use in (Markdown) Shell Recorder tests has to start with the text tests_.

@markus2330

This comment has been minimized.

Show comment
Hide comment
@markus2330

markus2330 Aug 18, 2018

Contributor

A constant prefix might reduce the likelihood but in my case it would not have been helpful: all the cases I had up to now where leftovers from unsuccessful or aborted previous test runs.

Shellrecorder is the only place where we can give guarantees about files not being there and being cleaned up in the end.

Contributor

markus2330 commented Aug 18, 2018

A constant prefix might reduce the likelihood but in my case it would not have been helpful: all the cases I had up to now where leftovers from unsuccessful or aborted previous test runs.

Shellrecorder is the only place where we can give guarantees about files not being there and being cleaned up in the end.

@sanssecours

This comment has been minimized.

Show comment
Hide comment
@sanssecours

sanssecours Aug 18, 2018

Contributor

…all the cases I had up to now where leftovers from unsuccessful or aborted previous test runs.

In that case the second solution you proposed above:

  • the shell recorder prefixes the names of the config files

does not fix the problem either.

Contributor

sanssecours commented Aug 18, 2018

…all the cases I had up to now where leftovers from unsuccessful or aborted previous test runs.

In that case the second solution you proposed above:

  • the shell recorder prefixes the names of the config files

does not fix the problem either.

@markus2330

This comment has been minimized.

Show comment
Hide comment
@markus2330

markus2330 Aug 18, 2018

Contributor

Depends on the prefix: mktemp or uuid should suffice for any practical purpose.

Contributor

markus2330 commented Aug 18, 2018

Depends on the prefix: mktemp or uuid should suffice for any practical purpose.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment