Skip to content

Conversation

@kmvanbrunt
Copy link
Member

@kmvanbrunt kmvanbrunt commented Jun 10, 2019

  • Text scripts now run immediately instead of adding their commands to cmdqueue. This allows easy capture of the entire script's output.
  • Transcript generation/runs now stops if any command it runs returns True for stop.
  • Added member to CommandResult called stop which is the return value of onecmd_plus_hooks after it runs the given command line.
  • Removed internally used eos command that was used to keep track of when a text script's commands ended
  • Removed cmd2 member called _STOP_AND_EXIT since it was just a boolean value that should always be True
  • Removed cmd2 member called _should_quit since PyscriptBridge now handles this logic
  • Removed support for cmd.cmdqueue
  • Fixed issue where _cmdloop() suppressed exceptions by returning from within its finally code
  • allow_cli_args is now an argument to init instead of a cmd2 class member
  • Fixed UnsupportedOperation on fileno error when a shell command was one of the commands run while generating a transcript

Fixes #687
Fixes #688
Fixes #692

kmvanbrunt and others added 26 commits May 23, 2019 15:30
…s whether a command returned stop.

This means the command loop will stop once transcript generation completes if a command did return stop.
Otherwise there is a risk in continuing to run if the application's state has already been marked to close.
… stop value. The command loop will also terminate.
…oolean constant and was only used internally
…lication to quit

Implemented an antediluvian TODO
… order to present a better error message to user
…o return whether a command returned True for stop.

Added stop to CommandResult so pyscripts can now know the return value of a command's do_* function.
@codecov
Copy link

codecov bot commented Jun 10, 2019

Codecov Report

Merging #696 into master will increase coverage by <.01%.
The diff coverage is 96.15%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #696      +/-   ##
==========================================
+ Coverage   95.46%   95.47%   +<.01%     
==========================================
  Files          11       11              
  Lines        3241     3248       +7     
==========================================
+ Hits         3094     3101       +7     
  Misses        147      147
Impacted Files Coverage Δ
cmd2/constants.py 100% <ø> (ø) ⬆️
cmd2/argparse_completer.py 90.26% <ø> (ø) ⬆️
cmd2/utils.py 97.67% <100%> (+0.08%) ⬆️
cmd2/pyscript_bridge.py 100% <100%> (ø) ⬆️
cmd2/transcript.py 93.69% <90%> (ø) ⬆️
cmd2/cmd2.py 95.63% <96.15%> (-0.02%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f64f9d5...756d8d3. Read the comment docs.

Copy link
Member

@tleonhardt tleonhardt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good, but I'd recommend changing how we deal with running commands from the cmdqueue in the main loop so that it is safe for multi-threaded applications.

self.poutput('{}{}'.format(self.prompt, line))

Example: cmd_obj.runcmds_plus_hooks(['load myscript.txt'])
if self.onecmd_plus_hooks(line):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, so now we just run all of the commands in a loop instead of appending them to the cmdqueue. That seems reasonable. I'm fearful that there could be some really obscure edge case where this would behave very differently, but I can't think of any right now.

@kmvanbrunt
Copy link
Member Author

kmvanbrunt commented Jun 11, 2019

Uh oh, I'm getting some push back here on 2 issues. Here is my thinking on both.

Command Queue Issue

I agree the code to run commands in cmdqueue as written in this PR is wrong. My intention was for cmd2 to only use cmdqueue for startup commands. I should have actually moved the code outside of the while loop.

The problem is that cmd2 has at least 6 places that could be considered a command loop.

  1. _cmdloop()
  2. history -r
  3. load
  4. pyscript
  5. transcript running
  6. transcript generation

Only _cmdloop() checks the contents of cmdqueue before running the next command. For this reason, running commands from the command line will behave differently in cases where a command appends to the command queue.

For instance, suppose I have a command called save that adds a command called print_results to the queue.

Then at the command line, the user runs these commands:
save
help

My history would be:
1 save
2 print_results
3 help

If I then do, history -r, commands will run this way:
save
print_results
help
print_results

cmdqueue is a remnant of cmd, which only runs commands in one place. We run commands from the main loop, history, scripts, and transcripts. Therefore I propose eliminating support of cmdqueue and instead create a member list called _startup_commands. These commands will run before the while loop in _cmdloop().

Transcript Generation Issue

This is a tricky one. The behavior before this PR was wrong in that the value of stop returned by onecmd_plus_hooks was never read and therefore transcript generation kept going.

My PR added code to stop transcript generation when any command returned True. That value gets passed up to the command loop, which then stops. Technically the whole application doesn't stop. Just the command loop does.

Since transcript generation is actually running commands, then behavior should be identical to running these same commands from the CLI or scripts. I agree that it seems weird to terminate the command loop after transcript generation, but it also seems weird to put quit in a transcript.

What I'm trying to do is have consistent behavior across all ways of running commands. Basically, if a command tells the command loop to stop, then it should in all cases, regardless of how it was run.

I wrote this in issue #688. Perhaps I didn't explain well enough.

Generating a transcript will only exit the command loop if one of the commands it runs returns True for its Stop value. For instance, running quit while generating a transcript will stop both transcript generation and cmd2's command loop..

@kmvanbrunt
Copy link
Member Author

kmvanbrunt commented Jun 11, 2019

@kotfu @tleonhardt
Since adding stop handling to transcript generation is new behavior, I took my best guess. After reading my reasons, if you still disagree with stopping the whole command loop, let me know what the desired behavior should be.

  1. Should I just terminate the transcript generation loop?
  2. Should I also print a message to the user? Right now it already prints a message using pfeedback() saying how many command ran. If it stops early, I could also print which command caused a stop.

allow_cli_args is now an argument to __init__ instead of a cmd2 class member
@tleonhardt
Copy link
Member

tleonhardt commented Jun 12, 2019

The current unit test failures are pretty weird and I don't understand them, but I have been able to replicate them locally.

If I just run py.test without any arguments, all tests pass.

However, if I run py.test with any arguments, such as py.test -v, then the same 6 tests fail and they do so because the arguments to py.test get passed as commands at invocation during these tests.

@kotfu Do these unit test failures make sense to you?

@tleonhardt
Copy link
Member

I made a partial reversion of the last commit to get the unit tests passing again until such as time as we can figure out WTF was going wrong.

self.broken_pipe_warning = ''

# Commands that will run at the beginning of the command loop
self._startup_commands = []
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing to the use of the self._startup_commands list instead of using self.cmdqueue inherited from cmd solves the "Command queue issue" alluded to in the comments.

Overall, I think we should do a better job in the Sphinx docs of calling out the elements in cmd which are not used or supported in cmd2. We have a "Note" for this in our Overview. But I think it is long overdue that we expand upon that.

tleonhardt
tleonhardt previously approved these changes Jun 13, 2019
@tleonhardt
Copy link
Member

@kotfu If you get a chance to look at this, there are a lot of changes. I think @kmvanbrunt addressed all of the concerns we both had, but I'd appreciate any further feedback you might have.

The major fundamental change is that self.cmdqueue is now completely unused in cmd2 and its use has been replaced by self._startup_commands. But self._startup_commands is is only used for supporting startup scripts and commands at invocation and is completely unused during the main REPL. A side affect is that load no longer queues commands, but runs them directly.

I can't think of any negative unintended consequences. But this is a major change in implementation, so I'd appreciate another brain thinking about it.

@tleonhardt
Copy link
Member

@teto Can you verify that this PR is getting you most of what you want in terms of the transcript testing and generation changes you were looking for?

Copy link
Member

@tleonhardt tleonhardt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recent changes look OK.

@kmvanbrunt kmvanbrunt merged commit ddd07f9 into master Jun 14, 2019
@kmvanbrunt kmvanbrunt deleted the script_refactor branch June 14, 2019 21:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

None prompt exits cmdloop flag to exit cmd2/stop the loop on command failure Handle return code of onecmd_plus_hooks

5 participants