monketype fails to stub the initial script

[Hnasar]

I want to add types to myscript.py,

so I do monkeytype run myscript.py

but monkeytype stub myscript.py says No traces found

so I dumped monkeytype.sqlite3 and see it has traces under the module named __main__,

but monkeytype stub __main__ says ERROR: Failed decoding trace: Module '__main__' has no attribute 'main'

[carljm]

Yeah, when you run myscript.py as a script, its module name is __main__; when you run monkeytype stub myscript, its module name is myscript. It may be possible to hack around this by mangling the name of traces recorded with module __main__ (if we can figure out the actual module name for the __main__ module, which might not be easy to do reliably). I have mixed feelings about it; monkeytype needs to import the modules it annotates, and importing a script that isn't carefully written could have unintended side effects. I think I'd rather just have MonkeyType clearly intended for annotating the modules used by the script; if you want to annotate myscript.py itself, treat it as a module and make another short script that imports and calls it, and run monkeytype with that script.

For now I'll add a documentation note to that effect. But I'll leave this open as a low-pri enhancement; if someone submits a PR that implements it in a way that looks robust, I'd consider it.

[carljm]

Doc clarification added in de7868a

Thanks for the report! This was definitely an oversight in the documentation, it wasn't at all clear which code monkeytype run was intended to trace.

[bhrutledge]

if you want to annotate myscript.py itself, treat it as a module and make another short script that imports and calls it, and run monkeytype with that script.

Here's a simple but concrete example of this workaround:

https://github.com/bhrutledge/til/blob/main/python/monkeytype-script.md

[ghost]

@bhrutledge maybe this workaround not working anymore..?

I have now this,

./
├── src/
│  └── sort.py
└── run_script.py

run_script.py

import sys
sys.path.insert(1,'./src')
import sort
sort.main()

## this also not work
# from src import sort
# sort.main()

then, monkeytype run run_script.py run correcty.

however,

❯ monkeytype -v stub sort
WARNING: Failed decoding trace: No module named 'sort'
WARNING: Failed decoding trace: No module named 'sort'
No traces found for module sort

stub and apply causes error.

[bhrutledge]

@Ryuta69 monkeytype stub is failing because the src directory isn't in the module search path, so it can't find the sort module. Instead of modifying the path in run_script.py, you could:

• Move run_script.py to src and execute the monkeytype commands from there

• Or, add the src directory to your PYTHONPATH prior to running the monkeytype commands, e.g.

$ PYTHONPATH=src:$PYTHONPATH

$ monkeytype run run_script.py

$ monkeytype stub sort

Here's an example of the second option: https://repl.it/@bhrutledge/monkeytyperunscript

[ghost]

@bhrutledge oh, great! Now it works. Thank you very much!

[dgutson]

What about adding an option so MonkeyType creates a temp python file turning the "main" into a module? That is, to "automate" the workaround and no need for any bash.
Maybe some python parsing is needed to create the temp calling file.

@arieltorti

[carljm]

What about adding an option so MonkeyType creates a temp python file turning the "main" into a module?

I'm open to something like this in principle as a resolution of this ticket, but I'm not clear how it should work in general. Does it assume that the script module has a main function to call? What if that function takes arguments?

It really seems like the "runner" script has to be created with some knowledge of the script itself.

[dgutson]

I think it should parse the file and look for __main__ (the typical if...) and forward the arguments if provided in the cli. This will in term be a validation. In any case I'm convinced that a minimum static analysis is needed.

[carljm]

I think it should parse the file and look for main (the typical if...) and forward the arguments if provided in the cli.

I'm not sure what this means; we can't "call" an if __name__ == "__main__": clause from another module, so the only way this approach works is if that clause simply calls a main() or similar function, which we can import and call.

TBH the whole thing sounds quite finicky and unreliable to me; I'm not going to work on it, but if somebody has a concrete plan I'm willing to evaluate whether it seems workable and potentially look at a PR.

[dgutson]

I think it should parse the file and look for main (the typical if...) and forward the arguments if provided in the cli.

I'm not sure what this means; we can't "call" an if __name__ == "__main__": clause from another module, so the only way this approach works is if that clause simply calls a main() or similar function, which we can import and call.

This was exactly my "parsing the code" proposal.

TBH the whole thing sounds quite finicky and unreliable to me; I'm not going to work on it, but if somebody has a concrete plan I'm willing to evaluate whether it seems workable and potentially look at a PR.

great! let me bring somebody ;)