Import files to script

[toert]

How can I import my custom files?
I need to use my files inside the script.
print(os.path.dirname(sys.argv[0])) gives me /opt/DO_wooey/media/wooey_scripts, I decided to put my custom files in this directory, but it does not work, the script still gives me this error: OSError: SavedModel file does not exist at: test_build/{saved_model.pbtxt|saved_model.pb} .

[Chris7]

Hi @toert,

Local files have long been an open question for Wooey -- from scripts that have relative imports, to this use case now. There are a few complicating factors that doesn't make this trivial -- notably that there is no guarantee of where the script is being run having the same environment of where the worker is. The worker only downloads the script to execute locally.

The easiest solution I can think of would be using a NFS to store common data files (if you are using a server setup), or giving more information about the location of your files and where wooey/celery are running.

[SSchott]

Make sure to use absolute paths into your script. In my case I'm using multiple modules and even other programs, and I specified everything relative to the absolute path of the script itself, so that I can copy it anywhere, as long as the relative location of the extra files is maintained.

[pymenow]

@Chris7 , "from scripts that have relative imports" , I am trying this out and failing. Whats the best work around for this ? Will a python zip work ?

[Chris7]

The python zip idea is interesting. I'll look into what that may entail.

[pymenow]

Here is some info I just read up on.
https://blogs.gnome.org/jamesh/2012/05/21/python-zip-files/

[pymenow]

@Chris7 , thanks for considering this !

OK So here is what I did :
I bundle up my project as below
lib/
__main__py

the lib folder has all libs specific to my project.
I renamed my main script to __main__.py

I then zip the contents up into reportgen.zip
I renamed this to reportgen.py

and now I can execute this directly as such : python reportgen.py locally.
However importing this into wooey gives the below error :

Request Method:	POST
http://URL/admin/wooey/script/add/
1.9.13
UnicodeDecodeError
'utf-8' codec can't decode byte 0x9a in position 12: invalid start byte
/opt/anaconda/anaconda3/lib/python3.6/codecs.py in decode, line 321
/opt/anaconda/anaconda3/bin/python
3.6.3

[Chris7]

Thanks for the trail run! I haven't tested this yet, but I thought it would probably fail because there is no detection routine currently to detect if an uploaded file is a zip. This support needs to be added to clinto and Wooey may "just work" after that.

[pymenow]

@Chris7 this zip feature is most sought after in the team i am working with. Please let me know if I can help test the fix / change you may have for clinto.

[Chris7]

There is support for this in clinto via wooey/clinto#31, but it is blocked until the subparser bugs are sorted out.

[pymenow]

I should be able to take on the patch and test locally ?
Sorry but I am not sure if the sub parser bugs are a gate to this feature or for the merge ?

[Chris7]

It's a gate for a stable package release. So to use this, you would need to run off the master branch of Wooey and the patch branch of clinto. I would only do that if you are ok with your database possibly becoming invalid for the next official release.

[Chris7]

@pymenow This should be testable now , but I would still backup your database first.

[tomanizer]

wooey/clinto#31 Is closed. Does this mean zip support will work now?
If yes, we should update the docs. In can submit a PR for the docs if that helps.

[Chris7]

Zip support is in now. I would love a docs PR. Having docs updated by someone new to the project is always the best I've found. I tend to forget what things aren't obvious :)

[Chris7]

Added in v0.10

[pymenow]

Just ran into this on the 0.12 using docker -

Request Method:	POST
http://autoreports.atonarp.com/admin/wooey/script/add/
3.0.5
ParserError
Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/clinto/parsers/argparse_.py", line 253, in extract_parser ast_source = source_parser.parse_source_file(self.script_path) File "/usr/local/lib/python3.7/site-packages/clinto/ast/source_parser.py", line 39, in parse_source_file s = f.read() File "/usr/local/lib/python3.7/codecs.py", line 322, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xda in position 14: invalid continuation byte
/wooey_build/wooey/signals.py in script_version_postsave, line 64
/usr/local/bin/python
3.7.7
['/docker_wooey', '/docker_wooey', '/usr/local/lib/python37.zip', '/usr/local/lib/python3.7', '/usr/local/lib/python3.7/lib-dynload', '/usr/local/lib/python3.7/site-packages', '/wooey_build']

[SaadBazaz]

Hey! Can someone point me to the relevant documentation where this "zip" thing is mentioned? I can't seem to find it and it's a necessary use case of my script to have relative file imports.