bug fixes/typos in the docs for jekyll-web export #75

vsoch · 2022-08-21T23:40:48Z

This will address:

Signed-off-by: vsoch vsoch@users.noreply.github.com

Signed-off-by: vsoch <vsoch@users.noreply.github.com>

vsoch · 2022-08-21T23:49:04Z

Ping @NickleDave for review! Sorry for the noise of isort, but this PR should:

add a check for the needed envars (RSE_HOST) and print a better message if you try running the export command (for the flask site again)
add links to the jekyll-web template under tutorials and the command example
fix the bugs with bot.exit -> sys.exit. The bot actually does have an exit, but in some files I create a logger proper (which does not). This is probably a bad design choice and I'll come back and fix it, but not today!

NickleDave · 2022-08-22T00:04:47Z

To test I cloned, checked out the PR, and installed into the .venv for my bioacoustics-software feature branch.

"'Logger' object has no attribute 'exit'" from rse import #73
The error for the bad url is better, but still a bit overwhelming. Maybe raise an actual error with a traceback? I got

rse import --type google-sheet "https://docs.google.com/spreadsheets/d/e/2PACX-1vQkPsu14BG0bErrY0thXymfS55be0spEVX_WpWm2Yy3We8swMO0sIb3iD4Sg-i1lWnxSsiiN5JmWAD-/pub\?gid\=0\&single\=true\&output\=csv"
Issue parsing https://docs.google.com/spreadsheets/d/e/2PACX-1vQkPsu14BG0bErrY0thXymfS55be0spEVX_WpWm2Yy3We8swMO0sIb3iD4Sg-i1lWnxSsiiN5JmWAD-/pub\?gid\=0\&single\=true\&output\=csv: <!DOCTYPE html><html lang="en"><head><meta name="description" content="Web word processing, presentations and spreadsheets"><meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, minimum-scale=1.0, user-scalable=0"><link rel="shortcut icon" href="//docs.google.com/favicon.ico"><title>Page Not Found</title><meta name="referrer" content="strict-origin-when-cross-origin"><link href="//fonts.googleapis.com/css?family=Product+Sans" rel="stylesheet" type="text/css" nonce="sVH5syIhh8xZVkP7K-TzxA"><style nonce="sVH5syIhh8xZVkP7K-TzxA">/* Copyright 2022 Google Inc. All Rights Reserved. */
.goog-inline-block{position:relative;display:-moz-inline-box;display:inline-block}* html .goog-inline-block{display:inline}*:first-child+html .goog-inline-block{display:inline}#drive-logo{margin:18px 0;position:absolute;white-space:nowrap}.docs-drivelogo-img{background-image:url('//ssl.gstatic.com/images/branding/googlelogo/1x/googlelogo_color_116x41dp.png');background-size:116px 41px;display:inline-block;height:41px;vertical-align:bottom;width:116px}.docs-drivelogo-text{color:#000;display:inline-block;opacity:0.54;text-decoration:none;font-family:'Product Sans',Arial,Helvetica,sans-serif;font-size:32px;text-rendering:optimizeLegibility;position:relative;top:-6px;left:-7px;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale}@media (-webkit-min-device-pixel-ratio:1.5),(min-resolution:144dpi){.docs-drivelogo-img{background-image:url('//ssl.gstatic.com/images/branding/googlelogo/2x/googlelogo_color_116x41dp.png')}}</style><style type="text/css" nonce="sVH5syIhh8xZVkP7K-TzxA">body {background-color: #fff; font-family: Arial,sans-serif; font-size: 13px; margin: 0; padding: 0;}a, a:link, a:visited {color: #112ABB;}</style><style type="text/css" nonce="sVH5syIhh8xZVkP7K-TzxA">.errorMessage {font-size: 12pt; font-weight: bold; line-height: 150%;}</style></head><body><div id="outerContainer"><div id="innerContainer"><div style="position: absolute; top: -80px;"><div style="margin: 18px 0; position: absolute; white-space: nowrap;"><a href="//support.google.com/docs/"><img height="35px" src="//ssl.gstatic.com/docs/common/product/spreadsheets_lockup2.png" alt="Google logo"/></a></div></div><div align="center"><p class="errorMessage" style="padding-top: 50px">Sorry, unable to open the file at this time.</p><p> Please check the address and try again. </p><div style="background: #F0F6FF; border: 1px solid black; margin-top: 35px; padding: 10px 125px; width: 300px;"><p><strong>Get stuff done with Google Drive</strong></p><p>Apps in Google Drive make it easy to create, store and share online documents, spreadsheets, presentations and more.</p><p>Learn more at <a href="https://drive.google.com/start/apps">drive.google.com/start/apps</a>.</p></div></div></div></div></body><style nonce="sVH5syIhh8xZVkP7K-TzxA">html {height: 100%; overflow: auto;}body {height: 100%; overflow: auto;}#outerContainer {margin: auto; max-width: 750px;}#innerContainer {margin-bottom: 20px; margin-left: 40px; margin-right: 40px; margin-top: 80px; position: relative;}</style></html>

✔️ for the second case in #73 the error is now correctly formatted so I can see what the missing field is:

$ rse import --type google-sheet "https://docs.google.com/spreadsheets/d/e/2PACX-1vQkPsu14BG0bErrY0thXymfS55be0spEVX_WpWm2Yy3We8swMO0sIb3iD4Sg-i1lWnxSsiiN5JmWAD-/pub?gid=0&single=true&output=csv"      
Sheet https://docs.google.com/spreadsheets/d/e/2PACX-1vQkPsu14BG0bErrY0thXymfS55be0spEVX_WpWm2Yy3We8swMO0sIb3iD4Sg-i1lWnxSsiiN5JmWAD-/pub?gid=0&single=true&output=csv is missing required field title.

This is confusing though, because I do have a column "Title" that is copied directly from the example spreadsheet so I know it's formatted the same way

TypeError during rse export but still got "Export is complete!" #72
Now when I try to rse export I get an error about missing env variables. I did see something about env vars in the traceback before the fix, so I assume this is expected behavior that makes what was happening explicit

$ rse export --type static-web docs/
Server initialized for gevent.
INFO:engineio.server:Server initialized for gevent.
Starting export for http://127.0.0.1:5000
Research Software Encyclopedia: running on http://127.0.0.1:5000
INFO:geventwebsocket.handler:127.0.0.1 - - [2022-08-21 20:02:30] "HEAD / HTTP/1.1" 200 118 0.031579
Missing environment variables! Here is an example:
#!/bin/bash
export RSE_HOST=https://rseng.github.io
export RSE_URL_PREFIX=/software/
export RSE_CONFIG_FILE=rse.ini
rse export --type repos-txt repos.txt --force
rse export --type static-web docs/

vsoch · 2022-08-22T01:15:48Z

@NickleDave some comments!

For this one:

The error for the bad url is better, but still a bit overwhelming. Maybe raise an actual error with a traceback? I got

I'll instead write the response.reason (push coming soon for you to test again!)

And this one:

This is confusing though, because I do have a column "Title" that is copied directly from the example spreadsheet so I know it's formatted the same way

The first row of the sheet seems to be a comment:

Did you add this / not use the template? In the above, the title is in the second row. My example looks like this:

Now when I try to rse export I get an error about missing env variables. I did see something about env vars in the traceback before the fix, so I assume this is expected behavior that makes what was happening explicit

Yes exactly! Instead of moving forward without the error and raising an exception, it shows exactly how the command is typically used.I'm going to add another newline so it's separate from the first part.

Signed-off-by: vsoch <vsoch@users.noreply.github.com>

vsoch · 2022-08-22T01:17:13Z

Here is the example template sheet, if you don't have it handy. https://docs.google.com/spreadsheets/d/1ZW2kOsBOfSpRSH_9Efvz-ytn7dQ2m1DmYDBdIVNGY4c/edit#gid=0

NickleDave · 2022-08-24T02:03:20Z

The first row of the sheet seems to be a comment

The first row is somehow hidden in the copy I made of Tessa's spreadsheet?

I didn't even know you could do that in Google Sheets. This is yet another reason why spreadsheets are the devil

Did you add this / not use the template?

Sorry if it wasn't clear -- I already tested on the template. Works great 👌
I am now trying to build a site from Tessa's actual spreadsheet, so I can submit a PR to that repo adding the database + a site.

Yes exactly! Instead of moving forward without the error and raising an exception

I tested and there's a space here now.

Why not raise an error?
Without an error, it just hangs for ever, admonishing me for not having set up env vars

vsoch · 2022-08-24T02:05:11Z

Huh, I'm not sure why it's hanging - I do a sys.exit, see https://github.com/rseng/rse/pull/75/files#diff-d322afafceee4094f785edb1cd91db3ec9edc923aec9d76af34ec326bc6edfd9R67.

vsoch · 2022-08-24T02:06:35Z

Did you press enter after (and perhaps it did exit and the terminal just needs to catch up?)

NickleDave · 2022-08-24T02:07:31Z

No, I'm pretty sure it's just hanging

NickleDave · 2022-08-24T02:09:03Z

This is what I get when I Ctrl+C to break out of it, in case it helps:

(.venv)  ✘ pimienta@pop-os  ~/Documents/repos/coding/opensci/bioacoustics/bioacoustics-software   add-rse ±
$ rse export --type static-web docs/
Server initialized for gevent.
INFO:engineio.server:Server initialized for gevent.
Starting export for http://127.0.0.1:5000
Research Software Encyclopedia: running on http://127.0.0.1:5000
INFO:geventwebsocket.handler:127.0.0.1 - - [2022-08-23 22:07:18] "HEAD / HTTP/1.1" 200 118 0.012994
Missing environment variables! Here is an example:

export RSE_HOST=https://rseng.github.io
export RSE_URL_PREFIX=/software/
export RSE_CONFIG_FILE=rse.ini
rse export --type repos-txt repos.txt --force
rse export --type static-web docs/

^CException ignored in atexit callback: <function _exit_function at 0x7f9f085d24d0>
Traceback (most recent call last):
KeyboardInterrupt
  File "/home/linuxbrew/.linuxbrew/opt/python@3.10/lib/python3.10/multiprocessing/util.py", line 357, in _exit_function
2022-08-24T02:08:07Z
Process Process-1:
    p.join()
  File "/home/linuxbrew/.linuxbrew/opt/python@3.10/lib/python3.10/multiprocessing/process.py", line 149, in join
    res = self._popen.wait(timeout)
  File "/home/linuxbrew/.linuxbrew/opt/python@3.10/lib/python3.10/multiprocessing/popen_fork.py", line 43, in wait
    return self.poll(os.WNOHANG if timeout == 0.0 else 0)
  File "/home/linuxbrew/.linuxbrew/opt/python@3.10/lib/python3.10/multiprocessing/popen_fork.py", line 27, in poll
    pid, sts = os.waitpid(self.pid, flag)
KeyboardInterrupt: 
Traceback (most recent call last):
  File "/home/linuxbrew/.linuxbrew/opt/python@3.10/lib/python3.10/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/linuxbrew/.linuxbrew/opt/python@3.10/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/pimienta/Documents/repos/coding/opensci/bioacoustics/rse/rse/app/server.py", line 66, in start
    socketio.run(app, port=port, debug=debug, host=host)
  File "/home/pimienta/Documents/repos/coding/opensci/bioacoustics/bioacoustics-software/.venv/lib/python3.10/site-packages/flask_socketio/__init__.py", line 664, in run
    self.wsgi_server.serve_forever()
  File "/home/pimienta/Documents/repos/coding/opensci/bioacoustics/bioacoustics-software/.venv/lib/python3.10/site-packages/gevent/baseserver.py", line 400, in serve_forever
    self._stop_event.wait()
  File "src/gevent/event.py", line 163, in gevent._gevent_cevent.Event.wait
  File "src/gevent/_abstract_linkable.py", line 521, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait
  File "src/gevent/_abstract_linkable.py", line 487, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait_core
  File "src/gevent/_abstract_linkable.py", line 490, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait_core
  File "src/gevent/_abstract_linkable.py", line 442, in gevent._gevent_c_abstract_linkable.AbstractLinkable._AbstractLinkable__wait_to_be_notified
  File "src/gevent/_abstract_linkable.py", line 451, in gevent._gevent_c_abstract_linkable.AbstractLinkable._switch_to_hub
  File "src/gevent/_greenlet_primitives.py", line 61, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
  File "src/gevent/_greenlet_primitives.py", line 65, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
  File "src/gevent/_gevent_c_greenlet_primitives.pxd", line 35, in gevent._gevent_c_greenlet_primitives._greenlet_switch
KeyboardInterrupt

NickleDave · 2022-08-24T02:11:23Z

Aside from that, I can confirm this fixes #71 #73 and #74

vsoch · 2022-08-24T02:54:52Z

Testing again now! Hopefully I can reproduce and fix this up!

vsoch · 2022-08-24T02:58:59Z

okay reproduced! Will push a fix shortly.

Signed-off-by: vsoch <vsoch@users.noreply.github.com>

vsoch · 2022-08-24T03:07:20Z

okay should be fixed! d440401

NickleDave · 2022-08-24T03:13:19Z

Confirmed that this exits as expected now

$ rse export --type static-web docs/
Server initialized for gevent.
INFO:engineio.server:Server initialized for gevent.
Starting export for http://127.0.0.1:5000
Research Software Encyclopedia: running on http://127.0.0.1:5000
INFO:geventwebsocket.handler:127.0.0.1 - - [2022-08-23 23:11:28] "HEAD / HTTP/1.1" 200 118 0.044321
Missing environment variables! Here is an example:

export RSE_HOST=https://rseng.github.io
export RSE_URL_PREFIX=/software/
export RSE_CONFIG_FILE=rse.ini
rse export --type repos-txt repos.txt --force
rse export --type static-web docs/

(.venv)  ✘ pimienta@pop-os

vsoch · 2022-08-24T03:15:22Z

yay! Okay to merge and release then?

vsoch · 2022-08-24T03:15:58Z

is your computer named pimienta? That's super cute! 🌶️

NickleDave · 2022-08-24T03:25:28Z

yay! Okay to merge and release then?

LGTM! 👍

is your computer named pimienta? That's super cute! 🌶️

yes. long story 😛

vsoch · 2022-08-24T03:33:26Z

I hope I can hear it someday! 😆

https://pypi.org/project/rse/0.0.44/

NickleDave · 2022-08-24T03:41:44Z

https://www.storytel.com/sg/en/books/2399500-Cecilia-Valdes-o-La-loma-del-Angel

bug fixes/typos in the docs for jekyll-web export

d6da666

Signed-off-by: vsoch <vsoch@users.noreply.github.com>

vsoch mentioned this pull request Aug 21, 2022

Set up website + document updates automatically rhine3/bioacoustics-software#3

Open

print response reason instead, one more line to export example

4bd3fda

Signed-off-by: vsoch <vsoch@users.noreply.github.com>

NickleDave mentioned this pull request Aug 24, 2022

ERROR:rse.utils.urls:Cannot find endpoint when running rse import --type google-sheets #76

Closed

ensure export exits on fail

d440401

Signed-off-by: vsoch <vsoch@users.noreply.github.com>

vsoch merged commit f020ee9 into master Aug 24, 2022

vsoch deleted the docs/tweaks branch August 24, 2022 03:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bug fixes/typos in the docs for jekyll-web export #75

bug fixes/typos in the docs for jekyll-web export #75

vsoch commented Aug 21, 2022

vsoch commented Aug 21, 2022

NickleDave commented Aug 22, 2022 •

edited

vsoch commented Aug 22, 2022

vsoch commented Aug 22, 2022

NickleDave commented Aug 24, 2022

vsoch commented Aug 24, 2022

vsoch commented Aug 24, 2022

NickleDave commented Aug 24, 2022

NickleDave commented Aug 24, 2022

NickleDave commented Aug 24, 2022

vsoch commented Aug 24, 2022

vsoch commented Aug 24, 2022

vsoch commented Aug 24, 2022

NickleDave commented Aug 24, 2022

vsoch commented Aug 24, 2022

vsoch commented Aug 24, 2022

NickleDave commented Aug 24, 2022

vsoch commented Aug 24, 2022

NickleDave commented Aug 24, 2022

bug fixes/typos in the docs for jekyll-web export #75

bug fixes/typos in the docs for jekyll-web export #75

Conversation

vsoch commented Aug 21, 2022

vsoch commented Aug 21, 2022

NickleDave commented Aug 22, 2022 • edited

vsoch commented Aug 22, 2022

vsoch commented Aug 22, 2022

NickleDave commented Aug 24, 2022

vsoch commented Aug 24, 2022

vsoch commented Aug 24, 2022

NickleDave commented Aug 24, 2022

NickleDave commented Aug 24, 2022

NickleDave commented Aug 24, 2022

vsoch commented Aug 24, 2022

vsoch commented Aug 24, 2022

vsoch commented Aug 24, 2022

NickleDave commented Aug 24, 2022

vsoch commented Aug 24, 2022

vsoch commented Aug 24, 2022

NickleDave commented Aug 24, 2022

vsoch commented Aug 24, 2022

NickleDave commented Aug 24, 2022

NickleDave commented Aug 22, 2022 •

edited