UPGRADE: myst-parser 1.0 #479
Conversation
aleivag
force-pushed
the
upgrade-to-1
branch
2 times, most recently
from
83f264b
to
2f77516
Compare
|
I've tried this. It forced to downgrade and I got this error |
|
hi, what did you did to get this error (how do i reproduce)?
Don't know, nothing in the PR sets the version of sphinx, so i guess is within the allowed versions , if you give me the command you used i'm happy to check |
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## master #479 +/- ##
==========================================
- Coverage 81.47% 80.84% -0.63%
==========================================
Files 29 30 +1
Lines 2618 2663 +45
==========================================
+ Hits 2133 2153 +20
- Misses 485 510 +25
Flags with carried forward coverage won't be shown. Click here to find out more.
☔ View full report in Codecov by Sentry. |
|
@aleivag thanks for this contribution! We'll get around to reviewing it soon, and keep you updated :) |
|
I took a look at the CI jobs and noticed a few errors: mypy errors in our pre-commit jobApparently MyST-parser introduced a new way to construct Here's the error: Regression test breakagesThis one is probably simple to fix, there are some regression test changes detected: For example: - <footnote_reference auto="1" ids="footnote-reference-1" refid="a">
+ <footnote_reference auto="1" ids="id1" refid="a">
1
- <footnote_reference auto="1" ids="footnote-reference-2" refid="b">
+ <footnote_reference auto="1" ids="id2" refid="b">
2To fix these you generally just need to delete the corresponding regression test file, and then re-run the tests. |
|
I've been trying to use that PR on top of 0.17.2 but looks like PR needs to be rediffed + /usr/bin/gzip -dc /home/tkloczko/rpmbuild/SOURCES/python-myst-nb-0.17.2.tar.gz
+ /usr/bin/tar -xof -
+ STATUS=0
+ '[' 0 -ne 0 ']'
+ cd MyST-NB-0.17.2
+ /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w .
+ /usr/bin/cat /home/tkloczko/rpmbuild/SOURCES/python-myst-nb-UPGRADE-myst-parser-1.0.patch
+ /usr/bin/patch -p1 -s --fuzz=0 --no-backup-if-mismatch -f
1 out of 2 hunks FAILED -- saving rejects to file myst_nb/core/read.py.rej
1 out of 6 hunks FAILED -- saving rejects to file myst_nb/core/render.py.rej
1 out of 4 hunks FAILED -- saving rejects to file myst_nb/docutils_.py.rej
2 out of 3 hunks FAILED -- saving rejects to file myst_nb/sphinx_.py.rej
2 out of 2 hunks FAILED -- saving rejects to file pyproject.toml.rej |
|
I need to take a closer look at how |
|
@chrisjsewell I have tried to mirror the new Would be interested in thoughts on how to resolve this if you've got time. Thanks. |
|
In short term, would it be acceptable to just suppress these warnings with |
|
I'm personally fine with that - it feels like the pain people are feeling with a version capped and outdated MyST parser is stronger than the pain they might feel if some problem occurs because of this type mismatch. I'd suggest we just try to get these changes in quickly and cut a release. Then keep an eye for new issues that might be related to this and provide guidance (inc telling people to version cap) if so. |
|
Thanks for making that effort @Yoshanuikabundi - in that case I'd recommend the following course of action:
@Yoshanuikabundi what were the behavior changes that you noticed? |
Oh there were a few warnings aleivag missed that I had to convert over to the new system, and I had to move some control flow around to handle the new warnings. I just meant that I had to do more than just tweak type annotations. So behaviour changes relative to this PR, not relative to the main branch. |
|
(FYI I'm not currently maintaining this repo: executablebooks/meta#961) |
|
OK I've cherry-picked and pushed the two commits from @Yoshanuikabundi's PR to this branch. Let's see how that impacts tests and such. EDIT: @Yoshanuikabundi it looks like there are a few errors in there: If you can get |
|
@choldgraf I was initially confused about this error because I don't see how anything I did could cause it (and also I don't understand it at all), so I checked out the commit on this branch before I did any work (a150582) and I get a very similar error locally there: So I think this might be from a change in a dependency? I'm looking into it but I'm in unfamiliar territory. EDIT: Looks like it's both; I've created an environment with all the dependencies from the last time tests passed on this PR and that environment passes tests on a150582 but not 8f7f59e. Conda environment from the last time tests passedchannels:
- conda-forge
dependencies:
- python=3.8
- pip
- pip:
- Jinja2==3.1.2
- MarkupSafe==2.1.2
- Pygments==2.15.1
- alabaster==0.7.13
- babel==2.12.1
- certifi==2023.5.7
- charset-normalizer==3.1.0
- docutils==0.19
- idna==3.4
- imagesize==1.4.1
- importlib-metadata==6.6.0
- packaging==23.1
- pytz==2023.3
- requests==2.30.0
- snowballstemmer==2.2.0
- sphinx==5.3.0
- sphinxcontrib-applehelp==1.0.4
- sphinxcontrib-devhelp==1.0.2
- sphinxcontrib-htmlhelp==2.0.1
- sphinxcontrib-jsmath==1.0.1
- sphinxcontrib-qthelp==1.0.3
- sphinxcontrib-serializinghtml==1.1.5
- urllib3==2.0.2
- zipp==3.15.0
- GitPython==3.1.31
- anyio==3.6.2
- argon2-cffi==21.3.0
- argon2-cffi-bindings==21.2.0
- arrow==1.2.3
- asttokens==2.2.1
- attrs==23.1.0
- backcall==0.2.0
- beautifulsoup4==4.12.2
- bleach==6.0.0
- cffi==1.15.1
- click==8.1.3
- colorama==0.4.6
- coverage==7.2.5
- cycler==0.11.0
- decorator==5.1.1
- defusedxml==0.7.1
- exceptiongroup==1.1.1
- executing==1.2.0
- fastjsonschema==2.16.3
- fonttools==4.39.4
- fqdn==1.5.1
- gitdb==4.0.10
- greenlet==2.0.2
- importlib-resources==5.12.0
- iniconfig==2.0.0
- ipykernel==5.5.6
- ipython==8.4.0
- ipython-genutils==0.2.0
- ipywidgets==8.0.6
- isoduration==20.11.0
- jedi==0.18.2
- jsonpointer==2.3
- jsonschema==4.17.3
- jupyter-cache==0.6.1
- jupyter-client==8.2.0
- jupyter-core==5.3.0
- jupyter-events==0.6.3
- jupyter-server==2.5.0
- jupyter-server-mathjax==0.2.6
- jupyter-server-terminals==0.4.4
- jupyterlab-pygments==0.2.2
- jupyterlab-widgets==3.0.7
- jupytext==1.14.5
- kiwisolver==1.4.4
- markdown-it-py==2.2.0
- matplotlib==3.5.3
- matplotlib-inline==0.1.6
- mdit-py-plugins==0.3.5
- mdurl==0.1.2
- mistune==2.0.5
- mpmath==1.3.0
- myst-parser==1.0.0
- nbclient==0.7.4
- nbconvert==7.4.0
- nbdime==3.2.1
- nbformat==5.8.0
- numpy==1.24.3
- pandas==2.0.1
- pandocfilters==1.5.0
- parso==0.8.3
- pexpect==4.8.0
- pickleshare==0.7.5
- pillow==9.5.0
- pkgutil-resolve-name==1.3.10
- platformdirs==3.5.1
- pluggy==1.0.0
- prometheus-client==0.16.0
- prompt-toolkit==3.0.38
- ptyprocess==0.7.0
- pure-eval==0.2.2
- pycparser==2.21
- pyparsing==3.0.9
- pyrsistent==0.19.3
- pytest==7.3.1
- pytest-cov==4.0.0
- pytest-datadir==1.4.1
- pytest-param-files==0.3.5
- pytest-regressions==2.4.2
- python-dateutil==2.8.2
- python-json-logger==2.0.7
- pyyaml==6.0
- pyzmq==25.0.2
- rfc3339-validator==0.1.4
- rfc3986-validator==0.1.1
- send2trash==1.8.2
- six==1.16.0
- smmap==5.0.0
- sniffio==1.3.0
- soupsieve==2.4.1
- sqlalchemy==2.0.13
- stack-data==0.6.2
- sympy==1.12
- tabulate==0.9.0
- terminado==0.17.1
- tinycss2==1.2.1
- toml==0.10.2
- tomli==2.0.1
- tornado==6.3.2
- traitlets==5.9.0
- typing-extensions==4.5.0
- tzdata==2023.3
- uri-template==1.2.0
- wcwidth==0.2.6
- webcolors==1.13
- webencodings==0.5.1
- websocket-client==1.5.1
- widgetsnbextension==4.0.7
- -e .
```
|
|
I suggest we roll back or pin as many non-myst-parser versions as we can until we get the tests passing, and then merge and iterate selectively from there. This seems to be where the field that generates this error is coming from: MyST-NB/myst_nb/core/config.py Lines 138 to 146 in 5d97244 cc also @agoose77 in case any of this is familiar? I think he's looked through the myst-nb codebase before. |
* docs: fix use of sphinx-design * Update pyproject.toml
* Add ipywidgets javascript * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Angus Hollands <goosey15@gmail.com>
* fix: use jsdelivr CDN * chore: bump ipywidgets version to 8.0 * docs: add note about ipywidgets
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ebooks#504) Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…#505) Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ablebooks#509) Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
Co-authored-by: Josh Mitchell <yoshanuikabundi@gmail.com>
|
To be clear, the tests are failing because we broke something by moving to stringized annotations. I've reverted that part of the PR, which should make the tests pass 🤞 |
|
For sure that sounds like a functionality issue that needs to be addressed. I was thinking about having the tests be a bit more separated so it is easier to read. At some point the tests did not start to run because it detected bad typing errors, which does not tell us if the tests would actually fail for the user or not. Having those checks are great to have and address, but have them a bit separate so its easier to judge which one to prioritize and which ones can/should be resolved by dropping older version support |
|
Amazing! Thanks for this iteration folks. It looks like the only "failing" thing is the documentation having a few broken cross-reference links because the myst docs have changed. I would be fine merging this as-is in order to get it in. But if anyone wants to quickly fix those I'm happy to review and merge. |
|
I've pushed fixes for the Intersphinx links that are breaking RTD to my branch: https://github.com/Yoshanuikabundi/MyST-NB/commits/upgrade-to-1 The docs build locally for me without warnings and I've spot checked a few of the links to check they still go to the same places. Was just a matter of updating to the new syntax for specifying Intersphinx links and fixing some ambiguities. Thanks for taking a look @agoose77, I don't know that I ever would have figured that out! |
There was a problem hiding this comment.
OK I've pushed the latest patch from @Yoshanuikabundi and now all of the tests are happy - the codecov isn't but I think that's OK because this isn't adding functionality, just refactoring and updating because of upstream changes.
Thanks so much everybody for helping out here!
|
Congrats on your first merged pull request in this project! 🎉 |
This updates the myst-parser dependency to ~1.0
The changes are simple:
I know very little about how this uses myst-parser all i know is that my jupyter-book compiles and uses latest features of myst parser using this. the doc generator seems broken on master , so i cant really tell if this is me or just broken on base rev, please advice !