External Plugins v2 #1216

MinchinWeb · 2021-03-17T02:08:43Z

(This is a follow up to #1115 because in that one I pushed directly to develop, plus it's been squashed and rebased, and it proved easier in my head to make a new branch.)

The open "issues" from #1115:

Tests: not included yet. Any ideas on how to change the installed Python packages in a "new" testing environment?
handling of name conflicts: Running jrnl --version will show you the list of active plugins, include where they came from.
display plugin version: Also on jrnl --version
documentation: Done!

Sample jrnl --version output:

λ jrnl --version
jrnl version v2.7.2-beta

Copyright (C) 2012-2021 jrnl contributors

This is free software, and you are welcome to redistribute it under certain
conditions; for details, see: https://www.gnu.org/licenses/gpl-3.0.html

Active Plugins:
    Importers:
        jrnl : v2.7.2-beta from jrnl.plugins.importer.jrnl_importer
    Exporters:
        boxed : v2.7.2-beta from jrnl.plugins.exporter.fancy_exporter
        fancy : v2.7.2-beta from jrnl.plugins.exporter.fancy_exporter
        json : v2.7.2-beta from jrnl.plugins.exporter.json_exporter
        markdown : v2.7.2-beta from jrnl.plugins.exporter.markdown_exporter
        md : v2.7.2-beta from jrnl.plugins.exporter.markdown_exporter
        tags : v2.7.2-beta from jrnl.plugins.exporter.tag_exporter
        text : v2.7.2-beta from jrnl.plugins.exporter.text_exporter
        txt : v2.7.2-beta from jrnl.plugins.exporter.text_exporter
        xml : v2.7.2-beta from jrnl.plugins.exporter.xml_exporter
        yaml : v2.7.2-beta from jrnl.plugins.exporter.yaml_exporter

For documentation, I would love to take insert the sample plugins from external files, and use those same external files as a base for the testing. Is the first part (inserting an external file) possible to MkDocs?

Also, the template exporter got dropped because I couldn't figure out how to make it work with displaying the version numbers... Is that okay?

Check column widths update gitignore throw error when linewrap too small simply check for large enough linewrap value

make exception more informative update check_linewrap signature in src and test make check_linewrap a free function

test for unimplemented method

delete unused imports

MinchinWeb · 2021-05-18T04:01:45Z

OK, tests work and they pass! Hallelujah! I won't claim that it's beautiful, but it works, and I'm happy with that.

@wren

Why do we need to change the directory? I see a comment about plugins not being detected. Do we know why that is?

I think this is a combination of the plugins being namespace plugins and the main jrnl source being locally importable from the project root folder. Namespace plugins work be creating a "virtual" module out of several physical locations on disk; when importing the jrnl module from the project root (rather than from the site-packages folder where it's installed by pip), these namespace modules aren't recognized as namespace modules, and instead come up as empty.

To test this:

# navigate to the project root folder
# create a virtual environment and activate it
python -m venv .venv-test-plugins
source .venv-test-plugins/bin/activate
# on Windows: .\.venv-test-plugins\Scripts\activate
# install jrnl
pip install .
# install the test plugins
pip install ./tests/external_plugins_src/
# install behave
pip install behave
# run behave; note the number of steps completed
behave --format progress2
# 1738 steps passed, 0 failed, 163 skipped, 0 undefined
# now change up a folder and rerun behave
cd ..
behave --format progress2 ./jrnl/features
# 1664 steps passed, 0 failed, 237 skipped, 0 undefined

In any case, I think we're ready to merge! @micahellison

micahellison · 2021-05-22T19:10:30Z

Thanks @MinchinWeb! I see this is ready for review, and I'll plan on diving into this in a week. Also, we won't merge anything big in before that so there won't be any more major merge conflicts to deal with.

# Conflicts: # poetry.lock # pyproject.toml

…ng now

micahellison · 2021-05-29T21:29:21Z

Hi @MinchinWeb, I'm reviewing this while educating myself on namespace packages here and I see that there are three different approaches for namespace packages, but I can't tell which is being used in this PR. Or is there a completely different approach that it's using, and if so, what's the advantages of it?

MinchinWeb · 2021-05-29T21:45:35Z

Hi @micahellison These are "native namespace packages" (the first option mentioned on the page you link to).

Why this way? It seems to be the most "modern" option, and beyond getting the plugin files in the right place, there should be no packaging boilerplate for plugin authors.

micahellison · 2021-05-30T00:08:23Z

Got it, thanks. I agree that it makes sense -- I was just confused that the form of the sample plugin looked a little different from the documentation I linked. Then I learned more about __init__.py than I ever thought there was to know...

Along those lines, I'm chatting about this with @wren and we think we have a solution to this folder issue you documented, by removing the two __init__.py files from jrnl and moving their functionality elsewhere. We should have a patch coming in a bit.

micahellison · 2021-05-30T00:17:53Z

Actually, we're going to merge this into a feature branch so we can work on it a bit more asynchronously.

micahellison

✨🔌✨

MinchinWeb · 2021-06-16T01:38:54Z

@micahellison I'm curious about what more needs to be done/what else you want in place before merging this to master...

wren · 2021-06-19T19:18:44Z

cc: #1006

micahellison · 2021-06-19T19:19:48Z

Hi @MinchinWeb sorry for the slowness on this -- it's been a busy month. I just put my notes in the linked issue.

wren · 2021-06-19T19:26:33Z

@MinchinWeb Quick note: jrnl doesn't have a branch named master anymore. We renamed our main branches a while back to develop and release (where develop is the default branch that all PRs are pointed to) to avoid using language that could be hurtful to some people.

* behavior outline * FIrst pass at allow external plugins * remove template exporter * Add listing of active plugins to '--version' output * Documentation for plugins * [Docs] add custom imports and exporters to site TOC * [Docs] better linewrapping * enforce positive initial linewrap Check column widths update gitignore throw error when linewrap too small simply check for large enough linewrap value * delete unused error message * PR feedback make exception more informative update check_linewrap signature in src and test make check_linewrap a free function * delete unused function * delete else..pass block * newline for make format * Include dates_exporter * Use Base classes for importer and exporters. * [Docs] improve documentation of custom Importers and Exporters * [Testing] separate run with external plugin! * basic behavior test * prototype unittest for JSON Exporter test for unimplemented method * make format delete unused imports * Remove 'importer' or 'exporter' from filenames where not needed * [Test] run different tests with or without the external plugins installed * [Test] move test rot13 plugin into git tree from MinchinWeb/jrnl-rot13-exporter@0dc912a * consolidate demo plugins to common package * [Docs] name page for plugins * [Docs] include the sample plug in code files directly * style fixes * [test] determine whether to run external plug in tests based on installed packages * improved code documentation * style fixes for GitHub actions * Convert "short" and "pretty" (and "default") formaters to plugins further to #1177 * more code clean up tests pass locally...now for GitHub... * [tests] dynamically determine jrnl version for plugin tests * [GitHub Actions] direct install of testing plugins * Remove template code * [plugins] meta --> collector * [Docs] create scripted entries using an custom importer * (closer to) being able to run behave tests outside project root directory * We already know when exporter to use Don't re-calculate it! * [Tests] don't name test plugin 'testing" If so named, pip won't install it. * [Test] run behave tests with test plugins outside project root * [Test] behave tests pass locally * [Docs] fix typo * [GitHub Actions] run test commands from poetry's shell * black-ify code * [GitHub Actions] move downstream (rather than up) to run tests * [GitHub Actions] set shell to poetry * [GitHub Workflows] Manually activate virtual environment * [GitHub Actions] Skip Windows & Python 3.8 Can't seem to find Python exe? * [GiotHub Actions] explicitly use virtual env * [GitHub Actions] create virutal env directly * [GitHub Actions] better activate of Windows virtual env * [GitHub Actions] create virtual env on Mac * [Github Actions] install wheel and upgrade pip * [GitHub Actions] skip virtual environments altogether * [GitHub Actions] change directory for behave test * Remove Windows exclusions from CI as per note -- they should be working now Co-authored-by: Suhas <sugas182@gmail.com> Co-authored-by: Micah Jerome Ellison <micah.jerome.ellison@gmail.com>

c.f. jrnl-org/jrnl#1216 c.f. jrnl-org/jrnl#1282

c.f. jrnl-org/jrnl@e64ca2d This should fully merge jrnl-org/jrnl#1216

sriniv27 and others added 7 commits March 4, 2021 17:28

behavior outline

d3589fa

FIrst pass at allow external plugins

e25869e

remove template exporter

3795159

Add listing of active plugins to '--version' output

a550861

Documentation for plugins

a72115d

[Docs] add custom imports and exporters to site TOC

46c28d7

[Docs] better linewrapping

4eb0ab0

MinchinWeb mentioned this pull request Mar 17, 2021

[WIP] External Plugins #1115

Closed

micahellison force-pushed the develop branch 2 times, most recently from 9eb9990 to f50b430 Compare March 20, 2021 19:25

wren force-pushed the develop branch 5 times, most recently from 234e1f3 to de17222 Compare March 20, 2021 21:13

sriniv27 added 2 commits March 21, 2021 18:05

enforce positive initial linewrap

2d4ba1b

Check column widths update gitignore throw error when linewrap too small simply check for large enough linewrap value

delete unused error message

e063fa9

wren force-pushed the develop branch from ea37d4b to 50c053c Compare March 27, 2021 19:34

sriniv27 added 4 commits April 10, 2021 10:00

PR feedback

502d47f

make exception more informative update check_linewrap signature in src and test make check_linewrap a free function

delete unused function

2e09d88

delete else..pass block

8bf1da1

newline for make format

2b054d7

MinchinWeb mentioned this pull request May 1, 2021

External formatter plugins #1239

Open

MinchinWeb and others added 7 commits May 5, 2021 20:42

Include dates_exporter

51fc091

Use Base classes for importer and exporters.

2706314

[Docs] improve documentation of custom Importers and Exporters

a75f153

[Testing] separate run with external plugin!

cc682a3

basic behavior test

6c6929b

prototype unittest for JSON Exporter

c76b978

test for unimplemented method

make format

788b32b

delete unused imports

[GitHub Actions] create virtual env on Mac

86177be

MinchinWeb force-pushed the external-plugins branch from 6c91245 to 86177be Compare May 18, 2021 03:14

MinchinWeb added 3 commits May 17, 2021 21:25

[Github Actions] install wheel and upgrade pip

8afa3c0

[GitHub Actions] skip virtual environments altogether

ffe4a5f

[GitHub Actions] change directory for behave test

ab3b235

MinchinWeb force-pushed the external-plugins branch from 1595d5e to ab3b235 Compare May 18, 2021 03:49

MinchinWeb changed the title ~~[WIP] External Plugins v2~~ External Plugins v2 May 18, 2021

MinchinWeb mentioned this pull request May 18, 2021

Multiple entries with same timestamp should be grouped together #1247

Closed

Merge branch 'develop' into external-plugins

e061e78

# Conflicts: # poetry.lock # pyproject.toml

micahellison mentioned this pull request May 29, 2021

Remove asteval or set it as a dev dependency #1251

Open

Remove Windows exclusions from CI as per note -- they should be worki…

9f16382

…ng now

micahellison changed the base branch from develop to feature-plugins May 30, 2021 00:19

micahellison approved these changes May 30, 2021

View reviewed changes

micahellison merged commit 1a7e8d8 into jrnl-org:feature-plugins May 30, 2021

MinchinWeb deleted the external-plugins branch June 16, 2021 01:39

MinchinWeb mentioned this pull request Jul 9, 2021

display_format: pretty and display_format: short lead to crash #1263

Closed

MinchinWeb added a commit to MinchinWeb/minchin.jrnl that referenced this pull request Sep 22, 2023

Add listing of plugins on --diagnostic

24169e2

c.f. jrnl-org/jrnl#1216 c.f. jrnl-org/jrnl#1282

MinchinWeb added a commit to MinchinWeb/minchin.jrnl that referenced this pull request Sep 22, 2023

[test] improvements

25bcde0

c.f. jrnl-org/jrnl@e64ca2d This should fully merge jrnl-org/jrnl#1216

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

External Plugins v2 #1216

External Plugins v2 #1216

MinchinWeb commented Mar 17, 2021 •

edited

MinchinWeb commented May 18, 2021

micahellison commented May 22, 2021

micahellison commented May 29, 2021

MinchinWeb commented May 29, 2021

micahellison commented May 30, 2021

micahellison commented May 30, 2021

micahellison left a comment

MinchinWeb commented Jun 16, 2021

wren commented Jun 19, 2021

micahellison commented Jun 19, 2021

wren commented Jun 19, 2021

External Plugins v2 #1216

External Plugins v2 #1216

Conversation

MinchinWeb commented Mar 17, 2021 • edited

MinchinWeb commented May 18, 2021

micahellison commented May 22, 2021

micahellison commented May 29, 2021

MinchinWeb commented May 29, 2021

micahellison commented May 30, 2021

micahellison commented May 30, 2021

micahellison left a comment

Choose a reason for hiding this comment

MinchinWeb commented Jun 16, 2021

wren commented Jun 19, 2021

micahellison commented Jun 19, 2021

wren commented Jun 19, 2021

MinchinWeb commented Mar 17, 2021 •

edited