Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WordPress compiler plugin. #91

Merged
merged 20 commits into from Jul 5, 2015
Merged

WordPress compiler plugin. #91

merged 20 commits into from Jul 5, 2015

Conversation

@felixfontein
Copy link
Contributor

@felixfontein felixfontein commented Jun 27, 2015

This plugin allows to compile WordPress posts (i.e. posts directly copied out of a WordPress blog, without any modifications).

It has support for shortcodes. By default, it only supports the [code] shortcode, but plugins for other shortcodes can be added via a plugin interface. See wordpress/plugins/code.py for an example.

Review on Reviewable

__all__ = ['CompileWordpress', 'WordPressPlugin']


class WordPressPlugin(IPlugin):

This comment has been minimized.

This comment has been minimized.

@felixfontein

felixfontein Jul 2, 2015
Author Contributor

This is a class which shortcode plugins for the WordPress compiler should derive from. This is not the class for the WordPress page compiler itself. The WordPress page compiler is CompileWordpress and defined in wordpress.py.

This comment has been minimized.

@ralsina

ralsina Jul 2, 2015
Member

In any case, inherit at least nikola.plugin_categories.BasePlugin or you will be missing stuff.

This comment has been minimized.

@felixfontein

felixfontein Jul 2, 2015
Author Contributor

Why? The plugin handling for .wpplugin plugins (which inherit from WordPressPlugin) is done completely by the WordPress page compiler plugin. Nikola won't see these plugins.

This comment has been minimized.

@ralsina

ralsina Jul 2, 2015
Member

Oh, it will. Nikola's plugin loader will find them, unless you put them in a different place? Then they will be loaded twice.
Also, there's no reason for the compiler to load plugins, just let Nikola load them and get them from self.site.plugin_manager. Saves you a chunk of code, too.

This comment has been minimized.

@felixfontein

felixfontein Jul 2, 2015
Author Contributor

How about a "plugin plugin" category? The base class for that category would have a field for the plugin name this plugin is for, and the Nikola main object would get a function which allows to retrieve all plugins for a given plugin.
I can add something like that to Nikola if you want. The problem is that this won't be part of Nikola anytime soon (7.6.0 was just released), and so the WordPress plugin won't be useable until a newer Nikola version is released -- and in particular, it won't be useable for almost all v7 versions.

This comment has been minimized.

@felixfontein

felixfontein Jul 2, 2015
Author Contributor

Re registering plugin categories from within plugins: that would be really cumbersome, since we have to specify the plugin categories before any plugin is loaded. Except of course if we load plugins twice, but I don't really like that idea.

This comment has been minimized.

@Kwpolska

Kwpolska Jul 3, 2015
Member

class CompilerExtension(BasePlugin):
    compiler_name = ''

class RestExtension(CompilerExtension):
    compiler_name = 'rest'

class MarkdownExtension(CompilerExtension)
    compiler_name = 'markdown'

Your plugins would be CompilerExtensions with compiler_name = 'wordpress', and would filter for that.

We could make your plugin v8-only (and we really need to get our act together and land that release) to make compatibility easier.

This comment has been minimized.

@felixfontein

felixfontein Jul 3, 2015
Author Contributor

Yes, that's what I was thinking. Also very good idea to include RestExtension and MarkdownExtension into this scheme!

Apropos v8: is there a timeline when v8 should be done? For the earlytask branch, the main problem is that it is waiting for some features to appear in official doit before it an be merged.

This comment has been minimized.

@Kwpolska

Kwpolska Jul 4, 2015
Member

Well, it will take a while for v8 to become a thing, because there isn’t that much to change yet. However, I just implemented CompilerExtensions in getnikola/nikola@c65be6e. You could make your compiler v7.6.1+ and use them.

# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,

This comment has been minimized.

@ralsina

ralsina Jul 2, 2015
Member

It seems to me that this plugin has multiple licenses. It's easier, if it has a GPLv2 part, to just make it all GPL.

This comment has been minimized.

@felixfontein

felixfontein Jul 2, 2015
Author Contributor

I took the original copyright notice found in WordPress source files, and I took their license.txt (and added a part for this plugin).

This comment has been minimized.

@ralsina

ralsina Jul 2, 2015
Member

__init__.py has a MIT-style license, it's easier to just have one.

This comment has been minimized.

@ralsina

ralsina Jul 2, 2015
Member

Ah, my bad, it's also GPL it's just worded differently.

This comment has been minimized.

@felixfontein

felixfontein Jul 2, 2015
Author Contributor

Are you sure? The only __init__.py I can see in this PR has the same GPL boilerplate as all other files.

@ralsina
Copy link
Member

@ralsina ralsina commented Jul 2, 2015

Review status: 0 of 10 files reviewed at latest revision, 5 unresolved discussions, all commit checks successful.


v7/wordpress_compiler/README.md, line 12 [r1] (raw file):
If you put this in conf.py.sample it will be automatically shown here, and also shown as a suggestion to the user when he installs the plugin. For example here's what happens when you install markmin:

$ nikola plugin -i markmin
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): plugins.getnikola.com
[2015-07-02T13:49:30Z] INFO: plugin: Downloading: https://plugins.getnikola.com/v7/markmin.zip
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): plugins.getnikola.com
[2015-07-02T13:49:31Z] INFO: plugin: Extracting: markmin into plugins/
[2015-07-02T13:49:31Z] NOTICE: plugin: This plugin has a sample config file.  Integrate it with yours in order to make this plugin work!
Contents of the conf.py.sample file:

    # Add the markmin compiler to your COMPILERS dict.
    COMPILERS["markmin"] = ('.mm',)

    # Add markmin files to your POSTS, PAGES
    POSTS = POSTS + (("posts/*.mm", "posts", "post.tmpl"),)
    PAGES = PAGES + (("stories/*.mm", "posts", "post.tmpl"),)


v7/wordpress_compiler/wordpress/plugins/code.py, line 3 [r1] (raw file):
Since you wrote it, you shoud use your name in it :-)


v7/wordpress_compiler/wordpress/plugins/code.wpplugin, line 2 [r1] (raw file):
this name is waaaaay too generic and bound to conflict with something.


Comments from the review on Reviewable.io

@felixfontein
Copy link
Contributor Author

@felixfontein felixfontein commented Jul 2, 2015

Since you wrote it, you shoud use your name in it :-)

Well, I patched it together, but I took some parts from other people -- like you ;-)

this name is waaaaay too generic and bound to conflict with something.

Isn't this always local to the .wpplugin file?

@ralsina
Copy link
Member

@ralsina ralsina commented Jul 2, 2015

I try to keep the names consistent, same name in the metadata plugin file, the plugin itself, task names, etc. You never know what someone is going to try in the future :-)

@ralsina
Copy link
Member

@ralsina ralsina commented Jul 3, 2015

Ok, let's merge this whenever you want, and let's revisit it in a few months.

@felixfontein
Copy link
Contributor Author

@felixfontein felixfontein commented Jul 3, 2015

I'll add an remark for potential shortcode plugin authors that the plugin interface will be changed for v8, so they should be aware that their plugins have to be changed a bit.
I'll probably manage that by tomorrow.

@felixfontein
Copy link
Contributor Author

@felixfontein felixfontein commented Jul 5, 2015

The new version now uses the CompilerExtension interface of Nikola 7.6.1.

@felixfontein
Copy link
Contributor Author

@felixfontein felixfontein commented Jul 5, 2015

I guess the tests will fail until Nikola 7.6.1 is released.

from .wordpress import CompileWordpress


__all__ = ['CompileWordpress', 'WordPressPlugin']

This comment has been minimized.

@Kwpolska

Kwpolska Jul 5, 2015
Member

  • should be a tuple, not a list
  • where is WordPressPlugin?

This comment has been minimized.

@felixfontein

felixfontein Jul 5, 2015
Author Contributor

Thanks! I moved WordPressPlugin to plugin_interface, otherwise yapsy would (at least for me) load that definition as the plugin and ignore the page compiler...

class CompileWordpress(PageCompiler):
"""Compiles a subset of Wordpress into HTML."""

name = "latex"

This comment has been minimized.

@Kwpolska

Kwpolska Jul 5, 2015
Member

Are you sure?

This comment has been minimized.

@felixfontein

felixfontein Jul 5, 2015
Author Contributor

No :) Thanks for noticing!

_LOGGER = get_logger('compile_wordpress', STDERR_HANDLER)


class Context:

This comment has been minimized.

@Kwpolska

Kwpolska Jul 5, 2015
Member

class Context(object):, use new-style classes

This comment has been minimized.

@felixfontein

felixfontein Jul 5, 2015
Author Contributor

Fixed.

data = (context.get_file_dependencies_fragment(), context.get_file_dependencies_page(),
context.get_uptodate_dependencies_fragment(), context.get_uptodate_dependencies_page())
with io.open(deps_path, "wb") as file:
pickle.dump(data, file, pickle.HIGHEST_PROTOCOL)

This comment has been minimized.

@Kwpolska

Kwpolska Jul 5, 2015
Member

Please use JSON or some sane serialization format. Pickle is arbitrary code execution.

This comment has been minimized.

@felixfontein

felixfontein Jul 5, 2015
Author Contributor

Done.

@Kwpolska
Copy link
Member

@Kwpolska Kwpolska commented Jul 5, 2015

  • Replace by the contributors with , Felix Fontein.
  • Add a # -*- coding: utf-8 -*- declaration.
  • To fix the tests, you must duplicate even more code, the plugin test suite uses its own base.py file.
  • Get rid of pickles
  • Get rid of UnformattedLexer

@felixfontein
Copy link
Contributor Author

@felixfontein felixfontein commented Jul 5, 2015

Ok, there's a big problem with the new CompilerExtensions interface. How do I now import something from the original WordPress plugin? When it was using its own plugin system, I would just write import wordpress.xxx and everything would work fine. Now, I can't anymore.

import pygments.token


class UnformattedLexer(pygments.lexer.RegexLexer):

This comment has been minimized.

@Kwpolska

Kwpolska Jul 5, 2015
Member

Do you really need that? There is a text lexer available.

This comment has been minimized.

@felixfontein

felixfontein Jul 5, 2015
Author Contributor

Must have missed that one.

This comment has been minimized.

@felixfontein

felixfontein Jul 5, 2015
Author Contributor

Done.

@Kwpolska
Copy link
Member

@Kwpolska Kwpolska commented Jul 5, 2015

You could cheat and use import plugins.wordpress.plugin_interface, though this will make user-wide plugin installs impossible.

@felixfontein
Copy link
Contributor Author

@felixfontein felixfontein commented Jul 5, 2015

Doesn't work for me, either, since I'm including the wordpress plugin via EXTRA_PLUGINS_DIRS.
I'll Maybe I can expose the modules dynamically in the register method. Will be somewhat ugly, but should work. I'll play with that later...

@felixfontein
Copy link
Contributor Author

@felixfontein felixfontein commented Jul 5, 2015

Hmm, that won't work either, since the first thing I need to do is import wordpress.plugin_interface to be able to use wordpress.plugin_interface.WordPressPlugin. That sucks bigtime.

@Kwpolska
Copy link
Member

@Kwpolska Kwpolska commented Jul 5, 2015

Hack idea 2: use the plain CompilerExtension class and set compiler_name = 'wordpress'.

@felixfontein
Copy link
Contributor Author

@felixfontein felixfontein commented Jul 5, 2015

And define register() myself, and never be able to define any helper functionality in the plugin object itself. Very, very hacky. But probably the only thing that works... :(

@Kwpolska
Copy link
Member

@Kwpolska Kwpolska commented Jul 5, 2015

You could write some functions (≠methods) and pass them to every extension in your compiler, when you register them.

@felixfontein
Copy link
Contributor Author

@felixfontein felixfontein commented Jul 5, 2015

I'm now providing the modules. I'll do some more testing by porting my other plugins to this way...

@felixfontein
Copy link
Contributor Author

@felixfontein felixfontein commented Jul 5, 2015

Seems to work this way.

@Kwpolska
Copy link
Member

@Kwpolska Kwpolska commented Jul 5, 2015

Please complete the checklist a few posts above.

@felixfontein
Copy link
Contributor Author

@felixfontein felixfontein commented Jul 5, 2015

I now also adjusted the copyright lines (though I didn't do precisely the replacement you wanted to do). Is that ok?

@felixfontein
Copy link
Contributor Author

@felixfontein felixfontein commented Jul 5, 2015

Yeah, just noticed that too :)

Kwpolska added a commit that referenced this pull request Jul 5, 2015
@Kwpolska Kwpolska merged commit ea80c63 into getnikola:master Jul 5, 2015
1 check passed
1 check passed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@felixfontein
Copy link
Contributor Author

@felixfontein felixfontein commented Jul 5, 2015

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants