Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow plugins to provide metadata #1603

Merged
merged 1 commit into from Feb 2, 2015

Conversation

Projects
None yet
2 participants
@Carreau
Copy link
Contributor

commented Jan 28, 2015

This allow plugins to provide a way to extract metadata from file the way
they wish to.

Metadata will be search first from the default value of as before this
PR. Metadata extracted by the plugin will then be merged into the
metadata dict, which in turn will be extended by the value in the
.meta file if present.


See start of discussion from #1602

I tried to get the api to extract metadata by way of plugins, as close as possible as other get_meta calls.
I also implement this for 1 plugin (.ipynb) the one I'm interested in.

This should not change current behavior for reading metadata from other places.

@@ -136,6 +136,9 @@ def __init__(
self._dependency_uptodate_page = defaultdict(list)

default_metadata, self.newstylemeta = get_meta(self, self.config['FILE_METADATA_REGEXP'], self.config['UNSLUGIFY_TITLES'])
compiler_meta = self.compiler.read_metadata(self, self.config['FILE_METADATA_REGEXP'], self.config['UNSLUGIFY_TITLES'])

This comment has been minimized.

Copy link
@Kwpolska

Kwpolska Jan 28, 2015

Member

IMO, you should integrate this into get_meta instead.

@@ -245,6 +245,12 @@ def extension(self):
"""The preferred extension for the output of this compiler."""
return ".html"

def read_metadata(self, post, file_metadata_regexp=None, unslugify_titles=False, lang=None):
"""
Read the metadata from a post, and retur a metadata dict

This comment has been minimized.

Copy link
@Kwpolska

Kwpolska Jan 28, 2015

Member

return*

also, you should actually return an empty dict below

@Carreau Carreau force-pushed the Carreau:plugin-meta branch from d01227d to e22f644 Feb 1, 2015

@Carreau

This comment has been minimized.

Copy link
Contributor Author

commented Feb 1, 2015

Sorry for the time to respond, comment should be addressed.
moving things to get_meta seem a bit weird, also due to the cause that get_meta get passed self as first arg, but is not a method.

@Kwpolska

This comment has been minimized.

Copy link
Member

commented Feb 1, 2015

So what? read_metadata also gets the Post object, and also as self. read_metadata should not be put outside of the standard get_meta pipeline.

@Carreau Carreau force-pushed the Carreau:plugin-meta branch 2 times, most recently from 738080d to 6cd634c Feb 1, 2015

@Carreau

This comment has been minimized.

Copy link
Contributor Author

commented Feb 1, 2015

So what? read_metadata also gets the Post object, and also as self

No, i just ment that passing self as first argument just felt unnatural to me, I was thus thinking that from inside get_metadata I couldn't get access to self.compiler, but in fact I can, as it is name post.compiler from inside the function. Just me not used to the codebase, sorry if I express myself badly.

I've also forgot to push the cleanup commit that remove my debugs, and comments.
Tell me if you want me to squash everything into one commit.

@@ -928,6 +928,8 @@ def get_meta(post, file_metadata_regexp=None, unslugify_titles=False, lang=None)
meta.update(_)

if meta:
compiler_meta = post.compiler.read_metadata(post, file_metadata_regexp, unslugify_titles, lang)

This comment has been minimized.

Copy link
@Kwpolska

Kwpolska Feb 1, 2015

Member

why is this still there?

This comment has been minimized.

Copy link
@Carreau

Carreau Feb 1, 2015

Author Contributor

Still ? You asked me to move it in it's in get_meta which I did. Though get_metareturn in two places, so if the post.compiler.read_metadata want to have an effect in all casses, I need to call it before each of the returns. Which explain why the call still appear twice in the diff.

@Kwpolska Kwpolska added enhancement and removed enhancement labels Feb 1, 2015

@Kwpolska Kwpolska added this to the v7.3.1 milestone Feb 1, 2015

@Kwpolska Kwpolska self-assigned this Feb 1, 2015

@Kwpolska

This comment has been minimized.

Copy link
Member

commented Feb 1, 2015

  1. Failing tests: https://travis-ci.org/getnikola/nikola/jobs/49094459#L871 → dummy posts used for tests do not have a compiler attribute
  2. Failing flake8: https://travis-ci.org/getnikola/nikola/jobs/49094463#L136
  3. The part under if meta: should not be there; you should parse compiler meta only once in the code (don’t do it if an explicit .meta file exists? Refactor .meta loading?)
  4. Add this to CHANGES.txt (at the very top, under Features) and yourself to AUTHORS.txt.
@Carreau

This comment has been minimized.

Copy link
Contributor Author

commented Feb 1, 2015

Failing tests: https://travis-ci.org/getnikola/nikola/jobs/49094459#L871 → dummy posts used for tests do not have a compiler attribute
Failing flake8: https://travis-ci.org/getnikola/nikola/jobs/49094463#L136
The part under if meta: should not be there; you should parse compiler meta only once in the code (don’t do it if an explicit .meta file exists? Refactor .meta loading?)
Add this to CHANGES.txt (at the very top, under Features) and yourself to AUTHORS.txt.

Will do.

@Kwpolska

This comment has been minimized.

Copy link
Member

commented Feb 1, 2015

I recommend replacing the if meta: block with

if not meta:
    post.is_two_file = False

and then putting the compiler_meta code only in one place. It should also appear above the if lang is None: block. This sounds the most sensible to me.

@Carreau Carreau force-pushed the Carreau:plugin-meta branch from cf58091 to 5ebb99a Feb 1, 2015

Allow plugins to provide metadata.
This allow plugins to provide a way to extract metadata from file the way
they wish to.

Metadata will be search first from the default value of as before this
PR. Metadata extracted by the plugin will then be merged into the
metadata dict, which in turn will be extended by the value in the
`.meta` file if present.

@Carreau Carreau force-pushed the Carreau:plugin-meta branch from 5ebb99a to 90a005a Feb 1, 2015

@Carreau

This comment has been minimized.

Copy link
Contributor Author

commented Feb 1, 2015

Squashed, comment taken into account.
Does that looks better to you ?

@@ -374,6 +374,10 @@ They must provide:
If the compiler produces something other than HTML files, it should also implement ``extension`` which
returns the preferred extension for the output file.

These plugin can also be used to extract metadata from file. To do so, the

This comment has been minimized.

Copy link
@Kwpolska

Kwpolska Feb 2, 2015

Member

These plugins*

a dict containing the metadata*

source = post.source_path
with io.open(source, "r", encoding="utf8") as in_file:
nb_json = nbformat.read(in_file, current_nbformat)
# metadata shoudl always exist, but we never know

This comment has been minimized.

Copy link
@Kwpolska

Kwpolska Feb 2, 2015

Member
# metadata should always exist, but we never know
# if someone crafted the ipynb by hand.

Kwpolska added a commit that referenced this pull request Feb 2, 2015

Merge pull request #1603 from Carreau/plugin-meta
Allow plugins to provide metadata

@Kwpolska Kwpolska merged commit c87ffc6 into getnikola:master Feb 2, 2015

2 checks passed

continuous-integration/appveyor AppVeyor build succeeded
Details
continuous-integration/travis-ci The Travis CI build passed
Details
@Kwpolska

This comment has been minimized.

Copy link
Member

commented Feb 2, 2015

Code looks good, only needs some typo fixes in the documentation. Will fix myself.

Thanks for contributing!

@Kwpolska

This comment has been minimized.

Copy link
Member

commented Feb 2, 2015

(PS. I forgot to tell you squashing commits is unnecessary; we don’t really care.)

@Carreau Carreau deleted the Carreau:plugin-meta branch Feb 2, 2015

@Carreau

This comment has been minimized.

Copy link
Contributor Author

commented Feb 2, 2015

Thanks for contributing!

Thanks for maintaining Nikola and guiding me through the codebase !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.