Basic working version #5

xaker00 · 2022-02-03T19:46:39Z

Notion Blocks supported:

Bookmarks
Bulleted lists
Child pages (pages linked from database)
Code (fenced)
Divider
Headings
Images (download embedded, link to external)
Numbered Lists
Paragraphs
Quotes
Tables
Todo (checkboxes)
Toggle

Supported styling:

Bold
Italic
Inline code
strikethrough

Blocks can be extended or replaced using a plugin system.

Addresses issues #1, #2, & #3.

(not all block types supported yet)

(external links only)

+ refactored converted into objects + added image parser + add command line args related to images

johndgiese

Really nice work Ilya, the code is looking great! I played around with the Commandline tool a little bit. In addition to addressing my comments, if you could try pulling down this database:

https://www.notion.so/innolitics/01dee73fef7e4dd2a02a2a50b6848ba1?v=77043ea15048428ca095601714176e0f

That would be great. The access token is in 1password under the "Notion "Website" Integration Token" entry. The integration is already added to the page and has read-only access.

I think this is the most important comment: https://github.com/innolitics/n2y/pull/5/files#r799131270

I'd be happy to jump on a call to discuss tomorrow.

.github/workflows/tests.yml

n2y/main.py

johndgiese · 2022-02-04T01:23:56Z

n2y/main.py

+                              "  yaml - print yaml to stdout\n"
+                              "  markdown - create a mardown file per page"))
+    parser.add_argument("--image-path", help="Specify path where to save images")
+    parser.add_argument("--image-web-path", help="web path for images")


Small detail, but the case of the help strings is inconsistent; some are capitalized and some aren't.

johndgiese · 2022-02-04T01:25:30Z

n2y/main.py

+    if options.name_column not in raw_rows[0]:
+        print(f"Database does not contain the column \"{options.name_column}\". "
+              f"Please specify the correct name column using the --name-column flag.")
+        exit(1)


Instead of calling exit in these, I think it would be a little cleaner to return the exit code here and then again in main, that way all of the exits go through this line:

https://github.com/innolitics/n2y/pull/5/files#diff-83f6409c23eb83ba1be05c742aba1a0718002913060149a95dbbc5186b467e8fR100

This would let us test main, for example, if we wanted to.

n2y/main.py

README.md

n2y/simplify.py

n2y/main.py

johndgiese · 2022-02-04T03:22:01Z

n2y/main.py

    return 0


+def export_markdown(client, database_id, options):
+    raw_rows = client.get_database(database_id)


It seems like there's bit of duplicated code at the top of export_markdown and export_yaml that could be avoided by pulling this code up a level.

Duplicate code consolidated.

Co-authored-by: David Giese <johndgiese@gmail.com>

…kdown

xaker00

Ready for round 2

xaker00 · 2022-02-07T23:04:54Z

n2y/converter.py

+
+
+def load_plugins(filename):
+    global Bookmark, BulletedList, BulletedListItem, ChildPageBlock, CodeBlockFenced, \


I agree that this is repeating a lot of names, but I'm not sure how this can be done in a better way. Open to suggestions.

xaker00 · 2022-02-07T23:10:08Z

n2y/converter.py

+    plugin_spec = importlib.util.spec_from_file_location("plugins", abs_path)
+    plugin_module = importlib.util.module_from_spec(plugin_spec)
+    plugin_spec.loader.exec_module(plugin_module)
+    print(plugin_module)


n2y/converter.py

xaker00 · 2022-02-07T23:21:47Z

n2y/converter.py

+        self.type = 'to_do_list'
+        super().__init__(client, block, get_children)
+        if self.checked:
+            self.text.text[0].plain_text.text = '☒ ' + self.text.text[0].plain_text.text


As far as I can tell, this is a pandoc quirk. See http://boisgera.github.io/pandoc/markdown/#extension-task_lists
Pandoc will output GitHub flavored checkboxes when it processes them.

xaker00 · 2022-02-07T23:26:55Z

n2y/converter.py

+            self.url = obj['external']['url']
+
+    def download(self):
+        # TODO: append created time as hex to end of file to prevent collisions?


I'm not sure it's a good idea to change the URL of a web page every time the content changes. Another idea is to include Notion block id in the name. It's unique and constant. Date idea was floated because it would be shorter than Notion block id. Using the block id also has the advantage of being able to find the page in Notion from a markdown file.

n2y/notion.py

n2y/simplify.py

n2y/main.py

- Make sure the title column is not empty - Skip pages with no names and show a warning - Warn about entries without a page - Warn about duplicate file names

xaker00 · 2022-02-14T23:20:04Z

Updated per comments.

johndgiese

I had a few small suggestions, but feel free to merge once they're addressed!

johndgiese · 2022-02-15T14:15:44Z

n2y/converter.py

-            Toggle = value
+        if key in globals():
+            # plugins can only override classes in this file that are derrived from a Block
+            if globals()[key].__name__ in [b.__name__ for b in value.__bases__] \


I think this line can be simplified by using some intermediate variables (e.g., for globals()[key] and [b.__name__ for b in value.__bases__].

johndgiese · 2022-02-15T14:19:28Z

n2y/main.py

-        return 0
+    first_row_flattened = simplify.flatten_database_row(raw_rows[0])
+
+    def available_columns():


Why use a function for this instead of just calculating the value? Is this to avoid calculating the value if it doesn't end up being used?

Yes, there are two places where the available columns may be shown, but there is usually no need to generate this list. Also, I imagine this function will grow in future versions as we would want deeper checks.

xaker00 and others added 28 commits January 13, 2022 08:48

adding get_page and get_block_children methods

e84c137

adding pandoc dependency

03e8bed

adding Notion->pandoc converter

8ed13c1

(not all block types supported yet)

removed unused import

26aa331

correct whitespace

0ae25ed

install pandoc

cd29180

set pandoc output format to gfm & correct newlines

1d40e00

flix flake8 issue

c391224

resolved pandoc and markdown version issues

af3eb88

output to markdown files

fb0cd48

handle inline links and bookmarks

cf1b034

(external links only)

added horizontal rule (divider)

293a4e2

Added fenced code block & updated pandoc version

6894acf

Added block quotes

a1a16a3

Major changes

16c13dc

+ refactored converted into objects + added image parser + add command line args related to images

updated tests

85b83ee

fix formatting

fb897e9

correct typo

28cb0bf

Added plugin support

319a67c

fix line endings

fbdc35f

added support for tables

51cb39a

added support for toggle lists

6c0c6f8

added to-do list support

938ea63

add missing plugin loaders

f2351e4

change to GitHub markdown

9d72a00

Return None for child page if there is no content

94f02dd

add & correct command line options

908b839

removed raw example, added markdown example

6e94307

xaker00 requested a review from johndgiese February 3, 2022 20:19

added additional column types

6e1ee88

corrected typo

8ab4a62

johndgiese reviewed Feb 4, 2022

View reviewed changes

xaker00 and others added 3 commits February 7, 2022 15:39

Apply suggestions from code review

9e5b2b5

Co-authored-by: David Giese <johndgiese@gmail.com>

changes from code review

24c8e88

Merge branch 'markdown' of https://github.com/innolitics/n2y into mar…

a254a5a

…kdown

xaker00 commented Feb 8, 2022

View reviewed changes

xaker00 added 2 commits February 14, 2022 21:52

Added additional checks and warnings

fad3c4b

- Make sure the title column is not empty - Skip pages with no names and show a warning - Warn about entries without a page - Warn about duplicate file names

streamline plugin loader

202b5df

johndgiese approved these changes Feb 15, 2022

View reviewed changes

code readability improvement

a55aeb8

xaker00 merged commit 998726a into main Feb 15, 2022

johndgiese deleted the markdown branch March 16, 2022 21:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Basic working version #5

Basic working version #5

xaker00 commented Feb 3, 2022

johndgiese left a comment •

edited

Loading

johndgiese Feb 4, 2022

johndgiese Feb 4, 2022

johndgiese Feb 4, 2022

xaker00 Feb 14, 2022

xaker00 left a comment

xaker00 Feb 7, 2022

xaker00 Feb 7, 2022

xaker00 Feb 7, 2022

xaker00 Feb 7, 2022

xaker00 commented Feb 14, 2022

johndgiese left a comment

johndgiese Feb 15, 2022

johndgiese Feb 15, 2022

xaker00 Feb 15, 2022



		def load_plugins(filename):
		global Bookmark, BulletedList, BulletedListItem, ChildPageBlock, CodeBlockFenced, \

Basic working version #5

Basic working version #5

Conversation

xaker00 commented Feb 3, 2022

johndgiese left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xaker00 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xaker00 commented Feb 14, 2022

johndgiese left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

johndgiese left a comment •

edited

Loading