[1.7.0-beta1] 2022-02-25
Added
- Support for nimbus notes conversion. Conversion does not include converting to pandoc-markdown.
- The conversion process uses a different approach from previous conversions and does not use pandoc. Conversion is completely made using YANOM's own code
- see the end of this change log for table of nimbus features supported in conversion.
- The new methodology can be partially reused to reduce the time to convert another html based note format.
- yanom version on start up screen
- yanom version in conversion reports
- Some use of dictionaries to select functions to run in command line interface instead of if statements
- Use of Dict to select conversion process to be used in notes_converter.py
Nimbus features supported are
| Nimbus Feature | Conversion Support |
|---|---|
| Note data in 'blocks' (columns) | Yes, each block left to right is listed down the new note page in order Block 1 | Block 2 | Block 3 becomes Block 1 Block 2 Block 3 |
| Paragraph Text | Yes |
| Bold formatting | Yes |
| Underlined format | Yes, but using inline html with <u> tag where markdown format allows use of HTML |
| Italic format | Yes |
| Yes | |
| ==Highlighting== | Yes, using <mark> |
| Multiple "stacked" formats such as |
Yes |
| Coloured text | Yes, returned as inline html where supported |
| Placeholder format | No, plain text |
Inline code |
Yes |
| Headings | Yes, and text formatting as above |
| Checklists | Yes, including indentation levels and text formatting as above |
| Numbered lists | Yes, including indentation levels and text formatting as above |
| Bullet lists | Yes, including indentation levels and text formatting as above |
| Captions on images etc | Yes, and text formatting as above |
| Pasted in images | Yes |
| Inserted images | Yes, but note that the original file names are not maintained in nimbus and will be similar to 0UJGinY2jASIj13F.png |
| Embedded files - i.e. pdfs, videos, audio recordings that are visibly displayed on nimbus page | Yes. Original file name is restored. Audio file extensions that are incorrect from nimbus are corrected. Markdown uses  format that will display as an image/live file display/in a player where supported by the markdown viewer. HTML will be a simple link to the file. |
| Hyperlinks | Yes |
| Section break lines | Yes |
| Code Sections | Yes, including code type (Python etc) |
| Tables | Yes, option to keep the 'ABC' and '123' spreadsheet row and column labels. See below for table items |
| Table cell text and numbers | Yes |
| Table cell with inserted file link | No, Not exported by nimbus note |
| Table cell checkboxes | Yes, including checked status, but as inline html <input type="checkbox" checked> as MD does not support block level items in tables |
| Table cell select buttons | Yes, plain text |
| Table cell people mentions | Yes |
| Table cell collaborator mentions | Yes |
| Table cell date entry | Yes |
| Table cell hyperlinks | Yes |
| Table cell ratings | Yes as text Rating 3/5 stars |
| Table cell progress bar | Yes, but data is not exported from nimbus note, exports 0%. Functionality is coded, if nimbus fixes the export the correct % will appear. |
| Date stamps | Yes |
| Block quotes | Yes |
| Hints | Yes, become block quotes |
| Toggle block | Yes, becomes bullet list |
| Embedded iframe (e.g. youtube) | Yes |
| Embedded blockquote (e.g. twitter tweet) | Yes, becomes set of links, no java script |
| Bookmark | Yes, becomes link |
| Embedded audio recording | Yes, becomes link, file extension corrected |
| Embedded screen/video recording | Yes, becomes link |
| Button | Yes, becomes link |
| Outline | Yes, as a numbered table of contents that links to headers. For gfm markdown anchor links are human readable. For other formats links are id number and anchor is also placed next to headings Outline links will not be created for commonmark as are not supported, common mark.org says use gfm format instead. Working links will also depend on the markdown viewer program used as implementation is variable |
| Mentions of people | Yes, plain text or links where mention has an email address |
| Links to "workspaces" | Yes, but as link is to a folder it has to be an absolute URI in markdown. If the workspace name matches a folder in the export data set. If a workspace is renamed after link is created it can be re-linked if there is a link to the new name in one of the notes in the set of notes being converted. NOTE - these links DO NOT work at all in the original nimbus html files |
| Links to "folders" | Yes, but as link is to a folder it has to be an absolute URI in markdown. If the exported notes are organised in the same folder names and the mention matches a folder in the export data set, or if the folder has been renamed there is a link to the new folder name in one of the notes in the set of notes being converted. If there is more than one folder that matches the mention name links will be made to each folder. NOTE - these links DO NOT work at all in the original nimbus html files |
| Links to other notes | Yes - If there is a converted note file of the same name links will be regenerated. If the note has been renamed since the link was created it will be linked to the new name if there is at least one link to that new name in the set of notes being converted. If there is more than one note of the mention name a link will be made to each note of that name, to differentiate each of these links the displayed text also includes the folder the note is in. NOTE - these links DO NOT work at all in the original nimbus html files |
#tags |
Yes, tags and title will be in front matter section. But user action required to paste tags into top of note before export. |
| Files "Attached" above the note | Yes. Nimbus has an attachment option that "attaches" files separate from the note content. They are exported as assets but ARE NOT linked to in the exported nimbus HTML. These files are added as links to the end/bottom of the note content. This process will also catch any files that are in the nimbus HTML that could not be, or were not, processed. |
| Task list on right of nimbus webpage | No. This task list is not part of the note content exported from nimbus |
| Reminders | No. Reminders are not part of the note content exported from nimbus |
NOTE There is no longer a docker image or linux package for 1.6.0 onwards due to unfixable issue in pyinstaller.
Added
- User can now enter a source and export folder path in command line options, the interactive command line interface or config.ini. Paths can be relative to the yanom/data folder or an absolute path on the file system. For Docker the file system is only the docker container file system so paths must be relative to the shared data folder.
- When converting markdown or html files, conversions are now placed in the export folder rather than, being "converted in place" and, being placed in the source folder. The source and export folders can be the same folder to do a "conversion in place".
- When converting md or html files user can now specify an export location rather than converting in place and creating the new files' in the source directory. Attachment files are also copied to the export folder.
- Option to handle orphan files. Orphan files are files in the source path that are not used in any links in the notes. Option to ignore and not copy to the export folder, option to copy to the export folder, and an option to copy to an 'orphans' sub-directory in the export directory
- Option to change relative links to files outside the path of the source directory to absolute links
- For a Note Station NSX conversion check if an attachment uses the same md5 value and is a shared attachment between notes and no longer create a duplicated file on disk.
- Check for and filter out encrypted notes from nsx files. The encrypted notes are not converted.
- Check links in notes are valid for the current file system
- Conversion report displayed to screen and saved as markdown file. Report contains details of the conversion and issues identified with attachments.
- For obsidian image tags image height is now also supported.
- All link handling and link creation will output only posix formatted uri's.
Fixed
- File renaming for existing files with nsx export - was changing the exported file name now changes the existing file name.
- In nsx export duplicate attachment file names no longer overwrite each other.
- Creation time in file name is now fully implemented.
- Obsidian image tags with a path containing parenthesis are now parsed correctly.
- Progress bars no longer partially display in silent mode.
- Markdown to Markdown conversion of some specifically formatted autolink where the link and display text were the same would result in non-working link.
Other Changes
- Improve error message when unable to find file in nsx zip file
- Improvement to tests
[1.5.0] 2021-08-27
Added
- Checklists in NSX files formatted in html code other than the synology checklist format now also export as checklists.
- Checklists in HTML/NSX files formatted as unordered lists, with checkboxes as list items, now convert to Markdown.
- Checklist items now maintain links, such as to web page or email, as working links when converted to Markdown.
- Checklist items on the same line in HTML/NSX will be converted and in Markdown. Checklist items following the first one on a line will be indented as child checklist items
[x] An item [x] another item
becomes,
- [x] An item
- [x] another item
Fixed
- Checklist items in HTML/NSX on the same line will now be converted
- Markdown to HTML conversions - checklist items are now enabled in the exported html. Previously the status was set to disabled and were not clickable.
Other Changes
- Rewrite of checklist processing.
[1.4.1] 2021-08-26
Added
- Mac package is now code signed and notarised by apple. There should be no security alerts when run.
- YANOM now loops back to the command line interface after a conversion. Choose 'quit' to exit. If using the
-ioption for usingconfig.iniYANOM only runs once and exits.
Fixed
- Windows build script updated and is consistent in folders used for the build.
Other Changes
- Mac package now has a set of python based build modules to build, notarise and create a deployment zip file, instead of the bash script previously used.
- Mac and windows packages are now both one-file packages rather than one-directory packages.
[1.4.0] 20201-08-14
Added
- Options to select file and directory name cleaning
- Quick setting selections will keep most characters, spaces and uppercase characters.
- Manual option allows selection of cleaning options
- Allow unicode
- Allow uppercase
- Allow spaces
- Allow non-alphanumeric
- Enter a space replacement character
- Enter a maximum length for file names or directories. Limit can be set by user for non-Windows and Windows where long file paths is enabled. Maximum of 64 characters for Windows where long file paths are not enabled.
- Windows reserved file names are prepended with underscore on all operating systems.
- For nsx files if a file - image or attachment - does not have a file extension the file wil be read and if recognised a file extension is added. This may help where there are issues in the nsx file.
Fixed
- Command line interface may not ask user to accept a changed directory name for export and attachment folders. Now when name cleaned will always ask to accept new file name.
- When asking if user wished to accept the new attachment folder name if the user chose 'no' the export folder question would display not the attachment folder question.
- For nsx file updated image tag algorithm for new image tag format seen in nsx files
- 'gfm' quick setting was not setting export format to gfm, it was using the previous export setting.
Other Changes
- Improved error handling around directory creation with additional messages on Windows systems where long path names are not enabled.
- Moved global variables into a global class and use the instance object and properties for the fields.
- Re-write of image tag searching. Complete rewrite. Functions instead of classes. Reduced use of regex, now only a single regex is used, others replaced by beautiful soup or python string methods.
Release of v1.3.3
Increased stability of user interface and issues with nsx file format variations. Improved file name handling.
Added
- Mouse support in the interactive command line questions.
- NSX conversion.
- Filenames and directories can now have unicode characters.
- Filenames and directories from nsx with quoted characters will be unquoted.
Changed
- Change PyInquirer from imported pypi package to git submodule using fork of the original PyInquirer.
- Add mouse support for command line interface. Items can be mouse clicked, where there is a list of choices (radio butotns) or text entry the return key must be used to move to next question.
- In list questions, the last value used or value from config.ini is at its original place in the list rather than moving it to the top of list as the default option in the new PyInquirer code now works.
- Add check for valid working directory when saving config.ini
- Replace directory and filename cleaning functions.
- If cleaned name becomes and empty string generate random name.
- Allow unicode characters in names.
- Unquote non-ASCII (uses utf-8).
- Changes for logging and exception handling.
- Increased logging around changing file names and directories and any issues creating them on disk.
- Exceptions now log traceback.
- Warning level log file now created and populated.
- Unhandled exceptions are now handled and logged,
- Logging now commences slightly earlier in program startup,
- All calls to get values from json data now check the key exists and logs if it is not and handle the absence of the key. This allows for unknown versions of nsx files, that may have missing key values, attempt to continue conversion.
- If note json data does not have a key for encrypt the assumption is the note is not encrypted. Note text will appear encrypted after conversion if the note is actually encrypted.
- If notes or notebooks data is missing from the nsx file the note or notebook is skipped rather than exiting the program.
- Parent notebook key and id are missing from the nsx note data the note will be placed in the Recycle Bin notebook directory.
- If export folder or attachment folder names are blank in config.ini or the interactive command line tool YANOM defaults of 'notes' and 'attachments'.
- With an attempted read of an encrypted attachment from the nsx file sys.exit is no longer called. A warning is logged and execution continues.
- Tidy up docker files using best practices. No functional changes. Removed no longer used ubuntu docker file.
- Tests updated and improved as opportunities identified.
Fixed
- Fatal Error when user clicks with mouse in interactive command line. Now supports mouse, no longer reliant on user only using keyboard.
- Fixed unknown nsx format with missing encrypt key causes error. Add defensive checks around json data requests for potential unknown nsx formats.
- Fixed possible error due to name cleaning resulting in zero length name. Replaced file and directory cleaning functions that do not return zero length names.
- Fixed keyboard interrupt in a PyInquirer question printed traceback to screen. No longer prints traceback, traceback will be logged.
Added
- Refactor image tag processing - improved efficiency
Fixed
- Potential pandoc failure when converting single row tables from nsx file.
- Metadata keys named 'content' would make 3rd party library raise an exception. Now manually iterate the metadata items instead of using 3rd party function.
- Metadata key of 'charset' would make 3rd party library raise an exception.
- File renaming for duplicate attachment names for nsx export - fixes bug introduced in 1.3.0 that resulted in duplicate names over writing each other, now duplicates are renamed.
- Catch attachments set as null in nsx file, continue to convert note but there will be no attachments. log file records a WARNING message.
- Web version of note station can export encrypted notes. Check for and filter out encrypted notes from nsx files. The encrypted notes are not converted and a warning is placed in the log file.