Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include certain default plugins with the application #7934

Closed
4 tasks
laurent22 opened this issue Mar 16, 2023 · 34 comments · Fixed by #9360
Closed
4 tasks

Include certain default plugins with the application #7934

laurent22 opened this issue Mar 16, 2023 · 34 comments · Fixed by #9360
Assignees
Labels
desktop All desktop platforms enhancement Feature requests and code enhancements plugins Anything related to Joplin's plugin system

Comments

@laurent22
Copy link
Owner

laurent22 commented Mar 16, 2023

  • First decide the list of "chosen" plugins: Rich Markdown, Template, Backup, and more? Look through the list of plugins and see which ones we could include. Maybe check the most downloaded plugins as as guide: https://github.com/joplin/plugins/blob/master/stats.json
  • Then add these plugins to the app using makb's implementation.
    • We may have to create a pull request on the Simple Backup plugin so that it no longer require a binary component (7zip). Maybe we can use the wasm version of 7zip, or use a pure JS library that can be used to compress files, or make 7zip optional.
    • Likewise for the other plugins we include. We need to check they work on all platforms, without binary dependencies.

This work was started as part of GSoC 2022. See these files for what's currently working:

  • Doc: readme/spec/default_plugins.md
  • To bundle default plugins with the app: packages/tools/bundleDefaultPlugins.ts
  • Utilities to run and unpack plugins that have been bundled: packages/lib/services/plugins/defaultPlugins/defaultPluginsUtils.ts
  • And tests units: packages/app-cli/tests/services/plugins/defaultPluginsUtils.ts
@laurent22 laurent22 added enhancement Feature requests and code enhancements desktop All desktop platforms plugins Anything related to Joplin's plugin system labels Mar 16, 2023
@laurent22
Copy link
Owner Author

@palerdot, your change didn't do anything:

{
  "logFormatVersion": 1,
  "jobId": "f497b5f2-3205-461b-9aaf-2b088ed67f11",
  "status": "Invalid",
  "statusSummary": "Archive contains critical validation errors",
  "statusCode": 4000,
  "archiveFilename": "Joplin.zip",
  "uploadDate": "2023-03-30T16:50:08Z",
  "sha256": "e40c64579eaa13d1c8671a22cf5f1410d077df787352cdb2735d6bcd2c69664c",
  "ticketContents": null,
  "issues": [
    {
      "severity": "error",
      "code": null,
      "path": "Joplin.zip/Joplin.app/Contents/Resources/build/defaultPlugins/io.github.jackgruber.backup/plugin.jpl/7zip-bin/mac/x64/7za",
      "message": "The binary is not signed.",
      "docUrl": "https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution/resolving_common_notarization_issues#3087721",
      "architecture": "x86_64"
    },
    {
      "severity": "error",
      "code": null,
      "path": "Joplin.zip/Joplin.app/Contents/Resources/build/defaultPlugins/io.github.jackgruber.backup/plugin.jpl/7zip-bin/mac/x64/7za",
      "message": "The signature does not include a secure timestamp.",
      "docUrl": "https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution/resolving_common_notarization_issues#3087733",
      "architecture": "x86_64"
    },
    {
      "severity": "error",
      "code": null,
      "path": "Joplin.zip/Joplin.app/Contents/Resources/build/defaultPlugins/io.github.jackgruber.backup/plugin.jpl/7zip-bin/mac/x64/7za",
      "message": "The executable does not have the hardened runtime enabled.",
      "docUrl": "https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution/resolving_common_notarization_issues#3087724",
      "architecture": "x86_64"
    },
    {
      "severity": "error",
      "code": null,
      "path": "Joplin.zip/Joplin.app/Contents/Resources/build/defaultPlugins/io.github.jackgruber.backup/plugin.jpl/7zip-bin/mac/arm64/7za",
      "message": "The binary is not signed with a valid Developer ID certificate.",
      "docUrl": "https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution/resolving_common_notarization_issues#3087721",
      "architecture": "arm64"
    },
    {
      "severity": "error",
      "code": null,
      "path": "Joplin.zip/Joplin.app/Contents/Resources/build/defaultPlugins/io.github.jackgruber.backup/plugin.jpl/7zip-bin/mac/arm64/7za",
      "message": "The signature does not include a secure timestamp.",
      "docUrl": "https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution/resolving_common_notarization_issues#3087733",
      "architecture": "arm64"
    },
    {
      "severity": "error",
      "code": null,
      "path": "Joplin.zip/Joplin.app/Contents/Resources/build/defaultPlugins/io.github.jackgruber.backup/plugin.jpl/7zip-bin/mac/arm64/7za",
      "message": "The executable does not have the hardened runtime enabled.",
      "docUrl": "https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution/resolving_common_notarization_issues#3087724",
      "architecture": "arm64"
    }
  ]
}

laurent22 added a commit that referenced this issue Mar 30, 2023
@palerdot
Copy link
Contributor

This thread discusses this issue in detail - electron-userland/electron-builder#4656 (comment), and apparently they successfully solved the issue with https://github.com/electron/osx-sign

I'm still not sure if this has any relation to electron-notarize or if @electron/osx-sign is an extra step required to sign extra internal binaries.

@palerdot
Copy link
Contributor

palerdot commented Apr 10, 2023

Cumulative downloads stats for plugins (joplin/plugins@9a4f717)

[
  {
    "id":"com.andrejilderda.macOSTheme",
    "totalDownloads":21408,
    "description":"Native looking macOS theme for Joplin. Also works on non-macOS devices.",
    "url":"https://github.com/ajilderda/joplin-macos-native-theme"
  },
  {
    "id":"plugin.calebjohn.rich-markdown",
    "totalDownloads":21081,
    "description":"Helping you ditch the markdown viewer for good.",
    "url":"https://github.com/CalebJohn/joplin-rich-markdown#readme"
  },
  {
    "id":"outline",
    "totalDownloads":20221,
    "description":"Outline (TOC) sidebar plugin for joplin.",
    "url":"https://github.com/cqroot/joplin-outline"
  },
  {
    "id":"joplin.plugin.templates",
    "totalDownloads":17351,
    "description":"This plugin allows you to create and use templates in Joplin.",
    "url":"https://github.com/joplin/plugin-templates"
  },
  {
    "id":"joplin.plugin.note.tabs",
    "totalDownloads":17160,
    "description":"Allows to open several notes at once in tabs and pin them.",
    "url":"https://github.com/benji300/joplin-note-tabs"
  },
  {
    "id":"io.github.jackgruber.backup",
    "totalDownloads":15173,
    "description":"Plugin to create manual and automatic backups.",
    "url":"https://github.com/JackGruber/joplin-plugin-backup/blob/master/README.md"
  },
  {
    "id":"com.whatever.quick-links",
    "totalDownloads":15149,
    "description":"Create links to other notes",
    "url":"https://discourse.joplinapp.org/t/quick-links-plugin/14214"
  },
  {
    "id":"com.github.joplin.kanban",
    "totalDownloads":12003,
    "description":"Flexible kanban board plugin for all your tasks",
    "url":"https://github.com/joplin/plugin-kanban/wiki"
  },
  {
    "id":"io.treymo.LinkGraph",
    "totalDownloads":11237,
    "description":"Visualize the connections between Joplin notes.",
    "url":"https://github.com/treymo/joplin-link-graph"
  },
  {
    "id":"joplin.plugin.benji.favorites",
    "totalDownloads":9762,
    "description":"Save any notebook, note, to-do, tag, or search as favorite in an extra panel view for quick access.",
    "url":"https://github.com/benji300/joplin-favorites"
  },
  {
    "id":"com.gitlab.BeatLink.joplin-plugin-agenda",
    "totalDownloads":9569,
    "description":"An agenda/calendar/schedule panel for joplin that shows all uncompleted to-dos with a due date.",
    "url":"https://discourse.joplinapp.org/t/agenda/21158"
  },
  {
    "id":"net.rmusin.joplin-table-formatter",
    "totalDownloads":8950,
    "description":"Format the table under the cursor in Markdown editor",
    "url":"https://github.com/roman-r-m/joplin-plugin-table-formatter"
  },
  {
    "id":"joplin.plugin.ambrt.backlinksToNote",
    "totalDownloads":7588,
    "description":"Creates backlinks to opened note, also in automatic way",
    "url":"https://discourse.joplinapp.org/t/insert-referencing-notes-backlinks-plugin/13632"
  },
  {
    "id":"com.github.marc0l92.joplin-plugin-drawio",
    "totalDownloads":7012,
    "description":"Draw.io (aka Diagram.net) integration for Joplin",
    "url":"https://github.com/marc0l92/joplin-plugin-drawio#readme"
  },
  {
    "id":"com.septemberhx.Joplin.Enhancement",
    "totalDownloads":6403,
    "description":"Enhance the markdown editor with live preview for math, mermaid, link, image, and more. It also includes other features for markdown rendering.",
    "url":"https://github.com/SeptemberHX/joplin-plugin-enhancement.git"
  },
  {
    "id":"ylc395.noteLinkSystem",
    "totalDownloads":6392,
    "description":"A complete Link System for Joplin. Referrer(aka. backlink), Quick Link, Copy Anchor, Hover to preview, Url Icon, and much more feature",
    "url":"https://github.com/ylc395/joplin-plugin-note-link-system"
  },
  {
    "id":"plugin.calebjohn.MathMode",
    "totalDownloads":6218,
    "description":"Turn your notes into a powerful calculator with inline math.",
    "url":"https://github.com/CalebJohn/joplin-math-mode#readme"
  },
  {
    "id":"cx.evermeet.tessus.menu-shortcut-toolbar",
    "totalDownloads":6154,
    "description":"Additional menu items, shortcuts, and toolbar icons, which are not part of Joplin core",
    "url":"https://github.com/tessus/joplin-plugin-menu-shortcut-toolbar#readme"
  },
  {
    "id":"io.github.jackgruber.note-overview",
    "totalDownloads":6107,
    "description":"A note overview is created based on the defined search and the specified fields.",
    "url":"https://github.com/JackGruber/joplin-plugin-note-overview/blob/master/README.md"
  },
  {
    "id":"com.whatever.inline-tags",
    "totalDownloads":6025,
    "description":"Inline tags plugin",
    "url":"https://discourse.joplinapp.org/t/plugin-inline-tags/14192"
  },
  {
    "id":"sadmice.TextColorize",
    "totalDownloads":5957,
    "description":"Easily apply color to text.",
    "url":"https://github.com/sadmice/Joplin-Plugin-Text-Colorize"
  },
  {
    "id":"ylc395.betterMarkdownViewer",
    "totalDownloads":5594,
    "description":"Keep Cursor Synced Between Editor and MD Viewer",
    "url":"https://github.com/ylc395/joplin-plugin-better-markdown-viewer"
  },
  {
    "id":"com.github.marc0l92.joplin-plugin-plantUML",
    "totalDownloads":5457,
    "description":"Render PlantUML diagram inside your Joplin notes.",
    "url":"https://github.com/marc0l92/joplin-plugin-plantUML#readme"
  },
  {
    "id":"com.hieuthi.joplin.markdown-table-colorize",
    "totalDownloads":5369,
    "description":"Add colors to markdown table syntax and help distinguishing different columns",
    "url":"https://github.com/hieuthi/joplin-plugin-markdown-table-colorize"
  },
  {
    "id":"com.github.marc0l92.joplin-plugin-github-theme",
    "totalDownloads":4994,
    "description":"Joplin theme with colors of GitHub",
    "url":"https://github.com/marc0l92/joplin-plugin-github-theme#readme"
  },
  {
    "id":"joplin-plugin-knowledge-graph",
    "totalDownloads":4838,
    "description":"Notes as nodes. Explore your Joplin knowledge graph.",
    "url":"https://github.com/agerardin/joplin-plugin-knowledge-graph"
  },
  {
    "id":"joplin-plugin-conflict-resolution",
    "totalDownloads":4758,
    "description":"A plugin that makes process of resolving conflicts easier.",
    "url":"https://discourse.joplinapp.org/t/plugin-conflict-resolution/19204"
  },
  {
    "id":"org.joplinapp.plugins.admonition",
    "totalDownloads":4449,
    "description":"Enables markdownIt admonition plugin, which enables formatting notes",
    "url":"https://github.com/maxnegro/joplin-plugin-admonition"
  },
  {
    "id":"joplin.plugin.ambrt.fold-cm",
    "totalDownloads":4295,
    "description":"Allows to fold parts of markdown text displayed in editor",
    "url":"https://discourse.joplinapp.org/t/persistent-text-folding-in-editor/16183"
  },
  {
    "id":"io.github.jackgruber.combine-notes",
    "totalDownloads":4105,
    "description":"Combine one or more notes",
    "url":"https://github.com/JackGruber/joplin-plugin-combine-notes"
  },
  {
    "id":"org.joplinapp.plugins.ToggleSidebars",
    "totalDownloads":4041,
    "description":"Adds buttons to toggle note list and sidebar",
    "url":"https://github.com/laurent22/joplin/tree/dev/packages/plugins/ToggleSidebars"
  },
  {
    "id":"osw.joplin.markdowncalc",
    "totalDownloads":3886,
    "description":"Plugin for automatic calculations of markdown table formulas.",
    "url":"https://github.com/oswida/joplin-markdown-calc"
  },
  {
    "id":"plugin.calebjohn.todo",
    "totalDownloads":3855,
    "description":"Write TODOs *everywhere* and view them in one place.",
    "url":"https://github.com/CalebJohn/joplin-inline-todo#readme"
  },
  {
    "id":"com.hieuthi.joplin.markdown-table-sortable",
    "totalDownloads":3777,
    "description":"Sort rendered tables on the fly and apply it to markdown source text. Can recognize simple cases of number, money, date and sort accordingly.",
    "url":"https://github.com/hieuthi/joplin-plugin-markdown-table-sortable"
  },
  {
    "id":"com.gitlab.BeatLink.joplin-plugin-repeating-todos",
    "totalDownloads":3750,
    "description":"A powerful and comprehensive plugin for to-do repetition/recurrence",
    "url":"https://discourse.joplinapp.org/t/plugin-repeating-to-dos/16470"
  },
  {
    "id":"ylc395.joplinOcr",
    "totalDownloads":3744,
    "description":"offline OCR auto/manually for images, videos, pdf documents in your Joplin notes",
    "url":"https://github.com/ylc395/joplin-plugin-ocr"
  },
  {
    "id":"io.github.jackgruber.copytags",
    "totalDownloads":3702,
    "description":"Plugin to extend the Joplin tagging menu with a coppy all tags and a tagging dialog with more control. (Formerly Copy Tags).",
    "url":"https://github.com/JackGruber/joplin-plugin-tagging/blob/master/README.md"
  },
  {
    "id":"joplin.plugin.spoiler.cards",
    "totalDownloads":3655,
    "description":"Create inline spoilers and spoiler blocks with title and extendable body.",
    "url":"https://github.com/martinkorelic/joplin-plugin-spoilers"
  },
  {
    "id":"com.lki.homenote",
    "totalDownloads":3590,
    "description":"Plugin to open a choosen note each time joplin starts. It is like homepages on browsers.",
    "url":"https://github.com/adarsh-sgh/homenote#readme"
  },
  {
    "id":"retr0ve.EmojiPlugin",
    "totalDownloads":3524,
    "description":"A emoji picker.",
    "url":"https://discourse.joplinapp.org/t/plugin-emoji-picker/24228"
  },
  {
    "id":"joplin-insert-date",
    "totalDownloads":3450,
    "description":"Adds a button to insert the current date (without time) into the editor.",
    "url":"https://github.com/herdsothom/joplin-insert-date"
  },
  {
    "id":"joplin.plugin.benji.persistentLayout",
    "totalDownloads":3149,
    "description":"Save the editor layout (editor/split view/viewer/rich text) for each note separately with custom tags.",
    "url":"https://github.com/benji300/joplin-persistent-layout"
  },
  {
    "id":"io.github.jackgruber.hotfolder",
    "totalDownloads":3093,
    "description":"Monitors a locale folder and import the files as a new note.",
    "url":"https://github.com/JackGruber/joplin-plugin-hotfolder/blob/master/README.md"
  },
  {
    "id":"io.github.personalizedrefrigerator.js-draw",
    "totalDownloads":2958,
    "description":"Create and edit drawings with js-draw.",
    "url":"https://github.com/personalizedrefrigerator/joplin-draw"
  },
  {
    "id":"joplin.plugin.ambrt.convertToNewNote",
    "totalDownloads":2947,
    "description":"Converts highlighted text to new one in same folder",
    "url":"https://discourse.joplinapp.org/t/create-note-from-highlighted-text/12511"
  },
  {
    "id":"joplin.plugin.ambrt.embedSearch",
    "totalDownloads":2935,
    "description":"Embeds list of links specified by search inside of note",
    "url":"https://discourse.joplinapp.org/t/embed-any-search-with-content/14328"
  },
  {
    "id":"com.joplin.JSheets",
    "totalDownloads":2803,
    "description":"JSheets - use to write spreadsheets like excel in Joplin",
    "url":"https://github.com/ThibaultJanBeyer/joplin-sheets"
  },
  {
    "id":"com.shantanugoel.JoplinCMLineNumbersPlugin",
    "totalDownloads":2743,
    "description":"Enables line numbers for CodeMirror editor",
    "url":"https://github.com/shantanugoel/joplin-plugin-cm-linenumbers"
  },
  {
    "id":"com.s73ph4n.autolinker",
    "totalDownloads":2730,
    "description":"Adds an icon to the toolbar. When clicked, it creates a link to the note with a title matching the selected text.",
    "url":"https://github.com/S73ph4n"
  },
  {
    "id":"com.septemberhx.pluginBundle",
    "totalDownloads":2723,
    "description":"Table of Contents, Inline Todos, Daily Notes, and History aggregated in one panel with more eye-candy UI design.",
    "url":"https://github.com/SeptemberHX/joplin-plugin-bundle.git"
  },
  {
    "id":"joplin.plugin.forcewake.tags-generator",
    "totalDownloads":2461,
    "description":"Plugin for Joplin which can be used to extract keywords from note and assign them as a note's tags",
    "url":"https://github.com/forcewake/joplin-tags-generator"
  },
  {
    "id":"net.gagnepain.turnToChart",
    "totalDownloads":2460,
    "description":"Transforms a markdown or csv table into an HTML chart",
    "url":"https://github.com/Winbee/joplin-turn-to-chart"
  },
  {
    "id":"joplin.plugin.ambrt.goToItem",
    "totalDownloads":2452,
    "description":"Go to tag,notebook or note via links or via text",
    "url":"https://discourse.joplinapp.org/t/go-to-note-tag-or-notebook-via-highlighting-text-in-editor/12731"
  },
  {
    "id":"com.github.BeatLink.joplin-plugin-white-theme",
    "totalDownloads":2416,
    "description":"A white theme for joplin",
    "url":"https://discourse.joplinapp.org/t/joplin-white-theme"
  },
  {
    "id":"de.habelt.CodeSection",
    "totalDownloads":2376,
    "description":"Inserts a piece of code from code source",
    "url":"https://github.com/Mick2nd/Code-Section#readme"
  },
  {
    "id":"com.sctmes.kity-minder",
    "totalDownloads":2285,
    "description":"Kity Minder Mindmap Tools 思维导图(脑图插件) ",
    "url":"https://github.com/xeden3/joplin-plugin-kity-minder"
  },
  {
    "id":"com.eliasvsimon.email-note",
    "totalDownloads":2233,
    "description":"Send a note as an email",
    "url":"https://github.com/EliasVincent/joplin-email-note"
  },
  {
    "id":"joplin.plugin.quick.html.tags",
    "totalDownloads":2228,
    "description":"Create quick HTML tags.",
    "url":"https://github.com/martinkorelic/joplin-plugin-quick-html-tags"
  },
  {
    "id":"com.s73ph4n.day_review",
    "totalDownloads":2196,
    "description":"This plugin adds an icon to the toolbar. When clicked, it makes a review of the day's created/updated/completed notes and todos.",
    "url":"https://github.com/S73ph4n"
  },
  {
    "id":"de.habelt.CsvImport",
    "totalDownloads":2124,
    "description":"Imports a Csv file or pastes a Csv text from Clipboard as table",
    "url":"https://github.com/Mick2nd/CsvImport#readme"
  },
  {
    "id":"joplin.plugin.benji.quick-move",
    "totalDownloads":2090,
    "description":"Collection of commands to quickly move notes to other notebooks.",
    "url":"https://github.com/benji300/joplin-quick-move"
  },
  {
    "id":"com.xUser5000.bibtex",
    "totalDownloads":2085,
    "description":"Use locally stored BibTeX files to integrate citation into Joplin",
    "url":""
  },
  {
    "id":"joplin.plugin.remoods.theme",
    "totalDownloads":2069,
    "description":"3 theme modes, 36 single-tone theme colors, 5 major html custom elements, and more... (Recommend to read about the Getting Started Guide on GitHub before use.) ",
    "url":"https://github.com/Sinacs/Joplin.Plugin.ReMoods.Theme"
  },
  {
    "id":"com.hieuthi.joplin.slash-commands",
    "totalDownloads":2060,
    "description":"Execute several utilities by typing a command that starts with a slash.",
    "url":"https://github.com/hieuthi/joplin-plugin-slash-commands"
  },
  {
    "id":"com.hieuthi.joplin.life-calendar",
    "totalDownloads":2036,
    "description":"Life Calendar Plugin for Joplin",
    "url":"https://github.com/hieuthi/joplin-plugin-life-calendar"
  },
  {
    "id":"com.hieuthi.joplin.metis",
    "totalDownloads":1993,
    "description":"A Simple Task Manager Plugin for Joplin based on Todo.txt Specification",
    "url":"https://github.com/hieuthi/joplin-plugin-metis"
  },
  {
    "id":"net.rmusin.resource-search",
    "totalDownloads":1939,
    "description":"Search in Attached Resources",
    "url":"https://github.com/roman-r-m/joplin-plugin-resource-search"
  },
  {
    "id":"plugin.azamahJunior.note-statistics",
    "totalDownloads":1856,
    "description":"Plugin to get note statistics",
    "url":"https://github.com/Kaid00/joplin-note-statistics#readme"
  },
  {
    "id":"joplin.plugin.ambrt.copyNoteLink",
    "totalDownloads":1842,
    "description":"Adds entry to right click menu in editor to get link to active note",
    "url":"https://discourse.joplinapp.org/t/copy-markdown-link-to-active-note/14402"
  },
  {
    "id":"ylc395.pagesPublisher",
    "totalDownloads":1784,
    "description":"Generate static blog website from your picked Joplin notes, and publish to Github Pages, with a few mouse clicks. Support preview in local",
    "url":"https://github.com/ylc395/joplin-plugin-pages-publisher"
  },
  {
    "id":"com.leenzhu.journal",
    "totalDownloads":1782,
    "description":"Create or open note of today or selected date.",
    "url":"https://github.com/leenzhu/joplin-plugin-journal"
  },
  {
    "id":"shufo.markdown-prettier",
    "totalDownloads":1754,
    "description":"Format your markdown by prettier",
    "url":"https://github.com/shufo/joplin-plugin-markdown-prettier"
  },
  {
    "id":"com.wemakemachines.joplin.plugin.event-calendar",
    "totalDownloads":1718,
    "description":"A simple event calendar",
    "url":"https://github.com/WeMakeMachines/joplin-plugin-event-calendar"
  },
  {
    "id":"kensam.joplin.plugin.eztable",
    "totalDownloads":1694,
    "description":"Make markdown table easy",
    "url":"https://github.com/kensam94/joplin-plugin-eztable"
  },
  {
    "id":"com.github.BeatLink.joplin-plugin-untagged",
    "totalDownloads":1464
  },
  {
    "id":"io.github.makaanneo.Athena",
    "totalDownloads":1411,
    "description":"Import files from import a folder into joplin notebook as notes mit metadata (PDF text in note comment, no OCR)",
    "url":"https://github.com/makaanneo/joplin-plugin-athena"
  },
  {
    "id":"io.github.daeraxa.match-highlight",
    "totalDownloads":1397,
    "description":"Marks words that match the highlighted selection",
    "url":"https://discourse.joplinapp.org/t/highlight-matching-plugin/23747"
  },
  {
    "id":"com.hieuthi.joplin.copy-anchor-link",
    "totalDownloads":1389,
    "description":"This plugin add several icons next to the note headings that you can click on to copy markdown link of the specific heading.",
    "url":"https://github.com/hieuthi/joplin-plugin-copy-anchor-link"
  },
  {
    "id":"com.coderrsid.pasteSpecial",
    "totalDownloads":1369,
    "description":"A plugin used to paste special text into Joplin",
    "url":""
  },
  {
    "id":"com.github.marc0l92.joplin-plugin-jira-issue",
    "totalDownloads":1316,
    "description":"Retrieve Atlassian Jira issues information using their api in order to track the status of them from your Joplin notes.",
    "url":"https://github.com/marc0l92/joplin-plugin-jira-issue#readme"
  },
  {
    "id":"fd117a99-b165-4824-893c-5825439a842d",
    "totalDownloads":1283,
    "description":"Disables the PDF preview (and export) in Joplin for better performance",
    "url":"https://github.com/JJ-8/joplin-disable-pdf#readme"
  },
  {
    "id":"org.joplinapp.plugins.AbcSheetMusic",
    "totalDownloads":1248,
    "description":"Turns ABC text notation into sheet music",
    "url":"https://github.com/joplin/plugin-abc-sheet-music"
  },
  {
    "id":"joplin.plugin.alondmnt.history-panel",
    "totalDownloads":1170,
    "description":"Joplin note browsing history",
    "url":"https://github.com/alondmnt/joplin-plugin-history-panel#readme"
  },
  {
    "id":"io.github.manuerwin.attache",
    "totalDownloads":1155,
    "description":"Your attachment updater. Mass replacement of Joplin attachments (resources) such as resized image files, current music playlists, and any other attachments you need to one-off or regularly update within Joplin.",
    "url":"https://github.com/manuerwin/joplin-plugin-attache#readme"
  },
  {
    "id":"com.joplin.excalidraw",
    "totalDownloads":1155,
    "description":"use to write excalidraw in Joplin",
    "url":"https://github.com/artikell/joplin-excalidraw"
  },
  {
    "id":"joplin-plugin-dddot",
    "totalDownloads":1136,
    "description":"Recent notes, shortcuts, scratchpad, and .... in a single sidebar.",
    "url":"https://github.com/benlau/joplin-plugin-dddot"
  },
  {
    "id":"com.s73ph4n.complete_link",
    "totalDownloads":1051,
    "description":"This plugin adds an icon to the toolbar. When clicked, it completes the selected text into a note title (in a markdown link).",
    "url":"https://github.com/S73ph4n"
  },
  {
    "id":"com.s73ph4n.make_all_links",
    "totalDownloads":1037,
    "description":"Searches the current note for mentions of other notes, then makes the corresponding links. Just click the new icon in the toolbar.",
    "url":"https://github.com/S73ph4n"
  },
  {
    "id":"com.font.size.shortcut",
    "totalDownloads":1015,
    "description":"Adds shortcuts to increase/decrease font size",
    "url":"https://github.com/mak2002/joplin-font-size-shortcut.git"
  },
  {
    "id":"com.programming-emu.DependencyGraph",
    "totalDownloads":1011,
    "description":"Visualizes direct and indirect dependencies of a note as a graph.",
    "url":""
  },
  {
    "id":"com.DanteCoder.NoteVariables",
    "totalDownloads":975,
    "description":"A plugin to create variables that can be accesed through all the notes.",
    "url":"https://github.com/DanteCoder/JoplinPluginNoteVariables#readme"
  },
  {
    "id":"io.xargs.JoplinPluginAutoTag",
    "totalDownloads":955,
    "description":"Convert all octothorpe prefixed words into tags",
    "url":""
  },
  {
    "id":"com.github.uphy.PlantUmlPlugin",
    "totalDownloads":912
  },
  {
    "id":"org.joplinapp.plugins.AutoShowActiveNoteInSideBar",
    "totalDownloads":907,
    "description":"Auto show active note in sidebar",
    "url":"https://github.com/eyalag/joplin-plugin-auto-show-active-note"
  },
  {
    "id":"com.export-to-ssg.aman-d-1-n-only",
    "totalDownloads":848,
    "description":"Export a collection of notes to static site generator project.",
    "url":"https://github.com/aman-d-1-n-only/joplin-exports-to-ssg#readme"
  },
  {
    "id":"io.xargs.JoplinPluginAutoAlarm",
    "totalDownloads":834,
    "description":"Natural language entry of alarm date/times",
    "url":"github.com/zph/joplin-plugin-auto-alarm"
  },
  {
    "id":"joplin.plugin.alondmnt.jarvis",
    "totalDownloads":826,
    "description":"Joplin assistant running a very intelligent system (GPT-3, ChatGPT, GPT-4)",
    "url":"https://github.com/alondmnt/joplin-plugin-jarvis"
  },
  {
    "id":"org.joplinapp.plugins.Victor",
    "totalDownloads":811,
    "description":"Victor can be used to clear all your data - notes, notebooks, attachments, tags, etc. Convenient to start over.",
    "url":"https://github.com/joplin/plugin-victor#readme"
  },
  {
    "id":"Bishoy.EmailPlugin",
    "totalDownloads":800,
    "description":"Fetch your important emails as notes.",
    "url":"https://github.com/joplin/plugin-email"
  },
  {
    "id":"joplin-plugin-tag-links",
    "totalDownloads":799,
    "description":"Insert the link of tags at the footer automatically",
    "url":"https://github.com/benlau/joplin-plugin-tag-links"
  },
  {
    "id":"kensam.joplin.plugin.highlighter",
    "totalDownloads":734,
    "description":"Highlight/dehighlight the selected text",
    "url":"https://github.com/kensam94/joplin-plugin-simple-highlighter"
  },
  {
    "id":"joplin.plugin.space-indenter",
    "totalDownloads":721,
    "description":"Indent paragraphs with spaces instead of tabs",
    "url":"https://github.com/alondmnt/joplin-plugin-space-indenter"
  },
  {
    "id":"com.s73ph4n.automate_notes",
    "totalDownloads":716,
    "description":"Finds blocks of javascript in the current note (blocks have to start with '```javascript' and end with '```') and runs it as a function. ",
    "url":"https://github.com/S73ph4n"
  },
  {
    "id":"io.github.makaanneo.apollo",
    "totalDownloads":712,
    "description":"Joplin plugin to rename note attachments,",
    "url":"https://github.com/makaanneo/joplin-plugin-apollo"
  },
  {
    "id":"joplin.plugin.cmoptions",
    "totalDownloads":696,
    "description":"Adds some CodeMirror options to Joplin's settings, such as line numbers and line wrapping.",
    "url":"https://github.com/FelisDiligens/joplin-plugin-cmoptions"
  },
  {
    "id":"joplin.plugin.MultiMarkdownTableTools",
    "totalDownloads":669,
    "description":"A collection of tools to make editing tables easier. Also supports GitHub Flavored Markdown!",
    "url":"https://github.com/FelisDiligens/joplin-plugin-multimd-table-tools"
  },
  {
    "id":"joplin.plugin.anki-sync",
    "totalDownloads":665,
    "description":"Bidirectional sync between Joplin and Anki",
    "url":"https://github.com/chenlijun99/autoanki/packages/joplin-plugin"
  },
  {
    "id":"joplin.plugin.embeddedtags",
    "totalDownloads":651,
    "description":"embeded tags",
    "url":""
  },
  {
    "id":"joplin-plugin-search-and-replace",
    "totalDownloads":622,
    "description":"Adds a search and replace panel to the Markdown editor.",
    "url":"https://github.com/FelisDiligens/joplin-plugin-search-and-replace"
  },
  {
    "id":"org.yangby.joplin.plugins.enhanced-editing",
    "totalDownloads":612,
    "description":"An enhanced editing plugin for Joplin.",
    "url":"https://github.com/yangby/joplin-plugin-enhanced-editing"
  },
  {
    "id":"com.hieuthi.joplin.container-with-classes",
    "totalDownloads":600,
    "description":"Plugin for creating block-level containers with classes for customization",
    "url":"https://github.com/hieuthi/joplin-plugin-container-with-classes"
  },
  {
    "id":"aa.asrient.backstage",
    "totalDownloads":568,
    "description":"Scan and import files directly from your phone",
    "url":"https://asrient.github.io/backstage"
  },
  {
    "id":"b53da1f6-868c-468d-b60c-2897a27166ac",
    "totalDownloads":563,
    "description":"A Joplin plugin for custom links, like `jump://?query=blabla`.",
    "url":"https://github.com/Cologler/userlink-joplin"
  },
  {
    "id":"com.plugin.randomNotePlugin",
    "totalDownloads":531,
    "description":"Opens a random note from your vault",
    "url":"https://github.com/Kaid00/joplin-random-note/blob/master/README.md"
  },
  {
    "id":"com.DanteCoder.JoplinBibleQuote",
    "totalDownloads":521,
    "description":"Quote the bible direct in text editor",
    "url":"https://github.com/DanteCoder/JoplinBibleQuote#readme"
  },
  {
    "id":"com.github.marcgreen.joplin-plugin-semantically-similar-notes",
    "totalDownloads":515,
    "description":"Show user which notes are semantically similar to the one they are looking at",
    "url":"https://github.com/marcgreen/semantic-joplin/tree/master/similar-notes"
  },
  {
    "id":"com.joplin.copy.codeBlocks",
    "totalDownloads":515,
    "description":"Add a copy button to the code blocks.",
    "url":"https://github.com/LightAPIs/joplin-copy-code-blocks"
  },
  {
    "id":"com.hieuthi.joplin.quick-goto",
    "totalDownloads":469,
    "description":"Switch to designated notes using keyboard shortcuts",
    "url":"https://github.com/hieuthi/joplin-plugin-quick-goto"
  },
  {
    "id":"bluulo.JoplinCodeClipboardPlugin",
    "totalDownloads":455,
    "description":"Copy the content of a code block to the clipboard",
    "url":"https://github.com/bluulo/joplin-plugin-code-clipboard"
  },
  {
    "id":"com.remove.images.from.note",
    "totalDownloads":444,
    "description":"Allows to remove images from selected note",
    "url":"https://github.com/mak2002/joplin-remove-images-from-note.git"
  },
  {
    "id":"com.github.marc0l92.joplin-plugin-bytefield-svg",
    "totalDownloads":441,
    "description":"Add support for bytefield-svg diagrams used to describe network protocols, memory layouts and any other binary structure.",
    "url":"https://github.com/marc0l92/joplin-plugin-bytefield-svg#readme"
  },
  {
    "id":"com.jpa.joplin.timetagger",
    "totalDownloads":438,
    "description":"Time tracking with Joplin and TimeTagger",
    "url":"https://github.com/aranajuan/joplin-plugin-timetagger"
  },
  {
    "id":"joplin-plugin-note-encryption",
    "totalDownloads":423,
    "description":"A joplin plugin, encrypt selected notes on the joplin desktop client.",
    "url":"https://github.com/ZhangTe/joplin-plugin-encrypt-notes"
  },
  {
    "id":"cx.evermeet.tessus.folder-id",
    "totalDownloads":415,
    "description":"Copy/Show the ID of a notebook. For developers. A regular user won't need this.",
    "url":"https://github.com/tessus/joplin-plugin-get-notebook-id#readme"
  },
  {
    "id":"joplin.plugin.ambrt.sendSnippet",
    "totalDownloads":415,
    "description":"Sends snippet of text to any other note from any notebook",
    "url":"https://github.com/ambrt/joplin-plugin-send-snippet"
  },
  {
    "id":"albert.joplin.plugin.jl14",
    "totalDownloads":405,
    "description":"Delete unlinked resources",
    "url":"https://github.com/pnphuong29/joplin-plugin-jl14-pub"
  },
  {
    "id":"io.github.personalizedrefrigerator.joplin-vimrc",
    "totalDownloads":374,
    "description":"Configure CodeMirror‘s ViM emulation with a very-limited, vimrc-like file.",
    "url":"https://github.com/personalizedrefrigerator/joplin-plugin-codemirror-vimrc"
  },
  {
    "id":"com.joplin_plugin.nlr",
    "totalDownloads":365,
    "description":"将joplin在记笔记的同时,变成一个网文小说阅读器,a chinese net literature downloader and reader plugin for joplin.",
    "url":"https://github.com/fengqiaozhu/joplin_plugin_nlr.git"
  },
  {
    "id":"eu.xardbaiz.joplin.HackMD",
    "totalDownloads":343,
    "description":"Quick share notes on HackMD",
    "url":"https://discourse.joplinapp.org/t/plugin-share-note-online-via-hackmd/22048"
  },
  {
    "id":"joplin.plugin.alondmnt.suitcase",
    "totalDownloads":314,
    "description":"Change the capitalization of selected text",
    "url":"https://github.com/alondmnt/joplin-plugin-suitcase/#readme"
  },
  {
    "id":"com.ahaltindis.joplin.BulkNoteCreator",
    "totalDownloads":299,
    "description":"Create notes as bulk",
    "url":"https://github.com/ahaltindis/joplin-plugin-bulk-note-creator"
  },
  {
    "id":"com.hieuthi.joplin.function-plot",
    "totalDownloads":296,
    "description":"Joplin plugin based on Function Plot library to render function such as y = x * x",
    "url":"https://github.com/hieuthi/joplin-plugin-function-plot"
  },
  {
    "id":"org.yangby.joplin.plugins.pseudocode-support",
    "totalDownloads":232,
    "description":"Typesets pseudocode beautifully in Joplin.",
    "url":"https://github.com/yangby/joplin-plugin-pseudocode-support"
  },
  {
    "id":"com.hieuthi.joplin.rubi-and-furigana",
    "totalDownloads":198,
    "description":"Commands that insert appropriate <ruby> and <rt> tags to formatted text to display furigana.",
    "url":"https://github.com/hieuthi/joplin-plugin-rubi-and-furigana"
  },
  {
    "id":"com.arash28134.DiscordPresence",
    "totalDownloads":172,
    "description":"A Discord Rich Presence Plugin For Joplin",
    "url":""
  },
  {
    "id":"joplin-plugin-TimelineRender",
    "totalDownloads":156,
    "description":"render the timeline in the markdown",
    "url":"https://discourse.joplinapp.org/t/new-plugin-timeline/29408"
  },
  {
    "id":"net.noeto.JoplinBibleNotes",
    "totalDownloads":153,
    "description":"A Joplin plugin that helps find easily in which notes you spoke about a Bible reference.",
    "url":""
  },
  {
    "id":"de.habelt.NotesStationImport",
    "totalDownloads":150,
    "description":"This plugin imports notes from QNAP Notes Station",
    "url":"https://github.com/Mick2nd/Notes-Import-Joplin#readme"
  },
  {
    "id":"com.septemberhx.ReadCubePapers",
    "totalDownloads":140,
    "description":"Keep syncing with your ReadCube Papers library, help you create a note for one paper and render the paper note with a formatted paper information table, automatically generate markdown style reference, and allow you to insert your annotations into the note",
    "url":"https://github.com/SeptemberHX/joplin-plugin-readcube-papers.git"
  },
  {
    "id":"joplin-plugin-bidirectional-links",
    "totalDownloads":139,
    "description":"Create bidirectional to other notes",
    "url":"https://github.com/benlau/joplin-plugin-bidirectional-links"
  },
  {
    "id":"com.joshthor.export",
    "totalDownloads":127,
    "description":"Simple plugin to export your current note as an MD file to the directory of your choice",
    "url":"joshchavez.dev"
  },
  {
    "id":"app.nybble.joplin-plugin-wakatime",
    "totalDownloads":108,
    "description":"A Joplin plugin for WakaTime",
    "url":"https://github.com/uioporqwerty/joplin-plugin-wakatime"
  },
  {
    "id":"name.mikemueller.joplin2jira",
    "totalDownloads":93,
    "description":"Copies selected text or entire notes in Jira markup to the clipboard",
    "url":""
  },
  {
    "id":"com.cuibonobo.NoteRename",
    "totalDownloads":88,
    "description":"A utility for changing the titles of many notes at once via prefixes, suffixes, or search and replace.",
    "url":"https://github.com/cuibonobo/joplin-plugin-note-rename#readme"
  },
  {
    "id":"albert.joplin.plugin.jl7",
    "totalDownloads":79,
    "description":"Delete folder and notes without confirmation",
    "url":"https://github.com/pnphuong29/joplin-plugin-jl7-pub"
  },
  {
    "id":"ambrt.backlinksToNote",
    "totalDownloads":72
  },
  {
    "id":"com.gamalielbarboza.NoteVariables",
    "totalDownloads":68
  },
  {
    "id":"uk.co.brainsteam.JoplinHypothesis",
    "totalDownloads":59,
    "description":"Monitor and automatically add hypothesis annotations to your joplin notebooks",
    "url":"https://github.com/ravenscroftj/joplin-hypothesis"
  },
  {
    "id":"io.github.manuerwin.replace-resources",
    "totalDownloads":55
  },
  {
    "id":"albert.joplin.plugin7",
    "totalDownloads":52,
    "description":"Delete notes without confirmation",
    "url":"https://github.com/pnphuong29/joplin-plugin-jl7-pub7"
  },
  {
    "id":"MyPlugin",
    "totalDownloads":40
  },
  {
    "id":"org.joplin.bug-test",
    "totalDownloads":34
  },
  {
    "id":"com.github.paperless.file.importer",
    "totalDownloads":24
  },
  {
    "id":"io.github.makaanneo.joplin.paperless",
    "totalDownloads":21
  },
  {
    "id":"joplin-plugin-joplin-hotkey",
    "totalDownloads":17,
    "description":"hotkey in joplin",
    "url":"https://github.com/artikell/joplin-hotkey"
  },
  {
    "id":"hjonin.joplin.todoist-to-joplin",
    "totalDownloads":15,
    "description":"Import your Todoist data into Joplin",
    "url":"https://github.com/hjonin/todoist-to-joplin"
  },
  {
    "id":"com.export-to-noteson.shapkinaa",
    "totalDownloads":2,
    "description":"Export notes to NotesOn.",
    "url":"https://github.com/shapkinaa/noteson-joplin-plugin#readme"
  }
]

@palerdot
Copy link
Contributor

joplin-plugins-download-stats.txt
Downloadable JSON file for reference (.txt because of github limitations)

@laurent22
Copy link
Owner Author

Thanks, but how do I get to the plugin and get information about it? Can you help here?

What's your opinion about this list? Do you think we need to know about the plugin that's been downloaded twice? Maybe we do, but then you need to explain!

The task is to gather information and present it in a way that allows us to make a decision. A raw json list doesn't help much with this. It's sorted, which is better than nothing, but for many of these we don't even know what are these plugins since there's only an ID.

@palerdot
Copy link
Contributor

The list is updated with description/url.

@palerdot
Copy link
Contributor

plugin-stats.txt
File with downloads, id, url, description.

@mysticaltech
Copy link

That would be amazing! +1

@mak2002
Copy link
Contributor

mak2002 commented Aug 20, 2023

Hey there!! Happy to see this discussion. I would love to help here if @personalizedrefrigerator don't mind. Can I work on the Backup plugin (removing the binary component) part?

@personalizedrefrigerator
Copy link
Collaborator

Hey there!! Happy to see this discussion. I would love to help here if @personalizedrefrigerator don't mind. Can I work on the Backup plugin (removing the binary component) part?

Of course! I was thinking that an alternative to 7Zip might be to use the built-in JS CompressionStream API. For password protection, an option might be to use the window.crypto.subtle API.

@mak2002
Copy link
Contributor

mak2002 commented Aug 21, 2023

Sounds good! Will start working on it.

@personalizedrefrigerator
Copy link
Collaborator

personalizedrefrigerator commented Sep 20, 2023

It looks like joplin-desktop already depends on 7Zip. It might make sense to expose 7Zip to plugins in the same way that the plugin API exposes fs-extra, rather than modifying Simple Backup to work without 7Zip.

Edit: 7Zip is currently under optional dependencies, so it's unclear whether it is currently being bundled/signed for MacOS or not.

@laurent22
Copy link
Owner Author

I forgot why we have 7zip in there, maybe it is (or was) required by Electron-builder to build the Windows app?

@mak2002
Copy link
Contributor

mak2002 commented Oct 21, 2023

Hey there @personalizedrefrigerator, I think 7Zip is not being bundled for macOS as I checked on 2.12.18 (prod, darwin) by going into Joplin's Package Contents. So would it be better to modify Simple Backup plugin directly instead of bundling 7zip bin and then exposing it via API.

If so, I can go with either compression stream or Zlib as you suggested previously.

@personalizedrefrigerator
Copy link
Collaborator

personalizedrefrigerator commented Oct 21, 2023

Hey there @personalizedrefrigerator, I think 7Zip is not being bundled for macOS as I checked on 2.12.18 (prod, darwin) by going into Joplin's Package Contents. So would it be better to modify Simple Backup plugin directly instead of bundling 7zip bin and then exposing it via API.

If so, I can go with either compression stream or Zlib as you suggested previously.

Thank you for checking! I've re-opened the corresponding issue in the simple-backup plugin.

Note that when I run

> var node7z = require('7zip-bin-mac')
> await fsDriver.stat(node7z.path7za)

in Joplin's development tools (Joplin 2.12.19, MacOS 14.0/x86), I get

{
    "birthtime": "2023-10-21T14:06:01.671Z",
    "mtime": "2023-10-21T14:06:01.671Z",
    "path": "/Applications/Joplin.app/Contents/Resources/app.asar/node_modules/7zip-bin-mac/7za",
    "size": 1682640
}

So it looks like it's sometimes bundled with the application.

If so, I can go with either compression stream or Zlib as you suggested previously.

Because the 7zip functionality is currently optional in the simple backup plugin, it might make the most sense to start by supporting builds without 7zip (and preventing users from selecting these optional features).

@JackGruber Would you be open to a pull request that does this?

@mak2002
Copy link
Contributor

mak2002 commented Oct 21, 2023

Yep! my bad, I can also see the 7za executable on my macOS 14.0 (ARM). Thank you for looking into this!

@JackGruber
Copy link
Contributor

Because the 7zip functionality is currently optional in the simple backup plugin, it might make the most sense to start by supporting builds without 7zip (and preventing users from selecting these optional features).

I'm not excited about two plug-ins/two builds and I don't think I'd accept a PR for such a way.

@mak2002
Copy link
Contributor

mak2002 commented Oct 31, 2023

Okay, So now we have 2 options:

  1. Expose 7zip via API similar to fs-extra, sqlite
  2. Replace 7zip functions with some library in plugin's code

I like the idea of exposing it via API, this way very few changes will be required on plugin's side and will be helpful for future plugins as well. What are your thoughts?

@laurent22
Copy link
Owner Author

laurent22 commented Oct 31, 2023

That doesn't seem necessary to bundle 7zip with the app. If we want to expose a function to zip/unzip a file, we can probably do it using the build-in Node Zlib package: https://nodejs.org/api/zlib.html

Edit: In fact I see I've already implemented gunzipFile so I guess we just need gzipFile now.

@JackGruber
Copy link
Contributor

The zlib has no option to protect the archive with a password, or I have overseen the option.

@laurent22
Copy link
Owner Author

The zlib has no option to protect the archive with a password, or I have overseen the option.

I didn't realise there was this option. Node also include a crypto package so we could encrypt the data before gzipping it, and provide an option for this on the API

@laurent22
Copy link
Owner Author

It's on its way and should be available for download tomorrow. Hopefully that was a one-off 🤞

@jb261
Copy link

jb261 commented Nov 13, 2023

Maybe check the most downloaded plugins as as guide

That's one way to do it, but I think a lot of people download some of the 'exotic' plugins (Jarvis AI, graph/node plugins) just to check them out, but don't necessarily use them on a regular basis.

Maybe a multiple choice poll asking discourse forum users to select their top 5 'daily driver' plugins? I'd bet we'd see Favorites, Simple Backup, Note Tabs, and Outline in the cumulative top 5.

@laurent22
Copy link
Owner Author

Yes maybe, but polls are also not so reliable in general. Maybe a combination of both would work.

But I guess we'll work based on what seems obviously useful. For example the Backup plugin definitely is, and we'll see about others. It might end up being an ongoing discussion with the community rather than something based on hard numbers

@jb261
Copy link

jb261 commented Nov 13, 2023

Yes maybe, but polls are also not so reliable in general. Maybe a combination of both would work.

Good idea; that would weed out the 'false positives' generated by either method as a standalone screen.

@personalizedrefrigerator
Copy link
Collaborator

This is a summary of what, to my knowledge, needs to be done before this feature is ready for release.

1. Switch from NPM to custom-built versions of plugins

Currently, default plugins are downloaded from NPM.

This isn't ideal because .jpl files on NPM contain minified versions of the plugins (example) and are thus very difficult to review.

Because plugin authors can alter jpl files before uploading them to NPM, we either need to

  1. Review the minified code (hard), or
  2. Review the unminified code and dependencies, then build the jpl file ourselves. (Easier)

I propose we go with the second option. This can be done by:

  1. Creating a new package in the monorepo, desktop-default-plugins (or just default-plugins)
  2. Including copies of the default plugins as submodules
  3. Creating a script that runs in CI to build the plugins

2. Write a script to build the plugins

As another risk comes from malicious dependencies in a plugin's package.json (or perhaps malicious package resolutions in package-lock.json/yarn.lock) file. To make code review easier, I propose that the build script:

  1. Create a new build/ folder for each plugin
  2. Copy the plugin's src/ directory to the build/ folder
  3. Copy the default plugin build script (webpack.config.js) and package.json to build/
  4. Apply a patch to the contents of the build/ directory to add any additional dependencies to package.json (or modify the build script).

These extra steps make it harder to allow malicious dependencies run scripts during CI. This also allows us to build plugins like simple-backup that have dependencies that can't be bundled in a .jpl file.

Another thing we could do is build default plugins in a separate workflow (with more restricted permissions) and store the built JPL files as artifacts, then download these artifacts from the main workflow.

3. Modify bundleDefaultPlugins.ts to use the custom-built plugins

This might be as easy as copying the built JPL files from packages/default-plugins/ or, if we use a separate workflow for this, require using upload-artifact/download-artifact.

4. Create a pull request with several easy-to-bundle default plugins

For example, I expect Note Tabs to be easy to bundle.

5. (For simple-backup and possibly the outline plugin) Modify the list of require overrides to avoid bundling certain packages in the .jpl file

Simple Backup depends on 7Zip, which causes issues with code signing on MacOS. We already include 7Zip as a dependency for the main app. We can fix this by adding it to the list of packages we expose through require.

Optional/if including the outline plugin as one of the defaults: Similarly, the outline plugin includes markdown-it and katex as dependencies. As these are already dependencies of the main application, we should expose them to the plugin via require rather than bundling additional copies with the application.

@laurent22
Copy link
Owner Author

laurent22 commented Nov 21, 2023

another risk comes from malicious dependencies in a plugin's package.json

If we vendor the plugin, it means we've reviewed the code, and if we've done that we should also review the dependencies.

If a plugin requires a dependency that's not used by any other package, which means it's suspicious, we may decide not to bundle that plugin.

So i think this part 2 is not necessary.

@personalizedrefrigerator
Copy link
Collaborator

another risk comes from malicious dependencies in a plugin's package.json

If we vendor the plugin, it means we've reviewed the code, and if we've done that we should also review the dependencies.

If a plugin requires a dependency that's not used by any other package, which means it's suspicious, we may decide not to bundle that plugin.

So i think this part 2 is not necessary.

I suspect that part 2 will make bundling easier in some cases:

  • Some plugins install precommit hooks (e.g. freehand drawing, simple backup) that we don't want in the main repository.
    • We could fix this by copying the plugin into a temporary directory before running the install script (or disabling scripts on install).
    • Copying the plugin to a temporary directory before running npm install isn't that difficult.
  • Some plugins use very old versions of the plugin API that require special flags (e.g. export NODE_OPTIONS=--openssl-legacy-provider) that may complicate the build process
    • We could fix this by providing specific flags for different builds, forking plugins that use outdated versions of the plugin repository (some may be unmaintained), or patching the plugins.
  • Some plugins use yarn internally and some use npm. If we want to take advantage of yarn.lock or package-lock.json, we may need special logic to use either yarn or npm for different default plugins.
    • I think all of the plugins we've considered above use npm (so this point likely doesn't apply for now).

Of course, if we're vendoring the plugins anyway, we can update the plugin API ourselves in our version of the plugin repository.

@laurent22
Copy link
Owner Author

I would prefer a process that minimise our own maintenance work. The way I see it is that we check the dev repo and verify that the plugin doesn't do anything harmful, then in our repo we pull that commit and directly build it, without any patching.

I don't know how many default plugins we'll have but probably not that many, so those consideration about postinstall hooks, although sensible, might not apply.

Or, if they do, we can see at the time on a case by case basis. For example we can configure npm to build without running any postinstall script if needed, which is how we currently fetch the jpl files in the plugin repo.

@personalizedrefrigerator
Copy link
Collaborator

I'm currently working on bundling the note tabs plugin. By default, tabs are shown in a very large sidebar:
screenshot: very large note tabs sidebar

I don't think there's currently a way to set the default shape/position of a sidebar with the plugin API.

Here are some options:

  1. Update the default application layout to include the Note Tabs plugin's sidebar.
    • This solution is problematic: When the default plugin is installed for existing users of Joplin (i.e. users that aren't resetting the default layout), the Note Tabs sidebar will look as above (rather than use the new default).
  2. Extend the plugin API and patch/update the Note Tabs plugin to make use of this

@laurent22
Copy link
Owner Author

laurent22 commented Nov 22, 2023

Wait but for now we don't bundle the Tabs plugin. The one we want to bundle is the Backup one, and next one will probably be js-draw. Any other plugin will need to be discussed but that's for another time.

@personalizedrefrigerator
Copy link
Collaborator

personalizedrefrigerator commented Nov 22, 2023

Wait but for now we don't bundle the Tabs plugin. The one we want to bundle is the Backup one, and next one will probably be js-draw. Any other plugin will need to be discussed but that's for another time.

Got it!

Simple Backup is currently bundling successfully (branch with work-in-progress changes) on postinstall.

What still needs to be done:

  • For now, I'm still replacing package.json and other build scripts with alternate versions that exclude the 7zip dependency and configure Webpack to leave its import as a require. This should be converted to applying a patch to add this functionality.
  • Update the CI script to also clone the repository/checkout a commit
  • Mark default plugins as default (or recommended) in settings. Currently default plugins have no "recommended" or "default" badge in settings.
  • Other tasks (see below). Includes:
    • Auto-update plugins when Joplin updates.
    • For a default plugin, either: 1) don't show the Update button when a new NPM release is available (unless the non-default version is installed), or 2) change the behavior of Update.
      • Though maybe this behavior should be different?
    • Make sure that there's a way to install a non-default copy of the plugin. (Especially for development).

I plan to open a pull request after addressing item 1 in the above checklist.

@personalizedrefrigerator
Copy link
Collaborator

personalizedrefrigerator commented Nov 29, 2023

I'm running into an issue while implementing auto-update for default plugins (updating the plugin when Joplin updates). Below, I'll summarize the issue and two possible solutions. (Feedback is welcome!)

Currently, default plugins are copied on startup from the application bundle to the profile directory.

Diagram: io.github.jackgruber.backup.jpl is 500 KB in the app bundle, copied on startup to the profile directory

I'm currently working on auto-updating default plugins from the app bundle when a new version of Joplin is installed.

Currently, default plugins are installed before plugin manifests and thus plugin versions are loaded:

Diagram: Shows current behavior of copying based on a setting value, then later loading all plugins including versions

This makes it difficult to tell if the plugin stored in the app bundle is newer, older, or the same version while first installing default plugins. (Making auto-update difficult without unpacking plugins twice or refactoring).

Here are some potential solutions:

Solution 1: Update through the "Update" button in settings like other plugins

The "Update" button could be configured to copy a new version of the plugin from the app bundle. At this point, all plugins should be loaded.

This should (I think :) ) be easy to implement with the current code and require very little refactoring.

Solution 2: Load default plugins directly from the app bundle

PluginService's loadAndRunPlugins function can run plugins that are stored in other directories. The jpl file in the app bundle could be loaded, rather than creating a copy in the profile directory.

Diagram: Proposed behavior: Load and run plugins from both the app bundle and the profile directory, no copying

This has the benefit of automatically updating the plugin with the application, but the drawback of potential issues when downgrading the application (and thus attempting to downgrade the plugin). It also complicates updating/testing a newer version of the original plugin.

Note: Implementing a solution

It's difficult to test loading default plugins without having default plugins to load. As such, I propose either:

  1. Including this in Desktop: Resolves #7934: Add Simple Backup as a default plugin #9360, or
  2. Creating a follow-up pull request based on Desktop: Resolves #7934: Add Simple Backup as a default plugin #9360.

@personalizedrefrigerator
Copy link
Collaborator

Re-opening as update isn't handled yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
desktop All desktop platforms enhancement Feature requests and code enhancements plugins Anything related to Joplin's plugin system
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants