Full-text-search via Stork

[applejag]

Closes #324

Tasks

• [ ] srid: Handle concurrent reads of /-/stork.st (live server is not production friendly)
• srid: Stork search unicode issue in title #336 (may have to note this as known limitation)
• Finalize search input design for both book and note themes
• .tpl files: refactor, reorganize, avoid DRY
• Write documentation under ./guide

Changes

• By @jilleJr:
  • Added Stork search component, with modal popup inspired by the one found on https://tailwindcss.com/docs
• By @srid:
  • Adds Nix dependency on the Stork Nix package
  • Emanote exports /-/stork.st in live- and statically generated site by running stork build

Preview

Demo: https://jillejr.github.io/emanote/

I even did the blurred backdrop thing from tailwindcss' website :P

For the narrow layout it's using only the 🔍 emoji when the sidebar is collapsed.

I was not really sure what to do with the neuron-like page, it currently steals a lot of the vertical view-space:

Old version (kept for just preserving history)

It's also sticky: (difficult to show in a screenshot, but try it out on e.g https://jillejr.github.io/emanote/resources/vim)

Also works on mobile layout:

I used that published GitHub Pages to do some manual testing on my phone.

In a separate branch I made small changes to build the index in the CI pipeline and publish it to that above GitHub Pages site.

• Changes: applejag/emanote@feature/stork-search...jilleJr:emanote:feature/stork-search-poc

[applejag]

<rant>

One thing to consider here is that this has a lot of impact on how Emanote sites are used, and I predict they will also have a great impact on how they are written as well.

On the negative side, this search feature could:

• render tags useless.
• remove importance on creating proper folgezettel links to connect your knowledge.

This isn't just an added positive feature, it also dulls down the other features of Emanote.

On the other hand, the positives are things like:

• user can search for related topics if they've forgotten the main subject, and then navigate via folgezettel links to the one they sought.
• speeds up usage when you know ahead-of-time what subject you're searching for, as you don't have to navigate through collapsed folders in the sidebar.

---

Do we want to counteract the negatives somehow? Or document the pros/cons of using search vs tags & uplink trees?

This could actually be a "killer feature" that makes users jump to writing docs in Emanote instead of Hugo or Jekyll. Most programs are after all only as strong as their search feature.

</rant>

[srid]

Note to self: in addition to the UI changes in this PR, stork index generation should be done in postRun.

          STORK_TOML=./-/stork.toml
          STORK_INDEX=./-/stork.st
          echo '[input]' > $STORK_TOML
          echo 'files = [' >> $STORK_TOML
          grep '<title>' -A1 -r --exclude-dir '-' --include '*.html' | grep html- | sed 's/\(.*.html\)-  *\(.*\)/  {path = "\1", url="\1", title = """\2"""},/' | tee -a $STORK_TOML
          echo ']' >> $STORK_TOML
          /opt/stork build --input $STORK_TOML --output $STORK_INDEX

We want to use emanote's model internally rather than parsing HTML files. Also add stork as nix dependency and use staticWhich.

[srid]

@jilleJr Thanks! I'll hook the rest of the machinery up, and see if we can have emanote gen automate it all.

[applejag]

Thanks! I'll hook the rest of the machinery up, and see if we can have emanote gen automate it all.

Thank you so much! I can create docs for this later (after this PR is merged, when Emanote's behavior on this is decided)

[applejag]

Latest changes:

• Added support for neuron-like note layout
• Moved all CSS & JS to snippets
• Moved searchbox to component instead of template hook, and make it visible via CSS snippet

Demo of neuron-like note layout with search box: https://jillejr.github.io/emanote/demo/neuron-layout.html

[applejag]

On second thought, this is now how I envision to enable search, where it's a 2-step process:

1. Add command flag or YAML config value to make emanote create the /-/stork.st index file
2. Add the <snippet var="js.stork-search" /> to your page.headHtml in index.yaml

[srid]

@jilleJr Give 12cf3a1 a try. The stork index is built behind the scenes in both live server and generate mode.

Note to self:

• Performance must be bad; so we want to cache the index in memory instead of rebuilding it on every request.
• Multiple layer (-L) scenario probably is broken here. Need to fix that.

[Bipodos]

Thank you, @jilleJr, for checking up my project. This seems, thus, to be an issue on Intel Macs. I shall see on M1 (may be only in a few days). UTF-8 is default for Mac, thus I had no "invalid characters", and missing links will disappear when I end moving the project from OpenProject to markdown/Obsidian/Emanote. I shall let you know once compile Emanote on a M1 Mac.

[applejag]

This seems, thus, to be an issue on Intel Macs.

This is though a sample size of 1, which means there are a lot more variables at play here. I know some coworkers has Macs, maybe intel ones. I'll ask if any of them are willing to test this out as well.

[Bipodos]

I suppose the issue is about Intel Macs because I have an extremely limited installation of Nix, only Emanote in it, and I have very little set in respect of variables etc. But we shall see. Thank you once more!

[srid]

@Bipodos I tried https://github.com/Bipodos/Plotinus on my M1 mac, but under rosetta emulation (passing --option system x86_64-linux to nix) - and I cannot reproduce your issue.

[srid]

Note: if anybody wants to test this feature, run the following after cd'ing to your notebook:

nix run github:jilleJr/emanote/feature/stork-search

[Bipodos]

> @Bipodos I tried https://github.com/Bipodos/Plotinus on my M1 mac, but under rosetta emulation (passing `--option system x86_64-linux` to nix) - and I cannot reproduce your issue. > >

After having updated nix, the stork search now works on my Intel Mac.
Last week, I also used the most recent version of nix. After waiting a few days, I gave it a new try. Your updated versions of the resulted in Emanote producing the same 0-byte stork index, until I updated nix. Now index works both in static and run modes.

[applejag]

@srid Thank you for applying the changes!