Fixed some typos found via Clojure-LosAngeles.

[bombaywalla]

Some minor documentation typos.
Address a part of issue #4

I was not able to find the commodities.txt (and other such) file in the repo to fix the ids.
My guess is that the files are not downloaded from nextjournal into this repo.

[deobald]

Thanks, @bombaywalla ! (Nice nick, btw.)

I'll have a look on Monday for the commodities.txt — it was probably missed on the export. You're right that git should definitely be the source of truth for it, not Nextjournal. :)

Redeploying the tutorial is, unfortunately, a bit of a hassle. (There are a lot of actions that require the Nextjournal GUI and can't be automated.) Do you have more commits coming? If you do, I might batch them all up before pushing them to Nextjournal.

[bombaywalla]

I do not have further commits for now. Other than the id changes to commodities.txt and other such files, which are currently not in the repo.

[deobald]

I'm researching this, but I'll take notes while things are in flight since I'm not sure I'll be able to resolve this today. I found this document, which describes the Nextjournal Markdown format:

https://nextjournal.com/nextjournal/markdown-format-reference

I've also added the .txt file in question to the repo (under ./files) but I added a link in the Markdown that points straight to the GitHub raw URL and that doesn't work, so we'll have to figure out how to get the file back into https://nextjournal.com/data unfortunately.

[deobald]

@bombaywalla It really looks like there's some way to import the sample data files along with the Markdown import. I just can't tell what it is. I've emailed the @nextjournal folks asking for help.

[bombaywalla]

Thanks for the update @deobald

There are two other files that would need to be downloaded as well.
Both are confusingly called example_data.txt but they are different. One is associated with Tutorial 3 (Datalog Queries) and the other with Tutorial 5 (Match).

[deobald]

@bombaywalla The Nextjournal folks got back to me and suggested we try an experimental "GitHub Explorer" feature:

Hi Steven,

In your scenario, accessing tutorial notebooks from our (experimental) github explorer https://nextjournal.com/github/xtdb/ might turn out useful for you (note I just noticed we have a routing problem and that should be accessible from https://github.nextjournal.com/xtdb and we need to fix that).

Anyway, once you visit -- say -- https://nextjournal.com/github/xtdb/xtdb-tutorial/blob/main/1-getting-started.nextjournal.md

then your notebook's "working directory" is at the repo's home (this is visible in the clojure runtime settings in the sidebar). In a clojure cell you might for instance (slurp "files/commodities.txt") and get back the files content...

There's caveat here, you'll need to also add a deps.edn in the root folder of the repo (or mount each in-notebook deps.edn at /xtdb-tutorial/deps.edn from within the runtime settings and export again) in order for the deps to be taken into account.

As an alternative, starting from a regular notebook you can achieve the same result by using a github component (https://nextjournal.com/help/github)

to mount the tutorial repo in your notebook: in this case you can access the files in the repo by traversing the runtime filesystem instead of arguing by uploads.

I'm torn. I like the idea of automating everything using the GitHub Explorer (I'm not sure if a GitHub Component can be added via Markdown) but I'm a bit nervous about using an experimental feature.

The alternative would be to store the .txt files in git and manually copy them into the notebooks while deploying, though... which is far from ideal. The deploy instructions in the README already have quite a few manual steps.

Perhaps if one (or both) of us can try out the GitHub Explorer — and if we're happy with it — we can go that route.

[bombaywalla]

I am not familiar with NextJournal (other than running the tutorials).

My understanding of the above is:

1. Github Explorer. Keep all files on github. NextJournal will directly use the notebooks/files from github (rather than notebooks/files from NextJournal). Not sure what happens when someone "remixes" the notebook on NextJournal? I guess that NextJournal makes a deep copy into NextJournal and then if anyone writes anything it is into the user's file space on NextJournal. Not sure about the comment on deps.edn. What should be in that deps.edn? The deps for all the tutorials in that repo? Can we not have a deps.edn at all?
2. Github Component. Somehow the github repo is manually "copied" (?) over to NextJournal and then used from NextJournal. Not sure if the repo is "copied" or just "mounted". My guess is "mounted".

Does the above match your understanding?

Yeah, I see that the README has a lot of manual steps for maintainers to re-install the notebooks. So, manually copying over the files, for now, might be okay.

Will try out Github Explorer in the next few days and report back.

[deobald]

@bombaywalla I got another reply from the @nextjournal folks that I'll paste here so it's public on this thread:

We just fixed our github subdomain, so now you can actually use https://github.nextjournal.com/xtdb (or also https://githubnj.com/xtdb :-)

Is there a way to automatically mount a GitHub component in Markdown?

A hack could work: add a github repo in a blank notebook on nextjournal and mount it in a (even bash) cell (list its content to ensure the mount work) and check how that's represented in terms of markdown references by exporting the notebook. Then try to rebuild that in your markdown source, you might need to hack the {details}-metadata at the bottom, not sure it will work if you strip that out altogether, but it could be worth a try.

Otherwise you start from a nextjournal notebook, just export it once and try to track in git the relevant changes from the first export...

[deobald]

Not sure what happens when someone "remixes" the notebook on NextJournal? I guess that NextJournal makes a deep copy into NextJournal and then if anyone writes anything it is into the user's file space on NextJournal.

Yup! That's pretty much it: a deep copy into the user's namespace. For the Learn Datalog tutorials we can cheat a little bit because there is a hidden URL at nextjournal.com/try which doesn't require "remixing" but allows the user to execute the notebook directly. This only works if there's one notebook on the account, though, and the Space Adventure tutorials have 8: https://github.com/xtdb/learn-xtdb-datalog-today/#quickstart

What should be in that deps.edn? The deps for all the tutorials in that repo? Can we not have a deps.edn at all?

At the moment, all the deps.edn files for the tutorials are identical so we could "share" a single deps.edn that's used by all the tutorials. The deps.edn is essential, since it's how Nextjournal knows how to mount Clojure and XTDB, so we can't really avoid having it.

Not sure if the repo is "copied" or just "mounted". My guess is "mounted".

My understanding is that it's "mounted" into the notebook... but I have a feeling that involves a local copy? I haven't tried the GitHub Component thing yet though, so your guess is probably as good as mine. :)

Yeah, I see that the README has a lot of manual steps for maintainers to re-install the notebooks. So, manually copying over the files, for now, might be okay.

Let me introduce @johantonelli — she might have some cycles to help with this and maybe (just maybe) we can find a more automated way to do deployments while we're at it. (Perhaps by poking the NJ team some more.) If we can, it would be great to avoid any additional manual steps.

I've broken this conversation out into some separate issues since I'm really abusing @bombaywalla 's PR here. 😜

#7 - automated deployments
#8 - keeping sample data in the repo