Link to "Getting Started" broken

[cjpoor]

At the end of "Installing the tools" the link to "Getting Started" is broken.
It leads to https://github.com/gs but should lead to
https://github.com/ZettelGeist/zettelgeist/wiki/Getting-Started

[icornelius]

Thank you, Christopher! I have fixed that now. Let us know when you find other problems.

[cjpoor]

Thanks for your reply. I have tried to create a new database with a new directory for import.
I even copied one of your Baseball files into the directory. However I get this error when I search for "Arizona":
(zenv) cjpoor@EliteDesk:~/Zettelgeist_Notes/zettels/Academic$ zfind --database mlb.db --query-string 'summary:"Arizona"' --show-filename --show-summary
Traceback (most recent call last):
  File "/home/cjpoor/zenv/bin/zfind", line 8, in
    sys.exit(main())
  File "/home/cjpoor/zenv/lib/python3.8/site-packages/zettelgeist/zfind.py", line 59, in main
    gen = db.fts_query(ast[0])
  File "/home/cjpoor/zenv/lib/python3.8/site-packages/zettelgeist/zdb.py", line 127, in fts_query
    return self.cursor.execute(prepared_sql)
sqlite3.OperationalError: no such table: zettels

Here is the directory structure I created, hoping to replicate your example:
/home/cjpoor/Zettelgeist_Notes/zettels/Academic
the zettels, including arizona-diamondbacks.yaml are there and the Academic.db also.
Do you have any suggestions?
Many thanks,Chris Poor.

[icornelius]

Hm, have you tried rebuilding the database? Our preferred command for that is

zimport --create --database index.db --dir . --fullpath

(In your case replace index.db with Academic.db.) This command combines the functions of zcreate and zimport described in the current documentation. It should be run within the directory that contains your zettels.

[cjpoor]

This command returned the following:

(zenv) cjpoor@EliteDesk:~/Zettelgeist_Notes/zettels/Academic$ zimport --create --database Academic.db --dir . --/home/cjpoor/Zettelgeist_Notes/zettels/Academic
usage: zimport [-h] --database DATABASE --dir DIR [--fullpath] [--validate]
zimport: error: unrecognized arguments: --create --/home/cjpoor/Zettelgeist_Notes/zettels/Academic

[cjpoor]

I ran the two commands separately and succeeded. I am sorry but I realize that my original query was on the wrong database.
The output I now have is

(zenv) cjpoor@EliteDesk:~/Zettelgeist_Notes/zettels/Academic$ zfind --database Academic.db --query-string 'summary:"Arizona"' --show-filename --show-summary
/home/cjpoor/zenv/lib/python3.8/site-packages/zettelgeist/zettel.py:457: YAMLLoadWarning: calling yaml.load_all() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  for ydoc in self.ydocs:
summary: Arizona Diamondbacks

filename:
/home/cjpoor/Zettelgeist_Notes/zettels/Academic/arizona-diamondbacks.yaml

---

[cjpoor]

So now my problem is not being able to import my own zettels. I have attached a simplified example (it is a copy of the .yaml file since github discussion does not accept files of type yaml) to see if you can tell my why "YAML loaded but could not be processed"
benes_language_2006.txt

[icornelius]

(1/2) The YAML load warning is something I have seen before, I think, but not on the current release. Are you running the current version of ZettelGeist (1.1.1)? The command

pip uninstall zettelgeist

should print the current installed version and ask you whether you want to uninstall. If you are not running 1.1.1, type y, then

pip install zettelgeist

to update to the current version.

[icornelius]

(2/) In the zimport command, the flag --fullpath should be passed verbatim (not replaced with your file path). This flag instructs zimport to log the full file path for each zettel. It is not obligatory, but useful for performing other operations on zettels later in the game.

So, once you have ZettelGeist 1.1.1 installed, the zimport command should be

zimport --create --database Academic.db --dir . --fullpath

[icornelius]

(3/) in reply to your sample zettel (with apologies for out-of-order posts). The problems there are generated by a few errors in yaml format.

1. Multi-line strings in note (and any other field) need to be indented
2. The value of year needs to be string, not integer. I'm not sure why (@gkthiruvathukal might have insight on that) but the error is avoided if you quote value as shown in this example.

I have made the above changes in the following text and it imports successfully.

title: Nietzsche on language
summary: Nietzsche on language Nietzsche on the structural constraint and determination of language and its capacity for being subverted.
note: |
  "'For Nietzsche, however, recognizing the autonomy and power of language was not cause for concern. Human subjects could be liberated from the artificial, self-imposed constraints of metaphysics and conceptual thought. The subject gained new creative freedom and the ability to express his or her own desires using a language capable of subversion and deviance. The historicization of language may have threatened the traditional humanist idealization of autonomous subjects. But it opened the way for a poststructuralist celebration of contingency and linguistic relativity.'" ([Benes 2006:228](zotero://open-pdf/0_KQQ6UPCE/11)) - Compare this to Marx's argument for the working class, forced into its situation by capitalism, to overthrow this structure. See Chibber, 2013
comment: This requires the subject to grasp this freedom ([note on p.228](zotero://open-pdf/0_KQQ6UPCE/11))
tags: 
- Nietzsche
- language
- autonomy
dates:
 year: "2006"
cite:
 bibkey: benesLanguageCognitiveSubject2006
 page: p.228

[cjpoor]

Upgraded to 1.1.1 thanks. No warning now.
Thanks for YAML points. That file now works.
I will change the rest of them and see how they go.
Cheers.

[cjpoor]

Apparently, the zettel file cannot have the fields "author" or "authors"

Invalid field author found in Zettel

[cjpoor]

I guess the template is not designed for annotations

[cjpoor]

I checked the note format: I guess "bibkey" would be in the place of "author" :-)

[gkthiruvathukal]

@cjpoor Glad to see you are up and running! Please do keep us informed!
And thanks to @icornelius for helping our user community! 👍🏾

[cjpoor]

Thanks for the nifty app!

[gkthiruvathukal]

@cjpoor We're definitely interested to hear how you use the tools. I would be interested to hear more about what you meant by annotation. If it's just a citation, you can use the bibkey field and "define" it later.

When the note: field doesn't give you what you need, you can just use YAML as frontmatter and have a proper Markdown "document" after the frontmatter. Then you can use whatever is defined in Markdown or Pandoc's Markdown.

You can read about this on the Wiki.

[cjpoor]

@gkthiruvathukal I've been testing a number of note-taking tools for some time.
I used org-mode for a few years but then decided to use Zotero since it is where I keep my readings i.e. pdfs and their bibliographic info, annotations and notes made from annotations plus standalone notes.
Until Zotero implemented the integrated pdf reader I used Skim on Mac and more recently Okular on Linux to annotate pdfs. The highlighted text could be extracted into a Zotero note by zotxt. This is now done within the updated Zotero beta with its own pdf reader and note editor which can now be used to collect notes into a draft.
Much as I appreciate the power these new features bring to Zotero I also want to keep a backup of my notes in plain text for future-proofing and so as not to be locked in to Zotero. Once I finish my current dissertation I will quite likely go back to using org-mode and may use it for notes, but I wanted to check out other options.
Your Zettelgeist is appealing in that it allows the notes to be stored as plain text but uses the power of sqlite to perform queries.
One thing that would make it even more appealing would be a way of easily exporting notes from Zotero into the Zettelgeist format. No doubt this could be done with a script (bibtex>yaml) but I'm just a newbie so far and am not up to writing such a script yet. At present I use the Zotero mdnotes plugin to export notes from Zotero to plain text but these then need to manually formatted as zettels.
Since I am new to YAML I am a bit confused by the different formats that appear to be out there for using it. Is that just part of its flexibility?
Also, I am curious as to whether it is possible to add fields to Zettelgeist's zettel format. Although I recognize that keeping it as simple as practicable is best.

[cjpoor]

I did not understand what you meant by "define" here:
If it's just a citation, you can use the bibkey field and "define" it later.
Could you clarify?

[icornelius]

I use the Better BibTeX extension for Zotero to generate the values that I input into the ZettelGeist field bibkey. Since each bibkey value is defined within Zotero ("defined" = "coordinated with a full unique bibliographic record") I do not need to repeat any of the standard bibliographic fields (author name, publication date, etc.) in my zettels. The principle is the same as that behind citations in LaTeX and Markdown.

[gkthiruvathukal]

@cjpoor, @icornelius has done a great job of explaining the bibkey vs. bibtex here. Just to explain a bit about this, when @dbdennis and I were designing the Zettel format, the thought was to have a system where you could store all of the bibliographic data. We still think there is merit to this, but as @icornelius noted, some will prefer to keep their bibliography on a system like Zotero, which allows for full BibTeX export. @dbdennis and I also keep our bibliographic info in Zotero but can turn all bibliographic data into simple Zetteln, where each Zettel only contains bibkey and bibtex. Other Zetteln simply cite any bibkey as needed.

Our companion effort is to be able to drive writing from a collection of Zetteln (the plural for Zettels) and unite any citations with the full bibliography. You can see how this works in our zg-tutorial, where we show how to create documents from a ZettelGeist database.

To your other question about whether other fields can be added, the short answer is "yes". However, as all three of us have lived with using a subset of the various Zettel fields, we're increasingly of the belief that less is more. That said, we don't plan to eliminate any fields, since some users may find them useful for their workflow.

In an earlier incarnation of this work, I was looking at how to support a completely ad hoc note format. It would be driven entirely by user-defined fields. There would be standard fields as in "today's" Zettel format but you could also include any additional data you wish. I think it would be awesome but would also result in people spending too much time thinking about how to "structure" Zetteln. We prefer the notion of "one note per idea" and then performing queries to bring ideas together.

As for YAML, this too, was driven primarily by ease-of-use. Now that we support arbitrary Markdown as "the rest of the Zettel", we probably could allow just about any format, e.g. JSON or XML, as the frontmatter. After working with other humanists such as @dbdennis and @icornelius, I'm convinced that YAML is a great choice for writers/scholars, who may well understand more complex formats but would prefer to spend their time on writing instead of programming their notes. Now that I think of it, I'd rather spend most of my time writing, too.