Skip to content

Commit

Permalink
Docs cleanup
Browse files Browse the repository at this point in the history
  • Loading branch information
Breck Yunits authored and Breck Yunits committed Apr 21, 2024
1 parent 5d5126d commit 5671e7c
Show file tree
Hide file tree
Showing 3 changed files with 36 additions and 32 deletions.
49 changes: 27 additions & 22 deletions blog/datasets.scroll
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ groups index all

endSnippet

Scroll Datasets are normal plain text blog posts written in Scroll that also contain structured data and output that data into forms ready for visualization and analysis tools.
Scroll Datasets are normal plain text blog posts written in Scroll that also contain structured data and output that data into formats ready for data visualization and analysis tools.
https://scroll.pub/ Scroll
match 1

Expand Down Expand Up @@ -89,10 +89,28 @@ code
- Measurements are done like this `appeared: 2024`

# FAQ
? What did you say the benefits were?
- Have an LLM do the bulk of the work while humans supervise to remove hallucinations.
- Can store everything (documentation, schema, all concepts) in 1 clean plain text file or split into many files (using the `import` keyword).
- The Scroll Dataset syntax balances _looseness_ useful in creative thinking with the _tightness_ needed by tabular data visualization and analysis tools.
? Isn't the better idea to enhance existing spreadsheet GUIs with LLM generation capabilities?
Almost certainly. Using Scroll for datasets will be much slower and worse than future spreadsheet apps with carefully crafted LLM integrations.

However, it's important to also have simple, lower tech, timeless tools and Scroll Datasets is one of those.

? Can't you do this same thing with YAML and/or Markdown?
Yes! You can easily achieve the same thing as LLMs & Scroll Datasets using LLMs & YAML, or LLMs & YAML & Markdown.
https://yaml.org/ YAML
https://github.github.com/gfm/ Markdown

For YAML, just put your documentation and schema in YAML comments up top and then have a tiny script to read that YAML and dump CSV/TSV/JSON or whatever. YAML gives you loads of data structures to use and is widely supported in many languages. But generating HTML from the same file would require more work.

If you want to intermix markup content with your datasets, you can use Markdown to add the marked up content and then have code sections embedding the YAML and a tiny script to parse out those YAML blocks and write your data to disk.

? So, why use Scroll for storing datasets instead of YAML?
Either can do the job. I expect the Scroll design to end up being more ergonomic, but that might not be true or may be unimportant.
// ergonomic: relating to or designed for efficiency and comfort in the working environment.

If you don't like Scroll's (evolving) version and want to switch it will always be straightforward to automatically refactor to YAML.

? What other related work is out there?
This is a simple pattern to implement, so I'm sure it is likely it has been done a few times before. Please let me know so I can include links to--and learn from--any other prior art.

? What are the advanced features?
- Supports nested measures^roadmap
Expand All @@ -113,23 +131,10 @@ Scroll Datasets evolved out of TrueBase. Scroll Datasets have eliminated the nee

Although Scroll Datasets are designed for a world with LLMs, the design is meant to be useful without them as well, and would also have been mildly useful 30 years ago.

? Can't you do this same thing with YAML and/or Markdown?
Yes! You can easily achieve the same thing as LLMs & Scroll Datasets using LLMs & YAML, or LLMs & YAML & Markdown.
https://yaml.org/ YAML
https://github.github.com/gfm/ Markdown

For YAML, just put your documentation and schema in YAML comments up top and then have a tiny script to read that YAML and dump CSV/TSV/JSON or whatever. YAML gives you loads of data structures to use and is widely supported in many languages. But generating HTML from the same file would require more work.

If you want to intermix markup content with your datasets, you can use Markdown to add the marked up content and then have code sections embedding the YAML and a tiny script to parse out those YAML blocks and write your data to disk.

? What other related work is out there?
This is a simple pattern to implement, so I'm sure it is likely it has been done a few times before. Please let me know so I can include links to--and learn from--any other prior art.

? So, why use Scroll for storing datasets instead of YAML?
Either can do the job. I expect the Scroll design to end up being more ergonomic, but that might not be true or may be unimportant.
// ergonomic: relating to or designed for efficiency and comfort in the working environment.

If you don't like Scroll's (evolving) version and want to switch it will always be straightforward to automatically refactor to YAML.
? What were the design goals?
- Have an LLM do the bulk of the work while humans supervise to remove hallucinations.
- Can store everything (documentation, schema, all concepts) in 1 clean plain text file or split into many files (using the `import` keyword).
- The Scroll Dataset syntax balances _looseness_ useful in creative thinking with the _tightness_ needed by tabular data visualization and analysis tools.

? Why are measures and concepts root-level features and not indented?
The normal way to implement this in Scroll would be something like:
Expand Down
1 change: 1 addition & 0 deletions blog/header.scroll
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@ importOnly
import settings.scroll
metaTags
gazetteCss
homeLink ../index.html
pageHeader
18 changes: 8 additions & 10 deletions readme.scroll
Original file line number Diff line number Diff line change
Expand Up @@ -21,21 +21,19 @@ Scroll is an npm package:
code
npm install -g scroll-cli

? What has changed in recent versions?
View the Scroll Release Notes.
link releaseNotes.html Scroll Release Notes

? Is Scroll open source?
Yes. Scroll is public domain and the source code is on GitHub.
https://github.com/breck7/scroll source code

? Is there a FAQ page?
Yes.
link faq.html Yes.

? Is there a tutorial?
Yes.
? Where are the docs?
- FAQ
link faq.html
- Tutorial
link tutorial.html
- Blog
link blog/index.html
- Release Notes
link releaseNotes.html

? Is there a place I can experiment with the Scroll grammar?
Yes. You can edit the Scroll Language in Tree Language Designer.
Expand Down

0 comments on commit 5671e7c

Please sign in to comment.