-
Notifications
You must be signed in to change notification settings - Fork 13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[epic] MarkdownDB v0.1 #6
Comments
Update 2023-03-08:Pick a basic sqlite libraryThere seems to be no point in using the standard SQLite3 lib considering the features that come with But between
Note that we can also use PR with initial implementation + testI raised a PR with the initial implementation and a unit test for the lib file: https://github.com/datopian/datahub-next/pull/36 Summary of the changes:
Here are the results if I fetch and log the "files" table after the execution of the unit test: [
{
_id: 'e1a3a07bd9ba8cacba586a16356ebecb98df4c23',
_path: '__tests__/fixtures/markdowndb/blog/blog1.mdx',
frontmatter: '{"title":"My Test Mdx Blog 1"}',
filetype: 'mdx',
type: null
},
{
_id: '61540419fb3911ea43fdf2fdc2f2635450f330a9',
_path: '__tests__/fixtures/markdowndb/blog/blog2.mdx',
frontmatter: '{"title":"My Test Mdx Blog 2"}',
filetype: 'mdx',
type: null
},
{
_id: 'f01ca5bd5ef693fab40f331536e2dd5485414136',
_path: '__tests__/fixtures/markdowndb/index.mdx',
frontmatter: '{"title":"Homepage"}',
filetype: 'mdx',
type: null
}
] |
@demenech want to acknowledge this exceptional quality comment. Also very clean PR. |
Update 2023-03-09PR to include initial tags and querying supportPR: https://github.com/datopian/datahub-next/pull/39 Changes:
TypesI'm not sure how we want to handle types, since having it as a frontmatter field might not be the most ideal way because if we had a blog folder we'd have to add the type metadata to all the files individually. On const Blog = defineDocumentType(() => ({
name: "Blog",
filePathPattern: `${siteConfig.blogDir}/!(index)*.md*`,
contentType: "mdx",
fields: {
... I believe that's a good way of handling this. The caveat is that the path of a file is now determining its type and therefore folders with mixed types are impossible, although we could apply the pattern as something like The use case I'm imaging is something like (there are probably better examples than blog):
Idea: How could we index frontmatter into our db?My idea is to have another table for frontmatter, something like:
To increase performance, since we are going to have many more rows now, we can create a DB index on this table (using the file_id field) If done this way we are going to be able to query mdx files using frontmatter fields. E.g: (may not be exactly this) MyMdDb.query({ tags: [economy], frontmatter: { author: 'João' } }) TODO: update ticket description tomorrow |
Waiting for this PR https://github.com/datopian/datahub-next/pull/39 to be reviewed before closing |
Spike solution of an index of our markdown files so that I can quickly access the metadata and content I want.
See parent epic for details: #3
Acceptance
lib/markdowndb.js
Tasks
Design
API sketch
Minimal viable API
lib/db.ts
What's the db schema?
The text was updated successfully, but these errors were encountered: