Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 42 additions & 31 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,36 @@ Welcome to the monorepo for [Discourse Graphs](https://discoursegraphs.com). Dis

## Local development

### Turborepo

This repository uses [Turborepo](https://turbo.build/repo/docs) as a build system, enabling a streamlined and efficient workflow for managing multiple applications and shared packages in a monorepo setup.

Using Turborepo allows for things like:

- **Centralize shared resources**: Shared configurations, utilities, and components are maintained in a single place, reducing duplication and inconsistency.
- **Incremental builds**: Only changes in code are rebuilt, which speeds up development.
- **Parallel processing**: Tasks across applications and packages run concurrently, saving time.
- **Dependency graph management**: Turborepo tracks relationships between projects, ensuring that tasks run in the correct order.

Learn more about how monorepos improve development workflows [here](https://vercel.com/blog/monorepos) and [here](https://github.com/babel/babel/blob/master/doc/design/monorepo.md).

### Apps & Packages

`apps`

- [website](https://github.com/DiscourseGraphs/discourse-graph/tree/main/apps/website): The public-facing website for Discourse Graphs, available at [discoursegraphs.com](https://discoursegraphs.com). Uses Next.js.
- [roam](https://github.com/DiscourseGraphs/discourse-graph/tree/main/apps/roam): The Roam Research extension that implements the Discourse Graph protocol.
- [obsidian](https://github.com/DiscourseGraphs/discourse-graph/tree/main/apps/obsidian): The Obsidian plugin that implements the Discourse Graph protocol.

`packages`

- [tailwind-config](https://github.com/DiscourseGraphs/discourse-graph/tree/main/packages/tailwind-config): Shared tailwind config
- [typescript-config](https://github.com/DiscourseGraphs/discourse-graph/tree/main/packages/typescript-config): Shared tsconfig.jsons
- [eslint-config](https://github.com/DiscourseGraphs/discourse-graph/tree/main/packages/eslint-config): ESLint preset
- [ui](https://github.com/DiscourseGraphs/discourse-graph/tree/main/packages/ui): Core React components

### Getting Started

To get started with local development:

1. Clone the repository:
Expand All @@ -23,45 +53,26 @@ npm install
turbo dev
```

You can use the `--filter` flag to run a single application:
You can use the `--filter` flag to run a single application, eg:

```bash
turbo dev --filter roam
```

### Turborepo

This repository uses [Turborepo](https://turbo.build/repo/docs) as a build system, enabling a streamlined and efficient workflow for managing multiple applications and shared packages in a monorepo setup.

Using Turborepo allows for things like:

- Centralize shared resources: Shared configurations, utilities, and components are maintained in a single place, reducing duplication and inconsistency.
- Incremental builds: Only changes in code are rebuilt, which speeds up development.
- Parallel processing: Tasks across applications and packages run concurrently, saving time.
- Dependency graph management: Turborepo tracks relationships between projects, ensuring that tasks run in the correct order.

Learn more about how monorepos improve development workflows [here](https://vercel.com/blog/monorepos) and [here](https://github.com/babel/babel/blob/master/doc/design/monorepo.md).

### Apps & Packages

This Turborepo includes the following packages and applications:
#### Roam

`apps`

- [website](https://github.com/DiscourseGraphs/discourse-graph/tree/main/apps/website): The public-facing website for Discourse Graphs, available at [discoursegraphs.com](https://discoursegraphs.com). Uses Next.js.
- [roam](https://github.com/DiscourseGraphs/discourse-graph/tree/main/apps/roam): The Roam Research extension that implements the Discourse Graph protocol.

`packages`

- [tailwind-config](https://github.com/DiscourseGraphs/discourse-graph/tree/main/packages/tailwind-config): Shared tailwind config
- [typescript-config](https://github.com/DiscourseGraphs/discourse-graph/tree/main/packages/typescript-config): Shared tsconfig.jsons
- [eslint-config](https://github.com/DiscourseGraphs/discourse-graph/tree/main/packages/eslint-config): ESLint preset
- [ui](https://github.com/DiscourseGraphs/discourse-graph/tree/main/packages/ui): Core React components
- go to your graph, open up settings, and go to the extensions tab
- click "Enable developer mode (the settings cog icon)
- click "Load extension"
- and choose the `dist` folder on your computer which is in the `discourse-graph/apps/roam` directory
- you can set a hotkey to `Reload developer extension`

### Deployment
#### Obsidian

- The Next.js website is automatically deployed to Vercel.
- The Roam Discourse Graph extension is manually deployed to Vercel blob storage using `npm run deploy`. (this will be automated in the future)
- copy the `.env.example` file to `.env`
- fill in the `OBSIDIAN_PLUGIN_PATH` with the path to your Obsidian plugins folder
- run `turbo dev --filter @discourse-graphs/obsidian`
- install the [Plugin Reloader](https://obsidian.md/plugins?id=plugin-reloader) or [BRAT](https://obsidian.md/plugins?id=obsidian42-brat)/[Hot Reload](https://github.com/pjeby/hot-reload) to reload the plugin after changes

## Contributing

Expand Down
1 change: 1 addition & 0 deletions apps/obsidian/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# OBSIDIAN_PLUGIN_PATH="path/to/your/obsidian/plugins/folder"
5 changes: 5 additions & 0 deletions apps/obsidian/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Discourse Graphs

The Discourse Graph extension enables Roam users to seamlessly add additional semantic structure to their notes, including specified page types and link types that model scientific discourse, to enable more complex and structured knowledge synthesis work, such as a complex interdisciplinary literature review, and enhanced collaboration with others on this work.

For more information about Discourse Graphs, check out our website at [https://discoursegraphs.com](https://discoursegraphs.com)
10 changes: 10 additions & 0 deletions apps/obsidian/manifest.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"id": "@discourse-graph/obsidian",
"name": "Discourse Graph",
"version": "0.1.0",
"minAppVersion": "1.7.0",
"description": "Discourse Graph Plugin for Obsidian",
"author": "Discourse Graphs",
"authorUrl": "https://discoursegraphs.com",
"isDesktopOnly": false
}
29 changes: 29 additions & 0 deletions apps/obsidian/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
{
"name": "@discourse-graphs/obsidian",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't just use obsidian because the npm obsidian package is in our dependencies sad face. will probably namespace all of the plugins this way

"version": "0.1.0",
"description": "Discourse Graph Plugin for obsidian.md",
"main": "dist/main.js",
"private": true,
"scripts": {
"dev": "tsx scripts/dev.ts",
"build": "tsx scripts/build.ts"
},
"keywords": [],
"author": "",
"license": "MIT",
"devDependencies": {
"@types/node": "^16.11.6",
"@typescript-eslint/eslint-plugin": "5.29.0",
"@typescript-eslint/parser": "5.29.0",
"builtin-modules": "3.3.0",
"esbuild": "0.17.3",
"obsidian": "^1.7.2",
"tslib": "2.4.0",
"tsx": "^4.19.2",
"typescript": "4.7.4"
},
"dependencies": {
"react": "^19.0.0",
"react-dom": "^19.0.0"
}
}
27 changes: 27 additions & 0 deletions apps/obsidian/scripts/build.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
import { compile } from "./compile";
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will generalize all of the scripts/ to work for all apps (as best we can) later

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


const build = async () => {
process.env = {
...process.env,
NODE_ENV: process.env.NODE_ENV || "production",
};

console.log("Compiling ...");
try {
await compile({});
console.log("Compiling complete");
} catch (error) {
console.error("Build failed on compile:", error);
process.exit(1);
}
};

const main = async () => {
try {
await build();
} catch (error) {
console.error(error);
process.exit(1);
}
};
if (require.main === module) main();
180 changes: 180 additions & 0 deletions apps/obsidian/scripts/compile.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
import esbuild from "esbuild";
import fs from "fs";
import path from "path";
import { z } from "zod";
import builtins from "builtin-modules";
import dotenv from "dotenv";

dotenv.config();

const DEFAULT_FILES_INCLUDED = ["manifest.json"];
const isProd = process.env.NODE_ENV === "production";

const cliArgs = z.object({
out: z.string().optional(),
root: z.string().optional(),
format: z.enum(["esm", "cjs"]).optional(),
external: z.array(z.string()),
mirror: z.string().optional(),
});

type Builder = (opts: esbuild.BuildOptions) => Promise<void>;
export type CliOpts = Record<string, string | string[] | boolean>;

export const args = {
out: "main",
format: "cjs",
root: ".",
mirror: process.env.OBSIDIAN_PLUGIN_PATH,
external: [
"obsidian",
"electron",
"@codemirror/autocomplete",
"@codemirror/collab",
"@codemirror/commands",
"@codemirror/language",
"@codemirror/lint",
"@codemirror/search",
"@codemirror/state",
"@codemirror/view",
"@lezer/common",
"@lezer/highlight",
"@lezer/lr",
...builtins,
],
} as CliOpts;

const readDir = (directoryPath: string): string[] => {
try {
if (!fs.existsSync(directoryPath)) {
console.error(`Directory does not exist: ${directoryPath}`);
return [];
}

return fs
.readdirSync(directoryPath, { withFileTypes: true })
.flatMap((f) => {
const fullPath = `${directoryPath}/${f.name}`;
return f.isDirectory() ? readDir(fullPath) : [fullPath];
});
} catch (error) {
console.error(`Error reading directory ${directoryPath}:`, error);
return [];
}
};

const appPath = (p: string): string =>
path.resolve(fs.realpathSync(process.cwd()), p);

export const compile = ({
opts = args,
builder = async (opts) => {
await esbuild.build(opts);
},
}: {
opts?: CliOpts;
builder?: Builder;
}) => {
const { root = ".", out, format, external, mirror } = cliArgs.parse(opts);

const srcRoot = path.join(root, "src");
const entryTs = "index.ts";
const outdir = path.resolve(process.cwd(), root, "dist");
const stylesDir = path.join(root, "src", "styles");
const outputStylesFile = path.join(outdir, "styles.css");

Comment on lines +83 to +85
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Gracefully handle missing styles directory.
This script expects a styles directory to exist. If the directory is absent, an error occurs when reading style files. Consider adding a guard check or fallback behavior when stylesDir is missing to avoid build-time exceptions.

fs.mkdirSync(outdir, { recursive: true });

const buildPromises = [] as Promise<void>[];
buildPromises.push(
builder({
absWorkingDir: process.cwd(),
entryPoints: [path.join(srcRoot, entryTs)],
outdir,
bundle: true,
format,
sourcemap: isProd ? undefined : "inline",
minify: isProd,
entryNames: out,
external: external,
plugins: [
{
name: "log",
setup: (build) => {
build.onEnd((result) => {
console.log(`built with ${result.errors.length} errors`);
});
},
},
{
name: "combineStyles",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

setup(build) {
build.onEnd(async () => {
const styleFiles = fs
.readdirSync(stylesDir)
.filter((file) => file.endsWith(".css"));
const combinedStyles = styleFiles
.map((file) =>
fs.readFileSync(path.join(stylesDir, file), "utf8"),
)
.join("\n");
fs.writeFileSync(outputStylesFile, combinedStyles);
});
},
},
{
name: "copyDefaultFiles",
setup(build) {
build.onEnd(async () => {
DEFAULT_FILES_INCLUDED.map((f) => path.join(root, f))
.filter((f) => fs.existsSync(f))
.forEach((f) => {
fs.cpSync(f, path.join(outdir, path.basename(f)));
});
});
},
},
{
name: "mirrorFiles",
setup(build) {
build.onEnd(async () => {
if (!mirror) return;

const normalizedMirrorPath = path.normalize(mirror);
const resolvedMirrorPath = path.resolve(
root,
normalizedMirrorPath,
);

if (!fs.existsSync(resolvedMirrorPath)) {
fs.mkdirSync(resolvedMirrorPath, { recursive: true });
}

readDir(outdir)
.filter((file) => fs.existsSync(appPath(file)))
.forEach((file) => {
const destinationPath = path.join(
resolvedMirrorPath,
path.relative(outdir, file),
);
fs.cpSync(appPath(file), destinationPath);
});
});
},
},
],
}),
);

return Promise.all(buildPromises);
};

const main = async () => {
try {
await compile({});
} catch (error) {
console.error(error);
process.exit(1);
}
};
if (require.main === module) main();
37 changes: 37 additions & 0 deletions apps/obsidian/scripts/dev.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
import esbuild from "esbuild";
import dotenv from "dotenv";
import { compile, args } from "./compile";

dotenv.config();

const dev = () => {
process.env.NODE_ENV = process.env.NODE_ENV || "development";
return new Promise<number>((resolve) => {
compile({
opts: args,
builder: (opts: esbuild.BuildOptions) =>
esbuild.context(opts).then((esb) => {
esb.watch();
// Cleanup on process termination
const cleanup = () => {
esb.dispose();
resolve(0);
};
process.on('SIGINT', cleanup);
process.on('SIGTERM', cleanup);
return esb;
}),
});
process.on("exit", resolve);
});
};

const main = async () => {
try {
await dev();
} catch (error) {
console.error(error);
process.exit(1);
}
};
if (require.main === module) main();
Loading
Loading