All files are rebuilt on save when watching with "codedoc serve"

[jamessimone]

I have 24 pages (including _toc.md, 404.md, and index.md), which doesn't feel like a lot, but it takes some time (not a ton on my desktop, but quite a bit on my laptop) on sequential, individual, file saves to rebuild the entire directory.

It would be awesome if only the files being saved were rebuilt!

[loreanvictor]

Makes sense, I suspect it should be relatively easy to keep track of the last build time in memory and then check last change time of each markdown file using fs.stat(), ignoring those not changed after.

Just out of curiosity, roughly how long does it take to rebuild all of those 24 pages?

[jamessimone]

love fs.stat() - it currently takes ~17 seconds per save on my Thinkpad with 8 gigs of RAM and an Intel i5.

It was really noticeable because I was going in to add the MetaOverride component into each file to fix the og:image issue we had been emailing about, but I have done these incremental changes a few times already in order to further "decorate" my files.

[loreanvictor]

wow thats a lot.

originally codedoc used to build all markdown files in parallel, which was pretty fast but also lead to memory issues (specifically on machines with lower RAM), so I had to change it. wonder if that should be a configurable thing (which of course would also require a better parallelization strategy than everything in parallel).

[loreanvictor]

this is deployed via @codedoc/core@0.2.0, in combination with @codedoc/cli@0.2.0. you can get the update and test by:

npm i -g @codedoc/cli@latesst
codedoc update latest

the update basically introduces tiered watching. main CODEDOC process now watches markdown files, while an external ts-node-dev process watches TypeScript files running the CODEDOC process.

• When a markdown file changes, it is queued, with a 1 second debounce time.
• All queued markdown files will be rebuilt after the debounce time.
• If the ToC is amongst the changed markdown files, all markdown files will be rebuilt.
• If new client-side components are used, the bundle will be repacked (since it needs to include additional code for new client-side components).
• If new components with additional styles are used, the stylesheets are rebuilt as well.
• If some files are changed while a rebuild is happening, they are queued for the next rebuild after current rebuild is finished.

[loreanvictor]

@jamessimone please check if with this update you get better reloading speeds while working on dev. the expectation naturally is drastically increased speeds, since in most cases only one or two markdown files should be rebuilt.

[jamessimone]

@loreanvictor I'm afraid that after upgrading, I'm getting a pretty crazy error while running codedoc serve:

# Serving ...
# Fetching project configuration ...
Using ts-node version 8.10.2, typescript version 3.9.5
# Serving docs on http://localhost:3000/joys-of-apex
# building ........ posts\404.md
# built:: ......... dist\404.html
# building ........ posts\apex-logging-service.md
# built:: ......... dist\apex-logging-service.html
# building ........ posts\apex-object-oriented-basics.md
# built:: ......... dist\apex-object-oriented-basics.html
# building ........ posts\batchable-and-queueable-apex.md
# built:: ......... dist\batchable-and-queueable-apex.html
# building ........ posts\building-a-better-singleton.md
# built:: ......... dist\building-a-better-singleton.html
# building ........ posts\continuous-integration-with-sfdx.md
# built:: ......... dist\continuous-integration-with-sfdx.html
# building ........ posts\dependency-injection-factory-pattern.md
# built:: ......... dist\dependency-injection-factory-pattern.html
# building ........ posts\enum-apex-class-gotchas.md
# built:: ......... dist\enum-apex-class-gotchas.html
# building ........ posts\extendable-apis.md
# built:: ......... dist\extendable-apis.html
# building ........ posts\future-method-callout-callback.md
# built:: ......... dist\future-method-callout-callback.html
# building ........ posts\idiomatic-salesforce-apex.md
# built:: ......... dist\idiomatic-salesforce-apex.html
# building ........ posts\index.md
# built:: ......... dist\index.html
# building ........ posts\intro.md
# built:: ......... dist\intro.html
# building ........ posts\lazy-iterators.md
# built:: ......... dist\lazy-iterators.html
# building ........ posts\lightweight-trigger-handler.md
# built:: ......... dist\lightweight-trigger-handler.html
# building ........ posts\lwc-composable-pagination.md
# built:: ......... dist\lwc-composable-pagination.html
# building ........ posts\mocking-dml.md
# built:: ......... dist\mocking-dml.html
# building ........ posts\picklist-validation.md
# built:: ......... dist\picklist-validation.html
# building ........ posts\react-versus-lightning-web-components.md
# built:: ......... dist\react-versus-lightning-web-components.html
# building ........ posts\repository-pattern.md
# built:: ......... dist\repository-pattern.html
# building ........ posts\sorting-and-performance-in-apex.md
# built:: ......... dist\sorting-and-performance-in-apex.html
# building ........ posts\test-driven-development-example.md
# built:: ......... dist\test-driven-development-example.html
# building ........ posts\writing-performant-apex-tests.md
# built:: ......... dist\writing-performant-apex-tests.html
# building ........ dist\styles\codedoc-styles.css
# built:: ......... dist\styles\codedoc-styles.css
# building ........ dist\bundle\codedoc-bundle.js
Hash: df486b832beb7a308534
Version: webpack 4.43.0
Time: 2861ms
Built at: 06/10/2020 2:46:27 PM
            Asset      Size  Chunks             Chunk Names
codedoc-bundle.js  2.25 MiB    main  [emitted]  main
Entrypoint main = codedoc-bundle.js
[./.codedoc/content/theme.ts] 2.78 KiB {main} [not cacheable] [built] [failed] [1 error]
[./.codedoc/node_modules/@codedoc/coding-blog-plugin/dist/es5/components/article-card/count-cards.js] 1 KiB {main} [built]
[./.codedoc/node_modules/@codedoc/coding-blog-plugin/dist/es5/components/article-card/index.js] 4.18 KiB {main} [built]
[./.codedoc/node_modules/@codedoc/coding-blog-plugin/dist/es5/components/author/index.js] 3.15 KiB {main} [built]
[./.codedoc/node_modules/@codedoc/core/dist/es5/components/code/line-hint/index.js] 1.98 KiB {main} [built]
[./.codedoc/node_modules/@codedoc/core/dist/es5/components/code/line-ref/index.js] 2.29 KiB {main} [built]
[./.codedoc/node_modules/@codedoc/core/dist/es5/components/code/same-line-length.js] 970 bytes {main} [built]
[./.codedoc/node_modules/@codedoc/core/dist/es5/components/code/selection.js] 5.33 KiB {main} [built]
[./.codedoc/node_modules/@codedoc/core/dist/es5/components/code/smart-copy.js] 1.68 KiB {main} [built]
[./.codedoc/node_modules/@codedoc/core/dist/es5/components/darkmode/index.js] 2.33 KiB {main} [built]
[./.codedoc/node_modules/@codedoc/core/dist/es5/components/heading/copy-headings.js] 1.38 KiB {main} [built]
[./.codedoc/node_modules/@codedoc/core/dist/es5/components/page/contentnav/highlight.js] 2.28 KiB {main} [built]
[./.codedoc/node_modules/@codedoc/core/dist/es5/components/page/toc/prevnext/index.js] 2.68 KiB {main} [built]
[./.codedoc/node_modules/@codedoc/core/dist/es5/components/page/toc/search/post-nav/index.js] 4.11 KiB {main} [built]
[./dist/bundle/codedoc-bundle.entry.js] 4.21 KiB {main} [built]
    + 589 hidden modules

ERROR in ./.codedoc/content/theme.ts
Module build failed (from ./.codedoc/node_modules/ts-loader/index.js):
Error: Cannot find module 'typescript'
Require stack:
- ~\.codedoc\node_modules\ts-loader\dist\utils.js
- ~\.codedoc\node_modules\ts-loader\dist\after-compile.js
- ~\.codedoc\node_modules\ts-loader\dist\instances.js
- ~\.codedoc\node_modules\ts-loader\dist\index.js
- ~\.codedoc\node_modules\ts-loader\index.js
- ~\.codedoc\node_modules\loader-runner\lib\loadLoader.js
- ~\.codedoc\node_modules\loader-runner\lib\LoaderRunner.js
- ~\.codedoc\node_modules\webpack\lib\NormalModule.js
- ~\.codedoc\node_modules\webpack\lib\NormalModuleFactory.js
- ~\.codedoc\node_modules\webpack\lib\Compiler.js
- ~\.codedoc\node_modules\webpack\lib\webpack.js
- ~\.codedoc\node_modules\@connectv\sdh\dist\es5\dynamic\webpack.js
- ~\.codedoc\node_modules\@connectv\sdh\dist\es5\dynamic\pack.js
- ~\.codedoc\node_modules\@connectv\sdh\dist\es5\dynamic\index.js
- ~\.codedoc\node_modules\@connectv\sdh\dist\es5\index.js
- ~\.codedoc\node_modules\@codedoc\core\dist\es5\build\index.js
- ~\.codedoc\node_modules\@codedoc\core\dist\es5\index.js
- ~\.codedoc\serve.ts
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:966:15)
    at Function.Module._load (internal/modules/cjs/loader.js:842:27)
    at Module.require (internal/modules/cjs/loader.js:1026:19)
    at require (internal/modules/cjs/helpers.js:72:18)
    at Object.<anonymous> (~\.codedoc\node_modules\ts-loader\dist\utils.js:6:20)
    at Module._compile (internal/modules/cjs/loader.js:1138:30)
    at Module._compile (~\AppData\Local\Yarn\Data\global\node_modules\source-map-support\source-map-support.js:547:25)
    at Module._extensions..js (internal/modules/cjs/loader.js:1158:10)
    at Object.nodeDevHook [as .js] (~\AppData\Local\Yarn\Data\global\node_modules\ts-node-dev\lib\hook.js:61:7)
    at Module.load (internal/modules/cjs/loader.js:986:32)
 @ ./dist/bundle/codedoc-bundle.entry.js 3:0-95 3:95-107
(node:2428) UnhandledPromiseRejectionWarning: [object Object]
(node:2428) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)  
(node:2428) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

[jamessimone]

@loreanvictor wow, just ignore that last comment. I think my npm / yarn versions lead to some crazy results. After reinstalling npm and re-running the commands, I am able to verify that this is working great for me. Nice job!

[loreanvictor]

I was worried for a moment that I have made a super crazy mistake somewhere. anyways thanks!