Skip to content

Implement on-demand source processing#31

Merged
Vladyslav-Kuksiuk merged 13 commits into
masterfrom
make-fragmentation-on-embedding
May 15, 2026
Merged

Implement on-demand source processing#31
Vladyslav-Kuksiuk merged 13 commits into
masterfrom
make-fragmentation-on-embedding

Conversation

@Vladyslav-Kuksiuk
Copy link
Copy Markdown
Collaborator

@Vladyslav-Kuksiuk Vladyslav-Kuksiuk commented May 13, 2026

This PR changes the source-processing flow from eager preprocessing to on-demand source resolution.

Changed source-processing flow

Previous flow:

  1. All source code files were scanned up front.
  2. Each source file was split into fragments, cleaned from fragment marker comments, normalized, and written into build/fragments.
  3. Target documents were scanned and embedded snippets were read from those prepared fragment files.

New flow:

  1. Target documents are scanned first.
  2. Source files are processed only when they are referenced by an <embed-code> instruction.
  3. Named fragments are resolved directly from source and cached in memory per source file.

Additional changes

  • Removed code-includes / code-excludes configuration, as it doesn't make sense anymore.
  • Removed fragments-path configuration.
  • Improved check mode outputs:
    image
    image

Issues

@Vladyslav-Kuksiuk Vladyslav-Kuksiuk self-assigned this May 13, 2026
@Vladyslav-Kuksiuk Vladyslav-Kuksiuk changed the title Make fragmentation in-memory Make fragmentation in demand May 13, 2026
@Vladyslav-Kuksiuk Vladyslav-Kuksiuk changed the title Make fragmentation in demand Make fragmentation on demand May 13, 2026
@Vladyslav-Kuksiuk Vladyslav-Kuksiuk changed the title Make fragmentation on demand Implement on-demand source processing May 13, 2026
Base automatically changed from improve-error-messages to master May 14, 2026 09:30
@Vladyslav-Kuksiuk Vladyslav-Kuksiuk marked this pull request as ready for review May 15, 2026 07:37
Copy link
Copy Markdown
Collaborator

@dmytro-kashcheiev dmytro-kashcheiev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Vladyslav-Kuksiuk Vladyslav-Kuksiuk merged commit 377e310 into master May 15, 2026
3 checks passed
@Vladyslav-Kuksiuk Vladyslav-Kuksiuk deleted the make-fragmentation-on-embedding branch May 15, 2026 14:12
@alexander-yevsyukov
Copy link
Copy Markdown
Contributor

@Vladyslav-Kuksiuk,

Improved check mode outputs:

All the errors come in the plain text flow. Can we, please, have each error on a separate line prefixed with -?

Please file an issue for this.

@dmytro-kashcheiev, FYI.

@alexander-yevsyukov
Copy link
Copy Markdown
Contributor

@Vladyslav-Kuksiuk,

Files outdated:

Please change this text to "Files to update:".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants