You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A streaming method stops at the caller's cancellation.ExtractBatchStreamAsync, the crawl and processing streams
and the crawler's stream read their results through a channel, and the channel's reader looks at the token only while it
waits. When the producer had run ahead, every result it had buffered still came after the caller cancelled (a caller that
cancelled at the fifth of 100 extractions received all 100), and the cancellation surfaced only after them. Now no
result follows the cancellation.
Documentation
docs/INTERFACES.md matches the library, and every C# example in docs/ is compiled by the tests.
The «Error Handling» example caught CrawlException, ExtractionException and ChunkingException, none of which
exists. It now shows WebExtractionException and its ErrorCode (ExtractErrorCodes), and says that cancellation
and batch extraction report failures differently.
The IEnrichedChunk and ISourceMetadata listings showed members the Flux.Abstractions interfaces do not have.
They are now copied from the interfaces, with their namespace.
The WebContentChunk listing now names every member it leaves out.
The ChunkingOptions listing showed Strategy as a string, an OverlapSize the type does not have, and two wrong
defaults. It now matches the class (ChunkingStrategyType Strategy, ChunkOverlap, and the semantic, header and
language settings). ICrawler.IsUrlAllowedAsync now shows its CancellationToken.
The type listings are now checked member by member against the real types.