Skip to content

v0.26.1

Latest

Choose a tag to compare

@github-actions github-actions released this 13 Oct 10:23

Fixed

  • A streaming method stops at the caller's cancellation. ExtractBatchStreamAsync, the crawl and processing streams
    and the crawler's stream read their results through a channel, and the channel's reader looks at the token only while it
    waits. When the producer had run ahead, every result it had buffered still came after the caller cancelled (a caller that
    cancelled at the fifth of 100 extractions received all 100), and the cancellation surfaced only after them. Now no
    result follows the cancellation.

Documentation

  • docs/INTERFACES.md matches the library, and every C# example in docs/ is compiled by the tests.
    • The «Error Handling» example caught CrawlException, ExtractionException and ChunkingException, none of which
      exists. It now shows WebExtractionException and its ErrorCode (ExtractErrorCodes), and says that cancellation
      and batch extraction report failures differently.
    • The IEnrichedChunk and ISourceMetadata listings showed members the Flux.Abstractions interfaces do not have.
      They are now copied from the interfaces, with their namespace.
    • The WebContentChunk listing now names every member it leaves out.
    • The ChunkingOptions listing showed Strategy as a string, an OverlapSize the type does not have, and two wrong
      defaults. It now matches the class (ChunkingStrategyType Strategy, ChunkOverlap, and the semantic, header and
      language settings). ICrawler.IsUrlAllowedAsync now shows its CancellationToken.
    • The type listings are now checked member by member against the real types.