docs: complete rustdoc pass — crate docs, all public items, #[must_use] - #97
Merged
Conversation
- Crate-level overview in lib.rs covering the three layers (parser/fetchers/spiders) with a runnable Selector quickstart doctest plus no_run fetch and minimal-Spider examples, and a feature-highlight list (::text/::attr via css_get/css_getall, adaptive relocation, stealth headers, robots.txt, AutoThrottle, checkpoints, dev cache, CSV/JSON/JSONL/XML export). - Enabled #![warn(missing_docs)] and documented every public item across all 36 source files: modules, structs, enums (incl. error variants), traits and their provided methods, public fields, and functions — with real semantics (Selector::find_by_text deepest-match rules, Scheduler enqueue-time dedup, TextHandler::clean whitespace/entity behavior, AttributesHandler ordering, AutoThrottle reserve/record, fingerprint options, robots.txt RFC 9309 matching, etc.). - #[must_use] on builder types (FetcherConfigBuilder, SpiderRequestBuilder, CrawlSpider/SitemapSpider/ShopifySpider builders, LinkExtractor, CrawlRule) and their build() methods, and on pure query/accessor methods whose dropped results are almost certainly bugs (Selector css/css_get/css_getall/text/get_all_text/ find_by_* etc., TextHandler/TextHandlers/AttributesHandler/Selectors accessors, Response accessors, stats getters). - No runtime behavior changes: docs, attributes, and doc examples only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KDFsMaKk764vogjUW3nqpk
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Batch 6b (audit finding #8: docs.rs rendered nearly empty — 275 undocumented public items, zero
#[must_use], one line of crate docs).Selectorquickstart doctest,no_runfetch and Spider examples, feature highlights.#[warn(missing_docs)]enabled and every public item across all 36 source files documented — module docs, structs, enums (incl. all error variants), public fields, the fullSpidertrait with per-method defaults/units/engine interaction, every public function. Docs were written from the actual function bodies (e.g.find_by_textdeepest-match semantics,Schedulerfingerprint-at-enqueue dedup,TextHandler::cleanexact behavior, AutoThrottle reserve/record protocol, RFC 9309 robots matching).#[must_use]on the five builder types +LinkExtractor/CrawlRuleand on pure query methods (Selector/Selectors/TextHandler/AttributesHandler/Response accessors, …); deliberately not on setters, hooks, or I/O.Gates:
cargo test344 passed (incl. doctests),clippy --all-targets --all-features -D warningsclean,fmt --checkclean,cargo doc --no-deps0 warnings.https://claude.ai/code/session_01KDFsMaKk764vogjUW3nqpk
Generated by Claude Code