improve docs, rework exports

[PSeitz]

Add top level architecture overview, so new users can orient themselves
Move top level exports so docs are easier to read.

• move snippet and advice
• make indexer pub, remove indexer reexports
• add top level architecture overview

Next steps would be to move segment and index out of core

[codecov-commenter]

Codecov Report

All modified lines are covered by tests ✅

Comparison is base (ecb9a89) 94.03% compared to head (ff05b66) 94.02%.
Report is 1 commits behind head on main.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2220      +/-   ##
==========================================
- Coverage   94.03%   94.02%   -0.02%     
==========================================
  Files         327      327              
  Lines       65156    65156              
==========================================
- Hits        61271    61262       -9     
- Misses       3885     3894       +9     

Files	Coverage Δ
src/core/index.rs	87.53% <ø> (ø)
src/directory/mmap_directory.rs	86.27% <ø> (ø)
src/indexer/merge_operation.rs	100.00% <ø> (ø)
src/indexer/mod.rs	100.00% <ø> (ø)
src/indexer/segment_writer.rs	97.82% <ø> (ø)
src/lib.rs	99.05% <ø> (ø)
src/snippet/mod.rs	95.29% <100.00%> (ø)
src/termdict/mod.rs	71.83% <ø> (ø)
tokenizer-api/src/lib.rs	100.00% <ø> (ø)

... and 2 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.