Skip to content

v0.2.0

Latest

Choose a tag to compare

@github-actions github-actions released this 13 Jun 19:45
96743ce

xhs can now read public Xiaohongshu data without a cookie, the profile login
wall is fixed, the exit codes are honest, crawl grew into a small scraping
engine, and the docs are rewritten around how to use each command.

Read without a cookie

The signed JSON API answers -101 needs login to an anonymous caller, but the
web client ships the data inside the server-rendered page as
window.__INITIAL_STATE__. xhs now reads that page first and only falls back
to the signed API when you pass a logged-in cookie.

  • note and feed work anonymously.
  • user, user --notes, and related work from a cold IP and get rate-walled
    once the IP is hot.
  • comments, search, suggest, tag, and me still need a cookie, since
    the site does not server-render them.

The new pkg/xhshtml package extracts __INITIAL_STATE__ from the HTML and
cleans up the undefined/NaN the page leaves in its JSON.

The profile login-wall fix

An in-site Referer header made the site treat a profile fetch as an in-app
navigation and bounce it to /login. Dropping the Referer on web reads
returns the full page. xhs also detects a redirect to /login and reports it
as a surface that needs a cookie instead of a confusing network error.

Honest exit codes

A walled or login-only surface now exits 3 (needs a cookie) instead of exiting
0 with empty output or 6 with a wrapped error. note and user return the
underlying error when a batch resolves nothing, so a fully blocked run fails
loudly.

A crawl engine

xhs crawl walks outward from seed notes and streams connected records as
JSONL, one file per kind, into an output directory. It writes notes.jsonl,
users.jsonl, and comments.jsonl as it goes, so a run that stops early still
leaves usable data on disk.

xhs crawl <note-id> --token <t> --out ./data --comments
xhs feed --category food -o url | xhs crawl - --out ./data

Usage-first docs

The docs lead with how to use each command. The new commands reference shows
every command with the call and a sample of what it prints.

Install

go install github.com/tamnd/xiaohongshu-cli/cmd/xhs@latest

Or download a prebuilt binary below, or run ghcr.io/tamnd/xhs:0.2.0.