v3.18.0 - Upload formats and whole Reddit threads

Added

• E-mail upload (.eml). Drop a saved e-mail onto the web UI, pick it with the file picker, or POST it to /api/file, and get the message as Markdown: the subject as title, From, To, Cc and Date below it, then the message text. The HTML part goes through the same extraction as POST /api/html; the plain-text part is used when there is no HTML part or it yields almost nothing, and a short HTML-only mail is converted directly. Attachments are listed by name only, not converted. Parsing happens inside pullmd (new dependency postal-mime), so it works without the markitdown sidecar. frontmatter=true fills title, author, published and source: email; lang=en switches the header labels to English (the web UI sends its current language). A file without mail headers returns 422. Like other uploads, nothing is cached or shared.
• More upload formats, all read without the markitdown sidecar. Outlook e-mails (.msg, new dependency @kenjiuno/msgreader) give the same result as .eml. Saved pages (.mhtml / .mht, Chrome's and Edge's "Webpage, single file") are extracted like POST /api/html, and the original URL stored in the file is passed along, so it appears as the frontmatter url and site recipes apply. Text and Markdown files (.txt, .md) are passed through with a title line and frontmatter; a note's own YAML frontmatter is kept as a fenced yaml block, and non-UTF-8 files are decoded as Windows-1252. Subtitles (.srt, .vtt) become readable paragraphs without numbers, timestamps or styling, with rolling auto-caption repeats collapsed. Files that do not match their format answer 422. The web UI's drop hint and file picker now say "file" instead of "HTML file".
• Even more upload formats, read without the markitdown sidecar. OpenDocument text, spreadsheets and presentations (.odt, .ods, .odp): headings, bold and italic, links, lists, tables and footnotes; one table per sheet; one section per slide with speaker notes. They are read with a small built-in ZIP reader that caps the unpacked size. RTF (.rtf): paragraphs, headings, bold and italic, links, lists and tables. Calendar files (.ics): start, end, location, organizer, attendees, recurrence and description per event, with invitations and cancellations marked. Contacts (.vcf, including vCard 2.1). Data files: .csv and .tsv as a table with the delimiter detected, .json and .xml as a code block, RSS and Atom feeds as a list of entries. Jupyter notebooks (.ipynb). Source code, config and log files (.py, .js, .yaml, .log, Dockerfile and many more) as a code block in the right language; .env is deliberately not accepted. Prose markup (.rst, .adoc, .org) is passed through like .txt. New source values: opendocument, rtf, calendar, contact, data, feed, notebook, code. Behaviour change: uploaded .csv, .json and .xml files used to go to the markitdown sidecar and now are read by pullmd itself, so they work without it; the markitdown sidecar also produced a table for CSV. Documents fetched by URL are unchanged.
• Whole Reddit threads: comment_depth=all. No depth limit, a first page of up to 500 comments instead of ~200, and the comments Reddit collapses ("load more comments", "continue this thread") are loaded and put back where they belong, with indentation counted from the top of the thread. Loading takes at most 20 extra Reddit requests per conversion with API credentials and 5 without (new PULLMD_REDDIT_MORE_REQUESTS, 0 turns it off) and stops early on a rate limit; whatever is still missing is noted below the comments. Works in /api, /api/stream and the MCP read_url tool (comment_depth: "all"); on Hacker News it lifts the depth limit. The web UI gets an "All" switch next to the depth field that greys the number out, and the comment controls (comments, depth, all) now form one group, set off from Frontmatter and PDF-OCR by a thin rule.
• Outlook .msg files whose HTML exists only inside the RTF body are now read from that HTML (new direct dependency @kenjiuno/decompressrtf, already used by @kenjiuno/msgreader), so links and formatting survive. Before, the plain-text body was used.

Fixed

• Missing Reddit comments are now noted. Reddit's comment JSON stops at a depth and size limit and puts the rest behind "load more comments" and "continue this thread" placeholders, which PullMD dropped without a word. A numeric comment_depth (1-10) is still a single request, but whatever is missing within that depth is now noted below the comments ("Nicht geladen: 3 weitere Kommentare" / "Not loaded: 3 more comments"); placeholders below comment_depth or beyond comment_limit are not counted. The count in the header can stay below Reddit's total, which includes deleted and removed comments.
• Replies to a deleted Reddit comment are no longer dropped. A deleted or removed comment took all its replies with it. It now shows as *[deleted]* / *[removed]* with its replies below, and is left out only when nothing below it remains.
• Negative Reddit comment scores read [-3] instead of [+-3].
• Page titles no longer pick up icon labels. The title was read from every <title> element on the page, including those inside inline SVG icons, so a page with payment logos came out as "MHTML - WikipediaPayPal LogoVisa Logo...". Only the document title is used now. Seen on a saved page; it applies to all HTML extraction.

Upgrade notes: Uploaded .csv, .json and .xml files are now read by pullmd itself instead of the markitdown sidecar. Reddit's comment_depth accepts all; numeric depths stay a single request. New optional env PULLMD_REDDIT_MORE_REQUESTS (passed through in both compose files).