Function Reference

Function Reference

Every top-level function and class method in the three core modules, for anyone reading or contributing to the source. Grouped by file, then roughly by purpose within each file. Descriptions are drawn from each function's own docstring where one exists.

generator/db.py

Dump import

• _split_by_string_literals(sql, backslash_escapes) — splits a SQL string into (chunk, is_literal) pairs at single-quoted string boundaries, so a caller can transform only the real-syntax chunks and leave literal (row data) chunks untouched.
• _apply_outside_strings(sql, transform, backslash_escapes) — applies a transform function to every non-literal chunk of sql, via _split_by_string_literals.
• _unescape_mysql_string_literals(sql) — resolves MySQL's backslash-escaped value content (\', \n, \\, etc.) into literal characters, so SQLite's own quoting rules apply cleanly afterward.
• import_mysql_dump(sql_path, db_path) — converts a mysqldump file into a SQLite database at db_path. See How It Works.

PhpbbDatabase — the query layer

• __init__(db_path, table_prefix) — opens the SQLite connection; table_prefix matches the dump's own $table_prefix (usually phpbb_).
• _table(name) — prefixes a bare table name (e.g. "topics" → "phpbb_topics").
• _query(sql, params) — runs a parameterized query, returns a list of dicts.
• _unescape_fields(rows) — undoes phpBB's stored htmlspecialchars() encoding on plain display fields (names, titles, subjects) so templates can HTML-escape them normally on render instead of double-encoding.
• get_forums() — every forum/category row.
• get_topics(forum_id) — a forum's visible topics (topic_visibility = 1), sorted topic-type-aware (announcements/stickies pinned above ordinary topics) then by last-post-time.
• get_global_announcements(exclude_forum_ids) — board-wide announcements (topic_type = 3, phpBB's TOPIC_GLOBAL) — shown atop every forum's listing, not just the one they're physically posted in.
• get_post_topic_id(post_id) — which topic a given post belongs to.
• get_posts(topic_id) — a topic's visible posts (post_visibility = 1), in post order.
• get_user(user_id) — a single user row.
• get_all_users() — every real user (excludes user_type = 2, phpBB's bot-account type).
• get_profile_fields() — custom profile field definitions visible to a public/anonymous visitor (excludes disabled, hidden, and admin-only fields), joined to their real display label.
• get_all_profile_field_values() — {user_id: {field_ident: value}} for every user with at least one profile field value set.
• get_attachments(post_id) — a post's attachments, excluding private-message attachments (in_message = 0) and ordered to match phpBB's own real retrieval order (attach_id DESC) so a stored [attachment=N] tag's index lines up correctly.
• get_poll_options(topic_id) — a topic's poll options and vote counts, if it has a poll.
• get_all_attachments(exclude_forum_ids) — every attachment across the whole archive, for building the physical-filename → real-filename map used when copying assets.
• get_password_protected_forum_ids() — forum ids with a real forum_password set.
• get_private_message_attachment_physical_filenames() — attachment filenames belonging to private messages, so they're never copied into assets/.
• get_attachment_physical_filenames_in_forums(forum_ids) — attachment filenames belonging to posts in the given forums, for excluding a forum's attachments entirely.
• get_all_post_texts(exclude_forum_ids) — every post's raw text, for external-image-URL discovery.
• get_all_poll_texts(exclude_forum_ids) — every poll's title/option text, for the same discovery pass (a poll can equally contain [img]).
• get_smilies() / get_ranks() / get_bbcodes() — lookup tables for smiley codes, user ranks, and custom BBCodes.
• get_config(name) — a single value from phpbb_config (site name, table prefix, script path, etc.).
• get_style_path(style_id) — resolves a style id to its real theme directory path.
• close() — closes the SQLite connection.

generator/bbcode.py

• _attachment_ext_badge(filename) — an inline SVG icon (Bootstrap Icons) matched to a non-image attachment's file extension, for the attachment link.

PhpbbBBCodeParser

• __init__(...) — takes every lookup table the conversion needs (smilies, attachments, custom BBCodes, external image cache, internal topic ids, board hosts, post-to-page map) up front.
• _is_same_board_host(url) — true if a URL has no host at all, or its host matches one of this board's known domains.
• _rewrite_internal_link(url) — rewrites a same-board viewtopic.php?t=N link to a relative topics/N.html link (page-aware — see How It Works); otherwise strips any phpBB session id and leaves it as a normal external link.
• _is_safe_url(url) — true if a URL is safe to render as a link's href (allowlisted scheme, or scheme-less).
• _strip_sid(url) — removes a phpBB session id (sid=<32 hex chars>) from a URL's query string.
• _append_trailing_attachments(result, post_id, original_text) — appends any attachment that had no inline [attachment=N] tag in the post body.
• convert(text, uid, post_id, enable_smilies) — the single entry point; detects legacy vs. XML format and dispatches.
• _convert_xml_markup(text, post_id, enable_smilies) — converts phpBB 3.2+'s XML markup format to HTML.
• _convert_smilies(text, enable_smilies) — replaces phpBB smiley HTML comments with <img> tags (or leaves the raw code as text when smilies are disabled for that post).
• _convert_attachments(text, post_id) — replaces [attachment=N]filename[/attachment] with the actual attachment link/image.
• _stash_code_blocks(text) / _restore_code_blocks(text, code_blocks) — protect [code]...[/code] content from every other substitution; see How It Works.
• _convert_bbcode(text) — converts standard legacy BBCode tags ([b], [i], [url], [quote], etc.) to HTML.
• _convert_list(content, ordered) — converts [*] items into <li> elements.
• _convert_custom_bbcodes(text) — applies any custom BBCodes defined in the dump's own phpbb_bbcodes table.

generator/generate.py

Small formatting/parsing helpers

• format_timestamp(ts) — formats a Unix timestamp as a human-readable UTC date string.
• strip_trailing_attachments(rendered_html) — removes a post's trailing attachments block, for building an og:description/twitter:description summary — without this, an attachment-only post's meta description is just the literal "Attachment: filename.ext" text instead of the post's own content.
• read_table_prefix(config_path) — extracts $table_prefix from phpBB's config.php.
• _read_style_cfg(style_dir) — parses a phpBB style.cfg file into a dict.
• resolve_default_style(db, dump) — resolves the dump's configured default style to a real theme directory path, following its parent-style chain.
• get_user_rank(user, ranks) — finds a user's display rank (special-assigned first, then by post count).
• build_edit_notice(post, poster_id, author_username, users, display_last_edited) — builds a post's "Last edited by X on <date>, edited N times in total." notice (or None), mirroring phpBB core's own display rule.
• humanize_profile_field_label(label) — title-cases an all-caps built-in profile field label (WEBSITE → Website), with confirmed real exceptions (ICQ stays ICQ).
• compute_post_pages(topic_posts, page_size) — builds the global {post_id: page_number} map.
• paginate_posts(posts, page_size) — splits an ordered post list into per-page chunks.
• paginate_topics(topics, page_size) — splits a forum's own normal/sticky topics into per-page chunks (announcements are handled separately — see How It Works).
• paginate_page_numbers(current, total) — the page-number sequence a pagination control should display, truncating with an ellipsis once there are more than 5 pages — ports phpBB 3.3.x's own real truncation algorithm from phpbb/pagination.php.
• build_forum_tree(forums) — nests a flat forum list into its real category/forum tree.
• prune_empty_categories(nodes) — drops a category node that ends up with no children (e.g. every child was excluded).
• process_forum_descs(forums, parser) — runs each forum's description through the BBCode/XML parser.
• build_jinja_env(template_dir) — sets up the Jinja2 environment (custom filters, template loader).

Exclusion and access control

• load_exclusions(path) — loads an --exclude JSON file into a set of seed forum/category ids.
• load_password_override(path) — loads a --password-override JSON file into a set of forum ids.
• expand_exclusions_recursively(seed_ids, all_forums) — expands a seed set of excluded ids to include every descendant.

Assets

• avatar_url(user, ...) — resolves a user's avatar to a relative URL, or None.
• _rewrite_css_imports(css_path) — strips ?hash=... query strings from @import URLs in a copied stylesheet, for static hosting.
• copy_avatars(dump_dir, output_dir) — copies uploaded avatars and the avatar gallery into assets/avatars/.
• copy_assets(dump_dir, output_dir, ...) — the main asset-copying pass: CSS, theme images, smilies, ranks, avatars, and attachments into assets/.
• build_attachments_map(db, post_ids) — {post_id: [attachment, ...]} for a set of posts.
• _attachment_is_valid(path, real_filename) — true if a file on disk decodes cleanly for its apparent type (image via Pillow, zip/rar via their own tools).
• find_bad_attachments(db, out, excluded_physical_filenames) — attachments that are missing or fail to decode.
• recover_bad_attachments(bad, recovery_dir, out) — replaces a bad attachment with a working copy from --attachment-recovery, if one exists there.
• find_bad_avatars(users, out) — avatar keys missing from assets/avatars/ or that fail to decode.
• find_bad_smilies(smilies, out) — smiley filenames missing from the copied smiley set.
• _image_ext(im) — maps a Pillow-detected image format to a file extension.

External URLs and fetching

• load_url_mirrors(path) / load_ignored_hosts(path) / load_board_hosts(path) — load the respective JSON config files.
• _fetch_image(url, timeout, url_mirrors, ignored_hosts) — resolves a URL to raw bytes + extension, via a mirror first, then the network.
• download_remote_avatars(users, out, url_mirrors, ignored_hosts, skip_fetch) — downloads avatar.driver.remote avatars into assets/avatars/.
• load_avatar_overrides(path) / apply_avatar_overrides(overrides, out) — load and apply an --avatar-overrides file.
• report_missing_avatars(users, bad_avatars, remote_avatar_exts, avatar_overrides, template_path) — prints every user whose avatar still doesn't resolve and writes the starter override template.
• find_image_urls(db, users, forums, exclude_forum_ids) — collects every external image URL referenced anywhere in the dump (posts, signatures, forum descriptions, polls).
• download_external_images(urls, out, url_mirrors, ignored_hosts, skip_fetch) — downloads external [img]/<IMG> URLs into assets/external/.
• _download_image_asset(url, dest_dir, stem, label, default_ext) — downloads a single asset (favicon/logo) from a live URL.
• _svg_natural_size(path) / _compute_logo_display_size(path, max_height, max_width) — read an image's real dimensions for correct logo sizing.

Page rendering

• render_index(env, out, forum_tree, total_posts, total_topics, total_members, site_name, announcement_html) — writes index.html.
• render_forums(env, out, db, users, parser, forums, site_name, announcement_html, exclude_forum_ids) — writes every forums/<id>.html (and -pN.html for paginated forums — see How It Works). Returns {forum_id: total pages} for every paginated forum.
• render_topics(env, out, db, users, ...) — writes every topics/<id>.html (and -pN.html for paginated topics). Returns (total_posts, post_id_to_page, multi_page_topics).
• render_users(env, out, db, ...) — writes every users/<id>.html.
• render_sitemap(out, db, site_url, forums) — writes sitemap.xml/robots.txt, including one entry per page of a paginated topic or forum.
• _paginated_redirect_blocks_apache(p, base_url, multi_page_topics) / _paginated_redirect_blocks_forums_apache(p, base_url, multi_page_forums) / _apache_redirects(old_prefix, base_url, multi_page_topics, multi_page_forums) — build the apache .htaccess redirect rules, including per-page rules for paginated topics (exact post-id alternation) and forums (exact start= offset).
• _paginated_redirect_blocks_nginx(base_url, multi_page_topics) / _paginated_redirect_blocks_forums_nginx(base_url, multi_page_forums) / _nginx_redirects(old_prefix, base_url, multi_page_topics, multi_page_forums) — build the nginx redirect config. nginx's if only accepts a bare variable as its left-hand operand, not an inline compound expression like $arg_t:$arg_p — the combined value is assigned to a variable with set first ($topic_page_key/$forum_page_key), then tested; see How It Works for why this matters (a real, previously-shipped bug here silently never matched).
• render_redirects(out, redirect_format, old_prefix, base_url, multi_page_topics, multi_page_forums) — dispatches to whichever redirect format was requested.
• render_search(env, out, site_name) — writes search.html.
• run_pagefind(out) — runs the Pagefind indexer as a subprocess after every other page is written.

Orchestration

• _open_db_and_copy_assets(dump_dir, output_dir, ...) — shared setup for generate() and every diagnostic mode: imports the dump, resolves exclusions, copies assets per copy_assets_mode.
• clean_disabled_feature_output(out, search, sitemap_url, redirect_format) — removes feature-gated output (search index, sitemap, redirect file) left over from an earlier --incremental run whose flags no longer request it.
• generate(dump_dir, output_dir, ...) — the full pipeline: import, copy assets, render every page type, write feature-gated output.
• find_missing_avatars(dump_dir, output_dir, avatar_overrides_path) — diagnostic mode -m.
• list_forums(dump_dir, output_dir) — diagnostic mode -l.
• check_images(dump_dir, output_dir, url_mirrors_path, ignored_hosts_path) — diagnostic mode -i.
• check_links(output_dir) — diagnostic mode -c.
• main() — the CLI entry point: parses arguments, dispatches to generate() or a diagnostic mode.