Skip to content

Releases: fangfufu/httpdirfs

1.3.3

Choose a tag to compare

@fangfufu fangfufu released this 11 Jun 20:12
Immutable release. Only release title and notes can be modified.
c67dea0

Added

  • ci: Add a dedicated CodeQL scanning job targeting GitHub Actions workflows while streamlining the C/C++ analysis job (d8e07c0, c02c0d1).

Changed

  • ci: Update pre-commit hook revisions for pre-commit-hooks, codespell, mdformat, and yamlfmt (8cf9d4c).
  • test: Update unity test framework subproject wrap file to track the latest master branch commit (2b4c765).
  • Update and expand USAGE.md to comprehensively document raw help options and FUSE/module flags, linking to Git commit 0855c0a (2bc5306).
  • ci: Replace legacy Prettier hooks with yamlfmt and mdformat (with GFM and alerts plugins) to format all Markdown and YAML files repo-wide, removing obsolete formatting bypass tags (a7d2931).
  • ci: Restrict automated pull request pre-commit hook runs to branches targeting the master branch (30dac3e).
  • ci: Modernize pre-commit configuration readability with clean multi-line block literal scripts and structure the PKG_CONFIG_PATH definition in build.yml incrementally (4c8f561).
  • doc: Fix grammar regarding parsing options in README.md (3746293).

Fixed

  • ci: Harden GitHub Actions workflows (build.yml and pre-commit.yml) by explicitly defining read-only contents permissions to resolve CodeQL workflow permissions alerts (81b87a0, 52b9baa).
  • doc: Fix typos in documentation (ca848fc).

1.3.2

Choose a tag to compare

@fangfufu fangfufu released this 29 May 20:06
1.3.2
f763d7f

Added

  • Add opt-in --external-links feature to parse absolute cross-origin directory
    links from HTML listings and expose them as local files and subdirectories,
    featuring first-wins deduplication and same-origin scoping of headers and
    credentials (edce0ca).
  • Add persistent caching support for external links, mapping and sanitizing
    cache paths dynamically to prevent directory traversal (855e134).
  • Introduce comprehensive unit and integration tests verifying external links
    detection, filename sanitization, same-origin credential isolation, cache
    mapping, and robust error handling (edce0ca, 855e134).
  • Add --cache-min-size and --cache-max-size command-line flags and
    configuration options to dynamically enforce file size thresholds for
    persistent caching (1547e39).
  • Introduce unit and integration tests verifying caching thresholds, including
    boundary conditions and bypass behavior (1547e39).

Changed

  • Update macOS build job in GitHub Actions workflow to only run unit tests and
    exclude integration tests, which are unsupported due to the lack of
    FUSE/macFUSE in the GitHub Actions runner environment (93bb8b5).
  • Optimize macOS CI workflow by removing redundant packages (openssl, curl,
    and pkg-config) from the brew installation step, as these are pre-installed
    in the runner environment (bd3503c).
  • Suppress zero-length file checks in the long integration test suite,
    preventing redundant skip messages when running integration tests under the
    long profile since zero-length files are only generated under the short
    profile (e8c6e16).
  • Rewrite and expand USAGE.md into a comprehensive reference manual detailing
    all 35 command-line flags, option syntax, security considerations, and
    defaults (37bb031).
  • Implement complete dynamic global state reset in tearDown() in
    tests/test_link.c to prevent configuration and root link table mutations
    from leaking between unit tests (1a13d41).
  • Improve integration test compatibility with BSD and macOS systems by replacing
    GNU-specific find -print -quit options with portable POSIX pipelines using
    head -n 1 (47d79de).

Fixed

  • Fix origin validation by canonicalizing default ports (HTTP/80 and HTTPS/443)
    in is_cross_origin() checks, ensuring same-origin URLs utilizing default
    ports are not misclassified as cross-origin (7ca2f2a).
  • Fix argument parsing and help output when parsing --help or --version
    options internally (860453a).
  • Harden argument parsing error handling by terminating early with
    exit(EXIT_FAILURE) instead of returning non-zero error codes from
    parse_arg_list (8d13093).

1.3.1

Choose a tag to compare

@fangfufu fangfufu released this 27 May 14:49
ed1aa85

Added

  • Add comprehensive unit and integration tests for duplicate URL handling,
    including unit tests for linknames_equal, hash_str, LinkHashSet
    operations, and LinkTable_parse_html (4534563).
  • Extend the system integration test HTTP server to inject duplicate URLs and
    directories, and implement mounted directory integration tests to verify
    robust FUSE duplicate deduplication (4534563).
  • Expose and document internal LinkHashSet operations and comparison helpers
    in link.h using Doxygen (4534563).

Changed

  • Optimize duplicate link detection during HTML parsing by replacing the
    $O(N^2)$ linear scan in HTML_to_LinkTable with a custom open-addressing hash
    set (LinkHashSet) using linear probing and a djb2-based hash_str function (26c0c5c).
  • Refine HTTP version negotiation behavior in Link_to_curl to gracefully fall
    back to HTTP/2 (CURL_HTTP_VERSION_2_0) if negotiation for HTTP/3
    (CURL_HTTP_VERSION_3) is unsupported or fails (5de0dd6).

Fixed

  • Fix formatting in error log outputs by adding missing newlines (\n) to
    multiple lprintf(error, ...) calls logging curl easy setup/info errors in
    src/link.c and src/network.c (2f5cf96).

Release 1.3.0

Choose a tag to compare

@fangfufu fangfufu released this 27 May 12:21
106d56a

Creating a new minor release, because of the following major changes:

  • The new threading model of the cache system, which allows multiple downloads (blocking foreground and background) to be done simultaneously.
  • When cache system is enabled, the downloading thread yields as soon as there is enough data to reply to the FUSE read call.
  • New pre-commit hooks to help developers to maintain a basic level of quality in each commit.
  • New unit tests which test individual software components.
  • New system integration tests which launch a HTTP server, then mount it for read testing. There are two variants:
    • A short test which tests the filesystem against different edge case filenames.
    • A long test which do multithread read on a 1GB file with different block sizes.

Here are the detailed changes:

Added

  • Dynamically scale the number of background prefetch worker threads to match up to half of the maximum concurrent connection capacity (CONFIG.max_conns capped at DEFAULT_NETWORK_MAX_CONNS), transitioning from a single hardcoded thread to improve performance when multiple concurrent active downloads are supported (2c39a7a).
  • Add a unit test test_Cache_alloc_num_bg_workers to verify correct background worker allocation scaling under different network connection capacities (2c39a7a).
  • Implement a thread-safe manual reference-counting mechanism (ActiveDownload_ref and ActiveDownload_unref) for ActiveDownload nodes to prevent Use-After-Free (UAF) vulnerabilities during concurrent read operations, background transfers, and cache teardown (9c34463).
  • Add a unit test test_Cache_free_active_downloads to verify that active download nodes are not prematurely freed while waiter threads are still active (9c34463).
  • Implement a robust shutdown handshake mechanism in Cache_free to coordinate safe cache teardown when reader threads are actively blocked on segment downloads, ensuring all threads unregister before mutexes and condition variables are destroyed (a01c9b1).
  • Add a unit test test_Cache_free_active_downloads_with_waiters to exercise and verify proper thread synchronization and early-exit cleanup during cache shutdown (a01c9b1).
  • Add standardized GPLv3/OpenSSL copyright headers and detailed Doxygen file-level \brief annotations to all C source and header files under the src/ directory to improve legal compliance and maintain consistent documentation headers (f621da7, 666d53a).
  • Add CURLOPT_PIPEWAIT setting to curl easy handles to optimize HTTP/2 multiplexing during concurrent file and directory transfers (4114977).
  • Support HTTP/3 (QUIC) protocol version negotiation with automatic and seamless fallback to HTTP/2 or HTTP/1.1 when unavailable (4114977).
  • Implement shutdown memory leak detection in DEBUG builds, recursively traversing the virtual filesystem at shutdown to tear down resources and verify that no memory leaks exist (30d7d10).
  • Add FUSE .opendir and .releasedir handlers to safely evict directories (LinkTables) from memory when they are no longer in use, using an atomic reference-counting strategy to prevent crashes in other subsystems (2fff24a).
  • Introduce a comprehensive unit test suite test_memory_tracking to validate correct allocation, reallocation, and free behavior of debug allocator wrappers (9b38614).
  • Implement asynchronous early-return download support in cache mode to improve read performance during ongoing background downloads (0fb2d06).
  • Introduce robust logging and error-checking wrappers for pthread condition variable APIs (PTHREAD_COND_INIT, PTHREAD_COND_DESTROY, PTHREAD_COND_BROADCAST, and PTHREAD_COND_WAIT) (af0e263).
  • Support concurrent background segment downloads in cache mode by replacing the single active download tracker with a linked list of ActiveDownload structs; add ActiveDownload_find, ActiveDownload_add, and ActiveDownload_remove helpers and implement double-checked locking in Cache_read_segment to prevent duplicate downloads under concurrent FUSE workers (7286314).
  • Add unit test for ActiveDownload_find verifying lookup behavior on both empty and populated active download lists (0441ced).
  • Define a custom sys_sem_t type and emulate unnamed POSIX semaphores (sem_init, sem_destroy, sem_wait, sem_post, and sem_trywait) using pthread mutex and condition variables on Apple/macOS platforms, where unnamed POSIX semaphores are unsupported (af24b36).
  • Add zero-length file coverage to integration and unit test suites, including a dedicated test group verifying size, content, and SHA-256 digest in both direct and cache modes; add unit tests for negative-len, NULL cf, and NULL cf->link guards in Cache_read (d02ac05).

Changed

  • Transition from a single global condition variable (dl_cond) to granular, block-specific condition variables integrated directly within ActiveDownload nodes to eliminate redundant wakeups of unrelated FUSE waiter threads and reduce thread scheduling overhead (a40bd74).
  • Optimize the cache wait loop inside Cache_read_segment to check status in $O(1)$ time by introducing an unlinked state flag on the ActiveDownload structure, avoiding expensive $O(N)$ list lookups on every loop iteration (42a4dd3).
  • Clean up and standardize #include directives across all source files, headers, and test suites, using forward struct declarations to break circular dependency chains (94ced41).
  • Extract common cache test setup logic into dedicated helper functions (setup_mock_link_table and setup_temp_cache_dir) inside tests/test_cache.c (2c39a7a).
  • Replace raw octal literals with symbolic permission constants (such as S_IRWXU) in test directory creation calls to address SonarCloud security hotspot alerts (2c39a7a).
  • Update README.md to document the permanent cache thread-safety improvements, including granular signaling, manual reference counting, and the shutdown handshake mechanism (7eea0ea).
  • Update USAGE.md to document the --proxy-capath and --capath flags, update the default value of --max-conns to 6, and update internal commit reference links (1864cce, 7bcbcb2, 7d76285).
  • Lower the default maximum network connection count (DEFAULT_NETWORK_MAX_CONNS) from 10 to 6 to align with the standard persistent connection limits agreed upon by all major modern web browsers (c049e6d).
  • Update tests/test_config.c configuration initialization test to assert max_conns against the configuration macro instead of a hardcoded value (c049e6d).
  • Optimize GitHub Actions workflows by separating short and long integration tests and removing redundant manual pre-commit hook stages to avoid duplicate, slow sequential CI execution (86b0e8e).
  • Refactor synchronization and memory wrappers (mutex, semaphore, condvar, and allocator) to remove trailing name string parameters and clean up debug log statements (efb8c24).
  • Remove all debug log statements (log_printf and lprintf calls under the debug log type) across FUSE, cache, and link modules to eliminate tracing overhead (efb8c24).
  • Eliminate unused helper variables (such as the static loop counter inside LinkTable_new) that were only required by the removed debug prints, preventing compiler warnings (efb8c24).
  • Optimize memory tracking in debug allocator wrappers by replacing the $O(N)$ linear linked list with a statically sized 8192-slot hash table with chaining (9b38614).
  • Optimize string sanitization loop in sanitise_LinkTable by replacing strlen() with a null-terminator check, eliminating $O(N^2)$ overhead (c8fbf51).
  • Refactor test suites into granular profiles (unit_test for C unit tests, integration_short for fast integration tests, and integration_long for intensive tests) and align documentation in src/README.md (39963f7).
  • Revise pre-commit hook configuratio...
Read more

Release 1.2.11

Choose a tag to compare

@fangfufu fangfufu released this 23 May 01:31
1.2.11
b4b6834

Added

  • Add support for --capath and --proxy-capath configuration options (da9b0e9).
  • Support --config=path to specify a configuration file (db65bb5).
  • Add core unit testing suite using the Unity framework (e50d690).
  • Add system integration test suite (89e5027).
  • Add a debug memory allocation tracker and hardened memory allocations (e86afe8).

Changed

  • Optimize Cache_open by caching file size locally and avoiding redundant system calls to Data_size (10e5208).
  • Exclude integration tests from default Meson test runs, executing them only in CI or when explicitly requested (f13559b, 91a8295).
  • Refactor fatal logging infrastructure (0753fc4).
  • Refactor and enhance the FREE macro and fix format specifiers (ef80e80).
  • Replace custom limit macros with standard limits.h constants (f9d4542).
  • Run cache tests with multiple block sizes and recreate cache directory between iterations (6f889ac, 9fe774a).
  • Install clang in build and pre-commit workflow environments, and upload Meson test logs as build artifacts (d6dd5d8, 4edc45b).
  • Update documentation in src/README.md (unit/integration test guides) and USAGE.md (4d67613, bde2dfa).

Fixed

  • Resolve concurrent cache initialization race in fs_open (64e8ea3).
  • Correct off-by-one error in cache segment count calculation (4cba404).
  • Prevent unbound MOUNT_DIR error in integration test cleanup (12b8cdd).
  • Check for backtrace in libc before searching for execinfo in build configuration (ded25f9).
  • Reset proxy_cafile correctly in Config_init (6a88d4d).
  • Handle Link_download errors in cache mode (23ec785).

Version 1.2.10

Choose a tag to compare

@fangfufu fangfufu released this 16 May 01:03
1.2.10
4a70e51

Fixed

  • Reuse populated LinkTable to prevent memory leaks and redundant I/O (0fd47e4).
  • Fix Sonic ID3 recursive root listing with artists by improving parser
    robustness (ffc3db6) (#213).

Release 1.2.9

Choose a tag to compare

@fangfufu fangfufu released this 13 May 00:01
1.2.9
4e963ad

Added

  • Fail early if mountpoint or FUSE module is missing (5c53ad3).

Fixed

  • Improve link initialization error reporting (8911119) (#207).
  • Return the correct failure exit code (5dd032a) (#208).
  • Resolve mountpoint to absolute path to prevent unmount failure on termination for relative mountpoints (3a53c40) (#177).
  • Removed useless NOLINT tags (c8abdec).

Changed

  • Update help's default value based on config.h (adf3bb9) (#150).
  • Update documentation in src/README.md and USAGE.md (1d7e8e5, 1b16c9d).

Release 1.2.8

Choose a tag to compare

@fangfufu fangfufu released this 10 May 02:45
1.2.8
da81681

What's Changed

New Contributors

Full Changelog: 1.2.7...1.2.8

1.2.7 - Fixed FreeBSD compilation

Choose a tag to compare

@fangfufu fangfufu released this 01 Nov 20:24
502dbec

Fixed

  • Fixed FreeBSD compilation
    (#165)

Full Changelog: 1.2.6...1.2.7

1.2.6

Choose a tag to compare

@fangfufu fangfufu released this 07 Oct 12:04
de9e594

What's Changed

Fixed

  • The refreshed LinkTable is now saved. (#141)
  • Only one LinkTable of the same directory is created when the cache mode is enabled. (#140)
  • Cache mode now works correctly with escaped URL. (#138)
  • Fixed --user-agent option (#159)
  • Allow leading ./ segments in links (#125)
  • Fix segfault due to missing LinkTable allocation (#155)
  • Fix Segmentation fault if HOME not set (#162)
  • Handle sites that use absolute links and sites that require the final slash (#121)

Added

  • Add --http-header option (#157)
  • Add --cache-clear option (#111)
  • Add --refresh-timeout to set refresh timeout for directory contents (#114)

Changed

  • Improved LinkTable caching. LinkTable invalidation is now purely based on timeout. (#147)
  • Replaced the GNU Autotools based build system with Meson. (#149)
  • Transitioned from using libfuse 2.x to to 3.x. (#116)
  • Updated OpenSSL MD5 checksum generation API usage. (#143)

New Contributors

Full Changelog: 1.2.5...1.2.6