v5.0.0

⚠️ API Changes

• date_range returned by get_info[_batch] was a datetime.datetime now returns a pandas.Timestamp with nanosecond precision (#1461)
• staged argument removed from write_pickle_batch (#1642)
• Set row_count/rows to None in get_description/get_info and batch versions thereof if symbol is pickled (#1664)
• Do not fallback to iterating snapshots if if version as_of int or timestamp not found in version chain in V2 API (#1672)
• Use float64 as the result type for all division operations in the processing pipeline (#1794)
• Make SymbolDescription return type match type hints (#1877)

🚀 Features

• Lazy dataframe implementation (#1703)
• Arbitrary clause ordering (#1860)
• Sort staged data before writing (#1869)
• Make library manager a LRU cache (#1930)
• Preserve nanoseconds and timezones in date range from get info (#1461)
• Do not fallback to iterating snapshots if if version as_of int or timestamp not found in version chain in V2 API (#1672)
• Use float64 as the result type for all division operations in the processing pipeline (#1794)

🐛 Fixes

• Improve full build time by 30% by adding PCH (#1764)
• Fix typo in docs flow (#1952)
• Fix entt dep management for conda builds (#1955)
• Run the pull request on the PR sha with the pull_request_target event (#1954)
• Fix the timers macro, configure it with a cmake option (#1914)
• Small fixes for mac arm compilation and fix some tests (#1905)
• Fixes for sort and finalize (#1763)
• Bugfix 1552: Set row_count/rows to None in get_description/get_info and batch versions thereof if symbol is pickled (#1664)
• Bugfix 1641: Remove unused staged arg from write_pickle_batch (#1642)
• Refactor 1749: make processing tests great again (#1758)
• Always use strict weak ordering in std::sort comparators (#1747)
• Use EncodedFieldImpl.blocks() rather than casting member variable _blocks (#1745)
• Fix batch_restore_version failing to use correct next_version_id (#1823)
• Bugfix arcticdb-man 96: Improve error message with pd.Timedelta columns (#1874)
• Bugfix 1865: Make SymbolDescription return type match type hints (#1877)
• Bugfix 1652: Improve error message when object dtype columns contains Timestamps with mixed timezones (#1880)
• Bugfix 1841: Correctly roundtrip None and empty string pd.Series names (#1878)
• Fix library options compatibility across ArcticDB versions (#1862)
• Bugfix 1830: fix resampling with multiindex (#1873)

Uncategorized

• Extend tests for merge sort (#1708)
• Check if object is an instance of QueryBuilder when comparing for equality in QueryBuilder.eq (#1754)
• Add resampling offset (#1743)
• Refactor/1722/remove composite from processing pipeline (#1741)
• Fix Segment use-after-move when replicating to NFS (#1756)
• Throw exception if appending using sort_and_finalize will create unordered index (#1760)
• Improve logging for stress tests (#1759)
• Remove rocksdb (#1761)
• Update BSL table for v4.5.0 (#1768)
• Handle KeyNotFoundException in recurse_index_key (#1766)
• Add Storage API to check for existence of a key matching a predicate (#1762)
• Revert "Handle KeyNotFoundException in recurse_index_key" (#1776)
• Add 1 Billion Row Challenge Notebook (#1774)
• Remove KeyNotFoundException catch in CopyCompressedInterStoreTask (#1778)
• Limit 3 item ref key bypass to only when loading undeleted versions (#1775)
• append docstring improvement: added that append on a new symbol will create it (#1787)
• Parallelize string handling, enable python type handlers, prepare for Arrow (#1698)
• Add interface to pass a cached entry to tombstone all (#1793)
• Fix sort index (#1796)
• Fix windpws CI by adding the string header in decimal.cpp (#1801)
• Remove unused RuntimeConfig feature (#1803)
• Add unimplemented increment counter (#1804)
• Pre arrow refactor (#1805)
• Fix regression in bool pickling (#1810)
• Add API to remove Prometheus metrics (#1806)
• Fix symbol list bug on compaction (#1324) (#1798)
• Allow metrics to be registered twice (#1816)
• Docs 1809: lazy dataframe documentation and example notebook (#1815)
• Bugfix 1818: Fix QueryBuilder equality checks (#1819)
• Support reading from prefixes that include a dot (#1820)
• Fix compilation errors for clang 18 (#1813)
• Add StagedDataFinalizeMethod to docs (#1744)
• Add demo video link to README.md (#1837)
• More sort and finalize fixes (#1799)
• Fix conda build (#1854)
• Fix row slicing with sort_and_finalize throw on column slicing when the column group is larger than the segment column size (#1838)
• Don't override the https value for backwards compatibility (#1840)
• Support the ignore missing key read option in recurse_index_keys (#1844)
• Add flag to clear staged data on failure to both finalize functions (#1856)
• Change test for library emptiness to allow specifying key types to exclude (#1861)
• Fixing clean builds (#1868)
• Extend LMDB instance lifetime (#1879)
• Allow customizing the most useful AWS SDK settings. (#1875)
• Refactor 1833:use entt as ecs (#1834)
• Fix mono CI issue (#1917)
• Fix lib_tool.read_to_keys (#1832)
• Skip lmdb compat tests for versions <= 4.5.0 (#1923)
• Use relative paths for compat tests (#1924)
• ci: Use mamba 2.0 via mamba-org/setup-micromamba@v2.0.0 (#1855)
• Remove unneeded step (#1929)
• Adds a type change utility for append data keys in lib_tool (#1932)
• fix introduction help (#1933)
• Rewrite the mkdocs template so the links point to the correct version (#1942)
• Change the pull_request trigger to pull_request_target (#1947)
• Revert pull_request_target back to pull_request (#1949)

New Contributors

• @grusev made their first contribution in #1933
• @maxim-morozov made their first contribution in #1947

Full Changelog: v4.5.1...v5.0.0

---

The wheels are on PyPI. Below are for debugging: