v0.8.35

Built against Mojo 1.0.0 (ed45d567).

A patch release. On the pandas side, MultiIndex, RangeIndex, TimedeltaIndex and Interval arrive, along with merge_asof, merge_ordered, align, pivot and pivot_table, crosstab, cut and qcut, read_json and to_json, to_csv, to_numpy, to_numeric, to_timedelta, date_range with calendar steps, and map, apply, agg and transform on frames and columns, and to_datetime reads formats the way pandas guesses them. In SQL, right, full, cross, ASOF and POSITIONAL joins run. The ClickBench group by got faster: a lasting tuple is written a key at a time, a stored text key is compared through a flat head the walk prefetches, the bucket pass counts through a pointer, and a grown table is zeroed on the cores, which together take about a tenth to a quarter off q15, q18, q33, q34 and q39 at 1M rows.

Changed: a grown lasting table is zeroed a part at a time on the cores

When a lasting map grows its table for a chunk that could be all new keys, the new table can be tens of megabytes, and it was zeroed on the calling thread with its page faults before the parts were moved in on the cores. Each part now zeroes its own share as it moves. On a six core Linux machine that was heavily loaded by other work, alternating the old and new driver over six rounds, ClickBench q15 went from 72 to 58 ms at best and q34 from 186 to 174, and every answer matched.

Changed: a lasting map counts and scatters its buckets through a pointer

The pass that sorts a chunk's rows by part indexed a list of counters on every row, which checked the bound and reloaded the list's length each time. It now keeps its morsel's counters through a pointer. On ClickBench q15 the two passes went from about 15% of the query's CPU to about 8%, and the best time from 31.8 to 29.4 ms on a six core Linux machine busy with other work, with every answer the same.

Added: to_datetime guesses a format from the first row the way pandas does

to_datetime with no format used to read ISO 8601 and refuse anything else. It now guesses a format from the first value that is not missing, as pandas' guess_datetime_format does, and holds every row to it, so 01/02/2026, Jan 2, 2024, 31.12.2024 and Sat, 06 Jan 2024 10:00:00 +0000 read as they do in pandas. When no format can be guessed each row is read on its own, as pandas does. A strptime format the core does not read, such as one with %B, %I and %p, %j or %Z, is read the same way, with pandas' words for a value that does not match, for data left over, and for a day past the end of its month. Labels in read_json are still read as ISO 8601 only, which is what pandas does there.

Added: tz_convert(None) and tz_convert with a tzinfo

tz_convert(None) on a DatetimeIndex or through .dt now moves the instants to UTC and takes the clock off, as pandas does, where it used to be refused. tz_convert also reads a zoneinfo.ZoneInfo or datetime.timezone.utc as the zone it names instead of refusing anything that is not a string. Converting instants that carry no zone raises a TypeError with pandas' own sentence, without the explanation the core adds after it.

Added: as_unit names the first instant nanoseconds cannot reach

as_unit("ns") on a DatetimeIndex or through .dt now raises OutOfBoundsDatetime with pandas' own sentence, Out of bounds nanosecond timestamp: followed by the first instant outside the range on the UTC wall clock and to the second, where it used to raise a type error from the core. An except clause written against pandas now catches it. Instants just inside either end of the range still restate.

Added: to_datetime answers an index for a list and an instant for one value

to_datetime now answers what pandas answers for each kind of input: a column for a column, a DatetimeIndex for a list, tuple or index, keeping the index name, and one instant for one value, with None where pandas gives NaT. A list with nothing to read is held at seconds, as in pandas. The DatetimeIndex constructor now reads text every row with its own format, the way pandas does, so a list of dates in different shapes builds an index instead of raising. A DatetimeIndex can be put in a frame or a column directly, and a column built from an index takes the index name.

Added: to_datetime reads every row with its own format

to_datetime(format="ISO8601") and to_datetime(format="mixed") used to be refused by name. Both now read each row with its own format, as pandas does. ISO8601 holds every row to the ISO 8601 shapes pandas reads. mixed also reads the common shapes dateutil reads: slashed and dotted dates, month first unless the first number cannot be a month, month and weekday names, a clock with AM or PM, and a two digit year in the century nearest today. The unit, the zone, utc, errors="coerce" and pandas' own errors for text that will not read and for rows at more than one offset all match pandas. A column whose rows share one shape still reads in a single pass through the core.

Added: at_time, between_time and asof on a column and a frame

Series and DataFrame now have at_time, between_time and asof. The first two choose rows by the time of day of labels that are instants, through the index indexers, and raise pandas' Index must be DatetimeIndex on any other labels. asof finds the last row at or before each label that holds values, skipping missing values on a column and rows with a missing value in subset on a frame. One label answers a value or a row named by the label, several answer a column or a frame on them, and rows that found nothing come back missing with whole numbers widened to floats, as in pandas.

Changed: the paths written for a dictionary encoded column ask for it by name

Fifty one kernels and string methods that have a fast path over a dictionary encoded column's codes used to take it for any column not held flat. They now ask is_coded(), so a column in an encoding they were not written for reaches the check in front of the values and is refused with a message naming decoded(), rather than having its buffer read as codes. Nothing is held in such an encoding yet. This is the ground the next encoding of #979, a column held as positions into another, needs before anything builds one.

Changed: an exchanged inner join sorts its pairs back on every core

When an inner join's left side is much shorter than its right, the join builds on the left and walks the right, then counting sorts the pairs back into left row order so the exchange stays invisible. That sort ran on one thread. On TPC-H q2, 2,000 European suppliers against 800,000 part supplies, it was 7% of the query's cycles. A pairing of at least 131,072 entries now gets cut into one run per worker. Each run counts its own left rows, the prefix sum goes over the left rows and, within a row, over the runs in order, and each run scatters into places no other run writes, so the order is the same stable one the single thread gave. The two output lists are no longer zero filled before the scatter overwrites them. On a busy 8 core VM the sort dropped to under 2% of q2's cycles, spread across the cores, and q2's best time went from 102.6 to 84.5 ms with the two builds run back to back.

Added: The members only an index of instants has

DatetimeIndex now has shift, snap, indexer_at_time, indexer_between_time, isocalendar, mean, std, to_julian_date, to_pydatetime, time, timetz, tzinfo and resolution. shift and snap count along calendar frequencies such as MS, W-MON and B with the steps date_range already uses, and a label between two landing dates rolls the way pandas rolls it. The time of day indexers read times the way pandas does, including the evening time pandas reads a missing label as, so a range that wraps past midnight takes it in as it does in pandas. Slicing or picking labels out of an index of instants or spans now keeps its type. shift without freq raises NullFrequencyError, since firepanda keeps no frequency on an index.

Added: Index join, asof and the members around them

Index and every index built on it now have join, asof, asof_locs, get_indexer_for, get_indexer_non_unique, sortlevel, groupby, view, array, shift and a str accessor that answers indexes. join follows pandas down each of its three paths, the sorted join, the merge join for repeated labels and the lookup join, so the rows and the positions come back in pandas' order, None included where pandas answers it. A join of text with numbers is refused, since the answer would be an index of mixed values. get_indexer on an index of instants or spans now reads labels written as text, where it used to fail.

Added: MultiIndex

MultiIndex is here, with every member pandas declares on it spelled the way pandas spells it. It is built with from_arrays, from_tuples, from_product, from_frame or out of levels and codes, and keeps what pandas keeps: sorted levels, codes with -1 for a gap, and level names. Lookups (get_loc, get_locs, get_loc_level, get_indexer, isin, slice_locs), reshaping of levels (droplevel, swaplevel, reorder_levels, set_levels, set_codes, remove_unused_levels), sorting, set operations, append, insert, drop and the duplicate members answer as pandas does, arrays as numpy arrays when numpy is installed. What needs a frame or column labelled by a MultiIndex, or a column of tuples, is refused by name for now: to_frame(index=True), to_series, to_flat_index and value_counts.

Added: cov, corr and aggregate on a decay

ExponentialMovingWindow gains cov, corr, aggregate and agg. cov and corr follow pandas' own recurrence, with adjust, ignore_na, min_periods and bias, and only the rows where both sides hold a value count. They pair a column with a column, a frame with a column, or two frames column by column, the same way the window cov and corr do, and the pairing and aggregate are now one piece of code shared by windows and decays.

Fixed: a window member with no answer is a NaN

first, last, nunique, apply, cov and corr on a window answered a gap where a window held too few values, and pandas and the kernel reductions answer NaN there. They answer NaN now.

Added: the window members that read every window

Rolling and Expanding gain first, last, nunique, apply, aggregate (and agg), pipe, cov and corr, placed the way pandas places each window, including center, closed, step and min_periods. apply hands the window over as a float64 column with its labels, or as a numpy array with raw=True. cov and corr pair a column with a column, a frame with a column, or two frames column by column. Answers that pandas gives with two levels of labels, a list of reductions over a frame and every pair of a frame's columns, are refused for now.

Fixed: convert_dtypes turns NaN into a gap

convert_dtypes now turns a NaN in a float column into a missing value, as pandas' nullable float does, before it looks for whole numbers.

Added: DataFrame.combine, DataFrame.corrwith and convert_dtypes

DataFrame.combine runs a function over each pair of columns across both frames' labels and columns, with pandas' casting, fill_value and overwrite, and whole number answers cast back the way pandas does. DataFrame.corrwith correlates each column, or each row with axis=1, with a column or with the shared columns or rows of another frame, with drop, numeric_only, min_periods, Pearson, Spearman and a callable method. DataFrame.convert_dtypes and Series.convert_dtypes answer the types here that hold a gap, which are pandas' nullable types under their plain names, and turn a float column of whole numbers into int64. A float column of whole numbers with a gap stays float64, where pandas answers Int64.

Added: the grouping made visible

DataFrameGroupBy and SeriesGroupBy now have ngroups, ndim, len, iteration, groups, indices, get_group and pipe, matching pandas 3. Each of them numbers the rows again when it is asked for and keeps nothing afterwards. A key written as one name is a value and a key written as a list is a tuple, as in pandas, a group with a missing key under dropna=False is keyed by nan and can be fetched with get_group(float("nan")), [[...]] narrows what a group hands out, and indices is numpy arrays when numpy is installed. groups is a plain dict rather than pandas' PrettyDict.

Added: Interval

• Interval(left, right, closed) with ends that are numbers, instants or spans, and left, right, closed, closed_left, closed_right, open_left, open_right, length, mid, is_empty and overlaps. Containment works for a point and for another interval, equality, ordering and hashing go by left, right and closed, and a number or a span moves both ends while a number scales them, all as in pandas. Bad ends, an unknown closed and a left end past the right one are refused in pandas' words.
• IntervalIndex and interval_range still wait for a column type that holds pairs.

Added: RangeIndex

• RangeIndex builds the labels of a range from a stop, a start and a stop, a step, a range or another RangeIndex, and RangeIndex.from_range takes a range. start, stop, step and the repr read as pandas' do, and whole numbers, the step and the dtype are checked in pandas' words. Every other method answers a plain Index, and a frame's default labels are not a RangeIndex yet.
• bdate_range takes unit among its keywords, which is how pandas' signature has it.

Added: calendar steps in date_range, and bdate_range

• date_range steps by business days (B, C), weeks on a weekday (W, W-WED), and the first or last day, plain or business, of every month, quarter and year (ME, MS, BME, BMS, QE, QS, BQE, BQS, YE, YS, BYE, BYS, with a month anchor like QE-JAN), with multiples and backwards steps. The start rolls forward to the first landing date and keeps its time of day, as in pandas, and a zone counts on the wall clock.
• bdate_range is date_range on business days with both ends moved to midnight, and C takes a week mask and holidays.
• The old names M, Q, Y, BY, BM, BQ, SM and CBM raise pandas' error pointing at the new name. Semi month, business hour and week of month steps are still refused with NotImplementedError.

Added: TimedeltaIndex and timedelta_range

• TimedeltaIndex builds an index of spans from text, whole nanoseconds, Timedelta and timedelta objects, or an index of spans, and reads days, seconds, microseconds, nanoseconds, components, total_seconds(), unit, asi8 and as_unit the way pandas does.
• timedelta_range takes any three of start, end, periods and freq, with closed, name and unit, picks the unit pandas picks from the ends and the step, and reads a compound step like 2D3h.
• The dt accessor on a column of spans gains seconds, microseconds, nanoseconds and components, with float64 where there are gaps as in pandas, and asking a span column for a calendar field it does not have raises AttributeError in pandas' words.
• A held freq, and dtype and copy on TimedeltaIndex, are refused with NotImplementedError.

Added: str.split, str.rsplit, str.join and str.wrap

• str.split and str.rsplit with expand=True cut every row at a separator, a regular expression or runs of whitespace and answer a frame with one column per piece, as wide as the row cut into the most pieces, with gaps padding the rest, the way pandas does. n limits the cuts, and the pattern is read as a regular expression by pandas' rule: when regex=True, when it is compiled, or when it is longer than one character and regex is left out. The columns are labelled with the text of their position, which is the registered integer column label divergence. expand=False answers a column of lists in pandas and is refused by name until there is a list column type.
• str.join puts a separator between the characters of every row, and str.wrap breaks every row into lines with textwrap and every argument pandas takes.

Added: ASOF JOIN

ASOF JOIN and ASOF LEFT JOIN written with ON now run, as in DuckDB. The condition is equal keys, which may be none, and exactly one inequality between a column on each side. For each left row the join takes the right rows whose keys agree and whose value passes the inequality, and of those the nearest one, so a.t >= b.t is the latest right row at or before the left row and a.t <= b.t the earliest at or after it. An inner ASOF join drops a left row that found nothing and an ASOF LEFT JOIN keeps it with the right side null. The compared columns can be numbers, dates or times. A new AsOf operator sorts the right side by key and value once and does a binary search per left row, and a filter on the left side can go under the join while one on the right side stays above it. ASOF with USING and ASOF RIGHT or FULL are still refused by name.

Added: POSITIONAL JOIN

SELECT ... FROM a POSITIONAL JOIN b now runs, as in DuckDB. Row i of the left side is put beside row i of the right side, and the answer is as long as the longer side, with the shorter side's columns null past its end. It takes no ON or USING. A new Positional operator holds the right side and counts the left rows as they go past, then hands out any right rows left over once the left side is done. A filter over the join stays above it, because moving it under would change which rows line up.

Added: a cross join onto more than one row runs

A cross join used to run only when its right side was a single row, and any other right side was refused when the plan was lowered. It now runs whatever the right side holds. A new Cross operator holds the right side whole and hands each left chunk on once per right row, and an empty right side gives an empty answer. Because a cross join is an inner join with nothing asked of the pair, predicate pushdown now sends a filter on its right side under it too, where before it only did that for the left side.

Added: right and full joins run

A RIGHT JOIN and a FULL JOIN now run instead of being refused when the plan is lowered. The join operator marks each build row a pairing used as the probe side streams past, and once the probe side is done it hands out the build rows nothing matched, with the other side null. Because of that mark the operator is fed one chunk at a time rather than run in parallel. A right join also takes a condition part that reads only the left side, which is tested below the join since a left row that fails it just pairs with nothing. A full join still takes equalities only, and a full join with USING is still refused.

Added: label lookup on instants and spans

• An index of instants now answers loc, at, [], get_loc, slice_indexer and in with a Timestamp, a datetime or text, the way pandas does. Text that names a year, quarter, month, day, hour or minute coarser than the labels picks every row inside that period, as a slice when the labels rise, and a slice with text bounds runs to the end of the period the upper bound names. Zoned labels read naive text on their own clock and refuse a naive Timestamp in pandas' words. An index of spans answers a Timedelta or its text the same way, and index[i] on either gives back the Timestamp or Timedelta.
• Timestamp(2020, 1, 1, 6, 30) spelled out by position now keeps the hour, minute, second and microsecond, where it used to read them one slot off.

Changed: an inner or semi join steps over probe rows that matched nothing

When the built side of an inner or semi join is small, most probe rows usually find nothing: TPC-H q8 probes six million lines against about 1,300 parts and under one in a hundred hits. The pairing walk still visited every row, twice, once to count and once to write. It now reads sixteen probe codes at a time and steps past the block in one go when all sixteen are the miss code, which has no rows behind it. Left, outer and anti joins walk every row as before, since a miss there still makes a row. On a busy 8 core VM this cut the instructions per run by about a fifth on q8 and a sixth on q5. The load on that box was too high for the wall clock to show it cleanly.

Added: merge_ordered

• firepanda.merge_ordered joins two frames sorted by the key, the way pandas.merge_ordered does, as an outer join by default and any of the four joins with how. fill_method="ffill" carries each side's last row down over the rows where it had no match, keeping an integer column integer once no gap is left, and left_by or right_by merges group by group and stacks the pieces in the order their values first appear. The mistakes pandas refuses raise its errors in its words.
• merge with an outer or right join on a key of instants, zoned instants or spans now keeps the key's type, where it used to come back as the integer counts underneath.

Added: merge_asof

• firepanda.merge_asof pairs each left row with the last right row at or before its key, or with direction="forward" or "nearest" the first at or after it or the closer one, only among right rows with the same by values and within tolerance, the way pandas.merge_asof does. Keys can be integers, floats or instants, as columns on both sides or as the labels of both sides, and allow_exact_matches=False skips equal keys. Unsorted keys, keys with gaps, mismatched key types and bad tolerances raise pandas' errors in pandas' words. A key on the labels of one side and a column of the other is refused for now.

Added: to_string

DataFrame.to_string and Series.to_string write the table of text pandas writes, from a port of its formatter. Floats get six digits after the point with the zeros trimmed across each column, or scientific form when a column needs it. Instants are dates alone when all are at midnight, and spans are days alone when none has a part of a day. Every column is padded to its widest cell, and a categorical column gets its categories line. columns, col_space, header, index, na_rep, formatters, float_format, index_names, justify, show_dimensions, decimal, max_colwidth and writing to a buffer or a path all work as in pandas. The row and column limits and line_width are refused, since the text is always the whole frame.

Added: to_timedelta

firepanda.to_timedelta turns text, numbers with a unit, Python timedeltas, lists and Series into spans the way pandas does. It picks the same resolution pandas picks (microseconds for text, the given unit for whole numbers, nanoseconds for floats), keeps the Series index and name, returns a timedelta column untouched, and supports errors="coerce". A bad unit, a boolean or a DataFrame raises the error pandas raises.

Fixed: read_json instants keep pandas' unit

read_json reads epoch columns in the unit it detects, so a column of epoch milliseconds is datetime64[ms] as in pandas rather than datetime64[ns]. Epochs with a fraction still go through nanoseconds.

Added: read_json

read_json reads what to_json writes, in every orient but table, as a frame or with typ="series" as a column. Each column goes through pandas' inference: text that is all numbers becomes numbers, whole floats become integers, flags with a gap become floats, and columns named like dates become instants when they read as epochs or ISO text, in the first unit that fits. Row labels are inferred the same way unless convert_axes=False. Floats are read with a port of pandas' own decoder, which is not always the nearest float, fused the way arm64 builds of pandas fuse it, unless precise_float asks for the nearest. dtype, convert_dates, keep_default_dates, date_unit, lines, nrows, encoding and compressed files work as in pandas. Column names stay text, and chunksize, the pyarrow engine and dtype_backend are refused by name.

Added: to_json on frames and columns

DataFrame.to_json and Series.to_json take pandas' whole signature and write the same text pandas writes. Floats go through a port of pandas' own encoder, so double_precision cuts and rounds the digits the way pandas does, and very large or very small values switch to exponent form at the same places. Every orient pandas knows except table works: columns, index, records, split and values, with lines, indent, force_ascii, default_handler, and index=False where pandas allows it. Instants and spans are counts in date_unit since the epoch, with pandas' deprecation warning, or ISO text with date_format="iso", zoned instants in UTC with a Z. The text is answered with no path, or written to a handle or a file, compressed by its name, and mode="a" appends lines. Misused options fail with pandas' messages.

Fixed: instants and spans cast to text

astype(str) on a column, a frame or an index of instants or spans wrote the count of units since the epoch, 1577836800000000, where pandas writes 2020-01-01. It now writes what pandas writes: the date alone when every instant is at midnight, as many fraction digits as the finest instant needs, a zone's offset at the end, and each span as 1 days 02:03:04, with missing values left missing.

Fixed: a frame from labels or names with no values

DataFrame(index=...), DataFrame(columns=...) and the two together dropped what they were given and built a frame with no rows and no columns. They now keep the rows and the columns the way pandas does, with every cell missing. pandas makes such columns objects, which firepanda does not have, so they are floats unless dtype= names another type.

Added: to_csv on frames and columns

DataFrame.to_csv and Series.to_csv write the text pandas writes. Floats come out the shortest way that reads back the same, or through float_format and decimal, missing values as na_rep, and instants with one shape for the whole column: the date alone when every instant is at midnight, and otherwise as many fraction digits as the finest one needs. sep, quoting, quotechar, escapechar, doublequote, lineterminator, header aliases, index_label, columns and date_format all follow pandas. With no path the text is answered. A path is written, appended to with mode="a", and compressed with gzip, bz2, xz or zip when its name or compression asks for it, and a text or byte handle gets the text.

Added: cut and qcut

cut puts each value in a bin, from a count of bins spread evenly over the values or from a list of edges, and qcut puts the edges at quantiles. Both follow pandas' rules for right, include_lowest, duplicates and retbins and answer pandas' errors. With labels=False they answer each bin's position, and with a list of text labels they answer an ordered category column. pandas labels bins with intervals by default, which firepanda does not have yet, so leaving labels out is refused with a message that says so.

Added: crosstab, from_dummies and lreshape

crosstab counts each pair of keys, or runs a named or Python function over the values of each pair. It supports pandas' margins, margins_name, dropna, rownames and colnames, and all four ways to normalize, including pandas' handling of the totals when normalising. Several keys on an axis, a missing or numeric column key, and totals beside numeric row labels are refused, since pandas holds those as a MultiIndex or as objects. from_dummies reads ones and zeros, or true and false, back into categories, with sep and default_category. lreshape stacks groups of columns into one column each. All three raise pandas' errors with pandas' messages.

Added: pandas' options

get_option, set_option, reset_option, describe_option, option_context and options work as they do in pandas 3.0. All 73 of pandas' options are registered with pandas' defaults and pandas' checks on a new value, a name is a pattern searched for in every option's name the way pandas searches, and the errors carry pandas' classes and messages. mode.copy_on_write and future.no_silent_downcasting give pandas' Pandas4Warning. info now reads display.max_info_columns, and the other options are kept but change nothing yet, since the printed form comes from the extension.

Added: pivot and pivot_table

DataFrame.pivot, DataFrame.pivot_table, firepanda.pivot and firepanda.pivot_table turn each value of one column into a column of its own, with the rows and columns sorted as pandas sorts them. pivot refuses two rows with one pair in pandas' words. pivot_table aggregates by name or with a Python function called on each group, fills a missing pair with fill_value, and turns whole numbers with a gap into floats as pandas does. firepanda refuses several keys, values left out with a columns key, lists of functions and margins, which pandas labels with a MultiIndex, and column values that are not text.

Added: to_numeric

firepanda.to_numeric reads values as numbers the way pandas does. Whole numbers in text are int64 and a decimal point or an exponent makes float64, space may surround a number but not an infinity, and empty text is NaN. A column answers a column, an index an index, a list or an array a numpy array and one value one number. errors="coerce" reads what is not a number as NaN, and downcast walks pandas' own list of types from the smallest up. dtype_backend and whole numbers past int64 in a column are refused.

Fixed: AbstractMethodError and PyperclipWindowsException are made as pandas makes them

AbstractMethodError now takes the object and the kind of method and says which class should have written it, and PyperclipWindowsException adds the last Windows error to its message, as pandas' classes do. Their signatures now match pandas', which the compat Board checks.

Added: sample, case_when, dot and compare

Series.sample and DataFrame.sample ask numpy's choice for positions the way pandas does, so the same random_state draws the same rows, with weights, frac, replace and ignore_index. Series.case_when applies its conditions last to first after moving the column to the type it shares with every replacement. Series.dot and DataFrame.dot, and @ on both, line the two sides up by label. Series.compare sets the values that differ side by side, with keep_shape, keep_equal and result_names. A common type of objects, the sides stacked on a MultiIndex and a product labelled with numbers are refused.

Fixed: filter reads a missing label as pandas does

filter with like or regex now matches a missing label as the text nan, or NaT for instants, because pandas renders every label with str before it looks. A search for a over text labels with a gap now keeps the gap, as pandas does.

Added: update, isetitem, from_dict, from_records and infer_objects on a frame, filter on a column and map on an index

DataFrame.update puts in another frame's or column's values column by column and label by label, with overwrite, filter_func and errors="raise" as pandas reads them, and refuses a join other than left with pandas' words. DataFrame.isetitem sets columns by position. DataFrame.from_dict reads a dict of columns, a dict of rows and pandas' tight form, and DataFrame.from_records reads tuples with columns or dicts, with index and exclude. Rows given as lists or tuples with no column names are refused, because their column names would be numbers. infer_objects on a frame and a column answers a copy. Series.filter keeps the values whose labels items, like or regex names. Index.map maps the labels and Index.ravel answers the index.

Added: the rest of pandas.errors

firepanda.errors now has a class for every class in pandas.errors, 40 more, from AbstractMethodError to ValueLabelTypeMismatch, each on the same builtins and under the same parents as pandas' class, so an except clause or a warnings filter that names one of them works unchanged. Most are raised by readers and writers firepanda does not have yet and are carried for the name.

Changed: a stored text key is compared through a flat head the walk can prefetch

LastingText found a stored key by a binary search over where each piece of keys starts and then read that piece's view, two dependent loads before the key's bytes, and the probe could not ask for any of them early. The store now keeps sixteen bytes a key beside the pieces, the length and first four bytes and then either the rest of a short key or where a long key's bytes are, so an ordinal names its head. The walk asks for a row's slot sixteen rows ahead, for the stored key's head eight rows ahead and for its bytes four rows ahead. Group also cuts its output into windows over the key columns instead of copying them, which rebuilt every text key a second time, and three scratch buffers that are written in full are no longer zeroed first. On a six core Linux machine that was busy with other work, alternating the old and new driver over eight rounds, q34 went from 121 to 93 ms at best, q33 from 169 to 146 and q18 from 187 to 173, and every answer matched.

Added: apply, agg, transform, mode and transpose on a frame, and agg and transform on a column

DataFrame.apply calls a function on each column, or on each row with axis=1, and puts the answers together as pandas does, values into a column and columns into a frame, with a name, a list or a dict going to agg. DataFrame.agg and aggregate take a name, a function, a list, which answers a row per function, or a dict, which picks the functions for each column and leaves NaN where a column was not asked for one. DataFrame.transform transforms each column. DataFrame.mode answers the most common values of each column, NaN after a column's last one. DataFrame.T and transpose swap rows and columns when the row labels are text, keeping the one type the columns share. Series.agg, aggregate and transform take a name or a function, and a list or a dict for agg. A transform that does not answer the same labels is refused with pandas' Function did not transform.

Fixed: a gap in map's numbers is NaN, and a mapping with no hits keeps its type

Series.map and everything built on it answer NaN rather than a null in a gap among numbers, because pandas' answer is a numpy column where NaN is the only gap. A mapping or a column used as a mapping that found no key answers a column of its own type, text staying text, as pandas does when it lines the mapping up against the values.

Added: map, apply, combine, update and transpose on a column, and map on a frame

Series.map takes a function, a mapping or another column read as a mapping from its labels, hands a missing value to the function as NaN unless na_action="ignore", and reads the answer's type from what came back, with whole numbers beside a gap made floats as pandas makes them. Series.apply calls a function on each value with args and keywords, or runs a reduction by name, or several by name as a column. Series.combine calls a function on each pair of values over both columns' labels with fill_value for a label one side lacks. Series.update puts in the other object's values that are not missing and refuses a value the column cannot hold with pandas' TypeError. Series.T and Series.transpose answer the column. DataFrame.map maps each column. firepanda.errors.InvalidIndexError is new, for a column with repeated labels used as a mapping. A category column's map is refused for now, because pandas maps the categories.

Added: to_numpy and values

DataFrame.to_numpy, Series.to_numpy and Index.to_numpy hand back a numpy array in the type pandas gives, with dtype and na_value as pandas reads them. Whole numbers with a gap become floats, instants with no zone stay instants, and text, categories and instants with a zone are objects. A frame's columns share one type, the wider number or objects. DataFrame.values is to_numpy(), and Series.values is numpy for numbers, flags and instants with no zone, and a FirepandaArray for the rest, where pandas has an extension array. numpy is imported only when one of these is called.

Added: index members a column already has

Index gains all, any, argmax, argmin, item, fillna, where, diff, round, value_counts, T, transpose, memory_usage, astype, droplevel, get_level_values, to_flat_index and infer_objects, with pandas' signatures. Each goes through the labels as a column and back, so the answer keeps the index's class and name. all and any count a missing label as true, as numpy does. item, argmax and argmin with a bad axis, droplevel and get_level_values raise pandas' errors. memory_usage is the number the labels weigh as a column, which counts the bits that mark missing values where pandas does not.

Fixed: filling a column of instants with an instant

fillna on a column of instants with no zone now takes a Timestamp or a datetime with no zone, or text naming an instant, and keeps the column's type and unit, as pandas does. It used to refuse all three.

Added: repeat, set_axis, DataFrame.idxmax and idxmin, and iterrows

Series.repeat and Index.repeat take each value and its label as many times as one count for all or one count each says, with a fraction cut to its whole part and numpy's errors for a negative count or counts that do not match the values. Series.set_axis and DataFrame.set_axis put new labels on the rows or the columns, an index bringing its name along, with pandas' length mismatch error. DataFrame.idxmax and idxmin answer each column's label of its extreme, or across a row the column that holds it, with skipna and numeric_only. DataFrame.iterrows walks the rows as columns named by their labels. Across a row, idxmax and idxmin refuse columns other than numbers, and set_axis refuses column labels that repeat.

Added: align

DataFrame.align and Series.align give two objects the same labels and answer the pair, as pandas does. An outer join takes every label, sorted when the sides differ, an inner join the labels both sides hold in this side's order, and left and right one side's labels, and both answers carry the leading side's index name. Two frames line up on both axes unless axis names one, a frame and a column line up on the axis given, and fill_value fills the new places. Labels that already agree come back unchanged. A join pandas does not know, a missing or wrong axis and an other that is neither a frame nor a column fail with pandas' error and message. Repeated labels that differ and a level other than 0 are refused with NotImplementedError.

Added: combine_first

DataFrame.combine_first and Series.combine_first fill every missing value from the same row and column of the other object, as pandas does. The rows are both sides' rows, sorted when the two differ, a frame keeps its own columns first and then adds the ones only the other frame has, and a column both sides hold takes the type the two have in common. Two types that pandas combines into an object column, and differing labels where one side repeats a label, are refused with NotImplementedError.

Fixed: searchsorted with several values answers an array

Series.searchsorted and Index.searchsorted given several values now answer a FirepandaArray of int64 positions, where they answered a list, since pandas answers a numpy array and a list is a different shape of answer. One value still answers one position.

Added: Series.searchsorted

Series.searchsorted answers where each value would go for the column to stay in order, with pandas' signature, as Index.searchsorted already did for labels. Both now take the values to search for as a list, a tuple, an index, a series or a numpy array, honour sorter, search a float among whole numbers and a whole number among floats, and refuse an unknown side with numpy's message, search side must be 'left' or 'right'.

Added: date_range

firepanda.date_range builds a DatetimeIndex of evenly stepped instants from any three of start, end, periods and freq, with pandas' rules for the unit, the zone, normalize, name and inclusive. A step of days with a zone keeps to the wall clock across a transition and every other step keeps to absolute time. With start, end and periods and no freq the points are spaced evenly. Calendar offsets such as months and weeks are refused with NotImplementedError, and the mistakes pandas refuses are refused with its classes and messages.

Added: value_counts on a category column

Series.value_counts answers a category column now instead of refusing. It counts by the categories as pandas does, so a category nothing uses is a row with a count of nought and the rows start in the order of the categories before the stable sort by count. The index is categorical with the column's categories and ordered flag, and a missing value is a row of its own when dropna=False and there is one to count. normalize, sort and ascending work as they do on any other column. The counting is a group by on the integer codes.

Added: str.cat joins others row by row

Series.str.cat(others) answers a column now instead of refusing. others is read the way pandas reads it: a series, a frame's columns, an index, a numpy array of one or two dimensions, a list of strings, or a list of series, indexes and arrays. The pieces meet by position when every label list is the column's own and by label under join otherwise, left, right, inner or outer in pandas' order. A row with a missing piece is missing unless na_rep stands in for it, and a piece that is not text, a list of the wrong length, a bare string and an unknown join raise pandas' class with pandas' message. The rows are joined in the interpreter, since the engine has no row by row concatenation yet.

A series built with a name that is not text, Series(values, name=4), answers the number now rather than '4', the way rename(4) already did.

Changed: a lasting tuple is written a key at a time

LastingTuple wrote each row's key tuple a row at a time across the keys and asked its lists which key was which on every row, paying a bounds check each time, which was about a third of the write on ClickBench q18. The size and write passes now go a key at a time down each morsel with a cursor per row, and a four or eight byte key is stored as one word rather than through memcpy. The bytes are the same as before. _bytes_equal also reads the bytes past its last whole word as one more overlapping word instead of one at a time. On a six core Linux machine that was busy with other work, alternating the old and new driver, q18 went from 228 to 200 ms at best and 267 to 230 at the median, q39 from 257 to 214 at best, and every answer matched.