Releases: GopalGB/pqtools
Release list
pqtools 0.8.0 - the container loop closes
Point at a .xlsx or .pbix, see the queries inside it, add one, save it back
into the file, and run it. That last half is what 0.8.0 adds.
pip install pqtools
pq list book.xlsx # Clients, Colors, Countries, Invoices, Stock
pq add book.xlsx --name "Top Colours" \
--source 'let S = #table({"Colour","N"},{{"Blue",5},{"Red",3}}) in S' --write
pq eval book.xlsx --member "Top Colours" --format csv
# Colour,N
# Blue,5
# Red,3New
pq list FILEnames every query in a container.pq add FILE --name N --source '...'adds a new one.--writesaves it in.--writenow works onformat/rename/replace-source/addfor
containers. Every container write copies the original to<file>.bakfirst.- Local-file sources run locally:
File.Contents,Csv.Document,
Text.FromBinary, and theBinary.*family. The engine boundary is about
capability, not the word "connector" -Sql.DatabaseandWeb.Contents
still refuse, by design. #tableand#binaryare registered, so a literal table written the way
a person actually writes it now runs.#shared/#sectionssay they need
the enclosing section document instead of reading like a typo.
Fixed - two bugs that had green test suites
- UTF-16 DataMashup. Real Excel writes
customXml/item1.xmlas UTF-16 LE
with a BOM. Every real workbook therefore reported "no DataMashup part
found". The test fixture wrote UTF-8, which is why nothing caught it. - Quoted identifiers were never unquoted.
[#"First Name"]looked up a
field literally named#"First Name". Quiet, because step names round-tripped
by matching themselves. Fixed at the root;containers.split_sharedhad its
own copy of the same bug.
How the write path was checked
680 tests, 89% coverage, ruff + mypy + the pinned Microsoft parser bridge clean.
Container writes are verified four ways: CRC intact, every zip member except the
DataMashup part byte-identical, the M reading back, and openpyxl - an
independent xlsx parser - opening the rewritten workbook to the same sheets and
cells as the original. Two real Excel-authored workbooks, 21,215 cells
compared, zero differences.
Residual risk, stated plainly: Excel itself has not opened a rewritten
workbook, because Excel was not installed where this was validated. That is why
every --write leaves a .bak.
Unofficial. Not affiliated with or endorsed by Microsoft. pq eval runs a
query's transformation chain locally; it is not a Power Query runtime and never
pretends to be one.
pqtools 0.6.0 - a real query runs with no --bind
Local-file sources run natively now. Csv.Document, File.Contents and
Text.FromBinary are implemented in pqtools itself, in pure Python, with no
new dependencies - so a query copied verbatim out of Power Query's Advanced
Editor, Source step included, evaluates without anything supplied.
$ pq eval report.pq --format csv
Region,Total,Orders
West,380,2
North,150,1
East,100,1
Why this changed
The old boundary treated "connector" as one indivisible thing only
Microsoft's Mashup Engine could cross. That was too broad. Sql.Database
genuinely needs an engine. Reading a local CSV is reading a local CSV, and
Csv.Document(File.Contents(...)) is the most common Source step in real
Power Query - refusing it forced everyone through --bind for the one case
that needs no help.
Added
- Csv.Document with the options-record and positional forms, multi-character
and whitespace delimiters, QuoteStyle.None, CsvStyle.QuoteAlways,
ExtraValues, named/typed/counted columns, and code-page decoding with a
UTF-8 BOM strip. It does NOT promote headers - it yields Column1..ColumnN
and Table.PromoteHeaders stays a separate step, exactly as Power Query
does. Auto-promoting would break every real query. - File.Contents, Text.FromBinary.
- Table.ReverseRows, Table.Repeat, Table.ExpandListColumn,
Table.SelectRowsWithErrors, Table.ReplaceErrorValues - found by probing the
115 functions the Power Query UI actually writes. 288 builtins total. - evaluate, EvalError and UnsupportedError are exported from the package
root. "from pqtools import evaluate" is the import everyone guesses first
and it did not work; a test now enforces that every import the README
shows resolves.
Still refused, and now named individually rather than by the word
"connector": Sql.Database, Web.Contents, SharePoint., Odbc.,
Oracle/PostgreSQL/MySQL.Database, Folder.*, Excel.Workbook. Credentials, a
network identity, driver type mapping and query folding have no honest local
stand-in, and each raises a typed error naming itself.
Deliberately still unimplemented, with reasons rather than silence:
Table.FuzzyNestedJoin (Microsoft's matching algorithm is unspecified),
Table.View (query folding with no backing source), Table.Schema (no declared
column types exist in this data model), Table.Profile and
Table.AggregateTableColumn (output column names unverified).
646 tests, 89% coverage, mypy and ruff clean.