v0.21.0

Added

• All Pathway types can now be serialized to CSV using pw.io.csv.write and deserialized back using pw.io.csv.read.
• pw.io.csv.read now parses null-values in data when it can be done unambiguously.

Changed

• BREAKING: Updated endpoints in pw.xpacks.llm.question_answering.BaseRAGQuestionAnswerer:
  • Deprecated: /v1/pw_list_documents, /v1/pw_ai_answer
  • New: /v2/list_documents, /v2/answer
• RAG methods under the pw.xpacks.llm.question_answering.RAGClient are re-named, and they now use the new endpoints. Old methods are deprecated and will be removed in the future.
  • pw_ai_summary -> summarize
  • pw_ai_answer -> answer
  • pw_list_documents -> list_documents
• When pw.io.deltalake.write creates a table, it also stores its metadata in the columns of the created Delta table. This metadata can be used by Pathway when reading the table with pw.io.deltalake.read if no schema is specified.
• The schema parameter is now optional for pw.io.deltalake.read. If the table was created by Pathway and the schema was not specified by user, it is read from the table metadata.
• pw.io.deltalake.write now aligns the output metadata with the existing table's metadata, preserving any custom metadata in the sink.
• BREAKING: The Bytes type is now serialized and deserialized with base64 encoding and decoding when the CSV format is used.
• BREAKING: The Duration type is now serialized and deserialized as a number of nanoseconds when the CSV format is used.
• BREAKING: The tuple and np.ndarray types are now serialized and deserialized as their JSON representations when the CSV format is used.

Fixed

• pw.io.csv.write now correctly escapes quote characters.