Skip to content

feat(loaders): stream JSON via ijson; replace peek+replay with seek(0) where possible #125

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
garlontas merged 11 commits into from
Apr 13, 2026
+263 −49
Merged

feat(loaders): stream JSON via ijson; replace peek+replay with seek(0) where possible #125

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
7cf00ee
refactor: make JSON and XML loaders truly streaming/lazy
Copilot Apr 13, 2026
17116c6
address code review: add comments and type annotation
Copilot Apr 13, 2026
18ca525
Add ijson to dependencies in tox.ini
garlontas Apr 13, 2026
74e81a1
refactor: fix docstrings and remove trailing blank lines
deepsource-autofix[bot] Apr 13, 2026
663680c
fix: resolve syntax/lint errors in xml and json loaders, run poetry lock
Copilot Apr 13, 2026
15b9269
Correct docstring formatting in read method
garlontas Apr 13, 2026
638a2d8
Refactor XML parsing with helper functions
garlontas Apr 13, 2026
a343fe6
Refactor XML parsing logic in __xml_loader.py
garlontas Apr 13, 2026
f4b4b5a
Fix XML parsing to free space completely
garlontas Apr 13, 2026
029e56b
Update __xml_loader.py
garlontas Apr 13, 2026
3e20fa8
Fix XML parsing by removing unnecessary element removal
garlontas Apr 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 109 additions & 2 deletions poetry.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 3 additions & 1 deletion pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ packages = [
python = ">=3.10,<4.0"
joblib = ">=1.2.0"
defusedxml = { version = ">=0.7,<0.8", optional = true }
ijson = { version = ">=3.1", optional = true }
pyyaml = "^6.0.1"
tomlkit = "^0.13.2"
setuptools = ">=70.0.0"
Expand All @@ -24,7 +25,8 @@ setuptools = ">=70.0.0"
xml_loader = ["defusedxml"]
yaml_loader = ["pyyaml"]
toml_loader = ["tomlkit"]
all = ["defusedxml", "pyyaml", "tomlkit"]
json_loader = ["ijson"]
all = ["defusedxml", "ijson", "pyyaml", "tomlkit"]

[tool.poetry.group.test.dependencies]
parameterized = "*"
Expand Down
Loading
Loading