Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Support for JSONL Writer #630

Merged
merged 4 commits into from Sep 27, 2022
Merged

Add Support for JSONL Writer #630

merged 4 commits into from Sep 27, 2022

Conversation

mzaeemz
Copy link
Contributor

@mzaeemz mzaeemz commented Sep 26, 2022

This PR has the objective of adding support for writing to JSONL files

Changes

  1. Updated tojson function in petl.io.json to write in JSONL format if lines argument is set to true

Checklist

Use this checklist for assuring the quality of pull requests that include new code and or make changes to existing code.

  • Source Code rules apply:
    • Includes unit tests
    • New functions have docstrings with examples that can be run with doctest
    • New functions are included in API docs
    • Docstrings include notes for any changes to API or behaviour
    • All changes are documented in docs/changes.rst
  • Versioning and history tracking rules apply:
    • Using atomic commits when possible
    • Commits are reversible when possible
    • There is no incomplete changes in the pull request
    • There is no accidental garbage added in source code
  • Testing rules apply:
    • Tested locally using tox / pytest
    • Automated testing passes (see CI)
    • Unit test coverage has not decreased (see Coveralls)
  • State of these changes is:
    • Just a proof of concept
    • Work in progress / Further changes needed
    • Ready to review
    • Ready to merge

@mzaeemz mzaeemz mentioned this pull request Sep 26, 2022
Closed
20 tasks
@coveralls
Copy link

Pull Request Test Coverage Report for Build 3126840750

  • 42 of 43 (97.67%) changed or added relevant lines in 2 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.02%) to 91.117%
Changes Missing Coverage Covered Lines Changed/Added Lines %
petl/io/json.py 10 11 90.91%
Totals Coverage Status
Change from base Build 3123360450: 0.02%
Covered Lines: 12750
Relevant Lines: 13993
💛 - Coveralls

github-advanced-security[bot]
github-advanced-security bot found potential problems Sep 26, 2022
View reviewed changes
petl/io/json.py
finally:
f.detach()


def _writeobj(encoder, obj, f, prefix, suffix):
def _writeobj(encoder, obj, f, prefix, suffix, lines=False):

Check warning

Code scanning / Pylint (reported by Codacy)

Argument name "f" doesn't conform to snake_case naming style

Argument name "f" doesn't conform to snake_case naming style
petl/io/json.py
finally:
f.detach()


def _writeobj(encoder, obj, f, prefix, suffix):
def _writeobj(encoder, obj, f, prefix, suffix, lines=False):

Check warning

Code scanning / Pylint (reported by Codacy)

Too many arguments (6/5)

Too many arguments (6/5)
petl/test/io/test_jsonl.py
assert result[2]['bar'] == 2


def test_tojson_2():

Check warning

Code scanning / Pylint (reported by Codacy)

Missing function docstring

Missing function docstring
petl/test/io/test_jsonl.py
@@ -49,3 +50,44 @@

ieq(expect, actual)
ieq(expect, actual) # verify can iterate twice


def test_tojson_1():

Check warning

Code scanning / Pylint (reported by Codacy)

Missing function docstring

Missing function docstring
petl/test/io/test_jsonl.py
['Alexa', [['two pair', '4S'], ['two pair', '9S']]],
['May', []],
['Deloise', [['three of a kind', '5S']]]]
f = NamedTemporaryFile(delete=False, mode='r')

Check warning

Code scanning / Pylint (reported by Codacy)

Variable name "f" doesn't conform to snake_case naming style

Variable name "f" doesn't conform to snake_case naming style
petl/test/io/test_jsonl.py
('a', 1),
('b', 2),
('c', 2))
f = NamedTemporaryFile(delete=False, mode='r')

Check warning

Code scanning / Pylint (reported by Codacy)

Variable name "f" doesn't conform to snake_case naming style

Variable name "f" doesn't conform to snake_case naming style
github-advanced-security[bot]
github-advanced-security bot found potential problems Sep 26, 2022
View reviewed changes
petl/test/io/test_jsonl.py
['Alexa', [['two pair', '4S'], ['two pair', '9S']]],
['May', []],
['Deloise', [['three of a kind', '5S']]]]
f = NamedTemporaryFile(delete=False, mode='r')

Check warning

Code scanning / Pylintpython3 (reported by Codacy)

Consider using 'with' for resource-allocating operations

Consider using 'with' for resource-allocating operations
petl/test/io/test_jsonl.py
('a', 1),
('b', 2),
('c', 2))
f = NamedTemporaryFile(delete=False, mode='r')

Check warning

Code scanning / Pylintpython3 (reported by Codacy)

Consider using 'with' for resource-allocating operations

Consider using 'with' for resource-allocating operations
petl/test/io/test_jsonl.py
assert result[2]['bar'] == 2


def test_tojson_2():

Check warning

Code scanning / Pylintpython3 (reported by Codacy)

Missing function or method docstring

Missing function or method docstring
petl/test/io/test_jsonl.py
['Alexa', [['two pair', '4S'], ['two pair', '9S']]],
['May', []],
['Deloise', [['three of a kind', '5S']]]]
f = NamedTemporaryFile(delete=False, mode='r')

Check warning

Code scanning / Pylintpython3 (reported by Codacy)

Variable name "f" doesn't conform to snake_case naming style

Variable name "f" doesn't conform to snake_case naming style
petl/test/io/test_jsonl.py
('a', 1),
('b', 2),
('c', 2))
f = NamedTemporaryFile(delete=False, mode='r')

Check warning

Code scanning / Pylintpython3 (reported by Codacy)

Variable name "f" doesn't conform to snake_case naming style

Variable name "f" doesn't conform to snake_case naming style
petl/test/io/test_jsonl.py
@@ -49,3 +50,44 @@

ieq(expect, actual)
ieq(expect, actual) # verify can iterate twice


def test_tojson_1():

Check warning

Code scanning / Pylintpython3 (reported by Codacy)

Missing function or method docstring

Missing function or method docstring
petl/io/json.py
finally:
f.detach()


def _writeobj(encoder, obj, f, prefix, suffix):
def _writeobj(encoder, obj, f, prefix, suffix, lines=False):

Check warning

Code scanning / Pylintpython3 (reported by Codacy)

Too many arguments (6/5)

Too many arguments (6/5)
petl/io/json.py
finally:
f.detach()


def _writeobj(encoder, obj, f, prefix, suffix):
def _writeobj(encoder, obj, f, prefix, suffix, lines=False):

Check warning

Code scanning / Pylintpython3 (reported by Codacy)

Argument name "f" doesn't conform to snake_case naming style

Argument name "f" doesn't conform to snake_case naming style
@juarezr juarezr added the Feature A nice to have thing that we don't have yet label Sep 27, 2022
@juarezr juarezr merged commit 9f065ba into petl-developers:master Sep 27, 2022
@juarezr
Copy link
Member

juarezr commented Sep 27, 2022

@mzaeemz,

Are you waiting for a release with this change?
Current releases in petl are issued based on contributions, not on a fixed schedule.
So we usually group some changes and release them.

@mzaeemz
Copy link
Contributor Author

mzaeemz commented Sep 27, 2022 •
edited

@mzaeemz,

Are you waiting for a release with this change? Current releases in petl are issued based on contributions, not on a fixed schedule. So we usually group some changes and release them.

@juarezr it's alright, I think the release can wait for a while.

However, there is an open issue for over 2 years now that would be resolved with the release.

@juarezr juarezr mentioned this pull request Sep 28, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@juarezr juarezr juarezr approved these changes

Assignees
No one assigned
Labels
Feature A nice to have thing that we don't have yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@mzaeemz @coveralls @juarezr @Wassey16