Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

State Saver #15

Merged
merged 10 commits into from
Oct 24, 2018
Merged

State Saver #15

merged 10 commits into from
Oct 24, 2018

Conversation

kenodegard
Copy link
Contributor

Simplified state saving/restoring.

Previously:

idx = self._save_idx()

# do peeking here
...

self._restore_idx(*idx)

Now:

with self._src.state:
    # do peeking here
    ...

And in cases where the marker also needs to be preserved:

with self._src.state(save_marker=True):
    # do peeking here
    ...

A bonus feature that may become valuable for number versus date parsing is the ability to save multiple states:

with self._src.state:
    # do some peeking here
    ...
    with self._src.state:
        # do some more peeking here
        ...

@kenodegard
Added state saving logic
887fc0b 
Instead of relying on the _save_idx and _resotre_idx functions, we introduced a context manager to handle the state storing and restoring for us.
@kenodegard
Added _Source
ca56679 
Added special unicode class that handles iterating through the source text. This unicode class provides all of the functionality (and speed) of the prior solution but also allows the parser to more easily restore to a previous point in the source text without needing to copy/tee the generator.
@kenodegard
Update variable name
7d79f90 
Updated marker variable to save_marker to be more descriptive of which marker variable this is.
@kenodegard
Switch index back to idx
5e66417 
Preserving prior variable conventions.
@kenodegard
Cleanup state handler
b5f0b79 
Moved state handler into src since it is directly related to the source text.
@kenodegard
Added back generators
189f69c 
After testing found that generators performed significantly better than indexing implementation that perserved TOMLChars.
@kenodegard
Addressed minor bugs
2782729 
Some bugs related to reverting back to the generator implementation.
@kenodegard
Fixed state saving to properly handle various restore cases
13df551 
Modified state preserver to only restore unless explicitly configured to restore or if an exception was raised during parsing. Introduced a new exception (Restore) which when raised within a state handler will cause the state to be restored, but then the exception is silently consumed by the state handler.
@kenodegard
Copy link
Contributor Author

You can see at these lines in _parse_value how I expect this state saver to be utilized.
https://github.com/sdispater/tomlkit/blob/13df55136bfe382aa29c365591799cff83567f27/tomlkit/parser.py#L631-L635

@kenodegard
Moved Source into a standalone file
ae34887 
Looking forward into the potential development of a more modular parsing. This shift further seeks to seperate the functionality between the source text handling and the actual parsing itself.
This was referenced Oct 22, 2018
Closed
Closed
Closed
@kenodegard
Copy link
Contributor Author

@sdispater anything I can do to get this rolling?

@kenodegard
Added consume method
92935b2 
Add a convenience method that is of great value to simplifying and streamlining the parsing functions.
@sdispater sdispater merged commit 3dcd42f into python-poetry:master Oct 24, 2018
@sdispater
Copy link
Member

Thanks a lot for the amount of work your put into this PR. I really appreciate it.

Looks good to me 👍

@kenodegard kenodegard deleted the state-saver branch October 24, 2018 15:26
capuanob pushed a commit to capuanob/tomlkit that referenced this pull request Mar 1, 2023
@sdispater
State Saver (python-poetry#15)
5249db0 
* Added state saving logic

Instead of relying on the _save_idx and _resotre_idx functions, we introduced a context manager to handle the state storing and restoring for us.

* Added _Source

Added special unicode class that handles iterating through the source text. This unicode class provides all of the functionality (and speed) of the prior solution but also allows the parser to more easily restore to a previous point in the source text without needing to copy/tee the generator.

* Update variable name

Updated marker variable to save_marker to be more descriptive of which marker variable this is.

* Switch index back to idx

Preserving prior variable conventions.

* Cleanup state handler

Moved state handler into src since it is directly related to the source text.

* Added back generators

After testing found that generators performed significantly better than indexing implementation that perserved TOMLChars.

* Addressed  minor bugs

Some bugs related to reverting back to the generator implementation.

* Fixed state saving to properly handle various restore cases

Modified state preserver to only restore unless explicitly configured to restore or if an exception was raised during parsing. Introduced a new exception (Restore) which when raised within a state handler will cause the state to be restored, but then the exception is silently consumed by the state handler.

* Moved Source into a standalone file

Looking forward into the potential development of a more modular parsing. This shift further seeks to seperate the functionality between the source text handling and the actual parsing itself.

* Added consume method

Add a convenience method that is of great value to simplifying and streamlining the parsing functions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@kenodegard @sdispater