Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Main loop api and singlelogparser class #22

Merged
merged 23 commits into from
Jan 13, 2022
Merged

WIP: Main loop api and singlelogparser class #22

merged 23 commits into from
Jan 13, 2022

Conversation

simonbowly
Copy link
Owner

@simonbowly simonbowly commented Jan 12, 2022

This adds:

  • SingleLogParser class to parse a full log by delegating to the section handlers
  • Main loop with both parse("*.log").summary() and get_dataframe(["*.log"]) alternatives for compatibility with the old API

@maliheha there is still a bit to do here but could you take a look at single_log_parser.py and api.py?

Some other changes:

  • Removed feather files, regression tests now run directly against the old code still in the repo
  • Updated grblogtools.py to v1.3.1 (merged in master branch) to test against this code directly

@simonbowly
Copy link
Owner Author

Also - integrating the termination parser with the different termination messages from barrier and nodelogs is looking tricky. Switching the termination parser settings on the fly once we determine the model type doesn't seem to fit very well. What do you think of the following?

  1. Include "Optimal objective" in the barrier parser
  2. Include "Best objective/best bound/solution count" as well as "Optimal solution found" in the nodelog parser, since they are specific to models with a node log
  3. Instead of the termination parser, have a status message parser which picks up "Time limit reached", "Interrupted" and so on (without interrupting other parsers).

In the case of a MIP where barrier solves the relaxation but the node processing times out, when merging all the log info, Status="TIME_LIMIT" from the status parser would then taken precedence over Status="OPTIMAL" coming from the barrier parser, and we would get the right result.

Copy link
Collaborator

@maliheha maliheha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for all the work! It looks very great!

I looked at the two modules api.py and single_log_parser.py and wrote three comments. In one of the comments, you will find a possible idea for addressing the termination parser.

There are a few TODOs in these tow files:

  • TODO fill_default_parameters could be much cleaner now, without column regexes
  • TODO extend this to also check multiple logs from one file.
  • TODO extend this with other args from the old api
  • Termination parser

I can take care of some of these. For example, if you agree with the termination idea, I can give it a try. I can also work on extending the logic to multiple logs in one file. We can probably turn the remaining TODOs into issues and work on them.

There are some inconsistency in the docstrings formats that we can take care of it at the very end.

src/grblogtools/api.py Outdated Show resolved Hide resolved
src/grblogtools/nodelog.py Outdated Show resolved Hide resolved
src/grblogtools/single_log_parser.py Show resolved Hide resolved
@simonbowly simonbowly merged commit db8cc94 into master Jan 13, 2022
@simonbowly simonbowly deleted the main-loop branch January 13, 2022 22:19
simonbowly pushed a commit that referenced this pull request Feb 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants