Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for Mantid Jacobians and created a Mantid development parser #1231

Merged
merged 19 commits into from Mar 12, 2024

Conversation

AnthonyLim23
Copy link
Contributor

@AnthonyLim23 AnthonyLim23 commented Jan 26, 2024

Description of work

This PR adds Jacobians to the mantid parsers. However, it does not work for multistart problems.

It also adds a new mantiddev parser, which removes the advantage from the mantid software when evaluating mantid functions. This does not remove the advantage from mantid jacobians.
Fixes #1228

Testing Instructions

I can send some test problems that use both mantid and mantiddev. It also shows a comparison with nist problems and has similar order of results.

Checklist

  • The title is of a format appropriate for a line in future release notes.
  • I have added the appropriate tags to the PR.
  • My PR represents one logical piece of work.
  • My PR fully fixes the issue linked. If new issues have been created, what are they?
  • I have added the appropriate tests to cover code that has been added.
  • I have updated the documentation in the relevant places to cover the changes.

@AnthonyLim23 AnthonyLim23 added the pr-feature Tag PR as new feature in notes label Jan 26, 2024
Copy link

github-actions bot commented Jan 26, 2024

Unit & System Test Results

  4 files  ±0    4 suites  ±0   5m 57s ⏱️ +5s
477 tests +7  469 ✅ +7   8 💤 ±0  0 ❌ ±0 
908 runs  +7  853 ✅ +7  55 💤 ±0  0 ❌ ±0 

Results for commit 3c602fa. ± Comparison against base commit fc83cde.

This pull request removes 14 and adds 21 tests. Note that renamed tests count towards both.
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_parsers[expected0-cutest-/__w/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/cutest/start_end_x.SIF]
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_parsers[expected0-nist-/__w/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/nist/start_end_x.dat]
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_parsers[expected1-cutest-/__w/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/cutest/basic.SIF]
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_parsers[expected1-nist-/__w/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/nist/basic.dat]
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_parsers[expected10-hogben-/__w/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/hogben/start_end_x.txt]
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_parsers[expected11-hogben-/__w/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/hogben/basic.txt]
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_parsers[expected2-nist-/__w/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/nist/start_end_x.dat]
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_parsers[expected3-nist-/__w/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/nist/basic.dat]
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_parsers[expected4-mantid-/__w/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/mantid/start_end_x.txt]
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_parsers[expected5-mantid-/__w/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/mantid/basic.txt]
…
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_factory[mantiddev-/__w/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/mantiddev/basic.txt]
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_factory[mantiddev-/__w/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/mantiddev/start_end_x.txt]
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_function_evaluation[/__w/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/mantiddev/function_evaluations.json-mantiddev]
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_hessian_evaluation[/__w/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/mantiddev/hessian_evaluations.json-mantiddev]
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_jacobian_evaluation[/__w/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/mantiddev/jacobian_evaluations.json-mantiddev]
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_parsers[expected0-cutest-/__w/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/cutest/basic.SIF]
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_parsers[expected0-nist-/__w/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/nist/basic.dat]
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_parsers[expected1-cutest-/__w/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/cutest/start_end_x.SIF]
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_parsers[expected1-nist-/__w/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/nist/start_end_x.dat]
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_parsers[expected10-hogben-/__w/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/hogben/basic.txt]
…

♻️ This comment has been updated with latest results.

@AnthonyLim23 AnthonyLim23 marked this pull request as ready for review January 26, 2024 12:19
Copy link
Contributor

@AndrewLister-STFC AndrewLister-STFC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. 1 question about the cached jacobians, but I think that's it

fitbenchmarking/parsing/mantiddev_parser.py Outdated Show resolved Hide resolved
fitbenchmarking/parsing/mantiddev_parser.py Outdated Show resolved Hide resolved
Co-authored-by: Andrew Lister <48282025+AndrewLister-STFC@users.noreply.github.com>
@coveralls
Copy link
Collaborator

coveralls commented Feb 14, 2024

Coverage Status

coverage: 90.225% (+0.07%) from 90.157%
when pulling 3c602fa on 1228_mantid
into fc83cde on master.

@AndrewLister-STFC AndrewLister-STFC changed the title Mantid Jacobians and development mantid parser Added support for Mantid Jacobians and created a development mantid parser Mar 1, 2024
@AndrewLister-STFC AndrewLister-STFC changed the title Added support for Mantid Jacobians and created a development mantid parser Added support for Mantid Jacobians and created a Mantid development parser Mar 1, 2024
@AndrewLister-STFC
Copy link
Contributor

All working now apart from that I can't use lru cache with BiComplex... will have to have a think about how best to do this.

@AndrewLister-STFC
Copy link
Contributor

Seems like the caching isn't needed in numdifftools (identical accuracy tables and these runtimes):
image
vs
image

@AndrewLister-STFC
Copy link
Contributor

@AnthonyLim23 I think this is ready? Can you check my understanding is right on the docs update I added: https://fitbenchmarking--1231.org.readthedocs.build/en/1231/users/options/jacobian_option.html

@AnthonyLim23
Copy link
Contributor Author

@AnthonyLim23 I think this is ready? Can you check my understanding is right on the docs update I added: https://fitbenchmarking--1231.org.readthedocs.build/en/1231/users/options/jacobian_option.html

yep. Looks good to me.

@AndrewLister-STFC AndrewLister-STFC merged commit b4c65df into master Mar 12, 2024
9 of 10 checks passed
@AndrewLister-STFC AndrewLister-STFC deleted the 1228_mantid branch March 12, 2024 11:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-feature Tag PR as new feature in notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Mantid improvements
3 participants