The integration of SasView #199

AtomAnu · 2019-08-21T14:55:57Z

Description of Work

This is the integration of SasView. There are five main additions for this integration.

An additional example script, example_runScripts_SasView.py has been added to use SasView as the software to fit the user-selected problem(s). At the start of the example script, there are three condition statements with each statement verifying if one of the three required SasView packages is installed. Bumps, sasmodels and sascalc are the three required packages. As explained in the FitBenchmarking README.md file, Bumps and sasmodels can be installed using pip while sascalc is not an independent package.

Additionally, the package sasmodels contains a script data.py which contains a module called load_data. This module requires sascalc in order to function. In this module's code, the line of code which imports sascalc is written below.

from sas.sascalc.dataloader.loader import Loader

Furthermore, sascalc now exists under the directory src/sas which resides in the SasView repository (https://github.com/SasView/sasview)

As a consequence, it was neccessary to copy the folder sas from the SasView repository and place it inside FitBenchmarking (fitbenchmarking/sas) in order to run SasView headless inside FitBenchmarking.
A parsing script, parse_sasview_data.py has been added to parse SasView problem definition files and their corresponding data files. This script can later be integrated with parse_fitbenchmark_data.py.

in parse_sasview_data.py, the eval_f and get_function modules work in the same manner as that of parse_nist_data.py and parse_fitbenchmark_data.py. However, the module eval_f for parse_sasview_data.py accepts the parameters list in two formats. The first format is a string which contains the parameter names and values. The second format is to accept the parameter values as multiple parameters to the module.

In parse_fitbenchmark_data.py, a new module, get_bumps_function, is created to format Mantid functions into the functions that are acceptable by Bumps. Functions/models that are acceptable by Bumps should not contain any arguments (*arg) or keyword arguments (**kwargs).
A data preparation script, sasview/prepare_data.py, has been created to prepare the data in the correct format. There are three main tasks done in this script. The first task is to set the data errors (data_e) if they are not given in the data file. The second task is to set the X data range if specified. If the X data range is not specified, the range would be set from the minimum X value to the maximum X value. The final task is to create a data object of type sasmodels.data.Data1D so that it can be passed onto Bumps.
A function definition preparation script, sasview/func_def.py, has been created to prepare the models/functions and their starting parameter values for Bumps fitting. The script also prepares the initial and the final function definition strings for each fit.
Finally, a script, sasview/main.py, has been added to perform fitting using Bumps. All the fitting occur in the module fit. One can observe that this module is quite lengthy. This is because the method of configuring the function/model is different for SasView models and other functions from Scipy and Mantid. Nevertheless, this module could be shorten up in the future.

This script also calculates the chi-squared value and the run time of each fit. Then, a result object is created for the graph and table creating modules. A single result object contains the fitted Y values, the fit status, the chi-squared value, the fit runtime, the minimiser name, the initial function definition string and the final function definition string.

Note that the unit tests for these scripts have not been completed.

Testing Instructions

run example_runScripts.py with problems of type NIST, FitBenchmark and SasView
run example_runScripts_mantid.py with problems of type NIST, FitBenchmark and SasView
run example_runScripts_SasView.py with problems of type NIST, FitBenchmark and SasView

…rve_fit`

…nchmarking The file `example_runScripts_SasView.py` has been updated so that it will prompt a message if Bumps, sasmodels or sascalc is/are not installed. The folder `sas/sascalc` is added as sascalc is not an independent package to be installed via `pip` yet. This folder should be removed once there is an installation method for sascalc.

A script to test the modules in `parse_sasview_data.py` has been added. A set of mock problem files has also been added.

…o check if sasmodels is installed

This is a unit test file for `sasview/prepare_data.py`

This will be remove once bumps and sasmodels can be installed by `python setup.py install`.

lxml is required by sasmodels

Anders-Markvardsen · 2019-08-22T20:45:30Z

@AtomAnu thanks for this PR and it detailed description. You write "As a consequence, it was neccessary to copy the folder sas from the SasView repository and place it inside FitBenchmarking (fitbenchmarking/sas) in order to run SasView headless inside FitBenchmarking.".

A questions:

In the sasview meeting tuesday did you discuss with the sasview team this aspects? - i.e. to include a copy of parts of there code until the sasview team has separated sascal out as separate package.
- assuming this is OK then as a minimum please add readme in that folder to specify when the copy was made and a copy of your above reasoning for adding it.

Obviously the above is not ideal, but with the sasview team actively working on separating out sascal it seems like the best workaround

Anders-Markvardsen · 2019-08-23T05:53:44Z

Thinking about it again. As an alternative to copy and paste have you considered code which

clone the sasview repo in read only mode, either a particular version of it directly or where after the repo has been cloned it is reset to a particular version. The latter is to guard against the expected future changes of this part of the sasview codebase
copy the sascal part, and this might be a tricky part: to a location where it is visible from the python path as appropriate
delete the cloned sasview repo

See https://stackoverflow.com/questions/2472552/python-way-to-clone-a-git-repository for suggestions for how a github repo can be cloned

README.md

example_scripts/example_runScripts.py

AtomAnu · 2019-08-23T07:11:34Z

Dear Anders, The intention was for the purpose of testing. The problem_sets variable should be set to [“NIST/low_difficulty”]. Kind regards, Atom

…

On Fri, Aug 23, 2019 at 8:08 AM Anders Markvardsen ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In example_scripts/example_runScripts.py <#199 (comment)> : > @@ -77,7 +78,7 @@ # problem_sets = ["Neutron_data", "NIST/average_difficulty"] # problem_sets = ["CUTEst", "Muon_data", "Neutron_data", "NIST/average_difficulty", "NIST/high_difficulty", "NIST/low_difficulty"] -problem_sets = ["NIST/low_difficulty"] +problem_sets = ["Muon_data"] @AtomAnu <https://github.com/AtomAnu> did you mean to change problem set up Muon_data in this script? I.e. with problem_sets = Muon_data this script with not run without mantid — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#199?email_source=notifications&email_token=AMN6IMHOT4NINSWMBS57NALQF6EHPA5CNFSM4IOIDXA2YY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCCPJMTA#pullrequestreview-278828620>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AMN6IMH4FIDICPCPYETLQC3QF6EHPANCNFSM4IOIDXAQ> .

Anders-Markvardsen · 2019-08-23T19:36:15Z

fitbenchmarking/fitting/sasview/main.py

+    return results_problem, best_fit
+
+def fit(problem, data, function, minimizer, init_func_def):
+    """


code comment please

I meant def comment please

Anders-Markvardsen · 2019-08-23T19:38:12Z

fitbenchmarking/fitting/sasview/main.py

+    t_start, t_end = None, None
+    model = function[0]
+
+    if hasattr(model, '__call__'):


A few code comments here may help

Anders-Markvardsen · 2019-08-23T19:40:25Z

fitbenchmarking/fitting/sasview/main.py

+
+    final_param_values = result.x
+
+    fin_func_def = get_fin_function_def(final_param_values, problem, init_func_def)


what does 'fin' in fin_func_def stand for

The word fin stands for final in this case.

Anders-Markvardsen · 2019-08-23T19:41:59Z

fitbenchmarking/fitting/sasview/prepare_data.py

+
+from utils.logging_setup import logger
+
+def prepare_data(problem, use_errors):


prepare data for what?

Anders-Markvardsen · 2019-08-23T19:59:21Z

fitbenchmarking/fitting/scipy/func_def.py

        popt = list(popt)
        params = init_function_def.split("|")[1]
-        params = re.sub(r"[-+]?\d+\.\d+", lambda m, rep=iter(popt):
+        params = re.sub(r"[-+]?\d+[.]\d+", lambda m, rep=iter(popt):


looks subtle - code comment please and e.g. line 61 also

Anders-Markvardsen · 2019-08-23T20:00:33Z

fitbenchmarking/fitting/scipy/func_def.py

+
+    problem_type = extract_problem_type(problem)
+
+    if not 'name=' in str(problem.equation):


please add comment to specify what use case this is. And for else below also

Anders-Markvardsen · 2019-08-23T20:03:02Z

fitbenchmarking/parsing/fitbenchmark_data_functions.py

    function_defs = [[fit_function, params]]

    return function_defs

+def get_fit_function_without_kwargs(fit_function, functions_string):
+    """


please add short def description

Anders-Markvardsen · 2019-08-23T20:03:24Z

fitbenchmarking/parsing/fitbenchmark_data_functions.py

+def get_fit_function_without_kwargs(fit_function, functions_string):
+    """
+
+    :param fit_function:


some parameter info please

Anders-Markvardsen · 2019-08-23T20:03:45Z

fitbenchmarking/parsing/fitbenchmark_data_functions.py

@@ -179,6 +204,7 @@ def get_fitbenchmark_ties(param_set, ties):
        else:
            tie = param_set[start + 1:comma]
        ties_per_function.append(tie.replace("=", "': "))
+        # ties_per_function.append(tie)


dead comment?

Anders-Markvardsen · 2019-08-23T20:04:39Z

fitbenchmarking/parsing/parse_fitbenchmark_data.py

@@ -59,7 +59,7 @@ def __init__(self, fname):
        #String containing the function name(s) and the starting parameter values for each function
        self._equation = entries['function']

-        self._starting_values = None
+        # self._starting_values = (entries['function'].split(',', 1))[1]


dead comment?

Anders-Markvardsen

Functional testing all passes.
many thanks for this work - a monster PR!
Please address suggestions for changes in comments

These fixes were done as discussed in the Pull Request

…ng/fitbenchmarking into integratingSasView

… that can specify multiple software

Anders-Markvardsen · 2019-08-29T19:48:32Z

All definitions works with Scipy and Mantid. All work for SasView except for FitBenchmark type problems. For example by setting problem_sets = ["Neutron"] in example_runscript_sasview gives the error

C:\Midlertiddig\fitbenchmarking\fitbenchmarking\fitting\sasview\main.py in fit(problem, data, function, minimizer, init_func_def)
    107             # The problem type is NIST
    108             #Formatting the function parameter names
--> 109             formatted_param_names = [param[0] for param in problem.starting_values]
    110
    111         param_values = function[1]

TypeError: 'NoneType' object is not iterable

Remove SasView_example.py (not obvious what purpose it has)

Anders-Markvardsen · 2019-09-01T16:19:19Z

@AtomAnu thanks for the changes and again for your work over the summer.

I have created work for follow up work that this PR has highlighted during testing.

AtomAnu added 11 commits August 20, 2019 10:16

Add SasView example_runScripts

49bd1ea

Create parse file for SasView

519174c

Update prob_def_1.txt

7f6d3a9

Update parse_sasview_data.py

eb3d8dd

Further integration of SasView

e1e3259

SasView Successful Integration

8cda081

Enable FitBenchmarking to fit SasView models using 'scipy.optimize.cu…

d29060b

…rve_fit`

Update misc.py

1f60683

Some errors fixed in the case of using SasView models in scipy fitting

4c70d12

Futher Integration of SasView

53113fa

AtomAnu added the Enhancement New feature or request label Aug 21, 2019

AtomAnu requested review from Anders-Markvardsen and wathen August 21, 2019 14:55

AtomAnu added 11 commits August 21, 2019 16:03

Minor adjustments

9ee4a31

Update test_func_def.py

5690ad5

Removal of the test data files and update on parse.py

7c3d303

Update parse.py

0da9bae

Update parse_fitbenchmark_data.py

f45c1db

Update test_parse_fitbenchmark_data.py

5cd4ea9

Unit tests file for parse_sasview_data.py

260eb4b

A script to test the modules in `parse_sasview_data.py` has been added. A set of mock problem files has also been added.

Temporary condition statement added in test_parse_sasview_data.py t…

bcb1df4

…o check if sasmodels is installed

Add sasview/tests/test_prepare_data.py

5d1f847

This is a unit test file for `sasview/prepare_data.py`

Update setup.py to install bumps and sasmodels

be5c5e3

This will be remove once bumps and sasmodels can be installed by `python setup.py install`.

Update setup.py to install lxml

70fcfd6

lxml is required by sasmodels

Anders-Markvardsen reviewed Aug 23, 2019

View reviewed changes

README.md Outdated Show resolved Hide resolved

Anders-Markvardsen reviewed Aug 23, 2019

View reviewed changes

example_scripts/example_runScripts.py Outdated Show resolved Hide resolved

Anders-Markvardsen reviewed Aug 23, 2019

View reviewed changes

Anders-Markvardsen requested changes Aug 23, 2019

View reviewed changes

AtomAnu and others added 6 commits August 26, 2019 22:37

Minor fixes

6caf380

These fixes were done as discussed in the Pull Request

Merge branch 'integratingSasView' of https://github.com/fitbenchmarki…

8c83520

…ng/fitbenchmarking into integratingSasView

remove some unused imports

9d9eb18

updated doc for two of the example script, incluidng using [] to show…

328b657

… that can specify multiple software

added more clear doc utility nist and fitbenchmarking files

85d7d9a

update doc for sasview example file

92834a9

fix tests

2875c80

This was referenced Aug 30, 2019

Replace copied sascalc with download of these files #203

Closed

Documentation of code out of sync in top level files #205

Closed

sasview not working with neutron problem definitions #206

Closed

User install instruction out of sync with code #207

Closed

Anders-Markvardsen added 2 commits September 1, 2019 12:08

More doc update and remove file

845f9db

Remove SasView_example.py (not obvious what purpose it has)

Describe new sasview format and give description of all problem folders

6a3e68b

This was referenced Sep 1, 2019

Merge existing FitBenchmarking and SASView problem formats #209

Closed

Have code for creating functions for different software in one place #210

Closed

Anders-Markvardsen merged commit 27e7992 into master Sep 1, 2019

Anders-Markvardsen deleted the integratingSasView branch September 1, 2019 16:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

The integration of SasView #199

The integration of SasView #199

AtomAnu commented Aug 21, 2019

Anders-Markvardsen commented Aug 22, 2019

Anders-Markvardsen commented Aug 23, 2019

AtomAnu commented Aug 23, 2019 via email

Anders-Markvardsen Aug 23, 2019

Anders-Markvardsen Aug 23, 2019

Anders-Markvardsen Aug 23, 2019

Anders-Markvardsen Aug 23, 2019

AtomAnu Aug 23, 2019

Anders-Markvardsen Aug 23, 2019

Anders-Markvardsen Aug 23, 2019

Anders-Markvardsen Aug 23, 2019

Anders-Markvardsen Aug 23, 2019

Anders-Markvardsen Aug 23, 2019

Anders-Markvardsen Aug 23, 2019

Anders-Markvardsen Aug 23, 2019

Anders-Markvardsen left a comment

Anders-Markvardsen commented Aug 29, 2019

Anders-Markvardsen commented Sep 1, 2019


		final_param_values = result.x

		fin_func_def = get_fin_function_def(final_param_values, problem, init_func_def)


		from utils.logging_setup import logger

		def prepare_data(problem, use_errors):


		problem_type = extract_problem_type(problem)

		if not 'name=' in str(problem.equation):

The integration of SasView #199

The integration of SasView #199

Conversation

AtomAnu commented Aug 21, 2019

Description of Work

Testing Instructions

Anders-Markvardsen commented Aug 22, 2019

Anders-Markvardsen commented Aug 23, 2019

AtomAnu commented Aug 23, 2019 via email

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Anders-Markvardsen left a comment

Choose a reason for hiding this comment

Anders-Markvardsen commented Aug 29, 2019

Anders-Markvardsen commented Sep 1, 2019