Global parameter by benjello · Pull Request #206 · liam2/liam2

benjello · 2016-01-13T15:01:34Z

For now I am able to introduce a DEBUG parameters.
@gdementen could you tell me if this seems ok and if I should proceed to the inclusion of any parameters set to a non iterable variable (or a variable that have with neither path nor defined in the node globals of the hdf file).

gdementen · 2016-01-29T15:59:48Z

liam2/entities.py

I'd prefer an isinstance check in this case

gdementen · 2016-01-29T16:20:31Z

Ok, so this PR looks good in principle. The implementation is a bit ugly to my taste (no real objective criticism), but I can fix that myself, but that validator thing is a no-go. Might need a separate PR for that. I would personally write that as a class:

class Or(object):
    def __init__(self, choices):
        ....

# and in the layout, you would use:
yaml_layout = {
         'import': None,
         'globals': {
          '*': Or(str, int, float, bool, {
                 'path': str,
                 'type': str,
                 'fields': [{
                     '*': None  # Or(str, {'type': str, 'initialdata': bool})
                 }],
                 'oldnames': {
                     '*': str
                 },
                 'newnames': {
                     '*': str
                 },
                 'invert': [str],
                 'transposed': bool
             })

or something like that...

benjello · 2017-04-19T14:41:43Z

@gdementen : I made some changes following your remarks. Feel free to ask more changes.

gdementen · 2017-04-19T15:09:31Z

liam2/data.py

                continue

            if name not in globals_node:
+                if isinstance(global_def, numbers.Number) or isinstance(global_def, bool):


wouldn't if np.isscalar(global_def) be more robust? I would move the check above, so that None of the code below is executed (it seems useless in this case). In fact, I would just add these lines just below globals_data = load_path_globals(globals_def):

globals_data.update({k: v for k, v in globals_def.iteritems() if np.isscalar(v)})

EDIT: In fact, I think this code is completely useless now that the constants are handled before the loop (currently in load_path_globals, but if you handle them in handle_constant_globals, that would be the same.

gdementen · 2017-04-19T15:52:15Z

liam2/simulation.py

+                    # provided in the file)
+                    v["fields"] = fields_yaml_to_type(v["fields"])
+                globals_def[k] = v
+            except TypeError:


I would rather use an np.isscalar(v) check than a try except

gdementen · 2017-04-19T15:56:06Z

liam2/data.py

    globals_data = {}
    for name, global_def in globals_def.iteritems():
-        if 'path' not in global_def:
+        if 'path' not in global_def and 'value' not in global_def:


"constant" globals shouldn't be handled in load_path_globals. Either in index_tables directly or in a separate method "handle_constant_globals".

gdementen · 2017-04-19T16:00:36Z

liam2/data.py

                for k, g_def in globals_def.iteritems():
+                    # Do not save global constants ie those who have a value
+                    # attribute
+                    if 'value' in g_def:


maybe change the test to if 'value' in g_def or 'path' in g_def: so that the if 'path' not in g_def test can be removed?

gdementen · 2017-04-19T16:01:58Z

liam2/importer.py

                        % ent_name)

+    if "value" in section_def:
+        return 'scalar', section_def["value"]


remove this change

gdementen · 2017-04-19T16:04:23Z

liam2/utils.py

        validate_dict(v, target, context)
    elif isinstance(target, list):
        validate_list(v, target, context)
+    elif isinstance(target, numbers.Number) or isinstance(target, bool):


this shouldn't be needed since you are using None as validator for the 'value' keyword

gdementen

Please also:

revert your changes to demo.h5 and small.h5 (unless you have a good reason, but then it needs to be documented)
add a changelog line
add the corresponding documentation

gdementen · 2017-09-22T11:37:38Z

liam2/data.py

                for k, g_def in globals_def.iteritems():
-                    if 'path' not in g_def:
-                        anyarray_to_disk(output_globals, k, globals_data[k])
+                    # Do not save global constants ie those who have a value


Do not save global constants (ie those who have a value attribute) nor globals loaded from external (.csv) files.

gdementen · 2017-09-22T11:38:57Z

liam2/tests/functional/simulation.yml


+            test_global_parameter:
+                - assertTrue(BOOL_CONSTANT)
+                - assertTrue(FLOAT_CONSTANT == 3.1415)


use assertEqual instead?

gdementen · 2017-09-22T11:50:07Z

liam2/data.py

+def handle_constant_globals(globals_def):
+    globals_data = {}
+    for name, global_def in globals_def.iteritems():
+        if 'value' not in global_def:


unsure restricting to scalars adds any value. Using constant array globals does not seem like such a bad idea.

the "or isinstance(value, bool)" part is useless (AFAICT) since np.isscalar(True) is True and np.isscalar(False) is True
So I would rewrite the content of the loop to something like:

if 'value' in global_def: globals_data[name] = global_def['value']

gdementen · 2017-09-22T11:54:34Z

liam2/entities.py

    symbols = {}
    for name, global_def in globals_def.iteritems():
-        global_type = global_def.get('fields')
+        if isinstance(global_def, dict):


this change is useless AFAICT: it would be useful if you could write:

BOOL_CONSTANT: True FLOAT_CONSTANT: 3.1415 INT_CONSTANT: 42

but the current "yaml validator" forbids that.

EDIT: I have just seen this is useful after all (because of the code in from_str) but I would rather have that changed. and Ideally, the case where value is given, type should be optional (ie inferred using type(value))

gdementen · 2017-09-22T12:00:37Z

liam2/simulation.py

        globals_def = {}
        for k, v in content.get('globals', {}).iteritems():
+            if 'value' in v:
+                if np.isscalar(v['value']) or isinstance(v['value'], bool):


unsure restricting to scalars adds any value

isinstance(v['value'], bool) seems useless

gdementen · 2017-09-22T12:12:33Z

liam2/simulation.py

        #                'MIG': {'type': int}}
        globals_def = {}
        for k, v in content.get('globals', {}).iteritems():
+            if 'value' in v:


this whole block of change could be removed IMO because it is mostly unneeded. All it does is transform {'INT_CONSTANT': {'value': 42, 'type': 'int'}} to {'INT_CONSTANT': 42}

I would rather avoid doing that and simply handle the "value" case in globals_symbols. Besides, I prefer if from_str is does not change the structure of the globals definition, just convert string (types) to real types.

benjello · 2017-09-22T16:48:58Z

Modifications and rebasing done.

gdementen · 2017-09-22T18:31:32Z

liam2/data.py

        if kind == 'table':
            fields, numlines, datastream, csvfile = info
            array = stream_to_array(fields, datastream, numlines)
+        elif kind == 'scalar':


are these two lines of any use? I do not see anything producing that kind = "scalar"???

gdementen · 2017-09-22T18:33:56Z

doc/usersguide/source/changes/version_0_12.rst.inc

 * implemented experimental align(link=) to use proportions in combination with the Chenard algorithm. It needs more
  testing though.

+* implemented a way of declaring global constants. These constant should be explicitely typed. (partially closes :issue:`203`): ::


why partially?

We cannot use:

INT_CONSTANT: 42

I consider the feature implemented, even though the syntax is not ideal. We can still improve the syntax later, but I will close the issue anyway.

benjello · 2017-09-24T22:20:02Z

Rebase done

gdementen

LGTM. Thanks!

gdementen reviewed Jan 29, 2016
View reviewed changes

liam2/entities.py Outdated

Copy link

Member

gdementen Jan 29, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer an isinstance check in this case

gdementen modified the milestone: 0.12 Apr 19, 2016

gdementen added the enhancement label Apr 19, 2016

benjello force-pushed the global_parameter branch from 6db15eb to 8452875 Compare April 19, 2017 09:01

gdementen reviewed Apr 19, 2017

View reviewed changes

gdementen reviewed Sep 22, 2017

View reviewed changes

Improve error message

9781618

benjello force-pushed the global_parameter branch 2 times, most recently from 3888b22 to e2f82b7 Compare September 24, 2017 22:16

Be able to set a global parameter in the yaml file

88542c0

benjello force-pushed the global_parameter branch from e2f82b7 to 88542c0 Compare September 24, 2017 22:25

gdementen approved these changes Sep 25, 2017

View reviewed changes

gdementen merged commit 08f6f0c into liam2:master Sep 25, 2017

benjello deleted the global_parameter branch September 25, 2017 10:17

benjello mentioned this pull request Sep 26, 2017

Set value of a global parameter in the yaml file should be possible #203

Closed

Conversation

benjello commented Jan 13, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gdementen commented Jan 29, 2016

Uh oh!

benjello commented Apr 19, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gdementen left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

benjello commented Sep 22, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

benjello commented Sep 24, 2017

Uh oh!

gdementen left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants