New Merge Method proposal

[alex-ber]

I've change your deep merge method to make another merge variant.
I don't want to list to be merge, I want them to be replaced.

My use case is the following: I have config.yml and config-dev.yml, etc (dev is profile). I want default configuration to be taken from config.yml and if I use dev profile appropriate configuration I want to be overridden by config-dev.yml. Spring Boot (this is Framework in Java) don't merge lists, it replaces them, so I want to mimic this behavior.

It will be nice, if you will incorporate this change in your code base. Thanks.

This is my variant of hiyapyco.HiYaPyCo.__deepmerge:

def __deepmerge(self, a, b):
    logger.debug('>' * 30)
    logger.debug('deepmerge %s and %s' % (a, b,))
    # FIXME: make None usage configurable
    if b is None:
        logger.debug('pass as b is None')
        pass

    #MYRPOPOSAL:, treat listTypes as primitiveTypes in merge
    #subsititues list, don't merge them

    if a is None or isinstance(b, primitiveTypes) or isinstance(b, listTypes):
        logger.debug('deepmerge: replace a "%s"  w/ b "%s"' % (a, b,))
        a = b

    elif isinstance(a, dict):
        if isinstance(b, dict):
            logger.debug('deepmerge: dict ... "%s" and "%s"' % (a, b,))
            for k in b:
                if k in a:
                    logger.debug('deepmerge dict: loop for key "%s": "%s" and "%s"' % (k, a[k], b[k],))
                    a[k] = self._deepmerge(a[k], b[k])
                else:
                    logger.debug('deepmerge dict: set key %s' % k)
                    a[k] = b[k]
        elif isinstance(b, listTypes):
            logger.debug('deepmerge: dict <- list ... "%s" <- "%s"' % (a, b,))
            for bd in b:
                if isinstance(bd, dict):
                    a = self._deepmerge(a, bd)
                else:
                    raise HiYaPyCoImplementationException(
                        'can not merge element from list of type %s to dict (@ "%s" try to merge "%s")' %
                        (type(b), a, b,)
                    )
        else:
            raise HiYaPyCoImplementationException(
                'can not merge %s to %s (@ "%s" try to merge "%s")' %
                (type(b), type(a), a, b,)
            )
    logger.debug('end deepmerge part: return: "%s"' % a)
    logger.debug('<' * 30)
    return a

[zerwes]

Hello. Would you please be so kind and provide me a patch / diff ?

[alex-ber]

Index: hiyapyco/init.py
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8

--- hiyapyco/init.py (revision 8ab38a0)
+++ hiyapyco/init.py (date 1582205392539)
@@ -65,9 +65,11 @@

you may set this to something suitable for you

jinja2env = Environment(undefined=Undefined)

-METHODS = { 'METHOD_SIMPLE':0x0001, 'METHOD_MERGE':0x0002 }
+METHODS = { 'METHOD_SIMPLE':0x0001, 'METHOD_MERGE':0x0002, 'METHOD_SUBSTITUTE':0x0003 }
METHOD_SIMPLE = METHODS['METHOD_SIMPLE']
METHOD_MERGE = METHODS['METHOD_MERGE']
+METHOD_SUBSTITUTE = METHODS['METHOD_SUBSTITUTE']
+

class HiYaPyCo():
"""Main class"""
@@ -75,8 +77,8 @@
"""
args: YAMLfile(s)
kwargs:

•      * method: one of hiyapyco.METHOD_SIMPLE | hiyapyco.METHOD_MERGE
  

•      * mergelists: boolean (default: True) try to merge lists (only makes sense if hiyapyco.METHOD_MERGE)
  

•      * method: one of hiyapyco.METHOD_SIMPLE | hiyapyco.METHOD_MERGE | hiyapyco.METHOD_SUBSTITUTE
  

•      * mergelists: boolean (default: True) try to merge lists (only makes sense if hiyapyco.METHOD_MERGE or hiyapyco.METHOD_SUBSTITUTE)
       * interpolate: boolean (default: False)
       * castinterpolated: boolean (default: False) try to cast values after interpolating
       * usedefaultyamlloader: boolean (default: False)
  

@@ -238,6 +240,8 @@
else:
if self.method == METHOD_SIMPLE:
self._data = self._simplemerge(self._data, ydata)

•            elif self.method == METHOD_SIMPLE:
  

•                self._data = self._deepmerge(self._data, ydata)
             else:
                 self._data = self._deepmerge(self._data, ydata)
             logger.debug('merged data: %s' % self._data)
  

@@ -329,6 +333,50 @@
(type(b), type(a), a, b,)
)
return a
+

• def __substmerge(self, a, b):

•    logger.debug('>' * 30)
  

•    logger.debug('deepmerge %s and %s' % (a, b,))
  

•    # FIXME: make None usage configurable
  

•    if b is None:
  

•        logger.debug('pass as b is None')
  

•        pass
  

•    # treat listTypes as primitiveTypes in merge
  

•    # subsititues list, don't merge them
  

•    if a is None or isinstance(b, primitiveTypes) or isinstance(b, listTypes):
  

•        logger.debug('deepmerge: replace a "%s"  w/ b "%s"' % (a, b,))
  

•        a = b
  

•    elif isinstance(a, dict):
  

•        if isinstance(b, dict):
  

•            logger.debug('deepmerge: dict ... "%s" and "%s"' % (a, b,))
  

•            for k in b:
  

•                if k in a:
  

•                    logger.debug('deepmerge dict: loop for key "%s": "%s" and "%s"' % (k, a[k], b[k],))
  

•                    a[k] = self._deepmerge(a[k], b[k])
  

•                else:
  

•                    logger.debug('deepmerge dict: set key %s' % k)
  

•                    a[k] = b[k]
  

•        elif isinstance(b, listTypes):
  

•            logger.debug('deepmerge: dict <- list ... "%s" <- "%s"' % (a, b,))
  

•            for bd in b:
  

•                if isinstance(bd, dict):
  

•                    a = self._deepmerge(a, bd)
  

•                else:
  

•                    raise HiYaPyCoImplementationException(
  

•                        'can not merge element from list of type %s to dict (@ "%s" try to merge "%s")' %
  

•                        (type(b), a, b,)
  

•                    )
  

•        else:
  

•            raise HiYaPyCoImplementationException(
  

•                'can not merge %s to %s (@ "%s" try to merge "%s")' %
  

•                (type(b), type(a), a, b,)
  

•            )
  

•    logger.debug('end deepmerge part: return: "%s"' % a)
  

•    logger.debug('<' * 30)
  

•    return a
  

  def _deepmerge(self, a, b):
  logger.debug('>'*30)
  @@ -436,8 +484,8 @@

  args: YAMLfile(s)
  kwargs:

•  * method: one of hiyapyco.METHOD_SIMPLE | hiyapyco.METHOD_MERGE
  

•  * mergelists: boolean (default: True) try to merge lists
  

•  * method: one of hiyapyco.METHOD_SIMPLE | hiyapyco.METHOD_MERGE | hiyapyco.METHOD_SUBSTITUTE
  

•  * mergelists: boolean (default: True) try to merge lists (only makes sense if hiyapyco.METHOD_MERGE or hiyapyco.METHOD_SUBSTITUTE)
   * interpolate: boolean (default: False)
   * castinterpolated: boolean (default: False) try to cast values after interpolating
   * usedefaultyamlloader: boolean (default: False)
  

[alex-ber]

My comment above is proposed patch. Sorry for formatting issues.

[zerwes]

sorry, I need something usable.
can you please fork the repo and commit your proposes changes into the fork?
and maybe you can even create a pull request then ...
Greetings

[alex-ber]

patch.txt

You should be more clear about this. Uploading patch as attachment.

[alex-ber]

Please, see #37

[zerwes]

PR #37 merged
released 0.4.16