Problems Adding External resources with references in JSON

[pablo-campillo]

I want to split the model in several files. However, when I try to join the models, attributes that are references to an external resource are None.

My toy example (all files attached at files.zip):

Code:

from functools import partial
import pyecore.ecore as Ecore
from pyecore.ecore import *


name = 'miniModel'
nsURI = 'http://www.example.org/miniModel'
nsPrefix = 'miniModel'

eClass = EPackage(name=name, nsURI=nsURI, nsPrefix=nsPrefix)

eClassifiers = {}
getEClassifier = partial(Ecore.getEClassifier, searchspace=eClassifiers)


class Version(EObject, metaclass=MetaEClass):

    name = EAttribute(eType=EString, unique=True, derived=False, changeable=True)

    def __init__(self, *, name=None):
        # if kwargs:
        #    raise AttributeError('unexpected arguments: {}'.format(kwargs))

        super().__init__()

        if name is not None:
            self.name = name
            
            
class Project(EObject, metaclass=MetaEClass):

    name = EAttribute(eType=EString, unique=True, derived=False, changeable=True)
    versions = EReference(eType=Version, ordered=True, unique=True, containment=True, derived=False, upper=-1)

    def __init__(self, *, versions=None, name=None):
        # if kwargs:
        #    raise AttributeError('unexpected arguments: {}'.format(kwargs))

        super().__init__()

        if name is not None:
            self.name = name

        if versions:
            self.versions.extend(versions)


class RefProject(EObject, metaclass=MetaEClass):

    name = EAttribute(eType=EString, unique=True, derived=False, changeable=True)
    version = EReference(eType=Version, ordered=True, unique=True, containment=False, derived=False)

    def __init__(self, *, name=None, version=None):
        # if kwargs:
        #    raise AttributeError('unexpected arguments: {}'.format(kwargs))

        super().__init__()

        if name is not None:
            self.name = name

        if version is not None:
            self.version = version

The test:

from pyecore.resources import ResourceSet, URI
from pyecore.resources.json import JsonResource

from .miniModel import Project, Version, RefProject


class TestTwoModelFiles:
    def test_create_model(self):
        v0 = Version(name="V0")
        v1 = Version(name="V1")
        
        p = Project(name='MyProject', versions=[v0, v1])

        rset = ResourceSet()
        rset.resource_factory['json'] = JsonResource

        rset = ResourceSet()
        rset.resource_factory['json'] = JsonResource
        resource = rset.get_resource(URI('miniModel.ecore'))
        mm_root = resource.contents[0]
        rset.metamodel_registry[mm_root.nsURI] = mm_root
        
        resource = JsonResource(URI('project.json'))
        resource.append(p)
        resource.save()

        ref_p = RefProject(name="RefMyProject", version=v1)
        resource = JsonResource(URI('ref_project.json'))
        resource.append(ref_p)
        resource.save()

        rset = ResourceSet()
        rset.resource_factory['json'] = JsonResource
        resource = rset.get_resource(URI('miniModel.ecore'))
        mm_root = resource.contents[0]
        rset.metamodel_registry[mm_root.nsURI] = mm_root

        rset.get_resource(URI('project.json'))
        resource = rset.get_resource(URI('ref_project.json'))
        ref_project = resource.contents[0]
        
        assert ref_project.version.name == "V1" 

The content of the serialized files:

{
  "eClass": "http://www.example.org/miniModel#//Project",
  "versions": [
    {
      "name": "V0"
    },
    {
      "name": "V1"
    }
  ],
  "name": "MyProject"
}

{
  "eClass": "http://www.example.org/miniModel#//RefProject",
  "name": "RefMyProject",
  "version": {
    "eClass": "http://www.example.org/miniModel#//Version",
    "$ref": "<pyecore.resources.resource.URI object at 0x7f871bc29460>//@versions.1"
  }
}

The error:

>       assert ref_project.version.name == "V1"
E       AttributeError: 'NoneType' object has no attribute 'name'

test.py:43: AttributeError

files.zip

[pablo-campillo]

The ecore file was missing, now it is included! :-)

files.zip

Thank you in advance!

[aranega]

Hi @pablo-campillo ,

Thanks for the ticket and the use-case! I think you found a nasty bug here, I can see from the serialized json that something is wrong in the computed href. I'll try to fix it asap. I suspect a regression over a not well covered aspect of the json serialization. I suppose XMI doesn't have this regression.

Thanks again, I will fix that very soon!

[aranega]

Hi @pablo-campillo

Thanks again for the use-case and the issue. I fixed it on develop, I cannot believe I didn't see it before, and I think I did because there was another bug in the JSON deserialization. Both bugs were hidding each others. Bottom line, I will improve my test set to catch well this case.

[pablo-campillo]

@aranega , Thanks to you!

[aranega]

Hi @pablo-campillo ,

Thanks for the report! I tried your use-case, but I cannot reproduce the problem on my side 😬, everything is going just fine... For the execution of the code, I'm on the develop branch with Python 3.10. What version of Python are you using ? The error would look like a loose of sync between the dynamic and static part of your metamodel, but I'm unsure how this could happen (could it be some tests that are loading metamodels two times or something like that?)

[pablo-campillo]

Hi @aranega,

I removed the issue because the problem was solved, I think I was loading the metamodel two times or something like that.

Now, I found a problem, when I deserialize a model instead of get instances of object generated static by pyecoregen I got EProxy objects, please, do you know why?

Thank you very much!

[aranega]

I'm glad this issue went away :)

Regarding the proxies, yes, everytime you deserialize models that are split into different resources, PyEcore puts a proxy to reference the external object. EMF does the same, but it has a mechanism that then transfers the real instance and reconnect things, destroying proxies. In PyEcore, the proxies stays, but acts like a transparent proxy, meaning that you can handle it as a normal instance and it will resolve the proxy on it's own when you access any feature of the object (loading the external resource automatically if this resource has not been loaded before). To avoid this, I should slightly change the way resources are deserialized by checking first if the external resource owning the object is already loaded in the resource set and resolve it from there. I think there is not so much work on that, but perhaps there is corner cases to deal with that. This joins a little bit what I can do to deal with #120 .

[pablo-campillo]

Ok, I understand. The problem is that isinstance(obj, Class) does not work, right?

[aranega]

It should work if the proxy can be resolved meaning if the external resource can be loaded (if it wasn't in the first place), EClass and special metaclass redefine __instancecheck__ and other dunder methods. I have this behavior on my side:

from pyecore.ecore import EClass, EProxy

A = EClass('A')
instance = A()
proxy = EProxy(wrapped=instance)

assert isinstance(instance, A)
assert isinstance(proxy, A)