Create transfer_private_objects.py

[wimgielis]

First code attempt to get private objects transferred in an automated way. Objects are subsets, views, application entries. Main difficulty: the script is rather slow due many calls to the REST API. Also, application entries cannot be retrieved for the moment.

[MariusWirtz]

I think this could live in the Utils module. We need to parameterize it and break it down.

We could have functions like these:

• sync_subset(tm1_s, tm1_t, dimension_name: str, hierarchy_name: str, subset_name: str, private: bool)
• sync_application(tm1_s, tm1_t, path: str, application: str, private: bool)

[wimgielis]

Thank you Marius. I will work on this and deliver a PR when ready.

[wimgielis]

I wrote some code, for 2 files, but I cannot test it due to a weird circular import error. Apologies for this way of working and any error that might come from not having tested.

I noticed that in the Application Service, an "update_or_create" could be useful. Below a proposal, untested.

FOR THE APPLICATION SERVICE:

    def update_or_create(self, application: Application, private: bool = False, **kwargs) -> Response:
        """ update if exists, else create

        :param application:
        :param private:
        :param kwargs:
        :return:
        """
        if self.exists(path=application.path, application_type=application.application_type, name=application.name, private=private, **kwargs):
            return self.update(application=application, private=private, **kwargs)

        return self.create(application=application, private=private, **kwargs)

FOR THE UTILS:

def sync_subset( tm1_s: TM1Service, tm1_t: TM1Service, dimension_name: str, hierarchy_name: str, subset_name: str, private: bool):
    """synchronize a subset, public or private, from a source TM1 model to a target TM1 model
    existing objects are overwritten
    Impersonation must be used in case the owner of the private object is not the user for the connection

    :tm1_s: TM1Service of Source
    :tm1_t: TM1Service of Target
    :dimension_name: str, subset to be synchronized is part of what dimension
    :hierarchy_name: str, subset to be synchronized is part of what hierarchy
    :subset_name: str, subset to be synchronized
    :private: bool, a private or public subset
    :return: a subset object
    """
    if tm1_s != tm1_t:
        subset_s = tm1_s.dimensions.subsets.get(subset_name, dimension_name, hierarchy_name, private)
        return tm1_t.dimensions.subsets.update_or_create(subset_s, private)


def sync_view( tm1_s: TM1Service, tm1_t: TM1Service, cube_name: str, view_name: str, private: bool):
    """synchronize a view, public or private, from a source TM1 model to a target TM1 model
    existing objects are overwritten
    Impersonation must be used in case the owner of the private object is not the user for the connection

    :tm1_s: TM1Service of Source
    :tm1_t: TM1Service of Target
    :cube_name: str, view to be synchronized is part of what cube
    :view_name: str, view to be synchronized
    :private: bool, a private or public view
    :return: a view object
    """
    if tm1_s != tm1_t:
        view_s = tm1_s.cubes.views.get(cube_name, view_name, private)
        return tm1_t.cubes.views.update_or_create(view_s, private)


def sync_application( tm1_s: TM1Service, tm1_t: TM1Service, path: str, application_type: Union[str, ApplicationTypes], application_name: str, private: bool):
    """synchronize an application, public or private, from a source TM1 model to a target TM1 model
    existing objects are overwritten
    Impersonation must be used in case the owner of the private object is not the user for the connection

    :tm1_s: TM1Service of Source
    :tm1_t: TM1Service of Target
    :path: str, path of the application to be synchronized
    :application_type: str, type of the application to be synchronized
    :application_name: str, name of the application to be synchronized
    :private: bool, a private or public application
    :return: an application object
    """
    if tm1_s != tm1_t:
        application_s = tm1_s.applications.get(path=path, application_type=application_type, name=application_name, private=private)
        return tm1_t.applications.update_or_create(application=application_s, private=private)

[tomasfelcman]

Hi @wimgielis,
I have accidentally created very similar function for Application service - update_or_create, but in your code I am missing the "else" statement for the create. Please see my PR #1071 to react to that (I have tested my code internally).
I would also really appreciate any response regarding the creation of the Websheets and its default "blob" nature...
Thank you,
Tomas

[MariusWirtz]

@wimgielis, sorry for the late reply.

I just merged @tomasfelcman's work on the update_or_create function.
Please feel free to submit the 3 sync functions in a separate Pull Request.

Regarding the circular import, if you start with the current master branch this issue should no longer exist.
if it still persists try to reorder the imports or move the troublesome import into a function scope.