migrate_tables_columns_to_semantic_model - source- and target semantic models in different workspaces

[MarcAHunt]

Hi m-kovalsky,

first of all thx for this great tool.

I have a small issue with migrating tables and columns to another semantic model.

It all works fine if source- and target-semantic-model reside in the same workspace,
but I can not seem to get it to work if my target-semantic-model is in a different workspace.

What I actually would want to achieve is building new semantic models in a dev-workspace, once I am fine with it, push them to a production-workspace.
Your tool indicates this could be possible.

Here is what I'm doing (copied of notebook cell):
import fabric_cat_tools as fct
import time

newDatasetName = 'targetSM' #Enter the new Direct Lake semantic model name
""" comment
-> I can create the newDataset in any workspace I want.
"""
fct.create_blank_semantic_model(newDatasetName, workspaceName='DevWS')

time.sleep(5)

""" comment
-> re-creates all tables and columns of source_semantic_model (=param 1) in target_semantic_model (=param 2)
-> lakehouseName and lakehouseWorkspaceName define the workspace and lakehouse where the data should be coming from. (Switching datasource, all fine!)

-> PROBLEM:
It does not seem to work if source- and target-semantic-model do not reside in the same workspace!
"""
fct.migrate_tables_columns_to_semantic_model( 'sourceSM', 'targeSM', workspaceName= 'DevWS', lakehouseName= 'LH_Gold', lakehouseWorkspaceName='ProdWS')

fct.migrate_model_objects_to_semantic_model('sourceSM', 'targetSM', workspaceName= 'DevWS')

I am not sure if that's an issue. Could be a feature.

Thx a million for a feedback.

Best Regards from Germany,
Marc

[m-kovalsky]

Yes, I am aware of this current limitation. This capability will be added in a future version (you will have another parameter to set the workspace for the new semantic model). Stay tuned...!

…

________________________________ From: MarcAHunt ***@***.***> Sent: Wednesday, April 10, 2024 4:05:36 PM To: m-kovalsky/fabric_cat_tools ***@***.***> Cc: Subscribed ***@***.***> Subject: [m-kovalsky/fabric_cat_tools] migrate_tables_columns_to_semantic_model - source- and target semantic models in different workspaces (Issue #2) Hi m-kovalsky, first of all thx for this great tool. I have a small issue with migrating tables and columns to another semantic model. It all works fine if source- and target-semantic-model reside in the same workspace, but I can not seem to get it to work if my target-semantic-model is in a different workspace. What I actually would want to achieve is building new semantic models in a dev-workspace, once I am fine with it, push them to a production-workspace. Your tool indicates this could be possible. Here is what I'm doing (copied of notebook cell): import fabric_cat_tools as fct import time newDatasetName = 'targetSM' #Enter the new Direct Lake semantic model name """ comment -> I can create the newDataset in any workspace I want. """ fct.create_blank_semantic_model(newDatasetName, workspaceName='DevWS') time.sleep(5) """ comment -> re-creates all tables and columns of source_semantic_model (=param 1) in target_semantic_model (=param 2) -> lakehouseName and lakehouseWorkspaceName define the workspace and lakehouse where the data should be coming from. (Switching datasource, all fine!) -> PROBLEM: It does not seem to work if source- and target-semantic-model do not reside in the same workspace! """ fct.migrate_tables_columns_to_semantic_model( 'sourceSM', 'targeSM', workspaceName= 'DevWS', lakehouseName= 'LH_Gold', lakehouseWorkspaceName='ProdWS') fct.migrate_model_objects_to_semantic_model('sourceSM', 'targetSM', workspaceName= 'DevWS') I am not sure if that's an issue. Could be a feature. Thx a million for a feedback. Best Regards from Germany, Marc — Reply to this email directly, view it on GitHub<#2>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AHBQBNXTF6QZV757VJ3TYP3Y4U2KBAVCNFSM6AAAAABGANGHDOVHI2DSMVQWIX3LMV43ASLTON2WKOZSGIZTKNJVGM2TKMY>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[MarcAHunt]

Great. Thx.
Any timeline?

Best Regards,
Marc

[m-kovalsky]

Would you try using a beta version of 0.3.0 in this folder, along with the accompanying notebook. Note that parameter names have changed so following the new notebook will help with that.

https://github.com/m-kovalsky/fabric_cat_tools/tree/main/testing

[MarcAHunt]

It works like charm. Can't emphasis how much this is appreciated. My test-case is as follows: In a dev-workspace I build a lakehouse and from that a new semantic-model and report. Now, I want to bring it in production (separation of data and report workspaces) I am setting these vars: *dataset_name = 'my_dev_semantic_model' #Original or source semantic-modelworkspace_name = 'my_dev_workspace' #Workspace were source semantic-model residesnew_dataset_name = 'my_new_semantic_model_to_be_created' #New target semantic-model to be creatednew_dataset_workspace_name = 'my_prod_workspace' #Workspace were new target semantic-model is to be created* *lakehouse_name = 'my_prod_lakehouse' #Production lakehouse, of course holding the same schema as dev-lakehouselakehouse_workspace_name = 'my_prod_workspace_for_lakehouse' #Workspace of prod-lakehousenew_report_name = 'my_new_report_copied_from_dev_report_but_still_bound_to_dev_semantic_model' #New report to be bound to new semantic-modelnew_report_workspace_name = 'my_prod_report_workspace' #Workspace of report* My findings: 1. In your example notebook rebinding one report is documented like this: fct.report_rebind( report = report_name ,new_dataset = new_dataset_name ,workspace=workspace_name) I found these working parameters to run it against a different workspace: f *ct.report_rebind( report = new_report_name ,dataset = new_dataset_name ,dataset_workspace = new_dataset_workspace_name ,report_workspace=new_report_workspace_name)* Would be great if example could be adjusted to all possible (and optional) parameters 2. The URL to install the beta-version is missing the testing directory. Instead of this: %pip install " https://raw.githubusercontent.com/m-kovalsky/fabric_cat_tools/main/fabric_cat_tools-0.3.0-py3-none-any.whl " it needs to be this: %pip install " https://raw.githubusercontent.com/m-kovalsky/fabric_cat_tools/main/testing/fabric_cat_tools-0.3.0-py3-none-any.whl " But hey! You saved my day and a lot of manual work in my job. Thank you so much, Marc Am Mo., 15. Apr. 2024 um 21:40 Uhr schrieb m-kovalsky < ***@***.***>:

…

Would you try using a beta version of 0.3.0 in this folder, along with the accompanying notebook. Note that parameter names have changed so following the new notebook will help with that. https://github.com/m-kovalsky/fabric_cat_tools/tree/main/testing — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BHXG2MMLG7YABE67XGOCZNDY5QUJJAVCNFSM6AAAAABGANGHDOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJXGY3DSNBUGI> . You are receiving this because you authored the thread.Message ID: ***@***.***>

-- *Marc A. Hunt* Rainerstr. 7 83059 Kolbermoor Mobil: +49 (0) 172 176 06 59 Email: ***@***.*** ***@***.***>mail.com

[m-kovalsky]

Great to hear and thank you for testing! The notebook and documented parameters will all be updated once I release this new version. The pip install was still aligned to the main site instead of /testing so glad you used the correct pip install link.

[MarcAHunt]

You are very welcome. One more question: Do you see any possibility to make this work for semantic-models based on warehouses instead of lakehouses? Currently it does seem to work. Thx Am Di., 16. Apr. 2024 um 09:59 Uhr schrieb m-kovalsky < ***@***.***>:

…

Great to hear and thank you for testing! The notebook and documented parameters will all be updated once I release this new version. The pip install was still aligned to the main site instead of /testing so glad you used the correct pip install link. — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BHXG2MMH254KFFAS3UFH3FLY5TK4PAVCNFSM6AAAAABGANGHDOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJYGQ3TENZWGU> . You are receiving this because you authored the thread.Message ID: ***@***.***>

-- *Marc A. Hunt* Rainerstr. 7 83059 Kolbermoor Mobil: +49 (0) 172 176 06 59 Email: ***@***.*** ***@***.***>mail.com

[m-kovalsky]

This is intended for migrating to Direct Lake which is predicated on a lakehouse, not a warehouse. If you are thinking of another idea, please explain further. Btw, 0.3.0 is available and the function documentation has been updated.

…

________________________________ From: MarcAHunt ***@***.***> Sent: Tuesday, April 16, 2024 10:14:08 AM To: m-kovalsky/fabric_cat_tools ***@***.***> Cc: Michael Kovalsky ***@***.***>; Comment ***@***.***> Subject: Re: [m-kovalsky/fabric_cat_tools] migrate_tables_columns_to_semantic_model - source- and target semantic models in different workspaces (Issue #2) You are very welcome. One more question: Do you see any possibility to make this work for semantic-models based on warehouses instead of lakehouses? Currently it does seem to work. Thx Am Di., 16. Apr. 2024 um 09:59 Uhr schrieb m-kovalsky < ***@***.***>:

Great to hear and thank you for testing! The notebook and documented parameters will all be updated once I release this new version. The pip install was still aligned to the main site instead of /testing so glad you used the correct pip install link. — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BHXG2MMH254KFFAS3UFH3FLY5TK4PAVCNFSM6AAAAABGANGHDOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJYGQ3TENZWGU> . You are receiving this because you authored the thread.Message ID: ***@***.***>

-- *Marc A. Hunt* Rainerstr. 7 83059 Kolbermoor Mobil: +49 (0) 172 176 06 59 Email: ***@***.*** ***@***.***>mail.com — Reply to this email directly, view it on GitHub<#2 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AHBQBNRX3KZHV4P4MOEVFO3Y5TMVBAVCNFSM6AAAAABGANGHDOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJYGUYDEOBYG4>. You are receiving this because you commented.Message ID: ***@***.***>