Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to "undelete" flows marked as deleted in ByoDL environment #7511

Closed
1 task done
shoffma1 opened this issue Jan 22, 2024 · 39 comments
Closed
1 task done

How to "undelete" flows marked as deleted in ByoDL environment #7511

shoffma1 opened this issue Jan 22, 2024 · 39 comments
Assignees
Labels
coe-starter-kit CoE Starter Kit issues question Further information is requested

Comments

@shoffma1
Copy link

shoffma1 commented Jan 22, 2024

Does this question already exist in our backlog?

  • I have checked and confirm this is a new question.

What is your question?

From before the January update, some 2000 of our flows from our main environment have been marked as deleted. Though they show up in the PowerBI report now, they are not in the default view in the Admin app.
How can we get remove the "marked as deleted"?

What solution are you experiencing the issue with?

Core

What solution version are you using?

4.22

What app or flow are you having the issue with?

Not sure

What method are you using to get inventory and telemetry?

Data Export

AB#2105

@shoffma1 shoffma1 added coe-starter-kit CoE Starter Kit issues question Further information is requested labels Jan 22, 2024
@Jenefer-Monroe Jenefer-Monroe self-assigned this Jan 22, 2024
@Jenefer-Monroe
Copy link
Collaborator

This should happen automatically with the dataflow refreshes. Can you please show a screenshot of the dataflows to see if they are passing?
image

@niwrejim21
Copy link

Hello @Jenefer-Monroe, we are also getting this issue. Flows, Apps and environments are being marked for deletion and most of the flow are under the "Deleted flows/apps/environment/AI Models" view in the Admin View. I was able to observe that this is happening after the scheduled refresh of dataflows. Once I do a manual refresh of the flow - the flows are again available and are removed from the deleted view.

Here is an example of our default environment under the "Deleted Environments Review" View.

image

@Jenefer-Monroe
Copy link
Collaborator

@niwrejim21 can you please share a screenshot like this?
Also what version of Core are you on?
image

@shoffma1
Copy link
Author

shoffma1 commented Jan 25, 2024

Screenshot 2024-01-25 074949

This is my view ...

@niwrejim21
Copy link

image

@Jenefer-Monroe
Copy link
Collaborator

I'm not sure how you got into this state, but if you go and delete records in the dataflow history table back to when you saw this issue happen, that should clear it up.
image
image
image

@shoffma1
Copy link
Author

I'm not sure how you got into this state, but if you go and delete records in the dataflow history table back to when you saw this issue happen, that should clear it up. image image image

Thanks for this. Would there perhaps be a more automated way to delete all rows from this table, as I have no idea when the issue started.

@niwrejim21
Copy link

@Jenefer-Monroe , it happened again today - it seems that this is an intermittent issue. No dataflow failed during the scheduled refresh but some of the resources including our default environment got marked for deletion again. Yesterday, the scheduled refresh worked fine. To be able to fix it, I had to do a refresh of the dataflows again.

@Jenefer-Monroe
Copy link
Collaborator

We did create a more automated way to do this in the Feb release. You'll now see a button which lets you delete back as far as you want
image

@Jenefer-Monroe
Copy link
Collaborator

@niwrejim21 I'm not sure what you mean by marked for deletion. Do you mean they are marked as deleted?
Please take a sample object that you think should not be marked deleted, and see if it exists in the dataflow file.

@niwrejim21
Copy link

niwrejim21 commented Jan 29, 2024

This was last week - our default environment is under the deleted view:

image

This is today:

image

Please also notice the number of Apps between the two snapshots (from 9k+ to 250). The missing apps are now under the Deleted Apps view and have a "app deleted on" date of 1/29/2024. Those apps are not actually deleted and if I run a manual refresh of the makers dataflow - it will eventually be corrected.

image

@Jenefer-Monroe
Copy link
Collaborator

Please go to the Admin Command Center > CoE Flows > Inventory. Validate that all flows are turned on and that there are no unmanaged layers as shown here.
image
Then see if there are any sync flow errors to investigate.
image

@Jenefer-Monroe
Copy link
Collaborator

ok another person posted the question and it pointed me to the issue. Please see #7562 (comment) for how to locally resolve. And let me know!

@shoffma1
Copy link
Author

ok another person posted the question and it pointed me to the issue. Please see #7562 (comment) for how to locally resolve. And let me know!

Not sure this will work for us, as this particular flows is not "On", and gives an error trying to do so. Sorry, I can't keep straight which flows should be on or off for the ByoDL solution vs the older.

@Jenefer-Monroe
Copy link
Collaborator

No problem. The instructions for the workaround are for a Dataflow. So if you are using DAta Export that one should be on. The issue is in the Dataflow (which brings the default ent back as default-abc123.... instead of Default-abc123...). So regardless of if you are using v3 or v4 of the check deleted cloud flow system, you would be impacted.

@niwrejim21
Copy link

@Jenefer-Monroe - confirming that the fix resolved our issue as well. Appreciate all the help!!!

@shoffma1
Copy link
Author

shoffma1 commented Feb 1, 2024

After applying the above code to our dataflow, and activating the Admin sync flow, the general number of flows seems to be correct, however all of the data is now missing in the active flows view, or when clicking into one of the flows from there ...
2024-02-01 07_11_15-Flows Active Flows - Power Apps and 2 more pages -  InPrivate  - Microsoft​ Edge

@Jenefer-Monroe
Copy link
Collaborator

What do you mean by this: and activating the Admin sync flow

@shoffma1
Copy link
Author

shoffma1 commented Feb 1, 2024

What do you mean by this: and activating the Admin sync flow

Currently all of our v4 Cleanup flows are off. I had activated this one for a short time: CLEANUP - Admin | Sync Template v4 (Check Deleted)CLEANUP - Admin | Sync Template v4 (Check Deleted)

@Jenefer-Monroe
Copy link
Collaborator

This is written to from another dataflow and mine is working as are the other folks who did the workaround. So I think there may be something unique to your envt.

  • Are all the records blank or just a few?
  • Are all the fields blank or just display name? In particular is the envt lookup field filled/

@shoffma1
Copy link
Author

shoffma1 commented Feb 2, 2024

As of this morning none of these flows are listed at all, even after removing the "Flow Deleted" filter.

2024-02-02 07_33_58-Environments Active Environments - Power Apps

@Jenefer-Monroe
Copy link
Collaborator

If you go look at the flow table itself, instead of this roll up number, do you see flows in default?

@shoffma1
Copy link
Author

shoffma1 commented Feb 2, 2024

No, nothing from the default environment. It almost seems like our "service" account had lost access to that environment, but it does have the "System Administrator" role there, and the "Power Platform Admin" role in Entra, though it is from a group.

@Jenefer-Monroe
Copy link
Collaborator

Can you go see if the file is present in the Data Export storage account?

Image

@shoffma1
Copy link
Author

shoffma1 commented Feb 5, 2024

Yes, our Default env is in the data export. In fact, the power bi report is coming through fine with all of our flows from that environment.
2024-02-05 07_04_03-fvpowerplat

@Jenefer-Monroe
Copy link
Collaborator

Great. Please view the dataflows.
First please confirm that you only have the Next Refresh on Maker (not on any other dataflow)
Then go to the refresh history for the Flows dataflow, is it passing?
image
image

@shoffma1
Copy link
Author

shoffma1 commented Feb 5, 2024

Correct, only the "CoE BYODL Makers" dataflow has a "Next refresh" that is not "N/A".

And the refresh history for "CoE BYODL Flows" is showing success as far back as that goes (12/18/2023).

@Jenefer-Monroe
Copy link
Collaborator

great, now please note the last modified date for the Default-abc... record for flows
image

Delete the records from the DataflowRefreshHistories table back through that date
image

Open the flow CoE BYODL Flows dataflow and see if you find the default envt flows in there, and if the name of the environment has the upper or lower casing for the word default: Default-abc... or default-abc...
You may need to filter to just default if they are hard to find like i did here
image

@shoffma1
Copy link
Author

shoffma1 commented Feb 6, 2024

Our data export file for our default environment has the current date (6-Feb). I did go ahead and delete today's refresh history items regardless.

Not sure if this is relevant, but we have two dataflows uploaded to powerbi ... one is the january release, which is showing our Default environment, with a capital D. The older dataflow shows the default environment with a lowercase d.

When I look from the "make.powerapps.com" portal, I'm getting the message, "This table has no rows".

NoRows

@Jenefer-Monroe
Copy link
Collaborator

We aren't doing anything with the PBI Dataflows here. The change above is not neeed for them.
We are only working with the Dataflows in the solution which write to the Dataverse tables.
Can you confirm we are on the same page?

@shoffma1
Copy link
Author

shoffma1 commented Feb 6, 2024

Yes, confirming only working with the Dataflows in the solution.

CoE_Environments

Just not sure why we are now seeing zero flows, for any environment here.

2024-02-08 10_17_49-Power Apps

@Jenefer-Monroe
Copy link
Collaborator

I've set up to try and repro locally, although I do not yet. Will let you know what I find next week

@Jenefer-Monroe
Copy link
Collaborator

ok I do not repro this issue. it could be that the local fix isnt applied correctly, or something else has changed in the build.
Please either upgrade to Feb and apply the local fix to see if it repairs, or wait for the March build when it will be released.

@shoffma1
Copy link
Author

This should probably be another issue report, but running the upgrade wizard after installing the 4.23 Core components solution, getting this on the first page after the modal windows.

Coe_Upgrade_Error

@shoffma1
Copy link
Author

shoffma1 commented Feb 13, 2024

After installing the 4.23 Core components, I was able to delete all the DataFlowRefreshHistory items. Thank you for that option to delete all!

Now I am getting this error, even after configuring the connections, etc., in the DataFlows ... this is the BYODL Environments dataflow ...

Screenshot 2024-02-13 103240

We do now have our flows populating, but still have a couple that are blank ...

Blank_Flows

@Jenefer-Monroe
Copy link
Collaborator

This means that the maker is not in the inventory. This typically happens when you have set the next reresh on all the dataflows instead of just the maker one.
Can you confirm that you only have it on for Maker?
image

@shoffma1
Copy link
Author

FYI, after our daily refresh(es) completed this morning, we have zero blank flows showing, and the correct number of flows from our default environment. I think we're good at this point. Thanks for your help!

@Jenefer-Monroe
Copy link
Collaborator

Wonderful news! I'm still testing a little more and have a few other folks testing as well so ill hold this till early next week and then hopefully close out feeling good about the March release :)

Thanks for your help!

@Jenefer-Monroe
Copy link
Collaborator

ok this is looking good for March release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
coe-starter-kit CoE Starter Kit issues question Further information is requested
Projects
Archived in project
Development

No branches or pull requests

3 participants