Web Admin UI

[Mimetis]

I'm currently working on a Web Admin UI, accessible from the Server side.
It will be published as a Sample I guess.
Is it something that can be interesting at some point for you ?
What kind of interaction do you want to find here ?

For now, I've just designed 2 main pages : Scopes & History

Ideas:

• Add a page for each client (each client is identified by its client scope id)
• Add a Force Reinitialize to any client on next Sync using OnGettingOperation interceptor
• Add a page for managing Snapshots (Add / Delete / Update)
• Add a page to add a new Scope / Setup (really simple page with just a list of tables to add and the scope name)

If you have any ideas, do not hesitate to discuss them here.
Remember that this admin tool is meant to be used on the server side :)

Scopes

History

[cdash]

This looks great and is similar to something we had started implementing on top of the base of Dotmim.

Just a couple questions from me:

1. I noticed the new tables defined in the context of the sample project that you checked in today, but it was unclear to me how you intend to populate them with data. Are those new tables going to be added into the base library and it will automatically start keeping track of session statistics or were you intending the sample project to implement all of the tracking? That is what we are doing right now but it definitely would be nice if at least the tables and tracking was built in.

2. I also see that you have a start time for the session but no finish time. Would it be possible to add that?

3. There seems to be a recent change to make all of the sync_scope_id for a single client the same GUID for different instances of scopes where before they were always different GUIDs. I assume the change was partly made for the purposes of the dashboard and tracking the scopes for a specific client and the new key is the combination of (sync_scope_id, sync_scope_name). That makes sense, only thought is the naming now is a little confusing when it is really representing the sync_client_id.

[Mimetis]

I'm really interested on what you've done so far on your implementation. If you can share something, that would be great !

1. The tables are created when you are plug in a new plugin (work in progress) to populates info about sync sessions.
   Basically the plugin will listen to some interceptors and will populate these tables.
   The tables will be created only if you are using this plugin
   Not a final version, but it will be, more or less, something like that:

[ApiController]
[Route("api/[controller]")]
public class SyncController : ControllerBase
{
    private WebServerAgent webServerAgent;

    public SyncController(WebServerAgent webServerAgent)
    {
        this.webServerAgent = webServerAgent;
    }

    [HttpPost]
    public async Task Post()
    {
        // Using remote interceptors to get some informations on the current sync
        await InteruptRemoteOrchestratorInterceptors(webServerAgent.RemoteOrchestrator);

        await webServerAgent.HandleRequestAsync(this.HttpContext);
    }

}

2. We have the StartTime value, and we have a duration value as well. I guess the EndTime value can be easily calculated :)
3. Yes, this is a major update from the last version. Now each client database has its own sync_scope_id and it will be unique across all scopes. It was made to handle the new multi scopes mechanism.
   And yes, the new key is now the composite key sync_scope_id / sync_scope_name

[cdash]

Not too much work was done before we got sidetracked with something else and we are only now starting to pick it back up again, but I can at least give you the general idea and tables that we created. The UI at this point was just some very basic grids showing the raw data in the database.

We were basically only interested in having a small dashboard that showed some statistic about specific users/clients/scopes. The scope definitions themselves are already defined in a preexisting custom configuration that was created for an old product that uses Sync Framework and are working on migrating to Dotmim.Sync, so we didn't need to do any of that in the dashboard.

1. We were already using multiple scopes per client so the first thing we did was create a table that allowed us to track not only a client id but also a user id to the multiple instances of scopes for that client. The client was then responsible for setting its client id and user id in the headers that were sent to the server.

CREATE TABLE [dbo].[sync_client_info] (
	[sync_client_id] [varchar](36) NOT NULL,
	[sync_user_id] [varchar](36) NOT NULL,
	[client_last_sync_time] [datetime] NULL,
	[client_initialized_time] [datetime] NULL,
	[client_blocked] [bit] NOT NULL
)
CREATE TABLE [dbo].[sync_client_scopes] (
	[sync_client_id] [varchar](36) NOT NULL,
	[sync_scope_id] [uniqueidentifier] NOT NULL
)

2. Then we also created a table for tracking statistics on a per client scope level. We did not go to the level you have on keeping track of every session but only an average. I really like having the per session and even per table per session data, but I do wonder if with a large number of clients syncing (2000+) if the data will need to be occasionally cleaned up.

CREATE TABLE [dbo].[scope_info_statistics](
	[sync_scope_id] [uniqueidentifier] NOT NULL,
	[total_sessions] [bigint] NOT NULL,
	[total_sessions_failed] [bigint] NOT NULL,
	[average_session_time] [float] NOT NULL,
	[average_upload_records] [float] NOT NULL,
	[average_download_records] [float] NOT NULL,
	[average_applied_records] [float] NOT NULL,
	[average_conflicts_resolved] [float] NOT NULL
)

3. We also added a couple features like you did for being able to force a scope to reinitialize or to block a specific client from being able to sync at all.

CREATE TABLE [dbo].[scope_info_reinitialize](
	[sync_scope_id] [varchar](36) NOT NULL,
	[completed] [bit] NOT NULL,
	[requested_time] [datetime] NOT NULL,
	[completed_time] [datetime] NULL
)

All of this was done through some combination of using interceptors or inheritance. I think probably the majority was done through interceptors.

[gentledepp]

@Mimetis
This looks great!
It would be great if it did show additional information like:

1. all the clients and when they were last connected.

• of course a "purge/delete" client action would also be handy in case we would like to get rid of an old client

2. in the history, I would like to see

• the time a sync took
• it should be flagged red, if there was a sync error (or list sync errors in another table "sync errors"?)

3. It would also be great, if threshholds could be configured for e.g. sync time

• if a sync takes longer than 2 seconds, it should be flagged with a warning in the history table ⚠️
• if it takes longer than 25 seconds (or close to the db connection timeout) it should show a more cirical warning ❗❗

4. starting up, maybe you could add a dashboard like hangfire does https://www.hangfire.io/

• one chart showing synchronization times over time (past 24 hours)
• another showing synchronization errors (past 24 hours)

I am a big fan of this idea in general.
What bogged me with the dashboard of hangfire.io is, that it is not extensible... except if you fiddle around in their source code. (add another chart, column to history grid, etc.) But that is another topic....

[carlosxjose]

I still have an old implementation with SQLCE Replication and SQLServer 2012; the replication monitor is what I use to see the performance of each client and I can do a lot of stuff from the server side.

[Mimetis]

can be interesting to make a quick summary / list of all performance counters you are watching from this dashboard
and also the action you can take (and are taking) from there
Would be a great feedback and will help us a lot

[Mimetis]

I'm continuing working on the web admin UI
Here is a quick update, just with some self-explained screenshots:

[Mimetis]

Adding

• Total duration (be careful, wrong column name. It's Duration instead of End Time)
• Orange line when duration > 10 sec
• Red overlay if an error has occured
• Total rows count for download / applies / failed / upload (details are always accessible through details icons)

[gentledepp]

Cool!

1. Seeing the client-id is nice. But it would be more helpful to have e.g. the username here.
2. We are running a multi-tenant system. The tenant-id is passed on every sync as a parameter "@TenantID". It would be nice having the posstibility to configure this view, so that only data of Tenant 1, 2, 3 or all tenants is shown.

Of course both are application-specific features and should not be covered by Dotmim.Sync.

First thing that comes to my mind is having two extension points.

For (1) it would be cool, if there was an extension point that allowed to add a column. Then for each page, shown on screen, we just get the scope-ids, query the database to get the usernames, and could add them to those rows. (1 db call per page to fetch all the usernames)

For (2) - this would be a predicate (be it sql or .net - I don't care) that allows to configure a filter, that the user cannot override.
When an admin of a particular tenant logs on, we could filter the sync results to that particular tenantid (filtering the parameter)
When the sys admin is logged on, we do not apply this filter and can see all the syncs.

Does this make sense to you? Would love to hear your thoughts!

[Mimetis]

All scopes table have a field called "properties" that can contains barely anything: It's used to store a JSON array.
I use it to store a json property for the friendly name.
This "properties" field is also contained in all sync logs table (that I use here)

I don't want to go to deep in the customization, since the web admin will be mainly just a sample, you can clone and edit as you want.
But if you want to add a new column, feel free to add this column as a json property in the properties field and add the column to the Web UI

[VagueGit]

Big thumbs up for this.
How do you plan on calling it?
Will this only be available in debug/localhost or would we able to use it for live insight into how our customers errors are being generated?

[Mimetis]

No, it will be available in any mode
It's just a website that retrieve all events raised by the different interceptors.
It will be published as an example.

[BluePalmTree]

Hi, this looks really interesting. I have two questions:

1. In the samples there is a WebAdmin and WebAdminTool Project. What is the differen?
2. Is there a little guide or documentation on how to implement this in a already existing Sync project?

[Mimetis]

It's an in-progress project sample.

I do it as a sample, and it will be available only as a source code example (and will not be published as a nuget package)
Once I'll reach a stage where it's started to work as expecting, I'll publish a short how-to description and doc.
Stay tuned

[Ssapkade]

can some one help me with the location of the sample project , i am not able to find in repository