RFE: master file server addition information e.g. ETag

[damon-atkins]

Description of Issue/Question

Can the backed be updated so instead of just returning simple information i.e. filename it always returns a dict
Now looks like

cp.list_master
- file1
- file2
- dir1\file3

Future look, however every command should receive a dict for each file name, may not use the info

cp.list_master detail=True
'file1':
   Time: seconds since 1970
   ETag: string
   Size: 
   Type: f(ile) or d(irectory)
'file2':
   Time: 7483278389
   ETag: jfds8jlfsjd8ereteghyrbvvdffeeejfljdl
   Size: 7184
   Type: f
'dir':
   Time: seconds since 1970
   ETag: <Maybe a checksum of all the filenames and modification str times>
   Size: <items in directory> or None
   Type: d
'dir1\file3':
   Time: 6367
   ETag: rfejlwjkldggdgfddgthdvhyhyytrfwe
   Size: 9373
   Type: f 

ETag its up to the client to record it so it knows if the file has change or not. The client does not care how it was generated and does not need to generate/calculate it itself. However the server must change it if the file changes on the server. ETag could be a commit tag or something else from git.

It's important all masters give out the same ETag for the same file. An Etag which uses inode number would fail as this would be different on different masters.

+1 if you like the idea.

The next stage would be for the clients to use this information as part of caching.

[gtmanfred]

@saltstack/team-core does anyone have any opinions about this?

Thanks,
Daniel

[thatch45]

I am fine with this as a feature addition as long as the default interface does not change. Any changes to the master/minion apis needs to be very carefully managed as we don't want to create and master/minion incompatibilities.

[damon-atkins]

It's a change to the salt:// protocol. Aim being that it will help make the cache on the minion faster as it can be smarter about what it does. And opens the master to decided what to use the for ETag (e.g. checksum on a flat file system or git info from a git repo)
i.e. pulls a file down and saves the "file meta information", then to check if the file is out of date it compares save "file meta information" with the current "meta information" from the master. (The client does not spend time using a checksum to determine if the file is out of date). Its basically how http proxy servers cache work.

[damon-atkins]

Does the current salt:// protocol have version of the protocol in the request, or the minion version including in the request? So the master knows how to respond in a compatible way.

[thatch45]

no, the current salt:// protocol is not versioned. I think that adding versioning would be a good idea, but we should also be able to specify modifiers in the form of arguments and implement the minion side to compensate.

[damon-atkins]

Can we label this with Fluorine, improve the meta information available so clients can make smarter informed decisions.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

[damon-atkins]

Most likey worth keeping this open

[damon-atkins]

ping @saltstack/team-windows as per today working group meeting.