Investigate GIT as PDM replacement using GVFS.

[ghost]

Greetings,

I would like to open a dialog regarding GVFS as the engineering vault.
Typically, Product data management systems are SQL driven virtual file systems.
This requires constant local SQL server activity.
The advantage of a GIT is that we push changes to a central repository.

Is it possible to Support a file locking system on the GVFS file so that only one person can change the locked file?

Exclusively for CAD file usage, could we interrupt the commit/push operation?
The issue we see is there exists few methods to cleanly merge changes to CAD files.

Thanks,
coast

[fpqc]

@mancoast The point of git is that each of your designers should be able to maintain their own branches, and you can do merges or partial merges of their branches into the master branch. It's a decentralized system, and it's specifically made to be non-locking and with conflicts to be resolved at merge time.

Because GVFS has really nice binary blob support (so you don't need to download and checkout every version of every blob), GVFS might meet your needs even without this locking. If your CAD files are text-based and not binary-based, you could potentially even use Git to merge changes in them.

[ghost]

Greetings @fpqc,

In my experiences, Git usage in the Software world provides nearly "perfect knowledge" for incremental change history. This is the standard with which our minds grow comfortable for engineering.
The issue I see in manufacturing is that CAD Models do not Merge [proprietary file formats].
Try to generate an exact difference report of two 3D Model with absolute certainty.

There exists options to export CAD Models into plain-text formats, but those file formats are analogous to binaries, not full source code. The plain-text CAD Models do not include the data required for quickly altering the model; it's more of a shell.

Let us think in essence of virtual file system, and set aside the traditional GIT archetypes.

Thanks,
coast

[fpqc]

@mancoast Well, if you can't do merging, you can continue using an SQL-based filesystem for sure. I'm not sure that git is necessarily the best choice for just managing CAD documents. If, however, you're also using Git to manage code as well, then maybe it might make sense. But as to your earlier question, no, I don't think Git supports file locking at all. That would go against the decentralized nature.

[ghost]

Greetings,

Consider usage of GIT under normal operation for Markdown formatted build sheets.
Considers thousands of lines of code for each customer's embedded systems.
Additionally, create a tidy organization for all 3D data in a side by side virtual system.
Files represent "nodes" in the tree for 3D data.
The decentralization of GIT is incredible.
The ability for everyone to easily have incremental history for all data in one system.

This would require the ability for non-traditional mutex at the file system level.
File locking for a Mechanical Engineer in an active Model essentially translates to: "revert this node; rebuild the rest of the model".
Models can be developed in parallel with sub-model nodes as granules.
The new file system mutex for GVFS similar to features of traditional NFS.

Thanks,
coast

[sanoursa]

@mancoast I definitely agree that there are scenarios where file locking makes a lot of sense. However, GVFS is not the appropriate place to enforce it. Remember that GVFS is purely a client-side optimization. Users can still do a normal git clone of the repo, or go make edits in the web, and in both of those cases, GVFS can't enforce any such rules.

In general, GVFS tries to not change anything about the behavior of git, and only tries to make git more efficient for large repos.

[ghost]

It would be amazing for Microsoft to open-source GvFlt driver to community.
Perhaps extension with lock?

[fpqc]

@mancoast What you really want is an open API for GvFLT. The problem with opensourcing the driver is that it needs to be signed by Microsoft in order to be loaded by Windows (64-bit Windows enforces driver signatures as a security measure), so it can't really be forked. Hopefully GvFLT can be used to implement some kind of FUSE functionality, which you could then use to build a virtual filesystem that enforces the features you'd like to see.

But it would have to be a custom virtual file system and not GVFS. GVFS is like an 'online mode' for Git that allows you to do very sparse and fast checkouts for large repositories with lots of binary blobs. There's nothing stopping you from extending the protocol to add locking, and it probably wouldn't need to be added in at the driver level. It would need to be added at the server usermode level and the client usermode level to implement a protocol extension.

[YueLinHo]

About file locking, perhaps you can try git LFS, see here.

[ghost]

@YueLinHo Thanks for pointing out Github and Git LFS support file locking with decentralized spirit.

With Git LFS there exists something along the lines: $ git lfs track "*.3DCAD" --lockable

It is my opinion, a proposal for file system mutex for GVFS falls within scope of GVFS as a purely client-side optimization of standard Git behavior in support of the distributed essence of large binary file collaboration.

Considering Pack files ~75GB for just Electrical, GVFS enables capacity to include additional Mechanical data.

In the manufacturing realm, requirements include both large repo, and large files.
Any thoughts?

[fpqc]

@mancoast Is Git-lfs incompatible with an underlying GVFS setup?

[sanoursa]

Visual Studio Team Services also supports Git LFS, and file locking via LFS. Unfortunately for now LFS and GVFS cannot be used together (long story short: GVFS requires that you don't have any clean or smudge filters configured, and LFS relies on those filters)

As I said before though, while GVFS could participate in a locking feature, GVFS itself can't enforce it so this is not the right place to build locking. It's too easy for people to access the repo without going through GVFS, so the only way to build reliable locking is to build it into the hosting service, not into the client tool.

I'm going to close this issue because this isn't a feature that we think makes sense in GVFS, but I'm happy to continue the discussion if you have other questions about the topic.

[fpqc]

@sanoursa Just one last question: Do you think that GVFS could be extended to support those clean/smudge filters in the future, or was this a real engineering problem you ran into?

[sanoursa]

Handling clean/smudge filters is tricky, but possible for certain limited scenarios.

The problem is that when you project a virtual file, you must also indicate what its size is (and that size has to be accurate or tools will think your file is corrupt). Today, we ask the server for the size of the uncompressed blob, and claim that as the size of the virtual file, and that works because we disallow filters. But git allows you to configure smudge filters that arbitrarily modify the contents of a file as you check it out, and those filters could be totally unknown to the server because they're driven by client-side configuration. So there's really no way to support arbitrary filters with a virtual file system.

However, the server could know about certain filters and understand how they manipulate the contents of a blob, so that when GVFS asks the server for the size of a blob, the server can respond with the decompressed and smudged size of the file. It's something we're thinking about as a feature down the road.

[fpqc]

Neat! I'll be following along as you update things and continue development.

[ghost]

Thanks for the discussion on the bright future for GVFS with the opportunity to evolve both the GVFS client, built on a windows driver, and back end, built on Visual Studio Team Services, to concurrently accommodate both large repository [GVFS] and large files [LFS] with file lock operations.
This potential value for further extension into the traditional 3D Model work-flow emulates the "check-in" and "check-out" operations given they operate with very large amounts of data on Visual Studio Team Services.