Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

why does gvfs pull to "src" folder? #50

Closed
mrcaron opened this issue Nov 8, 2017 · 9 comments

Comments

@mrcaron
Copy link

commented Nov 8, 2017

When I use GVFS to clone a repo on TFS 2018 to a specific directory, GVFS makes a "src" folder under that and puts the code there. Can I prevent that in some way?

@sanoursa

This comment has been minimized.

Copy link
Contributor

commented Nov 8, 2017

We added the src subfolder for a couple of reasons:

  • we wanted to place the .gvfs folder in a non-virtualized location (this was a much bigger deal back in our original design but it has stuck around)
  • there are perf implications to writing lots and lots of new files inside the virtualized src folder, so we wanted to be able to redirect all build outputs to a location outside the src folder

So we created the src subfolder, and that is where we attach the gvflt driver, so that all the siblings of src are unaffected by any virtualization overhead.

@mrcaron

This comment has been minimized.

Copy link
Author

commented Nov 9, 2017

Interesting. So I should organize my projects to build artifacts to a directory outside of the src location? What about intermediate files, like *.obj, and *.pdb? Should I be setting up a directory outside of src to capture those?

Is that something that should be mentioned in the Readme?

@sanoursa

This comment has been minimized.

Copy link
Contributor

commented Nov 9, 2017

That would be ideal, yes. The issue is that every time you create a new file inside src, GVFS has to add it to git's sparse-checkout and always_exclude files, because those files now exist on disk and so git must take care of updating them if those same file names ever show up in a commit. It's a corner case, but one that we need to handle.

To avoid that overhead, it's definitely best to create a folder that is a sibling of src, and direct all build outputs, including intermediates, to that location.

If you look in the GVFS codebase itself, we assume that the repo is cloned to \src, and we redirect all build outputs to \BuildOutput (e.g. see https://github.com/Microsoft/GVFS/blob/master/GVFS/GVFS/GVFS.csproj, line 21). We also place all NuGet packages in \packages (see https://github.com/Microsoft/GVFS/blob/master/nuget.config, line 4).

You're right that we should clarify this in the Readme.

@tomspilman

This comment has been minimized.

Copy link

commented Feb 21, 2018

This confused me as well on my first attempt to use GVFS. It really should be in the Readme.

It would be nice if this was all cleaner... we work on games and artists can be confused by things like this. Ideally in the future this src directory isn't needed.

@tomspilman

This comment has been minimized.

Copy link

commented Feb 21, 2018

Also what is the git.cmd for? Can it be documented too?

@sanoursa

This comment has been minimized.

Copy link
Contributor

commented Feb 21, 2018

We have no immediate plans to get rid of the src folder, but I'm curious to hear more thoughts on whether/why it is an issue. The performance benefits of putting your build outputs outside of the virtualized repo root are non-trivial, and once you update your build scripts to redirect their outputs and folks get used to the layout, it doesn't seem to be too surprising for anyone. Then again, our current customers are all developers, and I can understand that non-technical workers like artists want a simpler solution.

@tomspilman

This comment has been minimized.

Copy link

commented Feb 22, 2018

The performance benefits of putting your build outputs
outside of the virtualized repo root are non-trivial

We often have mixed content in folders with both things that are committed to the repo and things that are generated outputs. So for this in particular we really will have to rethink our repo layouts.

non-technical workers like artists want a simpler solution

All that really stood out to me is that "src" suggests "source code" and maybe a more generic term like "repo" would make it clear where the repository content is.

our current customers are all developers

Right. I'm testing GVFS for use in game development where we have a mixture of code, runtime game content, and large source content files from things like Photoshop or 3DSMax.

For us it is often a struggle of keeping one repository with all the dependencies to build the full game and serve both coders and artists. Coders want good/fast branching and merging without getting GBs of Photoshop files they don't need. Artists/designers never branch/merge and just want an easy to use versioning system.

We been using Git LFS, but wanted to try GVFS to see if it works better for us. In particular not having to pre-define what file extensions should be treated as large files is one less thing to have to manage.

@plynkus

This comment has been minimized.

Copy link

commented May 4, 2019

Greetings, all. Late to the party here, but I just took VFS for a spin this weekend on an existing repo, ran into this same question myself and wanted to add to the discussion.

The issue is that every time you create a new file inside src, GVFS has to add it to git's sparse-checkout and always_exclude files, because those files now exist on disk and so git must take care of updating them if those same file names ever show up in a commit. It's a corner case, but one that we need to handle.

It sounds at first as though the class of files we want to avoid this cost for (intermediates, outputs, etc.) are exactly the sort of things we generally place in .gitignore in the first place in order to prevent their inclusion in the repository. Would it not be achievable to simply honor .gitignore rules in the VFS processing you cite above & related filter driver activity (along with, implicitly, .gvfs itself)?

Structural changes to our repos to percolate all of this upward/outward (and even the src tree addition itself) are likely not something we are to pursue. We'd much rather our repos be agnostic to the git implementation in use and employ a mechanism to hint (even something new, if .gitignore is insufficient) what can/should be exempt from VFS consideration.

@Terry-Mao

This comment has been minimized.

Copy link

commented May 21, 2019

@sanoursa

We have no immediate plans to get rid of the src folder, but I'm curious to hear more thoughts on whether/why it is an issue.

I'm testing GVFS for our Go mono-repo. but when add src folder to our codebase, the original import package path will affect by the src folder.

something like

import (
  "monorepo/pkg/1"
)

// After add src folder
import (
  "monorepo/src/pkg/1"
)

all the go files MUST refactor.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.