Feature: Find files in repo

[rogerluo410]

Find files for the specified branch in repo.

Issue for the feature:
#766

[lafriks]

Imho it will be quite performance killer for large repositories, imho filtering should be done server side using ui something like github

[KN4CK3R]

Imho it will be quite performance killer for large repositories, imho filtering should be done server side using ui something like github

Github doesn't filter server side. On pageload there is a JSON request which returns all files of the repository. I noticed that the vendor directory is missing from the list but it doesn't look like they are using the .ignore file for this.

[silverwind]

I noticed that the vendor directory is missing from the list but it doesn't look like they are using the .ignore file for this.

It's probably the linguist attributes:

gitea/.gitattributes

Lines 2 to 3 in 54d7b0f

	/vendor/** -text -eol linguist-vendored
	/public/vendor/** -text -eol linguist-vendored

Not sure how far we are for Gitea for the .gitattributes parsing but it'd certainly be nice if both it and maybe also .ignore could be applied in this search. But probably not worth to add it now if the infrastructure to parse such files is not present yet.

[KN4CK3R]

In https://github.com/go-gitea/lgtm the vendor directory is stripped too without .gitattributes or something. Maybe they have just a list of common excludes. I have created a vendor directory in a test repo and that was stripped too.

Edit: https://docs.github.com/en/github/searching-for-information-on-github/finding-files-on-github

[silverwind]

Ugh, yeah if it's a hardcoded list, that's not going to be obvious. I suggest that if we do filtering based on files, either use .gitattributes or .ignore, ideally both.

[lafriks]

we can easily use go-enry there is method enry.IsVendor("path/to/filename") to filter out them

[lafriks]

Imho it will be quite performance killer for large repositories, imho filtering should be done server side using ui something like github

Github doesn't filter server side. On pageload there is a JSON request which returns all files of the repository. I noticed that the vendor directory is missing from the list but it doesn't look like they are using the .ignore file for this.

That's sounds more reasonable, as rendering all file list in html will be quite an overkill

[lunny]

I think maybe the solution may have better performance.

On find file page, when loading, we can get all entries of currently tree via ajax and filter on frontend.

When input a /, we can load all entires below this trees and filter on frontend.

All entries will be cached on this page, so that if you delete a /, then no data loading is necessary.

[rogerluo410]

In https://github.com/go-gitea/lgtm the vendor directory is stripped too without .gitattributes or something. Maybe they have just a list of common excludes. I have created a vendor directory in a test repo and that was stripped too.

Edit: https://docs.github.com/en/github/searching-for-information-on-github/finding-files-on-github

I noticed that the vendor directory is missing from the list but it doesn't look like they are using the .ignore file for this.

It's probably the linguist attributes:

gitea/.gitattributes

Lines 2 to 3 in 54d7b0f

	/vendor/** -text -eol linguist-vendored
	/public/vendor/** -text -eol linguist-vendored

Not sure how far we are for Gitea for the .gitattributes parsing but it'd certainly be nice if both it and maybe also .ignore could be applied in this search. But probably not worth to add it now if the infrastructure to parse such files is not present yet.

I didn't think much about excluded directories. That is really a problem.

So, what is the best solution of it?

If repo has .gitattributes or .ignore exists, we use them to exclude directories, otherwise, use hardcoded list to exclude instead ??

[silverwind]

If repo has .gitattributes or .ignore exists, we use them to exclude directories, otherwise, use hardcoded list to exclude instead ??

I think just supporting .gitattributes via go-enry should be good enough for a start. Maybe it could be a feature request to go-enry to support .ignore as well for search ignores, e.g. enry.IsSearchIgnored("path/to/filename").

[rogerluo410]

If repo has .gitattributes or .ignore exists, we use them to exclude directories, otherwise, use hardcoded list to exclude instead ??

I think just supporting .gitattributes via go-enry should be good enough for a start. Maybe it could be a feature request to go-enry to support .ignore as well for search ignores, e.g. enry.IsSearchIgnored("path/to/filename").

Ok, I will solve it, thank you.

[lunny]

@lafriks needs your review again.

[rempas]

Finally, it is merged!!!! Personally thanks everyone that worked in the feature!!! You people are awesome!!! ❤️