Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] REPO_INDEXER won't index mirrored repository after sync #15367

Closed
2 of 6 tasks
ghost opened this issue Apr 9, 2021 · 6 comments · Fixed by #24634
Closed
2 of 6 tasks

[BUG] REPO_INDEXER won't index mirrored repository after sync #15367

ghost opened this issue Apr 9, 2021 · 6 comments · Fixed by #24634
Labels

Comments

@ghost
Copy link

ghost commented Apr 9, 2021

  • Gitea version (or commit ref): 1.13.1
  • Git version: Not relevant
  • Operating system:docker image
  • Database (use [x]):
    • PostgreSQL
    • MySQL
    • MSSQL
    • SQLite
  • Can you reproduce the bug at https://try.gitea.io:
    • Yes (provide example URL)
    • No
  • Log gist: Not relevant

Description

REPO_INDEXER won't index mirrored repository after sync

Related: #12380 #11200 #11479 #5329

@zeripath
Copy link
Contributor

zeripath commented Apr 9, 2021

func (r *indexerNotifier) NotifySyncPushCommits(pusher *models.User, repo *models.Repository, opts *repository.PushUpdateOptions, commits *repository.PushCommits) {
	if setting.Indexer.RepoIndexerEnabled && opts.RefFullName == git.BranchPrefix+repo.DefaultBranch {
		code_indexer.UpdateRepoIndexer(repo)
	}
	if err := stats_indexer.UpdateRepoIndexer(repo); err != nil {
		log.Error("stats_indexer.UpdateRepoIndexer(%d) failed: %v", repo.ID, err)
	}
}

It might be that you don't have a default branch set?

Go to the Repository Settings and change/re-set the default branch.

@ghost
Copy link
Author

ghost commented Apr 10, 2021

I tired resetting the default branch but it doesn't work.

Also I didn't see any error in the logs. I am sure that I have RepoIndexerEnabled, as the code search works for plain repos instead of the mirrored ones.

@ghost ghost changed the title REPO_INDEXER won't index mirrored repository, even if the NotifySyncPushCommits is called REPO_INDEXER won't index mirrored repository Apr 10, 2021
@ghost
Copy link
Author

ghost commented Apr 10, 2021

@zeripath

Tired on 1.13.7 with many different mirrored repos, but the problem persists.
Also I tried deleting repos.bleve to rebuild the index. The REPO_INDEXER will properly index the current default branch of the mirrored repo, but it's not going to index new commits after sync

@ghost ghost changed the title REPO_INDEXER won't index mirrored repository [BUG] REPO_INDEXER won't index mirrored repository after sync Apr 10, 2021
@ghost
Copy link
Author

ghost commented Apr 26, 2021

@zeripath Same on v1.14.1

@ghost
Copy link
Author

ghost commented May 13, 2021

Same with v1.14.2

@noerw noerw added the type/bug label May 14, 2021
@lflare
Copy link
Contributor

lflare commented Feb 22, 2023

Just to chime in that I am also noticing this issue on v1.18.5. Default branch is set as well.

I patched the original snippet of code to log out the test variables, in the following, and it seems that opts.RefFullName is not the same between push and sync pull.

func (r *indexerNotifier) NotifyPushCommits(pusher *user_model.User, repo *repo_model.Repository, opts *repository.PushUpdateOptions, commits *repository.PushCommits) {
	log.Info("stats_indexer.NotifyPushCommits(%d) RefFullName: %s, BranchPrefix: %s, DefaultBranch: %s", repo.ID, opts.RefFullName, git.BranchPrefix, repo.DefaultBranch)

	if setting.Indexer.RepoIndexerEnabled && opts.RefFullName == git.BranchPrefix+repo.DefaultBranch {
		code_indexer.UpdateRepoIndexer(repo)
	}
	if err := stats_indexer.UpdateRepoIndexer(repo); err != nil {
		log.Error("stats_indexer.UpdateRepoIndexer(%d) failed: %v", repo.ID, err)
	}
}

func (r *indexerNotifier) NotifySyncPushCommits(pusher *user_model.User, repo *repo_model.Repository, opts *repository.PushUpdateOptions, commits *repository.PushCommits) {
	log.Info("stats_indexer.NotifySyncPushCommits(%d) RefFullName: %s, BranchPrefix: %s, DefaultBranch: %s", repo.ID, opts.RefFullName, git.BranchPrefix, repo.DefaultBranch)

	if setting.Indexer.RepoIndexerEnabled && opts.RefFullName == git.BranchPrefix+repo.DefaultBranch {
		code_indexer.UpdateRepoIndexer(repo)
	}
	if err := stats_indexer.UpdateRepoIndexer(repo); err != nil {
		log.Error("stats_indexer.UpdateRepoIndexer(%d) failed: %v", repo.ID, err)
	}
}

When pushing,

2023/02/22 11:07:06 ...n/indexer/indexer.go:128:NotifyPushCommits() [I] [63f5f75a] stats_indexer.NotifyPushCommits(200) RefFullName: refs/heads/main, BranchPrefix: refs/heads/, DefaultBranch: main

When sync pulling,

2023/02/22 11:06:50 ...n/indexer/indexer.go:139:NotifySyncPushCommits() [I] [63f5f749-17] stats_indexer.NotifySyncPushCommits(199) RefFullName: main, BranchPrefix: refs/heads/, DefaultBranch: main

So clearly the comparison of setting.Indexer.RepoIndexerEnabled && opts.RefFullName == git.BranchPrefix+repo.DefaultBranch might need to be modified.

techknowlogick pushed a commit that referenced this issue May 26, 2023
…nc bugs (#24634)

This PR replaces all string refName as a type `git.RefName` to make the
code more maintainable.

Fix #15367
Replaces #23070 
It also fixed a bug that tags are not sync because `git remote --prune
origin` will not remove local tags if remote removed.

We in fact should use `git fetch --prune --tags origin` but not `git
remote update origin` to do the sync.

Some answer from ChatGPT as ref.

> If the git fetch --prune --tags command is not working as expected,
there could be a few reasons why. Here are a few things to check:
> 
>Make sure that you have the latest version of Git installed on your
system. You can check the version by running git --version in your
terminal. If you have an outdated version, try updating Git and see if
that resolves the issue.
> 
>Check that your Git repository is properly configured to track the
remote repository's tags. You can check this by running git config
--get-all remote.origin.fetch and verifying that it includes
+refs/tags/*:refs/tags/*. If it does not, you can add it by running git
config --add remote.origin.fetch "+refs/tags/*:refs/tags/*".
> 
>Verify that the tags you are trying to prune actually exist on the
remote repository. You can do this by running git ls-remote --tags
origin to list all the tags on the remote repository.
> 
>Check if any local tags have been created that match the names of tags
on the remote repository. If so, these local tags may be preventing the
git fetch --prune --tags command from working properly. You can delete
local tags using the git tag -d command.

---------

Co-authored-by: delvh <dev.lh@web.de>
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 11, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants