Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix minio removeobjects #371

Merged
merged 1 commit into from
Aug 27, 2023
Merged

fix minio removeobjects #371

merged 1 commit into from
Aug 27, 2023

Conversation

cute-angelia
Copy link
Contributor

现象:存储 minio, post 带有多张图片,删除post,多张图片仍在minio内,无法删除

参考官方用法:
https://github.com/minio/minio-go/blob/master/examples/s3/removeobjects.go

pr 修复了 minio 多文件删除的问题

@alimy
Copy link
Collaborator

alimy commented Aug 26, 2023

试试下面👇这种改法是否可行, 如果可行的话,就按这种改法,可以省一次协程的创建(Minio-sdk 内部实现还新建了两协程,就为一次删除操作,搞那么多协程,太他喵奢侈了,真是不当家不知柴米油盐贵,协程再便宜那也是资源啊, 协程魔怔了属于是😂)

func (s *minioServant) DeleteObjects(objectKeys []string) (err error) {
	objectsCh := make(chan minio.ObjectInfo, len(objectKeys))

	resCh := s.client.RemoveObjects(context.Background(), s.bucket, objectsCh, minio.RemoveObjectsOptions{})
	for _, objectKey := range objectKeys {
		objectsCh <- minio.ObjectInfo{
			Key: objectKey,
		}
	}
        // 记得一定要close,否则会被卡死,退出不了函数,造成资源泄露!!!
	close(objectsCh)

	// 宽松处理所有错误,只记录最后一次发生的错误
	for result := range resCh {
		if result.Err != nil {
			err = result.Err
		}
	}
	return
}

@alimy
Copy link
Collaborator

alimy commented Aug 27, 2023

我在本机minio上试了一下,这种改法是有效的。我先合并你的改法,再进行优化一下。

试试下面👇这种改法是否可行, 如果可行的话,就按这种改法,可以省一次协程的创建(Minio-sdk 内部实现还新建了两协程,就为一次删除操作,搞那么多协程,太他喵奢侈了,真是不当家不知柴米油盐贵,协程再便宜那也是资源啊, 协程魔怔了属于是😂)

func (s *minioServant) DeleteObjects(objectKeys []string) (err error) {
	objectsCh := make(chan minio.ObjectInfo, len(objectKeys))

	resCh := s.client.RemoveObjects(context.Background(), s.bucket, objectsCh, minio.RemoveObjectsOptions{})
	for _, objectKey := range objectKeys {
		objectsCh <- minio.ObjectInfo{
			Key: objectKey,
		}
	}
        // 记得一定要close,否则会被卡死,退出不了函数,造成资源泄露!!!
	close(objectsCh)

	// 宽松处理所有错误,只记录最后一次发生的错误
	for result := range resCh {
		if result.Err != nil {
			err = result.Err
		}
	}
	return
}

@alimy alimy merged commit 0f2e0b2 into rocboss:main Aug 27, 2023
@cute-angelia
Copy link
Contributor Author

棒! 优化的很好

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants