Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

optimize add cache first number page of post in custom configure #78

Merged
merged 5 commits into from
Jun 11, 2022

Conversation

alimy
Copy link
Collaborator

@alimy alimy commented Jun 10, 2022

  • optimize add cache first number page of post in custom configure
  • add core.CacheIndexService to abstract cache index posts
  • LoggerFile/LoggerZinc support config level of logrus

添加缓存泡泡广场前几页消息流的功能,优化前几页页面加载速度,也减轻后端数据库负担。可以按需启用缓存功能,开启方法为
在配置文件conf.yaml的 Features 节中添加CacheIndex 功能。

为什么添加前几页消息流缓存功能?
前几页消息流 是请求热点,通过缓存消息流来优化请求速度,同时也能减轻一点后端数据库的查询压力。目前实现是一个简单的立足单节点服务的版本,后续再优化完善。

实现缓存的思路是什么?

  1. 开一个goroutine用于 消息流缓存更新,通过chan 来传递更新事件避免加锁;
  2. 周期性自动检查是否需要重新从数据库拉取新数据更新消息流,目前默认60s周期性循环检查是否需要更新;
  3. 周期性强制过期消息流缓存, 这个是考虑到多节点部署的情况下其他节点更新 新文章,需要同步一下, 这里采用简单处理,周期性强制过期消息流,等待重新进入步骤2; 默认300s也就是5分钟为一个周期;
  4. 创建/删除文章、创建/删除评论、置顶/取消置顶等更新文章 都会强制过期消息流缓存,等待进入步骤2;
  5. 缓存有效时就从缓存取消息流,否则直接从数据库拉取新数据。

@alimy alimy requested a review from rocboss June 10, 2022 15:06
Copy link
Owner

@rocboss rocboss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job! 这么晚了,还在commit👍🏻

建议可以通过redis getset/setnx简单维护一个latestVersion(比如用时间戳),多实例部署的情况下,单实例不停轮询latestVersion(间隔可以小一点),如果latestVersion > localVersion,则本地重新生成缓存且更新localVersion。
这样应该可以省掉第2、3步。

@rocboss
Copy link
Owner

rocboss commented Jun 10, 2022

我本地没有测,要是测过没问题的话就合入主干吧

@alimy
Copy link
Collaborator Author

alimy commented Jun 10, 2022

建议可以通过redis getset/setnx简单维护一个latestVersion(比如用时间戳),多实例部署的情况下,单实例不停轮询latestVersion(间隔可以小一点),如果latestVersion > localVersion,则本地重新生成缓存且更新localVersion。
这样应该可以省掉第2、3步。

已经抽象出core.CacheIndexService, 目前实现了一个着眼于单实例部署的简单版本,后续可以按需要实现基于redis版本或基于消息队列的版本。 本机 上测试按预期路径执行,配置不使用CacheIndexService的话功能和以前保持一样,不过还没有编写压力测试用例,后续再跟上, 这里先合并到主线(目前项目处在开荒期就先随意点~),欢迎测试优化完善。

@alimy alimy merged commit 70e1c58 into rocboss:main Jun 11, 2022
alimy added a commit to alimy/paopao-ce that referenced this pull request Jun 11, 2022
alimy added a commit that referenced this pull request Jun 11, 2022
optimize #78 change CacheIndex in conf.yaml to SimpleCacheIndex
alimy added a commit to alimy/paopao-ce that referenced this pull request Jun 11, 2022
alimy added a commit that referenced this pull request Jun 11, 2022
optimize #78 support disable expire cache index if needed
alimy added a commit to alimy/paopao-ce that referenced this pull request Jun 11, 2022
alimy added a commit that referenced this pull request Jun 11, 2022
optimize #78 support custom configure ActionQPS
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants