Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: snapshot attributes and post publish #2709

Merged
merged 10 commits into from Nov 18, 2022

Conversation

guqing
Copy link
Member

@guqing guqing commented Nov 16, 2022

What type of PR is this?

/kind improvement
/area core
/milestone 2.0

What this PR does / why we need it:

  1. 简化 Snapshot 的职责,去除 Snapshot 中关于 version,publishTime 等与发布相关的东西,Snapshot本身没有发布概念
  2. 对应修改文章和自定义页面,将版本的概念转移到拥有 Snapshot 的模型上,如 Post 和 SinglePage

⚠️此 PR 为破坏性更新,对旧数据不兼容,可能导致旧数据文章内容无法显示问题

Console 端需要做出一些修改:

  • 将文章发布前调用保存内容的 API 修改为 /posts/{name}/content
  • 将自定义页面发布前调用保存内容的 API 修改为 /singlepages/{name}/content

发布接口提供了一个 async 参数,默认值为 false此时发布API需要等待发布成功后才返回结果,如果等待超时则提示Publishing wait timeout., 如果传递 async=true 表示异步发布,更新完数据就成功,reconciler 在后台慢慢执行。

Special notes for your reviewer:

how to test it?

  1. 新创建一篇文章,测试点击保存是否成功
  2. 新创建一篇文章,测试编辑内容后直接发布是否成功
  3. 测试发布过的文章的取消发布状态是否显示未发布状态
  4. 对于 Snapshot,新建文章点保存会创建一个snapshot,点击发布则是更新之前点保存创建的一条,所以记录数不会多,发布之后进入编辑随便写点内容直接点发布后查询snapshot则会多一条记录。
  5. 文章的 status 里的 conditions 最新添加的都在第一个即头插入,如果连续两次发布则不会在增加 condition,长度超过20条会把旧的 condition 删除掉
  6. 自定义页面上同

/cc @halo-dev/sig-halo

Does this PR introduce a user-facing change?

Action Required: 简化 Snapshot 模型的职责,去除版本相关属性

@f2c-ci-robot f2c-ci-robot bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/improvement Categorizes issue or PR as related to a improvement. labels Nov 16, 2022
@f2c-ci-robot f2c-ci-robot bot added this to the 2.0 milestone Nov 16, 2022
@f2c-ci-robot f2c-ci-robot bot added the area/core Issues or PRs related to the Halo Core label Nov 16, 2022
@f2c-ci-robot f2c-ci-robot bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 16, 2022
@guqing
Copy link
Member Author

guqing commented Nov 16, 2022

/kind api-change

@f2c-ci-robot f2c-ci-robot bot added the kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API label Nov 16, 2022
…ttribute

# Conflicts:
#	src/main/resources/extensions/role-template-post.yaml
@f2c-ci-robot f2c-ci-robot bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 16, 2022
@guqing guqing changed the title [WIP] refactor: snapshot attributes and post publish refactor: snapshot attributes and post publish Nov 17, 2022
@f2c-ci-robot f2c-ci-robot bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 17, 2022
Copy link
Member

@ruibaby ruibaby left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@f2c-ci-robot f2c-ci-robot bot added the lgtm Indicates that a PR is ready to be merged. label Nov 18, 2022
@ruibaby
Copy link
Member

ruibaby commented Nov 18, 2022

@guqing 需要处理一下冲突。

@JohnNiang
Copy link
Member

JohnNiang commented Nov 18, 2022

实际测试时,可能会发生如下错误:

2022-11-18T16:29:12.193+08:00 ERROR 4940 --- [     parallel-7] a.w.r.e.AbstractErrorWebExceptionHandler : [87be3966-69]  500 Server Error for HTTP PUT "/apis/api.console.halo.run/v1alpha1/posts/0386ce9c-56fd-4cbb-9020-889010c51866/publish"

reactor.core.Exceptions$RetryExhaustedException: Retries exhausted: 10/10
	at reactor.core.Exceptions.retryExhausted(Exceptions.java:306) ~[reactor-core-3.5.0.jar:3.5.0]
	Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Assembly trace from producer [reactor.core.publisher.FluxConcatMapNoPrefetch] :
	reactor.core.publisher.Flux.concatMap(Flux.java:3877)
	reactor.util.retry.RetryBackoffSpec.lambda$generateCompanion$6(RetryBackoffSpec.java:545)
Error has been observed at the following site(s):
	*__________Flux.concatMapat reactor.util.retry.RetryBackoffSpec.lambda$generateCompanion$6(RetryBackoffSpec.java:545)
	|_      Flux.contextWriteat reactor.util.retry.RetryBackoffSpec.lambda$generateCompanion$6(RetryBackoffSpec.java:609)
	*____Flux.deferContextualat reactor.util.retry.RetryBackoffSpec.generateCompanion(RetryBackoffSpec.java:543)
	*__________Mono.retryWhenat run.halo.app.core.extension.endpoint.PostEndpoint.lambda$publishPost$16(PostEndpoint.java:200)
	*____________Mono.flatMapat run.halo.app.core.extension.endpoint.PostEndpoint.publishPost(PostEndpoint.java:189)
	|_           Mono.flatMapat run.halo.app.core.extension.endpoint.PostEndpoint.publishPost(PostEndpoint.java:203)
	|_               Mono.mapat org.springframework.web.reactive.function.server.support.HandlerFunctionAdapter.handle(HandlerFunctionAdapter.java:62)
	|_           Mono.flatMapat org.springframework.web.reactive.DispatcherHandler.handleRequestWith(DispatcherHandler.java:184)
	*____________Mono.flatMapat org.springframework.web.reactive.DispatcherHandler.handle(DispatcherHandler.java:154)
	*______________Mono.deferat org.springframework.web.server.handler.DefaultWebFilterChain.filter(DefaultWebFilterChain.java:106)
	|_          Mono.doOnEachat org.springframework.web.filter.reactive.ServerHttpObservationFilter.filter(ServerHttpObservationFilter.java:108)
	|_        Mono.doOnCancelat org.springframework.web.filter.reactive.ServerHttpObservationFilter.filter(ServerHttpObservationFilter.java:118)
	|_      Mono.contextWriteat org.springframework.web.filter.reactive.ServerHttpObservationFilter.filter(ServerHttpObservationFilter.java:122)
	*__Mono.transformDeferredat org.springframework.web.filter.reactive.ServerHttpObservationFilter.filter(ServerHttpObservationFilter.java:101)
	|_             checkpointorg.springframework.web.filter.reactive.ServerHttpObservationFilter [DefaultWebFilterChain]
	*______________Mono.deferat org.springframework.web.server.handler.DefaultWebFilterChain.filter(DefaultWebFilterChain.java:106)
2022-11-18T16:34:26.993+08:00 ERROR 4940 --- [         task-2] reactor.core.publisher.Operators         : Operator called default onErrorDropped

reactor.core.Exceptions$ErrorCallbackNotImplemented: java.lang.IllegalArgumentException: value must not be null
Caused by: java.lang.IllegalArgumentException: value must not be null
	at org.apache.lucene.document.Field.<init>(Field.java:235) ~[lucene-core-9.4.1.jar:9.4.1 810817993e0956e63e29906e78a245949501b77d - ivera - 2022-10-20 15:29:24]
	Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Assembly trace from producer [reactor.core.publisher.MonoPeekFuseable] :
	reactor.core.publisher.Mono.doOnNext(Mono.java:2643)
	run.halo.app.search.post.PostEventListener.lambda$handlePostPublished$1(PostEventListener.java:36)
Error has been observed at the following site(s):
	*___Mono.doOnNextat run.halo.app.search.post.PostEventListener.lambda$handlePostPublished$1(PostEventListener.java:36)
	*____Mono.flatMapat run.halo.app.search.post.PostEventListener.handlePostPublished(PostEventListener.java:35)
	|_ Mono.doFinallyat run.halo.app.search.post.PostEventListener.handlePostPublished(PostEventListener.java:44)
Original Stack Trace:
		at org.apache.lucene.document.Field.<init>(Field.java:235) ~[lucene-core-9.4.1.jar:9.4.1 810817993e0956e63e29906e78a245949501b77d - ivera - 2022-10-20 15:29:24]
		at org.apache.lucene.document.StoredField.<init>(StoredField.java:116) ~[lucene-core-9.4.1.jar:9.4.1 810817993e0956e63e29906e78a245949501b77d - ivera - 2022-10-20 15:29:24]
		at run.halo.app.search.post.LucenePostSearchService.convert(LucenePostSearchService.java:157) ~[classes/:na]
		at run.halo.app.search.post.LucenePostSearchService.lambda$addDocuments$0(LucenePostSearchService.java:100) ~[classes/:na]
		at java.base/java.lang.Iterable.forEach(Iterable.java:75) ~[na:na]
		at run.halo.app.search.post.LucenePostSearchService.addDocuments(LucenePostSearchService.java:99) ~[classes/:na]
		at run.halo.app.search.post.PostEventListener.lambda$handlePostPublished$0(PostEventListener.java:38) ~[classes/:na]

测试步骤:

  1. 创建一篇空的文章
  2. 直接右上角发布按钮
  3. 填写好标题、别名、分类和标签并点击发布按钮

…ttribute

# Conflicts:
#	src/main/java/run/halo/app/config/ExtensionConfiguration.java
@f2c-ci-robot f2c-ci-robot bot removed the lgtm Indicates that a PR is ready to be merged. label Nov 18, 2022
@guqing
Copy link
Member Author

guqing commented Nov 18, 2022

实际测试时,可能会发生如下错误:

  1. 第一个错误与 Reconiler 执行有关,调用 update 更新数据后触发 reconciler, 由于保存数据也是异步的,可能会导致 Reconciler 触发完成后 update sql 才被执行,这种情况下 reconciler 获取到旧数据无法发布,需要手动点击,可能需要关注 Reconciler 触发时机或者剔除机制,此错误不会导致数据丢失和无法发布
  2. 第二个错误应该是旧数据可以重新创建环境重试,此 PR 是破坏性更新

@f2c-ci-robot f2c-ci-robot bot added the release-note-action-required Denotes a PR that introduces potentially breaking changes that require user action. label Nov 18, 2022
@f2c-ci-robot f2c-ci-robot bot removed the release-note Denotes a PR that will be considered when it comes time to generate release notes. label Nov 18, 2022
@JohnNiang
Copy link
Member

以上问题建议在下一个 PR 优化。

/approve

@f2c-ci-robot
Copy link

f2c-ci-robot bot commented Nov 18, 2022

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: JohnNiang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@f2c-ci-robot f2c-ci-robot bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 18, 2022
f2c-ci-robot bot pushed a commit to halo-dev/console that referenced this pull request Nov 18, 2022
#### What type of PR is this?

/kind improvement
/milestone 2.0

#### What this PR does / why we need it:

优化文章和自定义页面的保存内容和发布逻辑。详情可查阅:halo-dev/halo#2709

#### Special notes for your reviewer:

/cc @halo-dev/sig-halo-console 

测试流程:

1. Halo 需要切换到 halo-dev/halo#2709 分支。
2. 测试文章和自定义页面的新建、更新内容,以及发布。

#### Does this PR introduce a user-facing change?

```release-note
优化文章和自定义页面的保存内容和发布逻辑。
```
Copy link
Member

@ruibaby ruibaby left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@f2c-ci-robot f2c-ci-robot bot added the lgtm Indicates that a PR is ready to be merged. label Nov 18, 2022
@f2c-ci-robot f2c-ci-robot bot merged commit cca95cb into halo-dev:main Nov 18, 2022
@guqing guqing deleted the refactor/snapshot-attribute branch November 18, 2022 13:35
JohnNiang pushed a commit to JohnNiang/halo that referenced this pull request Mar 2, 2023
…v/console#696)

#### What type of PR is this?

/kind improvement
/milestone 2.0

#### What this PR does / why we need it:

优化文章和自定义页面的保存内容和发布逻辑。详情可查阅:halo-dev#2709

#### Special notes for your reviewer:

/cc @halo-dev/sig-halo-console 

测试流程:

1. Halo 需要切换到 halo-dev#2709 分支。
2. 测试文章和自定义页面的新建、更新内容,以及发布。

#### Does this PR introduce a user-facing change?

```release-note
优化文章和自定义页面的保存内容和发布逻辑。
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/core Issues or PRs related to the Halo Core kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API kind/improvement Categorizes issue or PR as related to a improvement. lgtm Indicates that a PR is ready to be merged. release-note-action-required Denotes a PR that introduces potentially breaking changes that require user action.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants