Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: using indexes to optimize queries #69

Merged
merged 4 commits into from
Feb 4, 2024
Merged

Conversation

guqing
Copy link
Member

@guqing guqing commented Jan 3, 2024

What this PR does?

使用索引来优化瞬间查询

how to test it?
基于 Halo 2.12.0
可以使用以下 jar 包测试(keyword 查询目前没有对内容建索引,暂时只可以根据 owner 用户名查询):

plugin-moments-1.0.1-SNAPSHOT.jar.zip

可以使用 postgre 数据库用以下脚本插入数据测试:

点击展开查看 SQL
CREATE OR REPLACE FUNCTION random_string(length integer) RETURNS text AS
$$
DECLARE
    chars  text[]  := array ['a','b','c','d','e','f','g','h','i','j','k','l','m',
        'n','o','p','q','r','s','t','u','v','w','x','y','z',
        'A','B','C','D','E','F','G','H','I','J','K','L','M',
        'N','O','P','Q','R','S','T','U','V','W','X','Y','Z',
        '0','1','2','3','4','5','6','7','8','9'];
    result text    := '';
    i      integer := 0;
BEGIN
    FOR i IN 1..length
        LOOP
            result := result || chars[1 + floor(random() * array_length(chars, 1))];
        END LOOP;
    RETURN result;
END;
$$ LANGUAGE plpgsql;

DO
$$
    DECLARE
        json_data  jsonb;
        bytea_data bytea;
        i          integer;
    BEGIN
        -- 可以修改此处的值来创建更大的数据集,默认 1w
        FOR i IN 1..10000
            LOOP
                -- 创建初始 JSON 数据
                json_data := '{
                  "spec": {
                    "content": {
                      "raw": "",
                      "medium": []
                    },
                    "releaseTime": "2024-01-09T07:24:46.077Z",
                    "visible": "PUBLIC",
                    "owner": "admin",
                    "tags": []
                  },
                  "status": {
                    "observedVersion": 0
                  },
                  "apiVersion": "moment.halo.run/v1alpha1",
                  "kind": "Moment",
                  "metadata": {
                    "finalizers": [
                      "moment-protection"
                    ],
                    "name": "",
                    "version": 0,
                    "creationTimestamp": "2024-01-09T07:33:56.172968174Z"
                  }
                }'::jsonb;
                -- 更新 'raw' 字段
                json_data :=
                        jsonb_set(json_data, '{spec, content, raw}', to_jsonb('<p>' || random_string(20000) || '</p>'),
                                  TRUE);
                json_data :=
                        jsonb_set(json_data, '{spec, content, html}', to_jsonb('<p>' || random_string(20000) || '</p>'),
                                  TRUE);
                -- 更新 'metadata.name' 字段
                json_data := jsonb_set(json_data, '{metadata, name}', to_jsonb('moment-' || i), TRUE);
                -- 将 JSON 数据转换为 bytea
                bytea_data := convert_to(json_data::text, 'UTF8');
                -- 插入数据
                INSERT INTO extensions (name, data, version)
                VALUES ('/registry/moment.halo.run/moments/moment-' || i, bytea_data, 0);
            END LOOP;
    END
$$;
适配索引功能来优化瞬间的查询请求以提高响应速度并降低内存占用

@f2c-ci-robot f2c-ci-robot bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. labels Jan 3, 2024
@f2c-ci-robot f2c-ci-robot bot requested review from LIlGG and wan92hen January 3, 2024 08:48
@guqing guqing force-pushed the refactor/index branch 6 times, most recently from 7ac7b70 to 9712978 Compare January 9, 2024 10:30
@guqing guqing force-pushed the refactor/index branch 6 times, most recently from b58a08f to 90988af Compare January 12, 2024 08:10
@guqing guqing force-pushed the refactor/index branch 2 times, most recently from bd0faad to 3ff8a22 Compare January 26, 2024 07:38
@guqing guqing marked this pull request as ready for review February 1, 2024 07:54
@f2c-ci-robot f2c-ci-robot bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 1, 2024
@LIlGG
Copy link
Member

LIlGG commented Feb 2, 2024

未对 keyword 建索引的情况下,左上角的搜索框就没有什么太大的意义了,个人建议暂时隐藏掉,后续有更好的解决方案之后再继续实现瞬间的 全文搜索。

@guqing
Copy link
Member Author

guqing commented Feb 2, 2024

未对 keyword 建索引的情况下,左上角的搜索框就没有什么太大的意义了,个人建议暂时隐藏掉,后续有更好的解决方案之后再继续实现瞬间的 全文搜索。

当前这个 PR 可以先不管这个搜索框 你可以重新提一个 pr 用来修改布局

Copy link
Member

@LIlGG LIlGG left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@f2c-ci-robot f2c-ci-robot bot added the lgtm Indicates that a PR is ready to be merged. label Feb 2, 2024
Copy link
Member

@ruibaby ruibaby left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

Copy link

f2c-ci-robot bot commented Feb 4, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: LIlGG, ruibaby

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@f2c-ci-robot f2c-ci-robot bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 4, 2024
@f2c-ci-robot f2c-ci-robot bot merged commit 2b2486c into main Feb 4, 2024
1 check passed
@guqing guqing deleted the refactor/index branch February 4, 2024 10:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants