Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix lock service ut #16517

Merged
merged 5 commits into from
Jun 3, 2024
Merged

fix lock service ut #16517

merged 5 commits into from
Jun 3, 2024

Conversation

iamlinjunhong
Copy link
Contributor

@iamlinjunhong iamlinjunhong commented May 30, 2024

User description

What type of PR is this?

  • API-change
  • BUG
  • Improvement
  • Documentation
  • Feature
  • Test and CI
  • Code Refactoring

Which issue(s) this PR fixes:

issue #15757

What this PR does / why we need it:

fix lock service ut


PR Type

Tests


Description

  • Changed timeout duration from seconds to milliseconds in TestLockSuccWithKeepBindTimeout and TestReLockSuccWithKeepBindTimeout to improve test precision.
  • Replaced time.Sleep calls with loops checking conditions in TestReLockSuccWithReStartCN to avoid arbitrary delays and improve test reliability.
  • Added additional assertions to ensure proper unlocking in TestReLockSuccWithReStartCN.

Changes walkthrough 📝

Relevant files
Tests
service_test.go
Improve lock service tests with precise timing and conditions

pkg/lockservice/service_test.go

  • Changed timeout duration from seconds to milliseconds in multiple
    tests.
  • Replaced sleep calls with loops checking conditions in tests.
  • Added additional assertions to ensure proper unlocking.
  • +18/-22 

    💡 PR-Agent usage:
    Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

    Copy link

    PR Review 🔍

    ⏱️ Estimated effort to review [1-5]

    2, because the changes are focused on improving test reliability and precision in a single file with a moderate amount of changes. The logic is straightforward, involving timing adjustments and replacing arbitrary sleep calls with condition checks.

    🧪 Relevant tests

    Yes

    ⚡ Possible issues

    Possible Bug: The loop in TestReLockSuccWithReStartCN might lead to an infinite loop if the condition l1.isStatus(pb.Status_ServiceCanRestart) never becomes true. Consider adding a timeout mechanism or a maximum iteration count to prevent this.

    🔒 Security concerns

    No

    @matrix-meow matrix-meow added the size/S Denotes a PR that changes [10,99] lines label May 30, 2024
    Copy link

    PR Code Suggestions ✨

    CategorySuggestion                                                                                                                                    Score
    Best practice
    Use t.Fatal instead of panic to fail the test gracefully

    Instead of using a panic statement, consider returning an error or using t.Fatal to fail
    the test gracefully.

    pkg/lockservice/service_test.go [1635]

    -panic("timeout bug")
    +t.Fatal("timeout bug")
     
    Suggestion importance[1-10]: 8

    Why: Replacing panic with t.Fatal in test code is a significant improvement as it allows the test framework to handle failures more gracefully and provide more informative test failure messages.

    8
    Possible issue
    Increase the timeout duration to reduce the risk of test flakiness

    Consider increasing the timeout duration from time.Millisecond to a slightly higher value
    to avoid potential flakiness in tests due to very short timeouts.

    pkg/lockservice/service_test.go [1350]

    -time.Millisecond,
    +time.Millisecond * 10,
     
    Suggestion importance[1-10]: 7

    Why: The suggestion to increase the timeout from time.Millisecond to time.Millisecond * 10 is valid as it addresses potential flakiness in tests due to very short timeouts. This is a practical improvement for stability in test environments.

    7
    Add a maximum number of iterations to prevent the loop from running indefinitely

    Add a check to ensure that the loop does not run indefinitely by setting a maximum number
    of iterations.

    pkg/lockservice/service_test.go [1629-1638]

    -for {
    +maxRetries := 100
    +for i := 0; i < maxRetries; i++ {
         if l1.isStatus(pb.Status_ServiceCanRestart) {
             break
         }
         select {
         case <-ctx.Done():
    -        panic("timeout bug")
    +        t.Fatal("timeout bug")
         default:
    +        time.Sleep(time.Millisecond * 10)
         }
     }
     
    Suggestion importance[1-10]: 7

    Why: Introducing a maximum number of iterations for the loop is a good practice to prevent potential infinite loops, enhancing the robustness of the test code. This suggestion also integrates well with the previous suggestion of using t.Fatal for error handling.

    7
    Performance
    Add a small sleep duration inside the loop to avoid busy-waiting

    Add a small sleep duration inside the loop to avoid busy-waiting, which can consume
    unnecessary CPU resources.

    pkg/lockservice/service_test.go [1629-1638]

     for {
         if l1.isStatus(pb.Status_ServiceCanRestart) {
             break
         }
         select {
         case <-ctx.Done():
             panic("timeout bug")
         default:
    +        time.Sleep(time.Millisecond * 10)
         }
     }
     
    Suggestion importance[1-10]: 6

    Why: Adding a sleep duration inside the loop can help reduce CPU usage due to busy-waiting. This is a good performance improvement, although not critical.

    6

    @mergify mergify bot merged commit ffab9d0 into matrixorigin:main Jun 3, 2024
    18 checks passed
    XuPeng-SH pushed a commit to XuPeng-SH/matrixone that referenced this pull request Jun 4, 2024
    * GC needs to consume all the mo_snapshot tables (matrixorigin#16539)
    
    Each tenant of the current mo has a mo_snapshot table to store snapshot information. GC needs to consume all mo_snapshot tables.
    
    Approved by: @XuPeng-SH
    
    * append log for upgrade and sqlExecutoer (matrixorigin#16575)
    
    append log for upgrader and sqlExecutor
    
    Approved by: @daviszhen, @badboynt1, @zhangxu19830126, @m-schen
    
    * [enhancement] proxy: filter CNs that are not in working state. (matrixorigin#16558)
    
    1. filter CNs that are not in working state.
    2. add some logs for migration
    
    Approved by: @zhangxu19830126
    
    * fix lock service ut (matrixorigin#16517)
    
    fix lock service ut
    
    Approved by: @zhangxu19830126
    
    * Add cost of GC Check (matrixorigin#16470)
    
    To avoid List() operations on oss, tke or s3, you need to add the Cost interface.
    
    Approved by: @reusee, @XuPeng-SH
    
    * optimize explain info for tp/ap query (matrixorigin#16578)
    
    optimize explain info for tp/ap query
    
    Approved by: @daviszhen, @ouyuanning, @aunjgr
    
    * Bvt disable trace (matrixorigin#16581)
    
    aim to exclude the `system,system_metrics` part case.
    changes:
    1. move `cases/table/system_table_cases` system,system_metrics part into individule case file.
    
    Approved by: @heni02
    
    * remove log print from automaxprocs (matrixorigin#16546)
    
    remove log print from automaxprocs
    
    Approved by: @triump2020, @m-schen, @ouyuanning, @aunjgr, @zhangxu19830126
    
    * rmTag15901 (matrixorigin#16585)
    
    rm 15901
    
    Approved by: @heni02
    
    * remove some MustStrCol&MustBytesCol (matrixorigin#16361)
    
    Remove some unnecessary MustStrCol, MustBytesCol calls.
    
    Approved by: @daviszhen, @reusee, @m-schen, @aunjgr, @XuPeng-SH
    
    * add bvt tag (matrixorigin#16589)
    
    add bvt tag
    
    Approved by: @heni02, @aressu1985
    
    * fix a bug that cause load performance regression issue (matrixorigin#16600)
    
    fix a bug that cause load performance regression issue
    
    Approved by: @m-schen
    
    * add case for restore pub_sub (matrixorigin#16602)
    
    add case for restore pub_sub
    
    Approved by: @heni02
    
    * add shard service kernel (matrixorigin#16565)
    
    Add shardservice kernel.
    
    Approved by: @reusee, @m-schen, @daviszhen, @XuPeng-SH, @volgariver6, @badboynt1, @ouyuanning, @triump2020, @w-zr, @sukki37, @aunjgr, @fengttt
    
    * [BugFix]: Use L2DistanceSq instead of L2Distance during IndexScan (matrixorigin#16366)
    
    During `KNN Select` and `Mapping Entries to Centroids via CROSS_JOIN_L2`, we can make use of L2DistanceSq instead of L2Distance, as it avoids `Sqrt()`. We can see the improvement in QPS for SIFT128 from 90 to 100. However, for GIST960, the QPS did not change much.
    
    L2DistanceSq is suitable only when there is a comparison (ie ORDER BY), and when the absolute value (ie actual L2Distance) is not required.
    - In the case of `CROSS JOIN L2` we find the nearest centroid for the Entry using `L2DistanceSq`. `CROSS JOIN L2` is used in both INSERT and CREATE INDEX.
    - In the case of `KNN SELECT`, our query has ORDER BY L2_DISTANCE(...), which can make use of `L2DistanceSq` as the L2Distance value is not explicitly required.
    
    **NOTE:** L2DistanceSq is not suitable in Kmenas++ for Centroid Computation, as it will impact the centroids picked.
    
    Approved by: @heni02, @m-schen, @aunjgr, @badboynt1
    
    * add sharding metrics (matrixorigin#16606)
    
    add sharding metrics
    
    Approved by: @aptend
    
    * fix data race (matrixorigin#16608)
    
    fix data race
    
    Approved by: @reusee
    
    * Refactor reshape (matrixorigin#15879)
    
    Reshape objects block by block.
    
    Approved by: @XuPeng-SH
    
    * refactor system variables to support account isolation (matrixorigin#16551)
    
    - system variable now is account isolated
    - table `mo_mysql_compatibility_mode` only saves delta info between account's and cluster's default system variable values
    - always use session variable except `show global variables`
    
    Approved by: @daviszhen, @aunjgr, @aressu1985
    
    * fix merge
    
    * [cherry-pick-16594] : fix moc3399 (matrixorigin#16611)
    
    When truncate table, if the table does not have any auto-incr col, there is no need to call the Reset interface of increment_service
    
    Approved by: @ouyuanning
    
    * bump go to 1.22.3, fix make compose and optimize ut script (matrixorigin#16604)
    
    1. bump go version from 1.21.5 to 1.22.3
    2. fix `make compose` to make it work
    3. `make ut` will read `UT_WORKDIR` env variable to store report, it will be `$HOME` if `UT_WORKDIR` is empty
    
    Approved by: @zhangxu19830126, @sukki37
    
    * remove isMerge from build operator (matrixorigin#16622)
    
    remove isMerge from build operator
    
    Approved by: @m-schen
    
    ---------
    
    Co-authored-by: GreatRiver <2552853833@qq.com>
    Co-authored-by: qingxinhome <70939751+qingxinhome@users.noreply.github.com>
    Co-authored-by: LiuBo <g.user.lb@gmail.com>
    Co-authored-by: iamlinjunhong <49111204+iamlinjunhong@users.noreply.github.com>
    Co-authored-by: nitao <badboynt@126.com>
    Co-authored-by: Jackson <xzxiong@yeah.net>
    Co-authored-by: Ariznawlll <ariznawl@163.com>
    Co-authored-by: Wei Ziran <weiziran125@gmail.com>
    Co-authored-by: YANGGMM <www.yangzhao123@gmail.com>
    Co-authored-by: fagongzi <zhangxu19830126@gmail.com>
    Co-authored-by: Arjun Sunil Kumar <arjunsk@users.noreply.github.com>
    Co-authored-by: Kai Cao <ck89119@users.noreply.github.com>
    Co-authored-by: Jensen <jensenojs@qq.com>
    Co-authored-by: brown <endeavorjia@gmail.com>
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    Review effort [1-5]: 2 size/S Denotes a PR that changes [10,99] lines Tests
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    None yet

    3 participants