Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: Remove transparent_decompress_chunk-[12,13,14,15] from Ignore List. #5479

Closed
shhnwz opened this issue Mar 23, 2023 · 2 comments · Fixed by #5546
Closed

[Bug]: Remove transparent_decompress_chunk-[12,13,14,15] from Ignore List. #5479

shhnwz opened this issue Mar 23, 2023 · 2 comments · Fixed by #5546
Assignees
@shhnwz
Labels
bug

Comments

@shhnwz
Copy link
Contributor

shhnwz commented Mar 23, 2023 •
edited

What type of bug is this?

Other

What subsystems and features are affected?

Compression

What happened?

I had to add transparent_decompress_chunk-[12,13,14,15] in the ignore list. This seems as a side effect of #5118 [Stats improvement code cleanup] planner unable to generate consistent plan across the versions and system. I will open an issue to make it more deterministic from the planner perspective. Please let me know if there's any concern towards this approach.

TimescaleDB version affected

2.10

PostgreSQL version used

14.7

What operating system did you use?

Ubuntu 20.04 x64

What installation method did you use?

Not applicable

What platform did you run on?

Not applicable

Relevant log output and stack trace

No response

How can we reproduce the bug?

make installcheck TESTS=transparent_decompress_chunk-13
@shhnwz shhnwz added the bug label Mar 23, 2023
@shhnwz shhnwz self-assigned this Mar 23, 2023
@shhnwz
Copy link
Contributor Author

shhnwz commented Mar 23, 2023

Version-wise Explain Diffs from github-workflow runs during PR Merge

     transparent_decompress_chunk-12 ... failed (ignored)      940 ms
diff -u /home/runner/work/timescaledb/timescaledb/tsl/test/shared/expected/transparent_decompress_chunk-12.out /home/runner/work/timescaledb/timescaledb/build/tsl/test/shared/results/transparent_decompress_chunk-12.out
--- /home/runner/work/timescaledb/timescaledb/tsl/test/shared/expected/transparent_decompress_chunk-12.out	2023-03-22 14:28:18.689038596 +0000
+++ /home/runner/work/timescaledb/timescaledb/build/tsl/test/shared/results/transparent_decompress_chunk-12.out	2023-03-22 14:42:02.219923635 +0000
@@ -811,16 +811,16 @@
    ->  Gather Merge (actual rows=100 loops=1)
          Workers Planned: 1
          Workers Launched: 1
-         ->  Sort (actual rows=90 loops=2)
+         ->  Sort (actual rows=50 loops=2)
                Sort Key: m1."time", m1.device_id, m2."time", m2.device_id
                Sort Method: top-N heapsort 
-               Worker 0:  Sort Method: top-N heapsort 
+               Worker 0:  Sort Method: quicksort 
                ->  Parallel Hash Left Join (actual rows=8995 loops=2)
                      Hash Cond: (m1."time" = m2."time")
                      Join Filter: (m1.device_id = 1)
                      Rows Removed by Join Filter: 7196
-                     ->  Custom Scan (DecompressChunk) on _hyper_X_X_chunk m1 (actual rows=8995 loops=2)
-                           ->  Parallel Seq Scan on compress_hyper_X_X_chunk (actual rows=10 loops=2)
+                     ->  Custom Scan (DecompressChunk) on _hyper_X_X_chunk m1 (actual rows=17990 loops=1)
+                           ->  Parallel Seq Scan on compress_hyper_X_X_chunk (actual rows=20 loops=1)
                      ->  Parallel Hash (actual rows=1799 loops=2)
                            Buckets: 4096 (originally 2048)  Batches: 1 (originally 1) 
                            ->  Custom Scan (DecompressChunk) on _hyper_X_X_chunk m2 (actual rows=3598 loops=1)

     transparent_decompress_chunk-13 ... failed (ignored)      994 ms
diff -u /home/runner/work/timescaledb/timescaledb/tsl/test/shared/expected/transparent_decompress_chunk-13.out /home/runner/work/timescaledb/timescaledb/build/tsl/test/shared/results/transparent_decompress_chunk-13.out
--- /home/runner/work/timescaledb/timescaledb/tsl/test/shared/expected/transparent_decompress_chunk-13.out	2023-03-22 14:28:21.023475860 +0000
+++ /home/runner/work/timescaledb/timescaledb/build/tsl/test/shared/results/transparent_decompress_chunk-13.out	2023-03-22 14:41:23.670274150 +0000
@@ -674,7 +674,7 @@
          ->  Gather Merge (actual rows=10 loops=1)
                Workers Planned: 1
                Workers Launched: 1
-               ->  Sort (actual rows=304 loops=2)
+               ->  Sort (actual rows=5 loops=2)
                      Sort Key: m1."time", m1.device_id
                      Sort Method: quicksort 
                      Worker 0:  Sort Method: quicksort 
@@ -705,7 +705,7 @@
                ->  Gather Merge (actual rows=10 loops=1)
                      Workers Planned: 1
                      Workers Launched: 1
-                     ->  Sort (actual rows=304 loops=2)
+                     ->  Sort (actual rows=5 loops=2)
                            Sort Key: m1."time", m1.device_id
                            Sort Method: quicksort 
                            Worker 0:  Sort Method: quicksort 

     transparent_decompress_chunk-14 ... failed (ignored)     1258 ms
diff -u /home/runner/work/timescaledb/timescaledb/tsl/test/shared/expected/transparent_decompress_chunk-14.out /home/runner/work/timescaledb/timescaledb/build/tsl/test/shared/results/transparent_decompress_chunk-14.out
--- /home/runner/work/timescaledb/timescaledb/tsl/test/shared/expected/transparent_decompress_chunk-14.out	2023-03-22 14:28:07.473935361 +0000
+++ /home/runner/work/timescaledb/timescaledb/build/tsl/test/shared/results/transparent_decompress_chunk-14.out	2023-03-22 14:38:09.613713036 +0000
@@ -705,7 +705,7 @@
                ->  Gather Merge (actual rows=10 loops=1)
                      Workers Planned: 1
                      Workers Launched: 1
-                     ->  Sort (actual rows=5 loops=2)
+                     ->  Sort (actual rows=304 loops=2)
                            Sort Key: m1."time", m1.device_id
                            Sort Method: quicksort 
                            Worker 0:  Sort Method: quicksort 
@@ -793,7 +793,7 @@
          ->  Gather Merge (actual rows=10 loops=1)
                Workers Planned: 1
                Workers Launched: 1
-               ->  Sort (actual rows=5 loops=2)
+               ->  Sort (actual rows=304 loops=2)
                      Sort Key: m1."time", m1.device_id
                      Sort Method: quicksort 
                      Worker 0:  Sort Method: quicksort 

  transparent_decompress_chunk-15 ... failed (ignored)     1244 ms
diff -u /home/runner/work/timescaledb/timescaledb/tsl/test/shared/expected/transparent_decompress_chunk-15.out /home/runner/work/timescaledb/timescaledb/build/tsl/test/shared/results/transparent_decompress_chunk-15.out
--- /home/runner/work/timescaledb/timescaledb/tsl/test/shared/expected/transparent_decompress_chunk-15.out	2023-03-22 14:28:12.870723070 +0000
+++ /home/runner/work/timescaledb/timescaledb/build/tsl/test/shared/results/transparent_decompress_chunk-15.out	2023-03-22 14:38:14.386624789 +0000
@@ -676,7 +676,7 @@
          ->  Gather Merge (actual rows=10 loops=1)
                Workers Planned: 1
                Workers Launched: 1
-               ->  Sort (actual rows=5 loops=2)
+               ->  Sort (actual rows=304 loops=2)
                      Sort Key: m1."time", m1.device_id
                      Sort Method: quicksort 
                      Worker 0:  Sort Method: quicksort

@shhnwz shhnwz changed the title [Bug]: Remove transparent_decompression_chunk-[12,13,14,15] from Ignore List. [Bug]: Remove transparent_decompress_chunk-[12,13,14,15] from Ignore List. Mar 23, 2023
@shhnwz shhnwz linked a pull request Apr 18, 2023 that will close this issue
Fixed transparent decompress chunk #5546
Merged
@shhnwz
Copy link
Contributor Author

shhnwz commented Apr 18, 2023

Root Cause Analysis

SORT STEP in GATHER_MERGE not calculating Actual Rows deterministically.


RUN-1
=======

diff -u /home/shhnwz/github/timescaledb/tsl/test/shared/expected/transparent_decompress_chunk-14.out /home/shhnwz/github/timescaledb/build_14_Debug/tsl/test/shared/results/transparent_decompress_chunk-14.out
--- /home/shhnwz/github/timescaledb/tsl/test/shared/expected/transparent_decompress_chunk-14.out        2023-04-12 14:07:25.594005000 +0530
+++ /home/shhnwz/github/timescaledb/build_14_Debug/tsl/test/shared/results/transparent_decompress_chunk-14.out  2023-04-12 14:22:21.166016394 +0530
@@ -700,7 +700,7 @@
          ->  Gather Merge (actual rows=10 loops=1)
                Workers Planned: 1
                Workers Launched: 1
-               ->  Sort (actual rows=588 loops=2)
+               ->  Sort (actual rows=586 loops=2)
                      Sort Key: m1."time", m1.device_id
                      Sort Method: quicksort
                      Worker 0:  Sort Method: quicksort
@@ -819,7 +819,7 @@
          ->  Gather Merge (actual rows=10 loops=1)
                Workers Planned: 1
                Workers Launched: 1
-               ->  Sort (actual rows=588 loops=2)
+               ->  Sort (actual rows=586 loops=2)
                      Sort Key: m1."time", m1.device_id
                      Sort Method: quicksort
                      Worker 0:  Sort Method: quicksort
@@ -845,7 +845,7 @@
    ->  Gather Merge (actual rows=100 loops=1)
          Workers Planned: 1
          Workers Launched: 1
-         ->  Sort (actual rows=51 loops=2)
+         ->  Sort (actual rows=90 loops=2)
                Sort Key: m1."time", m1.device_id, m2."time", m2.device_id
                Sort Method: top-N heapsort
                Worker 0:  Sort Method: top-N heapsort

RUN-2
======

diff -u /home/shhnwz/github/timescaledb/tsl/test/shared/expected/transparent_decompress_chunk-14.out /home/shhnwz/github/timescaledb/build_14_Debug/tsl/test/shared/results/transparent_decompress_chunk-14.out
--- /home/shhnwz/github/timescaledb/tsl/test/shared/expected/transparent_decompress_chunk-14.out        2023-04-12 14:07:25.594005000 +0530
+++ /home/shhnwz/github/timescaledb/build_14_Debug/tsl/test/shared/results/transparent_decompress_chunk-14.out  2023-04-12 14:25:35.930018746 +0530
@@ -700,7 +700,7 @@
          ->  Gather Merge (actual rows=10 loops=1)
                Workers Planned: 1
                Workers Launched: 1
-               ->  Sort (actual rows=588 loops=2)
+               ->  Sort (actual rows=586 loops=2)
                      Sort Key: m1."time", m1.device_id
                      Sort Method: quicksort
                      Worker 0:  Sort Method: quicksort
@@ -731,7 +731,7 @@
                ->  Gather Merge (actual rows=10 loops=1)
                      Workers Planned: 1
                      Workers Launched: 1
-                     ->  Sort (actual rows=586 loops=2)
+                     ->  Sort (actual rows=588 loops=2)
                            Sort Key: m1."time", m1.device_id
                            Sort Method: quicksort
                            Worker 0:  Sort Method: quicksort
@@ -819,7 +819,7 @@
          ->  Gather Merge (actual rows=10 loops=1)
                Workers Planned: 1
                Workers Launched: 1
-               ->  Sort (actual rows=588 loops=2)
+               ->  Sort (actual rows=586 loops=2)
                      Sort Key: m1."time", m1.device_id
                      Sort Method: quicksort
                      Worker 0:  Sort Method: quicksort

@shhnwz shhnwz closed this as completed Apr 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@shhnwz shhnwz

Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fixed transparent decompress chunk shhnwz/timescaledb
1 participant
@shhnwz