Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Revert fixes involving issue #99 (which makes Arel unusable in large datasets) #174

Merged
merged 2 commits into from

2 participants

@eduardordm

There are two commits involving issue #99 which should be reverted.

First, #99 is not an issue in Arel at all. Second, the fix provided pretty much destroys the purpose of pagination by causing full table scans.

The original code (it seems I can't simply revert the commits) is 900 times slower than this one.

/cc @tenderlove @rsim

With love,
Eduardo

Explain plans (txt)

SELECT * FROM (
   SELECT raw_sql_.*, rownum raw_rnum_
   FROM (SELECT "LANCAMENTOS".* FROM "LANCAMENTOS" ) raw_sql_
 )
 WHERE raw_rnum_ between 1 and 30


----------------------------------------------------------------------------------
| Id  | Operation           | Name        | Rows  | Bytes | Cost (%CPU)| Time     |
-----------------------------------------------------------------------------------
|   0 | SELECT STATEMENT    |             |  4636K|  2701M| 23442   (2)| 00:04:42 |
|*  1 |  VIEW               |             |  4636K|  2701M| 23442   (2)| 00:04:42 |
|   2 |   COUNT             |             |       |       |            |          |
|   3 |    TABLE ACCESS FULL| LANCAMENTOS |  4636K|   738M| 23442   (2)| 00:04:42 |
-----------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter("RAW_RNUM_"<=30 AND "RAW_RNUM_">=1)

   Statistics
-----------------------------------------------------------
               4  user calls
              13  physical read total multi block requests
       202588160  physical read total bytes
       202588160  cell physical IO interconnect bytes
               0  commit cleanout failures: block lost
               0  IMU commits
               0  IMU Flushes
               0  IMU contention
               0  IMU bind flushes
               0  IMU mbu flush






SELECT * FROM (
   SELECT raw_sql_.*, rownum raw_rnum_
   FROM (SELECT "LANCAMENTOS".* FROM "LANCAMENTOS" ) raw_sql_
   WHERE rownum <= 30
 )
 WHERE raw_rnum_ >= 0


-----------------------------------------------------------------------------------
| Id  | Operation           | Name        | Rows  | Bytes | Cost (%CPU)| Time     |
-----------------------------------------------------------------------------------
|   0 | SELECT STATEMENT    |             |    30 | 18330 |     2   (0)| 00:00:01 |
|*  1 |  VIEW               |             |    30 | 18330 |     2   (0)| 00:00:01 |
|*  2 |   COUNT STOPKEY     |             |       |       |            |          |
|   3 |    TABLE ACCESS FULL| LANCAMENTOS |    30 |  5010 |     2   (0)| 00:00:01 |
-----------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter("RAW_RNUM_">=0)
   2 - filter(ROWNUM<=30)

   Statistics
-----------------------------------------------------------
               4  user calls
               0  physical read total multi block requests
               0  physical read total bytes
               0  cell physical IO interconnect bytes
               0  commit cleanout failures: block lost
               0  IMU commits
               0  IMU Flushes
               0  IMU contention
               0  IMU bind flushes
               0  IMU mbu flush
@eduardordm

I'll fix the test (it expects the old sql statement)

@rafaelfranca

Interesting. Which arel version are you using? I think we will need to backport to the proper version.

@rafaelfranca

Also can I ask to add the content of the gist in the commit message? Just in case where you need to delete the gist or github choose to change the gist url again

@eduardordm

Ok :smile: (I fixed on master but the 3-0-stable branch is broken too)

@rafaelfranca

The commit message is still pointing to the gist. You can change it using rebase -i

eduardordm added some commits
@eduardordm eduardordm Revert issue #99 fixes
There are two commits involving issue #99 which should be reverted.

First of all, #99 is not an issue in Arel at all. Second, the fix provides pretty much destroys the purpose of pagination by cause full table scans.

The original code (it seems I can't simply revert the commits) is 900 times slower than this one.

```
SELECT * FROM (
   SELECT raw_sql_.*, rownum raw_rnum_
   FROM (SELECT "LANCAMENTOS".* FROM "LANCAMENTOS" ) raw_sql_
 )
 WHERE raw_rnum_ between 1 and 30

----------------------------------------------------------------------------------
| Id  | Operation           | Name        | Rows  | Bytes | Cost (%CPU)| Time     |
-----------------------------------------------------------------------------------
|   0 | SELECT STATEMENT    |             |  4636K|  2701M| 23442   (2)| 00:04:42 |
|*  1 |  VIEW               |             |  4636K|  2701M| 23442   (2)| 00:04:42 |
|   2 |   COUNT             |             |       |       |            |          |
|   3 |    TABLE ACCESS FULL| LANCAMENTOS |  4636K|   738M| 23442   (2)| 00:04:42 |
-----------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter("RAW_RNUM_"<=30 AND "RAW_RNUM_">=1)

   Statistics
-----------------------------------------------------------
               4  user calls
              13  physical read total multi block requests
       202588160  physical read total bytes
       202588160  cell physical IO interconnect bytes
               0  commit cleanout failures: block lost
               0  IMU commits
               0  IMU Flushes
               0  IMU contention
               0  IMU bind flushes
               0  IMU mbu flush

SELECT * FROM (
   SELECT raw_sql_.*, rownum raw_rnum_
   FROM (SELECT "LANCAMENTOS".* FROM "LANCAMENTOS" ) raw_sql_
   WHERE rownum <= 30
 )
 WHERE raw_rnum_ >= 0

-----------------------------------------------------------------------------------
| Id  | Operation           | Name        | Rows  | Bytes | Cost (%CPU)| Time     |
-----------------------------------------------------------------------------------
|   0 | SELECT STATEMENT    |             |    30 | 18330 |     2   (0)| 00:00:01 |
|*  1 |  VIEW               |             |    30 | 18330 |     2   (0)| 00:00:01 |
|*  2 |   COUNT STOPKEY     |             |       |       |            |          |
|   3 |    TABLE ACCESS FULL| LANCAMENTOS |    30 |  5010 |     2   (0)| 00:00:01 |
-----------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter("RAW_RNUM_">=0)
   2 - filter(ROWNUM<=30)

   Statistics
-----------------------------------------------------------
               4  user calls
               0  physical read total multi block requests
               0  physical read total bytes
               0  cell physical IO interconnect bytes
               0  commit cleanout failures: block lost
               0  IMU commits
               0  IMU Flushes
               0  IMU contention
               0  IMU bind flushes
               0  IMU mbu flush
```
5399048
@eduardordm eduardordm Update test_oracle.rb 47d5302
@eduardordm

Sorry, should be fixed now. I used the web editor and messed my local repo. :horse:

@rafaelfranca rafaelfranca merged commit 933ea21 into from
@rafaelfranca

Thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Mar 15, 2013
  1. @eduardordm

    Revert issue #99 fixes

    eduardordm authored
    There are two commits involving issue #99 which should be reverted.
    
    First of all, #99 is not an issue in Arel at all. Second, the fix provides pretty much destroys the purpose of pagination by cause full table scans.
    
    The original code (it seems I can't simply revert the commits) is 900 times slower than this one.
    
    ```
    SELECT * FROM (
       SELECT raw_sql_.*, rownum raw_rnum_
       FROM (SELECT "LANCAMENTOS".* FROM "LANCAMENTOS" ) raw_sql_
     )
     WHERE raw_rnum_ between 1 and 30
    
    ----------------------------------------------------------------------------------
    | Id  | Operation           | Name        | Rows  | Bytes | Cost (%CPU)| Time     |
    -----------------------------------------------------------------------------------
    |   0 | SELECT STATEMENT    |             |  4636K|  2701M| 23442   (2)| 00:04:42 |
    |*  1 |  VIEW               |             |  4636K|  2701M| 23442   (2)| 00:04:42 |
    |   2 |   COUNT             |             |       |       |            |          |
    |   3 |    TABLE ACCESS FULL| LANCAMENTOS |  4636K|   738M| 23442   (2)| 00:04:42 |
    -----------------------------------------------------------------------------------
    
    Predicate Information (identified by operation id):
    ---------------------------------------------------
    
       1 - filter("RAW_RNUM_"<=30 AND "RAW_RNUM_">=1)
    
       Statistics
    -----------------------------------------------------------
                   4  user calls
                  13  physical read total multi block requests
           202588160  physical read total bytes
           202588160  cell physical IO interconnect bytes
                   0  commit cleanout failures: block lost
                   0  IMU commits
                   0  IMU Flushes
                   0  IMU contention
                   0  IMU bind flushes
                   0  IMU mbu flush
    
    SELECT * FROM (
       SELECT raw_sql_.*, rownum raw_rnum_
       FROM (SELECT "LANCAMENTOS".* FROM "LANCAMENTOS" ) raw_sql_
       WHERE rownum <= 30
     )
     WHERE raw_rnum_ >= 0
    
    -----------------------------------------------------------------------------------
    | Id  | Operation           | Name        | Rows  | Bytes | Cost (%CPU)| Time     |
    -----------------------------------------------------------------------------------
    |   0 | SELECT STATEMENT    |             |    30 | 18330 |     2   (0)| 00:00:01 |
    |*  1 |  VIEW               |             |    30 | 18330 |     2   (0)| 00:00:01 |
    |*  2 |   COUNT STOPKEY     |             |       |       |            |          |
    |   3 |    TABLE ACCESS FULL| LANCAMENTOS |    30 |  5010 |     2   (0)| 00:00:01 |
    -----------------------------------------------------------------------------------
    
    Predicate Information (identified by operation id):
    ---------------------------------------------------
    
       1 - filter("RAW_RNUM_">=0)
       2 - filter(ROWNUM<=30)
    
       Statistics
    -----------------------------------------------------------
                   4  user calls
                   0  physical read total multi block requests
                   0  physical read total bytes
                   0  cell physical IO interconnect bytes
                   0  commit cleanout failures: block lost
                   0  IMU commits
                   0  IMU Flushes
                   0  IMU contention
                   0  IMU bind flushes
                   0  IMU mbu flush
    ```
  2. @eduardordm

    Update test_oracle.rb

    eduardordm authored
This page is out of date. Refresh to see the latest.
Showing with 4 additions and 2 deletions.
  1. +2 −1  lib/arel/visitors/oracle.rb
  2. +2 −1  test/visitors/test_oracle.rb
View
3  lib/arel/visitors/oracle.rb
@@ -25,8 +25,9 @@ def visit_Arel_Nodes_SelectStatement o
SELECT * FROM (
SELECT raw_sql_.*, rownum raw_rnum_
FROM (#{sql}) raw_sql_
+ WHERE rownum <= #{offset.expr.to_i + limit}
)
- WHERE raw_rnum_ >= #{offset.expr.to_i + 1 } and rownum <= #{limit}
+ WHERE #{visit offset}
eosql
end
View
3  test/visitors/test_oracle.rb
@@ -102,8 +102,9 @@ module Visitors
SELECT * FROM (
SELECT raw_sql_.*, rownum raw_rnum_
FROM (SELECT) raw_sql_
+ WHERE rownum <= 20
)
- WHERE raw_rnum_ >= 11 and rownum <= 10
+ WHERE raw_rnum_ > 10
}
end
Something went wrong with that request. Please try again.