Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Schema loading performance regression after update from rails 5.1 to rails 5.2 #1720

Closed
sudoremo opened this issue May 3, 2018 · 39 comments
Closed

Comments

@sudoremo
Copy link

sudoremo commented May 3, 2018

Steps to reproduce

Background is that we're experiencing massive performance issues since updating to rails 5.2. Looking at the logs, it seems that structural database statements make the difference. To isolate this, we performed db:cache:dump multiple times, both with rails 5.1 and rails 5.2.

The difference is startling (see next 2 sections).

This may be related to #1673.

Expected behavior

Executing db:cache:dump of a large application under Rails 5.1:

unknown             :   3936 ms
Sequence            :    320 ms
Primary Key         :   9551 ms
-------------------------------
total               :  13807 ms

Corresponding statement log

The results are grouped by query type (the string before the parenthesized time in the rails log) and are the average over multiple runs.

Actual behavior

Executing db:cache:dump of the same application under Rails 5.2:

tables              :    809 ms
views               :    504 ms
synonyms            :   1987 ms
Sequence            :   6270 ms
Primary Key         : 134574 ms
Column definitions  :  30425 ms
current schema      :      6 ms
table exists        :    181 ms
unknown             :      9 ms
-------------------------------
total               : 174765 ms

Corresponding statement log

This is over 12x slower than with Rails 5.1.

System configuration

Rails version: 5.1 / 5.2

Oracle enhanced adapter version: 1.8.2 / 5.2.2

Ruby version: jruby 9.1.14.0 (2.3.3) 2017-11-08 2176f24 Java HotSpot(TM) 64-Bit Server VM 25.66-b17 on 1.8.0_66-b17 +jit [darwin-x86_64]

Oracle Database version: Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production

Final thoughts

As always a ton of thanks to you for looking at this and I'm sorry I couln't provide an isolated performance test – I hope that comparing the database statements of the two log files will do the trick!

@sudoremo
Copy link
Author

sudoremo commented May 3, 2018

I've now been able to isolate this issue with a script. It shows even more drastic numbers than the example above.

To use this script

  1. Adapt the DB_CONFIG constant or provide the necessary environment variables

  2. Run the script twice, once for 5.1 and once for 5.2:

    # Using rails 5.1.6 and enhanced adapter 1.8.2
    jruby oracle_enhanced_1720.rb
    
    # Using rails 5.2 and enhanced adapter 5.2.2
    NEW=true jruby oracle_enhanced_1720.rb

    This can also be run with MRI (you may need to add oci8 to the list of gems), but for some reason I was unable to get it to connect to our database so I made it for jRuby.

Results: Rails 5.1 / Adapter 1.8.2

BENCHMARK:
  8.270000   0.210000   8.480000 ( 13.275247)
.

Finished in 13.293265s, 0.0752 runs/s, 0.0000 assertions/s.

1 runs, 0 assertions, 0 failures, 0 errors, 0 skips
jruby oracle_enhanced_1720.rb  54.68s user 1.90s system 134% cpu 42.108 total

Total execution time: 42.108s Log

Results: Rails 5.2 / Adapter 5.2.2

BENCHMARK:
  8.820000   0.280000   9.100000 (136.128926)
.

Finished in 136.141032s, 0.0073 runs/s, 0.0000 assertions/s.

1 runs, 0 assertions, 0 failures, 0 errors, 0 skips
NEW=true jruby oracle_enhanced_1720.rb  55.83s user 1.93s system 28% cpu 3:21.29 total

Total execution time: 201.29s Log

Conclusion

Not only loading the model classes (that leads to structural queries) but also the migration takes way longer with Rails 5.2 than with Rails 5.1. This leads to extremely poor performance in our Rails 5.2 applications.

The total execution time is 4.7 times slower in 5.2 than in 5.1.

Thank you a lot for having a look at this 👍

@sudoremo
Copy link
Author

sudoremo commented May 3, 2018

To rule out an issue with Rails that is independent of the oracle enhanced adapter, I've also created a script for MySQL.

Results: Rails 5.1

BENCHMARK:
  3.290000   0.070000   3.360000 (  0.704586)
.

Finished in 0.725115s, 1.3791 runs/s, 0.0000 assertions/s.

1 runs, 0 assertions, 0 failures, 0 errors, 0 skips
jruby oracle_enhanced_1720_mysql.rb  48.22s user 1.72s system 352% cpu 14.170 total

Total execution time: 14.170s

Results: Rails 5.2

BENCHMARK:
  3.010000   0.090000   3.100000 (  0.680692)
.

Finished in 0.697151s, 1.4344 runs/s, 0.0000 assertions/s.

1 runs, 0 assertions, 0 failures, 0 errors, 0 skips
NEW=true jruby oracle_enhanced_1720_mysql.rb  59.73s user 2.17s system 363% cpu 17.014 total

Total execution time: 17.014s

Conclusion

  • There is much less difference between 5.1 and 5.2.
  • Both execution times are signifficantly shorter than with the oracle enhanced
    adapter (of course some of this difference may be because of the database and
    not due to the adapter). Oracle is 11x slower than MySQL.

@yahonda
Copy link
Collaborator

yahonda commented May 5, 2018

Thanks for providing test cases.

Would you upload the entire log files of Results: Rails 5.1 / Adapter 1.8.2 and Results: Rails 5.2 / Adapter 5.2.2

I'd like to know which statements taking time.

@sudoremo
Copy link
Author

sudoremo commented May 7, 2018

Thanks for your answer @yahonda. Sure, I've added links to the respective log files above, right after the total execution time.

The queries look very similar to me, i.e.:

5.1

Primary Key (3.0ms) SELECT cc.column_name FROM all_constraints c, all_cons_columns cc WHERE c.owner = 'RFRITZSCHE4' AND c.table_name = 'TABLE_19' AND c.constraint_type = 'P' AND cc.owner = c.owner AND cc.constraint_name = c.constraint_name

5.2

Primary Key (1948.0ms) SELECT cc.column_name FROM all_constraints c, all_cons_columns cc WHERE c.owner = 'RFRITZSCHE4' AND c.table_name = 'TABLE_19' AND c.constraint_type = 'P' AND cc.owner = c.owner AND cc.constraint_name = c.constraint_name

But for some reason, 5.2 is 650x slower for the same query. Is there maybe a general problem that affects performance in 5.2?

@yahonda
Copy link
Collaborator

yahonda commented May 10, 2018

It is likely due to execution plan change for this query

pks = select_values(<<-SQL.strip.gsub(/\s+/, " "), "Primary Key")
SELECT cc.column_name
FROM all_constraints#{db_link} c, all_cons_columns#{db_link} cc
WHERE c.owner = '#{owner}'
AND c.table_name = #{quote(desc_table_name)}
AND c.constraint_type = 'P'
AND cc.owner = c.owner
AND cc.constraint_name = c.constraint_name
SQL

Would you change each change to see if you see performance improvement.

  • Gather dictionary stats, which would require DBA privileges
SQL> exec dbms_stats.gather_dictionary_stats()
  • Update database.yml to use cursor_sharing: :force
  cursor_sharing: :force

Background:

Oracle enhanced adapter 5.2.0 #1498 introduces bind values
for dictionary queries then default value of cursor_sharing changed from force to using the database instance value, likely exact.

However, to address #1678 Oracle enhanced adapter 5.2.1 reverted some of changes made #1498 to use sql literals. The query for primary key is one of them.

Thank you.

@sudoremo
Copy link
Author

sudoremo commented May 14, 2018

Thanks for your answer. Here the results:

  • 5.2: New run without any changes: 152s
  • 5.2: Run with cursor_sharing: :force: 90s
  • 5.1: Run with cursor_sharing: :force: 13s
  • 5.2: Run with cursor_sharing: :force and dictionary stats: 10s

Edit: Now I've run it again and 5.2 with both "settings" now takes 85s again.

@yahonda
Copy link
Collaborator

yahonda commented May 15, 2018

Thanks for testing. I'd like to request you to upload the log files for these tests.

* 5.2: Run with cursor_sharing: :force and dictionary stats: 10s
* Edit: Now I've run it again and 5.2 with both "settings" now takes 85s again.

Thank you.

@sudoremo
Copy link
Author

Sure, I'll upload it as soon as possible. Wouldn't it be easier for you running the script yourself to avoid constant back and forth communication? I'm more than happy to run it for you though.

@yahonda
Copy link
Collaborator

yahonda commented May 15, 2018

Thanks. Actually, I did execute your script file before but unable to reproduce the slowness.

@sudoremo
Copy link
Author

Ah ok. Sure, I'll be getting back to you asap.

@sudoremo
Copy link
Author

Sorry for the delay. I've now performed the following sequence of actions:

  1. Database user dropped and newly created

  2. First script execution with forced cursor sharing
    Took 50s, 106s in total, Log

  3. Gathering dictionary statistics

  4. Second script execution with forced cursor sharing
    Took 16s, 47s in total, Log

  5. Third script execution without forced cursor sharing:
    Took 147s, 220s in total, Log

  6. Fourth script execution with forced cursor sharing again:
    Took 90s, 150s in total, Log

So the peculiar thing is that the run after turning off cursor sharing, but with cursor sharing, is very slow again.

Does this help? Thanks for looking into it.

@jlahtinen
Copy link
Contributor

We are also experiencing performance problems now when upgrading to 5.2. We found that queries to Oracle data dictionary took at least 10 times more when using default optimizer mode compared to RULE based optimizer.

Maybe you should find out if those dictionary queries are the problem here also.

I changed all select queries accessing data dictionary with /*+ RULE */ hint in oracle-enhanced project and now our application is quite performant again.

Before doing this I just added exec "alter session set optimizer_mode=RULE" in jdbc_connection.rb after cursor sharing and all looked promising. I removed alter session before adding rule hints.

I do not know if this approach is good or not. More information about optimizer hints https://docs.oracle.com/cd/B10500_01/server.920/a96533/hintsref.htm#4991 At least that deprecation warning worries me.

@yahonda
Copy link
Collaborator

yahonda commented Jun 20, 2018

Thanks for the update and useful information. Let me create a branch to add rule hints for some of the queries for Oracle database dictionary to see if this issue resolves. As far as I remember rule hint has not been "supported" since 10g but there is a similar issue reported dbeaver/dbeaver#360 then I hope it would help ours.

@jlahtinen
Copy link
Contributor

I am testing this with Oracle Database 12c Standard Edition Release 12.1.0.2.0 - 64bit Production

@jlahtinen
Copy link
Contributor

I was able to increase db:migration time a lot with https://github.com/jlahtinen/oracle-enhanced/tree/optimize_schema_statement

I used rule optimizer also, but left outer join increased performance from 25s to 1.5s in that modified query. It also seems like this query is used multiple times in 5.2 branch and not in 5.1 branch.

@yahonda maybe you are able to check if this is done correctly and merge it to your branches

@sitroxrmeier
Copy link

Are there any news on this subject? We'd like to upgrade to Rails 5.2 but this issue is standing in our way. Thank you for your work!

@InteNs
Copy link

InteNs commented Jul 16, 2018

right now using @jlahtinen's fork is the only way for us to even be able to use db:schema:dump, it hangs with an empty db/schema.rb otherwise

@scrozier
Copy link

We are experiencing this performance issue as well. Any news? Anything you'd like us to test/try?

@joshpencheon
Copy link
Contributor

This causing us headaches too, so another volunteer here if there is anything that can be helped with.

@jlahtinen
Copy link
Contributor

@joshpencheon you could verify if 51bc17b is a working fix

@joshpencheon
Copy link
Contributor

@jlahtinen I've been testing that, thank you - it does appear to help significantly in our case - a structure:dump takes 6 minutes rather than 60 minutes with your fix.

I have also experimented with including an additional :table_name predicate, as that method currently builds index data for all tables (a hangover from when this gem maintained its own internal schema cache, as far as I can tell) - doing so provided modest additional gains.

@jlahtinen
Copy link
Contributor

How about you @scrozier? Do you think 51bc17b is fixed correctly?

@scrozier
Copy link

@jlahtinen My colleague who is working on this is out this morning. I'll get with him this afternoon and we'll see if we can do our testing on 51bc17b. Thanks!

@gyut
Copy link

gyut commented Jul 20, 2018

@jlahtinen, I work with @scrozier. As I read through this issue, it appears to focus on Rails 5.2 with the 5.2.2 adapter, right? We're seeing performance issues with Rails 5.1.5 and the 1.8.2 adapter.

Our problem appears to involve CLOB columns in Oracle (text attributes in Rails). Across several applications, we're seeing query times of 400ms for queries that should take 20ms, but only if a CLOB attribute is part of the query. If we remove the CLOB from the query, the queries are back to 20ms.

Any suggestions? Is this a known issue?

Thanks for your help!

@jlahtinen
Copy link
Contributor

@gyut maybe report that as a separate issue. This issue might be more about db:cache:dump not sure thought

@yahonda
Copy link
Collaborator

yahonda commented Jul 23, 2018

Would someone update your Gemfile as follows and let us know if your performance problem solved.

gem "activerecord-oracle_enhanced-adapter", github: 'yahonda/oracle-enhanced', branch: "diag1720_release52"

This branch has some changes:

  • Restoring column cache feature per connection
  • Use cursor_sharing = force by default
  • Add rule hint for the query at column_definitions

Thank you.

@InteNs
Copy link

InteNs commented Jul 24, 2018

@yahonda running rails db:schema:dump on this branch results in utter slowness (compared to https://github.com/jlahtinen/oracle-enhanced/tree/optimize_schema_statement)

the main issue is retrieving indexes with the query as defined below.

  • This query gets executed multiple times during the schema dump
  • All these executions take more than 120.000 ms
  • Running this query through a database client also takes around 2 minutes.
  • the query takes around 1000 ms on @jlahtinen's fork

edit:

rails db:schema:dump RAILS_ENV=production

  • query takes more than 5000 ms

  • our production database is oracle 12c

  • our test database is oracle 11g express

    • this is where we normally run our schema:dump for migrations
SELECT LOWER(i.table_name)      AS table_name,
       LOWER(i.index_name)      AS index_name,
       i.uniqueness,
       i.index_type,
       i.ityp_owner,
       i.ityp_name,
       i.parameters,
       LOWER(i.tablespace_name) AS tablespace_name,
       LOWER(c.column_name)     AS column_name,
       e.column_expression,
       atc.virtual_column
FROM all_indexes i
       JOIN all_ind_columns c ON c.index_name = i.index_name AND c.index_owner = i.owner
       LEFT OUTER JOIN all_ind_expressions e
         ON e.index_name = i.index_name AND e.index_owner = i.owner AND e.column_position = c.column_position
       LEFT OUTER JOIN all_tab_cols atc
         ON i.table_name = atc.table_name AND c.column_name = atc.column_name AND i.owner = atc.owner AND
            atc.hidden_column = 'NO'
WHERE i.owner = :owner
  AND i.table_owner = :owner
  AND NOT EXISTS(SELECT uc.index_name
                 FROM all_constraints uc
                 WHERE uc.index_name = i.index_name
                   AND uc.owner = i.owner
                   AND uc.constraint_type = 'P')
ORDER BY i.index_name, c.column_position

@sudoremo
Copy link
Author

Thanks a lot for this update @yahonda. The schema dumping performance in our application is way better with this branch, yet still about 30% slower than on Rails 5.1.

On the other hand, the application is still quite unusably slow because the schema loading statements take way longer in the 5.2 adapter compared to 5.1.

To illustrate this, we've performed one action in our application on both 5.1 and 5.2:

In these logs you can see that certain statements take about 35x longer than compared to 5.1 (i.e. column definition load takes ~580ms instead of ~15ms). Note that these logs have been stripped to only show structural queries.

Is there a particular reason these changes must be there for the 5.2 adapter, or is there a way we could continue using the 5.1 adapter for Rails 5.2? Unfortunately, the performance regressions currently prohibit us from upgrading to 5.2 (which we desperately need because of concurrency issues).

Thanks a lot & let me know if I can be of any more help.

@joshpencheon
Copy link
Contributor

@yahonda thank you for taking a look at this. Using your branch, my findings were sadly similar to those of @InteNs - the indexes query takes ~6000ms each time, and runs for every table being dumped.

However, I found the further addition of:

- result = select_all(<<-SQL.strip.gsub(/\s+/, " "), "indexes", [bind_string("owner", owner), bind_string("owner", owner)])
+ result = select_all(<<-SQL.strip.gsub(/\s+/, " "), "indexes", [bind_string("owner", owner), bind_string("owner", owner), bind_string("table_name", table_name)])
    SELECT LOWER(i.table_name) AS table_name, LOWER(i.index_name) AS index_name, i.uniqueness,
      i.index_type, i.ityp_owner, i.ityp_name, i.parameters,
      LOWER(i.tablespace_name) AS tablespace_name,
      LOWER(c.column_name) AS column_name, e.column_expression,
      atc.virtual_column
    FROM all_indexes#{db_link} i
      JOIN all_ind_columns#{db_link} c ON c.index_name = i.index_name AND c.index_owner = i.owner
      LEFT OUTER JOIN all_ind_expressions#{db_link} e ON e.index_name = i.index_name AND
        e.index_owner = i.owner AND e.column_position = c.column_position
      LEFT OUTER JOIN all_tab_cols#{db_link} atc ON i.table_name = atc.table_name AND
        c.column_name = atc.column_name AND i.owner = atc.owner AND atc.hidden_column = 'NO'
    WHERE i.owner = :owner
       AND i.table_owner = :owner
+      AND i.table_name = :table_name
       AND NOT EXISTS (SELECT uc.index_name FROM all_constraints uc
        WHERE uc.index_name = i.index_name AND uc.owner = i.owner AND uc.constraint_type = 'P')
    ORDER BY i.index_name, c.column_position
  SQL

to the query reduced the execution time from ~6000ms to ~15ms - perhaps worth rolling in too? As far as I can see, because this query is no longer cached it needn't get results for all indexes every time.

@yahonda
Copy link
Collaborator

yahonda commented Jul 31, 2018

Thanks for the testing everyone.

I have made several changes: Actually my environment does not get performance improvement by adding a search condition.

gem "activerecord-oracle_enhanced-adapter", github: 'yahonda/oracle-enhanced', branch: "diag1720_release52"

Please give it a try and let me know the result.

Thank you.

@sudoremo
Copy link
Author

sudoremo commented Jul 31, 2018

Thanks for the update. The performance is now similar to 5.1 in our tests.

What is very weird though: I've prepared a script that uses multiple models and therefore causes the adapter to load the database structure of many tables (the same one that generated the log outputs of my previous post). Then I've tried the following:

  1. Ran the script using your new branch (17303ms)
  2. Ran the script using 5.1 (17202ms)
  3. Ran the script using 5.2 stable (108'422ms)

So far so good.

  1. Ran the script using 5.1 again (54'000ms)
  2. Ran the script using your new branch again (78'708ms)

So running the old version again seems to slow down subsequent queries by the new version significantly. Do you have an idea what could cause this effect? Does it have something to do with dictionary statistics?

@sue445
Copy link

sue445 commented Jul 31, 2018

@yahonda Thank you for turning.

I'm using multiple activerecord versions at my gem CI

AR 5.2.0 + oracle-enhanced v5.2.2 build is very very slow, but It seems this issue was fixed at #1720 (comment) (yahonda/oracle-enhanced@2dc2ef9)

Benchmark

c.f. sue445/index_shotgun#48

Thank you.

@yahonda
Copy link
Collaborator

yahonda commented Aug 1, 2018

One possible reason is changing the execution plan may cause a longer elapsed time.

If you are using Oracle Database 12c Release 1, how about disabling optimizer_adaptive_features explained in these articles:

Since these parameters are changing in every database versions, I am not thinking about setting these parameters in Oracle enhanced adapter itself.

yahonda added a commit to yahonda/oracle-enhanced that referenced this issue Aug 1, 2018
rsim#1490 removed Oracle enhanced adapter own column cache feature
to use Rails `db:schema:cache:dump`.  However, it caused slower performance
about metadata search, such as column information.

This commit restores Oracle enhanced adapter own column cache feature.
It used to be class level caching, it is connection level caching now
since it is much easier and cleaner than class level ones.

Related to rsim#1720
@InteNs
Copy link

InteNs commented Aug 2, 2018

yahonda/oracle-enhanced@2dc2ef9 seems to fix most performance problems we had.

thanks for looking into this everyone!

yahonda added a commit to yahonda/oracle-enhanced that referenced this issue Aug 2, 2018
it actually reverts rsim#1503

rsim#1498 implemented prepared statements for dictionary
queries. However rsim#1713 restored non-prepared statements
for `column_definitions` and `pk_and_sequence_for`, which likely have caused
longer elapsed time for these queries reported at rsim#1720

Those who needs 'cursor_sharing = exact' can specify its own value to .database.yml
yahonda added a commit to yahonda/oracle-enhanced that referenced this issue Aug 6, 2018
and restored one spec which has been removed at rsim#1672

Related to rsim#1720
yahonda added a commit to yahonda/oracle-enhanced that referenced this issue Aug 7, 2018
it actually reverts rsim#1503

rsim#1498 implemented prepared statements for dictionary
queries. However rsim#1713 restored non-prepared statements
for `column_definitions` and `pk_and_sequence_for`, which likely have caused
longer elapsed time for these queries reported at rsim#1720

Those who needs 'cursor_sharing = exact' can specify its own value to .database.yml
yahonda added a commit to yahonda/oracle-enhanced that referenced this issue Aug 7, 2018
rsim#1490 removed Oracle enhanced adapter own column cache feature
to use Rails `db:schema:cache:dump`.  However, it caused slower performance
about metadata search, such as column information.

This commit restores Oracle enhanced adapter own column cache feature.
It used to be class level caching, it is connection level caching now
since it is much easier and cleaner than class level ones.

Related to rsim#1720
yahonda added a commit to yahonda/oracle-enhanced that referenced this issue Aug 7, 2018
rsim#1490 removed Oracle enhanced adapter own column cache feature
to use Rails `db:schema:cache:dump`.  However, it caused slower performance
about metadata search, such as column information.

This commit restores Oracle enhanced adapter own column cache feature.
It used to be class level caching, it is connection level caching now
since it is much easier and cleaner than class level ones.

Related to rsim#1720
yahonda added a commit to yahonda/oracle-enhanced that referenced this issue Aug 7, 2018
rsim#1490 removed Oracle enhanced adapter own column cache feature
to use Rails `db:schema:cache:dump`.  However, it caused slower performance
about metadata search, such as column information.

This commit restores Oracle enhanced adapter own column cache feature.
It used to be class level caching, it is connection level caching now
since it is much easier and cleaner than class level ones.

Related to rsim#1720
@yahonda
Copy link
Collaborator

yahonda commented Aug 7, 2018

Oracle enhanced adapter 5.2.3 has been released which should address this issue.
Thanks for everyone who investigated and tested some test branches.

@sudoremo
Copy link
Author

sudoremo commented Aug 8, 2018

Huge thanks, it works like a charm 👍

@sudoremo sudoremo closed this as completed Aug 8, 2018
@alexishida
Copy link

The problem persist when I use db:schema:dump

ruby '2.6.3'
gem 'rails', '~> 5.2.3'
activerecord-oracle_enhanced-adapter (5.2.6)

indexes (9006.2ms) SELECT LOWER(i.table_name) AS table_name, LOWER(i.index_name) AS index_name, i.uniqueness, i.index_type, i.ityp_owner, i.ityp_name, i.parameters, LOWER(i.tablespace_name) AS tablespace_name, LOWER(c.column_name) AS column_name, e.column_expression, atc.virtual_column FROM all_indexes i JOIN all_ind_columns c ON c.index_name = i.index_name AND c.index_owner = i.owner LEFT OUTER JOIN all_ind_expressions e ON e.index_name = i.index_name AND e.index_owner = i.owner AND e.column_position = c.column_position LEFT OUTER JOIN all_tab_cols atc ON i.table_name = atc.table_name AND c.column_name = atc.column_name AND i.owner = atc.owner AND atc.hidden_column = 'NO' WHERE i.owner = :owner AND i.table_owner = :owner AND i.table_name = :table_name AND NOT EXISTS (SELECT uc.index_name FROM all_constraints uc WHERE uc.index_name = i.index_name AND uc.owner = i.owner AND uc.constraint_type = 'P') ORDER BY i.index_name, c.column_position

@TehCorwiz
Copy link

TehCorwiz commented Dec 3, 2019

Hi @remofritzsche,
I'm experiencing this issue still as well.

ruby '2.6.3'
gem 'rails', '5.2.3'
gem 'activerecord-oracle_enhanced-adapter', '5.2.6'

rails db:schema:dump seems to take an extremely long time.

We have a large (40+ tables) application in which this is a significant issue, taking some minutes. I'm including debugging from a smaller project that's also slow to perform schema operations.

In the example below there are only 12 tables, none of which has more than 10 columns a piece, and none of which have more than 4 indexes. The few tables that do have that many are 'X-to-many' tables, one index for primary key, one index per foreign key (x2), and one unique index for the pair. It's these tables that take the longest. Total run time of the below log was just shy of 45 seconds.

The indexes query seems to be eating up the bulk of the time, taking anywhere from 910ms to 3674ms. Further, it appears that each of these queries runs twice for each table.

(specific schema/table names stripped for privacy.)

D, [2019-12-03T11:50:27.971178 #23130] DEBUG -- :   current schema (2.8ms)  SELECT SYS_CONTEXT('userenv', 'current_schema') FROM dual
D, [2019-12-03T11:50:27.974508 #23130] DEBUG -- :   table exists (2.9ms)  SELECT owner, table_name FROM all_tables WHERE owner = :owner AND table_name = :table_name  
D, [2019-12-03T11:50:27.992514 #23130] DEBUG -- :   Column definitions (6.0ms)  SELECT cols.column_name AS name, cols.data_type AS sql_type, cols.data_default, cols.nullable, cols.virtual_column, cols.hidden_column, cols.data_type_owner AS sql_type_owner, DECODE(cols.data_type, 'NUMBER', data_precision, 'FLOAT', data_precision, 'VARCHAR2', DECODE(char_used, 'C', char_length, data_length), 'RAW', DECODE(char_used, 'C', char_length, data_length), 'CHAR', DECODE(char_used, 'C', char_length, data_length), NULL) AS limit, DECODE(data_type, 'NUMBER', data_scale, NULL) AS scale, comments.comments as column_comment FROM all_tab_cols cols, all_col_comments comments WHERE cols.owner = 'DEV_SCHEMA' AND cols.table_name = 'SCHEMA_MIGRATIONS' AND cols.hidden_column = 'NO' AND cols.owner = comments.owner AND cols.table_name = comments.table_name AND cols.column_name = comments.column_name ORDER BY cols.column_id
D, [2019-12-03T11:50:27.995233 #23130] DEBUG -- :    (1.7ms)  SELECT "SCHEMA_MIGRATIONS"."VERSION" FROM "SCHEMA_MIGRATIONS" ORDER BY "SCHEMA_MIGRATIONS"."VERSION" ASC
D, [2019-12-03T11:50:28.009523 #23130] DEBUG -- :   tables (14.1ms)  SELECT DECODE(table_name, UPPER(table_name), LOWER(table_name), table_name) FROM all_tables WHERE owner = SYS_CONTEXT('userenv', 'current_schema') AND secondary = 'N'
D, [2019-12-03T11:50:28.011326 #23130] DEBUG -- :   materialized views (1.6ms)  SELECT LOWER(mview_name) FROM all_mviews WHERE owner = SYS_CONTEXT('userenv', 'current_schema')
D, [2019-12-03T11:50:28.021528 #23130] DEBUG -- :   Column definitions (8.5ms)  SELECT cols.column_name AS name, cols.data_type AS sql_type, cols.data_default, cols.nullable, cols.virtual_column, cols.hidden_column, cols.data_type_owner AS sql_type_owner, DECODE(cols.data_type, 'NUMBER', data_precision, 'FLOAT', data_precision, 'VARCHAR2', DECODE(char_used, 'C', char_length, data_length), 'RAW', DECODE(char_used, 'C', char_length, data_length), 'CHAR', DECODE(char_used, 'C', char_length, data_length), NULL) AS limit, DECODE(data_type, 'NUMBER', data_scale, NULL) AS scale, comments.comments as column_comment FROM all_tab_cols cols, all_col_comments comments WHERE cols.owner = 'DEV_SCHEMA' AND cols.table_name = 'EXPERIMENT_METHODS' AND cols.hidden_column = 'NO' AND cols.owner = comments.owner AND cols.table_name = comments.table_name AND cols.column_name = comments.column_name ORDER BY cols.column_id
D, [2019-12-03T11:50:28.025918 #23130] DEBUG -- :   Primary Keys (3.1ms)  SELECT cc.column_name FROM all_constraints c, all_cons_columns cc WHERE c.owner = :owner AND c.table_name = :table_name AND c.constraint_type = 'P' AND cc.owner = c.owner AND cc.constraint_name = c.constraint_name order by cc.position  
D, [2019-12-03T11:50:28.027666 #23130] DEBUG -- :   temporary table (1.5ms)  SELECT temporary FROM all_tables WHERE table_name = :table_name and owner = SYS_CONTEXT('userenv', 'session_user')  
D, [2019-12-03T11:50:28.041827 #23130] DEBUG -- :   Table comment (12.6ms)  SELECT comments FROM all_tab_comments WHERE owner = :owner AND table_name = :table_name  
D, [2019-12-03T11:50:28.044965 #23130] DEBUG -- :   default tablespace (1.2ms)  SELECT LOWER(default_tablespace) FROM user_users WHERE username = SYS_CONTEXT('userenv', 'current_schema')
D, [2019-12-03T11:50:28.997630 #23130] DEBUG -- :   indexes (952.4ms)  SELECT LOWER(i.table_name) AS table_name, LOWER(i.index_name) AS index_name, i.uniqueness, i.index_type, i.ityp_owner, i.ityp_name, i.parameters, LOWER(i.tablespace_name) AS tablespace_name, LOWER(c.column_name) AS column_name, e.column_expression, atc.virtual_column FROM all_indexes i JOIN all_ind_columns c ON c.index_name = i.index_name AND c.index_owner = i.owner LEFT OUTER JOIN all_ind_expressions e ON e.index_name = i.index_name AND e.index_owner = i.owner AND e.column_position = c.column_position LEFT OUTER JOIN all_tab_cols atc ON i.table_name = atc.table_name AND c.column_name = atc.column_name AND i.owner = atc.owner AND atc.hidden_column = 'NO' WHERE i.owner = :owner AND i.table_owner = :owner AND i.table_name = :table_name AND NOT EXISTS (SELECT uc.index_name FROM all_constraints uc WHERE uc.index_name = i.index_name AND uc.owner = i.owner AND uc.constraint_type = 'P') ORDER BY i.index_name, c.column_position  
D, [2019-12-03T11:50:29.000101 #23130] DEBUG -- :   default tablespace (1.2ms)  SELECT LOWER(default_tablespace) FROM user_users WHERE username = SYS_CONTEXT('userenv', 'current_schema')
D, [2019-12-03T11:50:29.965063 #23130] DEBUG -- :   indexes (964.7ms)  SELECT LOWER(i.table_name) AS table_name, LOWER(i.index_name) AS index_name, i.uniqueness, i.index_type, i.ityp_owner, i.ityp_name, i.parameters, LOWER(i.tablespace_name) AS tablespace_name, LOWER(c.column_name) AS column_name, e.column_expression, atc.virtual_column FROM all_indexes i JOIN all_ind_columns c ON c.index_name = i.index_name AND c.index_owner = i.owner LEFT OUTER JOIN all_ind_expressions e ON e.index_name = i.index_name AND e.index_owner = i.owner AND e.column_position = c.column_position LEFT OUTER JOIN all_tab_cols atc ON i.table_name = atc.table_name AND c.column_name = atc.column_name AND i.owner = atc.owner AND atc.hidden_column = 'NO' WHERE i.owner = :owner AND i.table_owner = :owner AND i.table_name = :table_name AND NOT EXISTS (SELECT uc.index_name FROM all_constraints uc WHERE uc.index_name = i.index_name AND uc.owner = i.owner AND uc.constraint_type = 'P') ORDER BY i.index_name, c.column_position  
D, [2019-12-03T11:50:29.968923 #23130] DEBUG -- :   Primary Key Trigger (2.2ms)  SELECT trigger_name FROM all_triggers WHERE owner = :owner AND trigger_name = :trigger_name AND table_owner = :owner AND table_name = :table_name AND status = 'ENABLED'  
D, [2019-12-03T11:50:29.997163 #23130] DEBUG -- :   Column definitions (26.8ms)  SELECT cols.column_name AS name, cols.data_type AS sql_type, cols.data_default, cols.nullable, cols.virtual_column, cols.hidden_column, cols.data_type_owner AS sql_type_owner, DECODE(cols.data_type, 'NUMBER', data_precision, 'FLOAT', data_precision, 'VARCHAR2', DECODE(char_used, 'C', char_length, data_length), 'RAW', DECODE(char_used, 'C', char_length, data_length), 'CHAR', DECODE(char_used, 'C', char_length, data_length), NULL) AS limit, DECODE(data_type, 'NUMBER', data_scale, NULL) AS scale, comments.comments as column_comment FROM all_tab_cols cols, all_col_comments comments WHERE cols.owner = 'DEV_SCHEMA' AND cols.table_name = 'EXPERIMENTS' AND cols.hidden_column = 'NO' AND cols.owner = comments.owner AND cols.table_name = comments.table_name AND cols.column_name = comments.column_name ORDER BY cols.column_id
D, [2019-12-03T11:50:30.001681 #23130] DEBUG -- :   Primary Keys (2.5ms)  SELECT cc.column_name FROM all_constraints c, all_cons_columns cc WHERE c.owner = :owner AND c.table_name = :table_name AND c.constraint_type = 'P' AND cc.owner = c.owner AND cc.constraint_name = c.constraint_name order by cc.position  
D, [2019-12-03T11:50:30.003352 #23130] DEBUG -- :   temporary table (1.5ms)  SELECT temporary FROM all_tables WHERE table_name = :table_name and owner = SYS_CONTEXT('userenv', 'session_user')  
D, [2019-12-03T11:50:30.072730 #23130] DEBUG -- :   Table comment (67.8ms)  SELECT comments FROM all_tab_comments WHERE owner = :owner AND table_name = :table_name  
D, [2019-12-03T11:50:30.077228 #23130] DEBUG -- :   default tablespace (1.7ms)  SELECT LOWER(default_tablespace) FROM user_users WHERE username = SYS_CONTEXT('userenv', 'current_schema')
D, [2019-12-03T11:50:31.846937 #23130] DEBUG -- :   indexes (1769.4ms)  SELECT LOWER(i.table_name) AS table_name, LOWER(i.index_name) AS index_name, i.uniqueness, i.index_type, i.ityp_owner, i.ityp_name, i.parameters, LOWER(i.tablespace_name) AS tablespace_name, LOWER(c.column_name) AS column_name, e.column_expression, atc.virtual_column FROM all_indexes i JOIN all_ind_columns c ON c.index_name = i.index_name AND c.index_owner = i.owner LEFT OUTER JOIN all_ind_expressions e ON e.index_name = i.index_name AND e.index_owner = i.owner AND e.column_position = c.column_position LEFT OUTER JOIN all_tab_cols atc ON i.table_name = atc.table_name AND c.column_name = atc.column_name AND i.owner = atc.owner AND atc.hidden_column = 'NO' WHERE i.owner = :owner AND i.table_owner = :owner AND i.table_name = :table_name AND NOT EXISTS (SELECT uc.index_name FROM all_constraints uc WHERE uc.index_name = i.index_name AND uc.owner = i.owner AND uc.constraint_type = 'P') ORDER BY i.index_name, c.column_position  
D, [2019-12-03T11:50:31.849859 #23130] DEBUG -- :   default tablespace (1.4ms)  SELECT LOWER(default_tablespace) FROM user_users WHERE username = SYS_CONTEXT('userenv', 'current_schema')
D, [2019-12-03T11:50:33.629966 #23130] DEBUG -- :   indexes (1779.9ms)  SELECT LOWER(i.table_name) AS table_name, LOWER(i.index_name) AS index_name, i.uniqueness, i.index_type, i.ityp_owner, i.ityp_name, i.parameters, LOWER(i.tablespace_name) AS tablespace_name, LOWER(c.column_name) AS column_name, e.column_expression, atc.virtual_column FROM all_indexes i JOIN all_ind_columns c ON c.index_name = i.index_name AND c.index_owner = i.owner LEFT OUTER JOIN all_ind_expressions e ON e.index_name = i.index_name AND e.index_owner = i.owner AND e.column_position = c.column_position LEFT OUTER JOIN all_tab_cols atc ON i.table_name = atc.table_name AND c.column_name = atc.column_name AND i.owner = atc.owner AND atc.hidden_column = 'NO' WHERE i.owner = :owner AND i.table_owner = :owner AND i.table_name = :table_name AND NOT EXISTS (SELECT uc.index_name FROM all_constraints uc WHERE uc.index_name = i.index_name AND uc.owner = i.owner AND uc.constraint_type = 'P') ORDER BY i.index_name, c.column_position  
D, [2019-12-03T11:50:33.632186 #23130] DEBUG -- :   Primary Key Trigger (1.0ms)  SELECT trigger_name FROM all_triggers WHERE owner = :owner AND trigger_name = :trigger_name AND table_owner = :owner AND table_name = :table_name AND status = 'ENABLED'  
D, [2019-12-03T11:50:33.639198 #23130] DEBUG -- :   Column definitions (6.0ms)  SELECT cols.column_name AS name, cols.data_type AS sql_type, cols.data_default, cols.nullable, cols.virtual_column, cols.hidden_column, cols.data_type_owner AS sql_type_owner, DECODE(cols.data_type, 'NUMBER', data_precision, 'FLOAT', data_precision, 'VARCHAR2', DECODE(char_used, 'C', char_length, data_length), 'RAW', DECODE(char_used, 'C', char_length, data_length), 'CHAR', DECODE(char_used, 'C', char_length, data_length), NULL) AS limit, DECODE(data_type, 'NUMBER', data_scale, NULL) AS scale, comments.comments as column_comment FROM all_tab_cols cols, all_col_comments comments WHERE cols.owner = 'DEV_SCHEMA' AND cols.table_name = 'GROUPS' AND cols.hidden_column = 'NO' AND cols.owner = comments.owner AND cols.table_name = comments.table_name AND cols.column_name = comments.column_name ORDER BY cols.column_id
D, [2019-12-03T11:50:33.642331 #23130] DEBUG -- :   Primary Keys (2.2ms)  SELECT cc.column_name FROM all_constraints c, all_cons_columns cc WHERE c.owner = :owner AND c.table_name = :table_name AND c.constraint_type = 'P' AND cc.owner = c.owner AND cc.constraint_name = c.constraint_name order by cc.position  
D, [2019-12-03T11:50:33.643754 #23130] DEBUG -- :   temporary table (1.3ms)  SELECT temporary FROM all_tables WHERE table_name = :table_name and owner = SYS_CONTEXT('userenv', 'session_user')  
D, [2019-12-03T11:50:33.658825 #23130] DEBUG -- :   Table comment (14.3ms)  SELECT comments FROM all_tab_comments WHERE owner = :owner AND table_name = :table_name  
D, [2019-12-03T11:50:33.661492 #23130] DEBUG -- :   default tablespace (1.2ms)  SELECT LOWER(default_tablespace) FROM user_users WHERE username = SYS_CONTEXT('userenv', 'current_schema')
D, [2019-12-03T11:50:34.594554 #23130] DEBUG -- :   indexes (932.8ms)  SELECT LOWER(i.table_name) AS table_name, LOWER(i.index_name) AS index_name, i.uniqueness, i.index_type, i.ityp_owner, i.ityp_name, i.parameters, LOWER(i.tablespace_name) AS tablespace_name, LOWER(c.column_name) AS column_name, e.column_expression, atc.virtual_column FROM all_indexes i JOIN all_ind_columns c ON c.index_name = i.index_name AND c.index_owner = i.owner LEFT OUTER JOIN all_ind_expressions e ON e.index_name = i.index_name AND e.index_owner = i.owner AND e.column_position = c.column_position LEFT OUTER JOIN all_tab_cols atc ON i.table_name = atc.table_name AND c.column_name = atc.column_name AND i.owner = atc.owner AND atc.hidden_column = 'NO' WHERE i.owner = :owner AND i.table_owner = :owner AND i.table_name = :table_name AND NOT EXISTS (SELECT uc.index_name FROM all_constraints uc WHERE uc.index_name = i.index_name AND uc.owner = i.owner AND uc.constraint_type = 'P') ORDER BY i.index_name, c.column_position  
D, [2019-12-03T11:50:34.598573 #23130] DEBUG -- :   default tablespace (2.1ms)  SELECT LOWER(default_tablespace) FROM user_users WHERE username = SYS_CONTEXT('userenv', 'current_schema')
D, [2019-12-03T11:50:35.509579 #23130] DEBUG -- :   indexes (910.8ms)  SELECT LOWER(i.table_name) AS table_name, LOWER(i.index_name) AS index_name, i.uniqueness, i.index_type, i.ityp_owner, i.ityp_name, i.parameters, LOWER(i.tablespace_name) AS tablespace_name, LOWER(c.column_name) AS column_name, e.column_expression, atc.virtual_column FROM all_indexes i JOIN all_ind_columns c ON c.index_name = i.index_name AND c.index_owner = i.owner LEFT OUTER JOIN all_ind_expressions e ON e.index_name = i.index_name AND e.index_owner = i.owner AND e.column_position = c.column_position LEFT OUTER JOIN all_tab_cols atc ON i.table_name = atc.table_name AND c.column_name = atc.column_name AND i.owner = atc.owner AND atc.hidden_column = 'NO' WHERE i.owner = :owner AND i.table_owner = :owner AND i.table_name = :table_name AND NOT EXISTS (SELECT uc.index_name FROM all_constraints uc WHERE uc.index_name = i.index_name AND uc.owner = i.owner AND uc.constraint_type = 'P') ORDER BY i.index_name, c.column_position  
D, [2019-12-03T11:50:35.512910 #23130] DEBUG -- :   Primary Key Trigger (1.6ms)  SELECT trigger_name FROM all_triggers WHERE owner = :owner AND trigger_name = :trigger_name AND table_owner = :owner AND table_name = :table_name AND status = 'ENABLED'  
D, [2019-12-03T11:50:35.529342 #23130] DEBUG -- :   Column definitions (14.5ms)  SELECT cols.column_name AS name, cols.data_type AS sql_type, cols.data_default, cols.nullable, cols.virtual_column, cols.hidden_column, cols.data_type_owner AS sql_type_owner, DECODE(cols.data_type, 'NUMBER', data_precision, 'FLOAT', data_precision, 'VARCHAR2', DECODE(char_used, 'C', char_length, data_length), 'RAW', DECODE(char_used, 'C', char_length, data_length), 'CHAR', DECODE(char_used, 'C', char_length, data_length), NULL) AS limit, DECODE(data_type, 'NUMBER', data_scale, NULL) AS scale, comments.comments as column_comment FROM all_tab_cols cols, all_col_comments comments WHERE cols.owner = 'DEV_SCHEMA' AND cols.table_name = 'INVOICES' AND cols.hidden_column = 'NO' AND cols.owner = comments.owner AND cols.table_name = comments.table_name AND cols.column_name = comments.column_name ORDER BY cols.column_id
D, [2019-12-03T11:50:35.532913 #23130] DEBUG -- :   Primary Keys (2.3ms)  SELECT cc.column_name FROM all_constraints c, all_cons_columns cc WHERE c.owner = :owner AND c.table_name = :table_name AND c.constraint_type = 'P' AND cc.owner = c.owner AND cc.constraint_name = c.constraint_name order by cc.position  
D, [2019-12-03T11:50:35.534523 #23130] DEBUG -- :   temporary table (1.4ms)  SELECT temporary FROM all_tables WHERE table_name = :table_name and owner = SYS_CONTEXT('userenv', 'session_user')  
D, [2019-12-03T11:50:35.604884 #23130] DEBUG -- :   Table comment (69.3ms)  SELECT comments FROM all_tab_comments WHERE owner = :owner AND table_name = :table_name  
D, [2019-12-03T11:50:35.608603 #23130] DEBUG -- :   default tablespace (1.8ms)  SELECT LOWER(default_tablespace) FROM user_users WHERE username = SYS_CONTEXT('userenv', 'current_schema')
D, [2019-12-03T11:50:37.404429 #23130] DEBUG -- :   indexes (1795.5ms)  SELECT LOWER(i.table_name) AS table_name, LOWER(i.index_name) AS index_name, i.uniqueness, i.index_type, i.ityp_owner, i.ityp_name, i.parameters, LOWER(i.tablespace_name) AS tablespace_name, LOWER(c.column_name) AS column_name, e.column_expression, atc.virtual_column FROM all_indexes i JOIN all_ind_columns c ON c.index_name = i.index_name AND c.index_owner = i.owner LEFT OUTER JOIN all_ind_expressions e ON e.index_name = i.index_name AND e.index_owner = i.owner AND e.column_position = c.column_position LEFT OUTER JOIN all_tab_cols atc ON i.table_name = atc.table_name AND c.column_name = atc.column_name AND i.owner = atc.owner AND atc.hidden_column = 'NO' WHERE i.owner = :owner AND i.table_owner = :owner AND i.table_name = :table_name AND NOT EXISTS (SELECT uc.index_name FROM all_constraints uc WHERE uc.index_name = i.index_name AND uc.owner = i.owner AND uc.constraint_type = 'P') ORDER BY i.index_name, c.column_position  
D, [2019-12-03T11:50:37.407571 #23130] DEBUG -- :   default tablespace (1.8ms)  SELECT LOWER(default_tablespace) FROM user_users WHERE username = SYS_CONTEXT('userenv', 'current_schema')
D, [2019-12-03T11:50:39.167445 #23130] DEBUG -- :   indexes (1759.6ms)  SELECT LOWER(i.table_name) AS table_name, LOWER(i.index_name) AS index_name, i.uniqueness, i.index_type, i.ityp_owner, i.ityp_name, i.parameters, LOWER(i.tablespace_name) AS tablespace_name, LOWER(c.column_name) AS column_name, e.column_expression, atc.virtual_column FROM all_indexes i JOIN all_ind_columns c ON c.index_name = i.index_name AND c.index_owner = i.owner LEFT OUTER JOIN all_ind_expressions e ON e.index_name = i.index_name AND e.index_owner = i.owner AND e.column_position = c.column_position LEFT OUTER JOIN all_tab_cols atc ON i.table_name = atc.table_name AND c.column_name = atc.column_name AND i.owner = atc.owner AND atc.hidden_column = 'NO' WHERE i.owner = :owner AND i.table_owner = :owner AND i.table_name = :table_name AND NOT EXISTS (SELECT uc.index_name FROM all_constraints uc WHERE uc.index_name = i.index_name AND uc.owner = i.owner AND uc.constraint_type = 'P') ORDER BY i.index_name, c.column_position  
D, [2019-12-03T11:50:39.170369 #23130] DEBUG -- :   Primary Key Trigger (1.6ms)  SELECT trigger_name FROM all_triggers WHERE owner = :owner AND trigger_name = :trigger_name AND table_owner = :owner AND table_name = :table_name AND status = 'ENABLED'  
D, [2019-12-03T11:50:39.179196 #23130] DEBUG -- :   Column definitions (7.7ms)  SELECT cols.column_name AS name, cols.data_type AS sql_type, cols.data_default, cols.nullable, cols.virtual_column, cols.hidden_column, cols.data_type_owner AS sql_type_owner, DECODE(cols.data_type, 'NUMBER', data_precision, 'FLOAT', data_precision, 'VARCHAR2', DECODE(char_used, 'C', char_length, data_length), 'RAW', DECODE(char_used, 'C', char_length, data_length), 'CHAR', DECODE(char_used, 'C', char_length, data_length), NULL) AS limit, DECODE(data_type, 'NUMBER', data_scale, NULL) AS scale, comments.comments as column_comment FROM all_tab_cols cols, all_col_comments comments WHERE cols.owner = 'DEV_SCHEMA' AND cols.table_name = 'LINE_ITEM_SAMPLES' AND cols.hidden_column = 'NO' AND cols.owner = comments.owner AND cols.table_name = comments.table_name AND cols.column_name = comments.column_name ORDER BY cols.column_id
D, [2019-12-03T11:50:39.182141 #23130] DEBUG -- :   Primary Keys (2.1ms)  SELECT cc.column_name FROM all_constraints c, all_cons_columns cc WHERE c.owner = :owner AND c.table_name = :table_name AND c.constraint_type = 'P' AND cc.owner = c.owner AND cc.constraint_name = c.constraint_name order by cc.position  
D, [2019-12-03T11:50:39.183287 #23130] DEBUG -- :   temporary table (1.0ms)  SELECT temporary FROM all_tables WHERE table_name = :table_name and owner = SYS_CONTEXT('userenv', 'session_user')  
D, [2019-12-03T11:50:39.248515 #23130] DEBUG -- :   Table comment (64.3ms)  SELECT comments FROM all_tab_comments WHERE owner = :owner AND table_name = :table_name  
D, [2019-12-03T11:50:39.251907 #23130] DEBUG -- :   default tablespace (1.6ms)  SELECT LOWER(default_tablespace) FROM user_users WHERE username = SYS_CONTEXT('userenv', 'current_schema')
D, [2019-12-03T11:50:42.755425 #23130] DEBUG -- :   indexes (3503.2ms)  SELECT LOWER(i.table_name) AS table_name, LOWER(i.index_name) AS index_name, i.uniqueness, i.index_type, i.ityp_owner, i.ityp_name, i.parameters, LOWER(i.tablespace_name) AS tablespace_name, LOWER(c.column_name) AS column_name, e.column_expression, atc.virtual_column FROM all_indexes i JOIN all_ind_columns c ON c.index_name = i.index_name AND c.index_owner = i.owner LEFT OUTER JOIN all_ind_expressions e ON e.index_name = i.index_name AND e.index_owner = i.owner AND e.column_position = c.column_position LEFT OUTER JOIN all_tab_cols atc ON i.table_name = atc.table_name AND c.column_name = atc.column_name AND i.owner = atc.owner AND atc.hidden_column = 'NO' WHERE i.owner = :owner AND i.table_owner = :owner AND i.table_name = :table_name AND NOT EXISTS (SELECT uc.index_name FROM all_constraints uc WHERE uc.index_name = i.index_name AND uc.owner = i.owner AND uc.constraint_type = 'P') ORDER BY i.index_name, c.column_position  
D, [2019-12-03T11:50:42.757386 #23130] DEBUG -- :   default tablespace (1.0ms)  SELECT LOWER(default_tablespace) FROM user_users WHERE username = SYS_CONTEXT('userenv', 'current_schema')
D, [2019-12-03T11:50:46.431910 #23130] DEBUG -- :   indexes (3674.3ms)  SELECT LOWER(i.table_name) AS table_name, LOWER(i.index_name) AS index_name, i.uniqueness, i.index_type, i.ityp_owner, i.ityp_name, i.parameters, LOWER(i.tablespace_name) AS tablespace_name, LOWER(c.column_name) AS column_name, e.column_expression, atc.virtual_column FROM all_indexes i JOIN all_ind_columns c ON c.index_name = i.index_name AND c.index_owner = i.owner LEFT OUTER JOIN all_ind_expressions e ON e.index_name = i.index_name AND e.index_owner = i.owner AND e.column_position = c.column_position LEFT OUTER JOIN all_tab_cols atc ON i.table_name = atc.table_name AND c.column_name = atc.column_name AND i.owner = atc.owner AND atc.hidden_column = 'NO' WHERE i.owner = :owner AND i.table_owner = :owner AND i.table_name = :table_name AND NOT EXISTS (SELECT uc.index_name FROM all_constraints uc WHERE uc.index_name = i.index_name AND uc.owner = i.owner AND uc.constraint_type = 'P') ORDER BY i.index_name, c.column_position  
D, [2019-12-03T11:50:46.434769 #23130] DEBUG -- :   Primary Key Trigger (1.5ms)  SELECT trigger_name FROM all_triggers WHERE owner = :owner AND trigger_name = :trigger_name AND table_owner = :owner AND table_name = :table_name AND status = 'ENABLED'  
D, [2019-12-03T11:50:46.442576 #23130] DEBUG -- :   Column definitions (6.6ms)  SELECT cols.column_name AS name, cols.data_type AS sql_type, cols.data_default, cols.nullable, cols.virtual_column, cols.hidden_column, cols.data_type_owner AS sql_type_owner, DECODE(cols.data_type, 'NUMBER', data_precision, 'FLOAT', data_precision, 'VARCHAR2', DECODE(char_used, 'C', char_length, data_length), 'RAW', DECODE(char_used, 'C', char_length, data_length), 'CHAR', DECODE(char_used, 'C', char_length, data_length), NULL) AS limit, DECODE(data_type, 'NUMBER', data_scale, NULL) AS scale, comments.comments as column_comment FROM all_tab_cols cols, all_col_comments comments WHERE cols.owner = 'DEV_SCHEMA' AND cols.table_name = 'LINE_ITEM_SERVICES' AND cols.hidden_column = 'NO' AND cols.owner = comments.owner AND cols.table_name = comments.table_name AND cols.column_name = comments.column_name ORDER BY cols.column_id
D, [2019-12-03T11:50:46.445320 #23130] DEBUG -- :   Primary Keys (1.8ms)  SELECT cc.column_name FROM all_constraints c, all_cons_columns cc WHERE c.owner = :owner AND c.table_name = :table_name AND c.constraint_type = 'P' AND cc.owner = c.owner AND cc.constraint_name = c.constraint_name order by cc.position  
D, [2019-12-03T11:50:46.446600 #23130] DEBUG -- :   temporary table (1.1ms)  SELECT temporary FROM all_tables WHERE table_name = :table_name and owner = SYS_CONTEXT('userenv', 'session_user')  
D, [2019-12-03T11:50:46.514970 #23130] DEBUG -- :   Table comment (67.5ms)  SELECT comments FROM all_tab_comments WHERE owner = :owner AND table_name = :table_name  
D, [2019-12-03T11:50:46.522671 #23130] DEBUG -- :   default tablespace (4.5ms)  SELECT LOWER(default_tablespace) FROM user_users WHERE username = SYS_CONTEXT('userenv', 'current_schema')
D, [2019-12-03T11:50:47.416083 #23130] DEBUG -- :   indexes (893.2ms)  SELECT LOWER(i.table_name) AS table_name, LOWER(i.index_name) AS index_name, i.uniqueness, i.index_type, i.ityp_owner, i.ityp_name, i.parameters, LOWER(i.tablespace_name) AS tablespace_name, LOWER(c.column_name) AS column_name, e.column_expression, atc.virtual_column FROM all_indexes i JOIN all_ind_columns c ON c.index_name = i.index_name AND c.index_owner = i.owner LEFT OUTER JOIN all_ind_expressions e ON e.index_name = i.index_name AND e.index_owner = i.owner AND e.column_position = c.column_position LEFT OUTER JOIN all_tab_cols atc ON i.table_name = atc.table_name AND c.column_name = atc.column_name AND i.owner = atc.owner AND atc.hidden_column = 'NO' WHERE i.owner = :owner AND i.table_owner = :owner AND i.table_name = :table_name AND NOT EXISTS (SELECT uc.index_name FROM all_constraints uc WHERE uc.index_name = i.index_name AND uc.owner = i.owner AND uc.constraint_type = 'P') ORDER BY i.index_name, c.column_position  
D, [2019-12-03T11:50:47.419058 #23130] DEBUG -- :   default tablespace (1.7ms)  SELECT LOWER(default_tablespace) FROM user_users WHERE username = SYS_CONTEXT('userenv', 'current_schema')
D, [2019-12-03T11:50:48.299821 #23130] DEBUG -- :   indexes (880.5ms)  SELECT LOWER(i.table_name) AS table_name, LOWER(i.index_name) AS index_name, i.uniqueness, i.index_type, i.ityp_owner, i.ityp_name, i.parameters, LOWER(i.tablespace_name) AS tablespace_name, LOWER(c.column_name) AS column_name, e.column_expression, atc.virtual_column FROM all_indexes i JOIN all_ind_columns c ON c.index_name = i.index_name AND c.index_owner = i.owner LEFT OUTER JOIN all_ind_expressions e ON e.index_name = i.index_name AND e.index_owner = i.owner AND e.column_position = c.column_position LEFT OUTER JOIN all_tab_cols atc ON i.table_name = atc.table_name AND c.column_name = atc.column_name AND i.owner = atc.owner AND atc.hidden_column = 'NO' WHERE i.owner = :owner AND i.table_owner = :owner AND i.table_name = :table_name AND NOT EXISTS (SELECT uc.index_name FROM all_constraints uc WHERE uc.index_name = i.index_name AND uc.owner = i.owner AND uc.constraint_type = 'P') ORDER BY i.index_name, c.column_position  
D, [2019-12-03T11:50:48.302882 #23130] DEBUG -- :   Primary Key Trigger (1.7ms)  SELECT trigger_name FROM all_triggers WHERE owner = :owner AND trigger_name = :trigger_name AND table_owner = :owner AND table_name = :table_name AND status = 'ENABLED'  
D, [2019-12-03T11:50:48.314815 #23130] DEBUG -- :   Column definitions (10.6ms)  SELECT cols.column_name AS name, cols.data_type AS sql_type, cols.data_default, cols.nullable, cols.virtual_column, cols.hidden_column, cols.data_type_owner AS sql_type_owner, DECODE(cols.data_type, 'NUMBER', data_precision, 'FLOAT', data_precision, 'VARCHAR2', DECODE(char_used, 'C', char_length, data_length), 'RAW', DECODE(char_used, 'C', char_length, data_length), 'CHAR', DECODE(char_used, 'C', char_length, data_length), NULL) AS limit, DECODE(data_type, 'NUMBER', data_scale, NULL) AS scale, comments.comments as column_comment FROM all_tab_cols cols, all_col_comments comments WHERE cols.owner = 'DEV_SCHEMA' AND cols.table_name = 'LINE_ITEMS' AND cols.hidden_column = 'NO' AND cols.owner = comments.owner AND cols.table_name = comments.table_name AND cols.column_name = comments.column_name ORDER BY cols.column_id
D, [2019-12-03T11:50:48.318823 #23130] DEBUG -- :   Primary Keys (2.6ms)  SELECT cc.column_name FROM all_constraints c, all_cons_columns cc WHERE c.owner = :owner AND c.table_name = :table_name AND c.constraint_type = 'P' AND cc.owner = c.owner AND cc.constraint_name = c.constraint_name order by cc.position  
D, [2019-12-03T11:50:48.320593 #23130] DEBUG -- :   temporary table (1.6ms)  SELECT temporary FROM all_tables WHERE table_name = :table_name and owner = SYS_CONTEXT('userenv', 'session_user')  
D, [2019-12-03T11:50:48.335222 #23130] DEBUG -- :   Table comment (13.4ms)  SELECT comments FROM all_tab_comments WHERE owner = :owner AND table_name = :table_name  
D, [2019-12-03T11:50:48.338773 #23130] DEBUG -- :   default tablespace (1.8ms)  SELECT LOWER(default_tablespace) FROM user_users WHERE username = SYS_CONTEXT('userenv', 'current_schema')
D, [2019-12-03T11:50:50.103966 #23130] DEBUG -- :   indexes (1764.8ms)  SELECT LOWER(i.table_name) AS table_name, LOWER(i.index_name) AS index_name, i.uniqueness, i.index_type, i.ityp_owner, i.ityp_name, i.parameters, LOWER(i.tablespace_name) AS tablespace_name, LOWER(c.column_name) AS column_name, e.column_expression, atc.virtual_column FROM all_indexes i JOIN all_ind_columns c ON c.index_name = i.index_name AND c.index_owner = i.owner LEFT OUTER JOIN all_ind_expressions e ON e.index_name = i.index_name AND e.index_owner = i.owner AND e.column_position = c.column_position LEFT OUTER JOIN all_tab_cols atc ON i.table_name = atc.table_name AND c.column_name = atc.column_name AND i.owner = atc.owner AND atc.hidden_column = 'NO' WHERE i.owner = :owner AND i.table_owner = :owner AND i.table_name = :table_name AND NOT EXISTS (SELECT uc.index_name FROM all_constraints uc WHERE uc.index_name = i.index_name AND uc.owner = i.owner AND uc.constraint_type = 'P') ORDER BY i.index_name, c.column_position  
D, [2019-12-03T11:50:50.107040 #23130] DEBUG -- :   default tablespace (1.7ms)  SELECT LOWER(default_tablespace) FROM user_users WHERE username = SYS_CONTEXT('userenv', 'current_schema')
D, [2019-12-03T11:50:51.890816 #23130] DEBUG -- :   indexes (1783.5ms)  SELECT LOWER(i.table_name) AS table_name, LOWER(i.index_name) AS index_name, i.uniqueness, i.index_type, i.ityp_owner, i.ityp_name, i.parameters, LOWER(i.tablespace_name) AS tablespace_name, LOWER(c.column_name) AS column_name, e.column_expression, atc.virtual_column FROM all_indexes i JOIN all_ind_columns c ON c.index_name = i.index_name AND c.index_owner = i.owner LEFT OUTER JOIN all_ind_expressions e ON e.index_name = i.index_name AND e.index_owner = i.owner AND e.column_position = c.column_position LEFT OUTER JOIN all_tab_cols atc ON i.table_name = atc.table_name AND c.column_name = atc.column_name AND i.owner = atc.owner AND atc.hidden_column = 'NO' WHERE i.owner = :owner AND i.table_owner = :owner AND i.table_name = :table_name AND NOT EXISTS (SELECT uc.index_name FROM all_constraints uc WHERE uc.index_name = i.index_name AND uc.owner = i.owner AND uc.constraint_type = 'P') ORDER BY i.index_name, c.column_position  
D, [2019-12-03T11:50:51.893819 #23130] DEBUG -- :   Primary Key Trigger (1.4ms)  SELECT trigger_name FROM all_triggers WHERE owner = :owner AND trigger_name = :trigger_name AND table_owner = :owner AND table_name = :table_name AND status = 'ENABLED'  
D, [2019-12-03T11:50:51.904662 #23130] DEBUG -- :   Column definitions (9.6ms)  SELECT cols.column_name AS name, cols.data_type AS sql_type, cols.data_default, cols.nullable, cols.virtual_column, cols.hidden_column, cols.data_type_owner AS sql_type_owner, DECODE(cols.data_type, 'NUMBER', data_precision, 'FLOAT', data_precision, 'VARCHAR2', DECODE(char_used, 'C', char_length, data_length), 'RAW', DECODE(char_used, 'C', char_length, data_length), 'CHAR', DECODE(char_used, 'C', char_length, data_length), NULL) AS limit, DECODE(data_type, 'NUMBER', data_scale, NULL) AS scale, comments.comments as column_comment FROM all_tab_cols cols, all_col_comments comments WHERE cols.owner = 'DEV_SCHEMA' AND cols.table_name = 'SAMPLES' AND cols.hidden_column = 'NO' AND cols.owner = comments.owner AND cols.table_name = comments.table_name AND cols.column_name = comments.column_name ORDER BY cols.column_id
D, [2019-12-03T11:50:51.907571 #23130] DEBUG -- :   Primary Keys (2.0ms)  SELECT cc.column_name FROM all_constraints c, all_cons_columns cc WHERE c.owner = :owner AND c.table_name = :table_name AND c.constraint_type = 'P' AND cc.owner = c.owner AND cc.constraint_name = c.constraint_name order by cc.position  
D, [2019-12-03T11:50:51.908669 #23130] DEBUG -- :   temporary table (1.0ms)  SELECT temporary FROM all_tables WHERE table_name = :table_name and owner = SYS_CONTEXT('userenv', 'session_user')  
D, [2019-12-03T11:50:51.922965 #23130] DEBUG -- :   Table comment (13.5ms)  SELECT comments FROM all_tab_comments WHERE owner = :owner AND table_name = :table_name  
D, [2019-12-03T11:50:51.926069 #23130] DEBUG -- :   default tablespace (1.4ms)  SELECT LOWER(default_tablespace) FROM user_users WHERE username = SYS_CONTEXT('userenv', 'current_schema')
D, [2019-12-03T11:50:53.767782 #23130] DEBUG -- :   indexes (1841.4ms)  SELECT LOWER(i.table_name) AS table_name, LOWER(i.index_name) AS index_name, i.uniqueness, i.index_type, i.ityp_owner, i.ityp_name, i.parameters, LOWER(i.tablespace_name) AS tablespace_name, LOWER(c.column_name) AS column_name, e.column_expression, atc.virtual_column FROM all_indexes i JOIN all_ind_columns c ON c.index_name = i.index_name AND c.index_owner = i.owner LEFT OUTER JOIN all_ind_expressions e ON e.index_name = i.index_name AND e.index_owner = i.owner AND e.column_position = c.column_position LEFT OUTER JOIN all_tab_cols atc ON i.table_name = atc.table_name AND c.column_name = atc.column_name AND i.owner = atc.owner AND atc.hidden_column = 'NO' WHERE i.owner = :owner AND i.table_owner = :owner AND i.table_name = :table_name AND NOT EXISTS (SELECT uc.index_name FROM all_constraints uc WHERE uc.index_name = i.index_name AND uc.owner = i.owner AND uc.constraint_type = 'P') ORDER BY i.index_name, c.column_position  
D, [2019-12-03T11:50:53.771484 #23130] DEBUG -- :   default tablespace (1.7ms)  SELECT LOWER(default_tablespace) FROM user_users WHERE username = SYS_CONTEXT('userenv', 'current_schema')
D, [2019-12-03T11:50:55.645878 #23130] DEBUG -- :   indexes (1874.1ms)  SELECT LOWER(i.table_name) AS table_name, LOWER(i.index_name) AS index_name, i.uniqueness, i.index_type, i.ityp_owner, i.ityp_name, i.parameters, LOWER(i.tablespace_name) AS tablespace_name, LOWER(c.column_name) AS column_name, e.column_expression, atc.virtual_column FROM all_indexes i JOIN all_ind_columns c ON c.index_name = i.index_name AND c.index_owner = i.owner LEFT OUTER JOIN all_ind_expressions e ON e.index_name = i.index_name AND e.index_owner = i.owner AND e.column_position = c.column_position LEFT OUTER JOIN all_tab_cols atc ON i.table_name = atc.table_name AND c.column_name = atc.column_name AND i.owner = atc.owner AND atc.hidden_column = 'NO' WHERE i.owner = :owner AND i.table_owner = :owner AND i.table_name = :table_name AND NOT EXISTS (SELECT uc.index_name FROM all_constraints uc WHERE uc.index_name = i.index_name AND uc.owner = i.owner AND uc.constraint_type = 'P') ORDER BY i.index_name, c.column_position  
D, [2019-12-03T11:50:55.649897 #23130] DEBUG -- :   Primary Key Trigger (2.1ms)  SELECT trigger_name FROM all_triggers WHERE owner = :owner AND trigger_name = :trigger_name AND table_owner = :owner AND table_name = :table_name AND status = 'ENABLED'  
D, [2019-12-03T11:50:55.661233 #23130] DEBUG -- :   Column definitions (9.7ms)  SELECT cols.column_name AS name, cols.data_type AS sql_type, cols.data_default, cols.nullable, cols.virtual_column, cols.hidden_column, cols.data_type_owner AS sql_type_owner, DECODE(cols.data_type, 'NUMBER', data_precision, 'FLOAT', data_precision, 'VARCHAR2', DECODE(char_used, 'C', char_length, data_length), 'RAW', DECODE(char_used, 'C', char_length, data_length), 'CHAR', DECODE(char_used, 'C', char_length, data_length), NULL) AS limit, DECODE(data_type, 'NUMBER', data_scale, NULL) AS scale, comments.comments as column_comment FROM all_tab_cols cols, all_col_comments comments WHERE cols.owner = 'DEV_SCHEMA' AND cols.table_name = 'USER_GROUPS' AND cols.hidden_column = 'NO' AND cols.owner = comments.owner AND cols.table_name = comments.table_name AND cols.column_name = comments.column_name ORDER BY cols.column_id
D, [2019-12-03T11:50:55.666699 #23130] DEBUG -- :   Primary Keys (3.8ms)  SELECT cc.column_name FROM all_constraints c, all_cons_columns cc WHERE c.owner = :owner AND c.table_name = :table_name AND c.constraint_type = 'P' AND cc.owner = c.owner AND cc.constraint_name = c.constraint_name order by cc.position  
D, [2019-12-03T11:50:55.669160 #23130] DEBUG -- :   temporary table (2.2ms)  SELECT temporary FROM all_tables WHERE table_name = :table_name and owner = SYS_CONTEXT('userenv', 'session_user')  
D, [2019-12-03T11:50:55.733561 #23130] DEBUG -- :   Table comment (63.0ms)  SELECT comments FROM all_tab_comments WHERE owner = :owner AND table_name = :table_name  
D, [2019-12-03T11:50:55.737948 #23130] DEBUG -- :   default tablespace (2.8ms)  SELECT LOWER(default_tablespace) FROM user_users WHERE username = SYS_CONTEXT('userenv', 'current_schema')
D, [2019-12-03T11:50:58.836821 #23130] DEBUG -- :   indexes (3098.6ms)  SELECT LOWER(i.table_name) AS table_name, LOWER(i.index_name) AS index_name, i.uniqueness, i.index_type, i.ityp_owner, i.ityp_name, i.parameters, LOWER(i.tablespace_name) AS tablespace_name, LOWER(c.column_name) AS column_name, e.column_expression, atc.virtual_column FROM all_indexes i JOIN all_ind_columns c ON c.index_name = i.index_name AND c.index_owner = i.owner LEFT OUTER JOIN all_ind_expressions e ON e.index_name = i.index_name AND e.index_owner = i.owner AND e.column_position = c.column_position LEFT OUTER JOIN all_tab_cols atc ON i.table_name = atc.table_name AND c.column_name = atc.column_name AND i.owner = atc.owner AND atc.hidden_column = 'NO' WHERE i.owner = :owner AND i.table_owner = :owner AND i.table_name = :table_name AND NOT EXISTS (SELECT uc.index_name FROM all_constraints uc WHERE uc.index_name = i.index_name AND uc.owner = i.owner AND uc.constraint_type = 'P') ORDER BY i.index_name, c.column_position  
D, [2019-12-03T11:50:58.839667 #23130] DEBUG -- :   default tablespace (1.7ms)  SELECT LOWER(default_tablespace) FROM user_users WHERE username = SYS_CONTEXT('userenv', 'current_schema')
D, [2019-12-03T11:51:01.770058 #23130] DEBUG -- :   indexes (2930.1ms)  SELECT LOWER(i.table_name) AS table_name, LOWER(i.index_name) AS index_name, i.uniqueness, i.index_type, i.ityp_owner, i.ityp_name, i.parameters, LOWER(i.tablespace_name) AS tablespace_name, LOWER(c.column_name) AS column_name, e.column_expression, atc.virtual_column FROM all_indexes i JOIN all_ind_columns c ON c.index_name = i.index_name AND c.index_owner = i.owner LEFT OUTER JOIN all_ind_expressions e ON e.index_name = i.index_name AND e.index_owner = i.owner AND e.column_position = c.column_position LEFT OUTER JOIN all_tab_cols atc ON i.table_name = atc.table_name AND c.column_name = atc.column_name AND i.owner = atc.owner AND atc.hidden_column = 'NO' WHERE i.owner = :owner AND i.table_owner = :owner AND i.table_name = :table_name AND NOT EXISTS (SELECT uc.index_name FROM all_constraints uc WHERE uc.index_name = i.index_name AND uc.owner = i.owner AND uc.constraint_type = 'P') ORDER BY i.index_name, c.column_position  
D, [2019-12-03T11:51:01.774474 #23130] DEBUG -- :   Primary Key Trigger (2.6ms)  SELECT trigger_name FROM all_triggers WHERE owner = :owner AND trigger_name = :trigger_name AND table_owner = :owner AND table_name = :table_name AND status = 'ENABLED'  
D, [2019-12-03T11:51:01.791489 #23130] DEBUG -- :   Column definitions (15.4ms)  SELECT cols.column_name AS name, cols.data_type AS sql_type, cols.data_default, cols.nullable, cols.virtual_column, cols.hidden_column, cols.data_type_owner AS sql_type_owner, DECODE(cols.data_type, 'NUMBER', data_precision, 'FLOAT', data_precision, 'VARCHAR2', DECODE(char_used, 'C', char_length, data_length), 'RAW', DECODE(char_used, 'C', char_length, data_length), 'CHAR', DECODE(char_used, 'C', char_length, data_length), NULL) AS limit, DECODE(data_type, 'NUMBER', data_scale, NULL) AS scale, comments.comments as column_comment FROM all_tab_cols cols, all_col_comments comments WHERE cols.owner = 'DEV_SCHEMA' AND cols.table_name = 'USERS' AND cols.hidden_column = 'NO' AND cols.owner = comments.owner AND cols.table_name = comments.table_name AND cols.column_name = comments.column_name ORDER BY cols.column_id
D, [2019-12-03T11:51:01.795284 #23130] DEBUG -- :   Primary Keys (2.4ms)  SELECT cc.column_name FROM all_constraints c, all_cons_columns cc WHERE c.owner = :owner AND c.table_name = :table_name AND c.constraint_type = 'P' AND cc.owner = c.owner AND cc.constraint_name = c.constraint_name order by cc.position  
D, [2019-12-03T11:51:01.797253 #23130] DEBUG -- :   temporary table (1.7ms)  SELECT temporary FROM all_tables WHERE table_name = :table_name and owner = SYS_CONTEXT('userenv', 'session_user')  
D, [2019-12-03T11:51:01.863809 #23130] DEBUG -- :   Table comment (65.1ms)  SELECT comments FROM all_tab_comments WHERE owner = :owner AND table_name = :table_name  
D, [2019-12-03T11:51:01.867164 #23130] DEBUG -- :   default tablespace (1.3ms)  SELECT LOWER(default_tablespace) FROM user_users WHERE username = SYS_CONTEXT('userenv', 'current_schema')
D, [2019-12-03T11:51:04.780922 #23130] DEBUG -- :   indexes (2913.5ms)  SELECT LOWER(i.table_name) AS table_name, LOWER(i.index_name) AS index_name, i.uniqueness, i.index_type, i.ityp_owner, i.ityp_name, i.parameters, LOWER(i.tablespace_name) AS tablespace_name, LOWER(c.column_name) AS column_name, e.column_expression, atc.virtual_column FROM all_indexes i JOIN all_ind_columns c ON c.index_name = i.index_name AND c.index_owner = i.owner LEFT OUTER JOIN all_ind_expressions e ON e.index_name = i.index_name AND e.index_owner = i.owner AND e.column_position = c.column_position LEFT OUTER JOIN all_tab_cols atc ON i.table_name = atc.table_name AND c.column_name = atc.column_name AND i.owner = atc.owner AND atc.hidden_column = 'NO' WHERE i.owner = :owner AND i.table_owner = :owner AND i.table_name = :table_name AND NOT EXISTS (SELECT uc.index_name FROM all_constraints uc WHERE uc.index_name = i.index_name AND uc.owner = i.owner AND uc.constraint_type = 'P') ORDER BY i.index_name, c.column_position  
D, [2019-12-03T11:51:04.784414 #23130] DEBUG -- :   default tablespace (1.6ms)  SELECT LOWER(default_tablespace) FROM user_users WHERE username = SYS_CONTEXT('userenv', 'current_schema')
D, [2019-12-03T11:51:07.819743 #23130] DEBUG -- :   indexes (3035.1ms)  SELECT LOWER(i.table_name) AS table_name, LOWER(i.index_name) AS index_name, i.uniqueness, i.index_type, i.ityp_owner, i.ityp_name, i.parameters, LOWER(i.tablespace_name) AS tablespace_name, LOWER(c.column_name) AS column_name, e.column_expression, atc.virtual_column FROM all_indexes i JOIN all_ind_columns c ON c.index_name = i.index_name AND c.index_owner = i.owner LEFT OUTER JOIN all_ind_expressions e ON e.index_name = i.index_name AND e.index_owner = i.owner AND e.column_position = c.column_position LEFT OUTER JOIN all_tab_cols atc ON i.table_name = atc.table_name AND c.column_name = atc.column_name AND i.owner = atc.owner AND atc.hidden_column = 'NO' WHERE i.owner = :owner AND i.table_owner = :owner AND i.table_name = :table_name AND NOT EXISTS (SELECT uc.index_name FROM all_constraints uc WHERE uc.index_name = i.index_name AND uc.owner = i.owner AND uc.constraint_type = 'P') ORDER BY i.index_name, c.column_position  
D, [2019-12-03T11:51:07.822981 #23130] DEBUG -- :   Primary Key Trigger (1.6ms)  SELECT trigger_name FROM all_triggers WHERE owner = :owner AND trigger_name = :trigger_name AND table_owner = :owner AND table_name = :table_name AND status = 'ENABLED'  
D, [2019-12-03T11:51:07.887763 #23130] DEBUG -- :   Foreign Keys (63.7ms)  SELECT r.table_name to_table ,rc.column_name references_column ,cc.column_name ,c.constraint_name name ,c.delete_rule FROM all_constraints c, all_cons_columns cc, all_constraints r, all_cons_columns rc WHERE c.owner = :owner AND c.table_name = :desc_table_name AND c.constraint_type = 'R' AND cc.owner = c.owner AND cc.constraint_name = c.constraint_name AND r.constraint_name = c.r_constraint_name AND r.owner = c.owner AND rc.owner = r.owner AND rc.constraint_name = r.constraint_name AND rc.position = cc.position ORDER BY name, to_table, column_name, references_column  
D, [2019-12-03T11:51:07.896157 #23130] DEBUG -- :   Foreign Keys (6.1ms)  SELECT r.table_name to_table ,rc.column_name references_column ,cc.column_name ,c.constraint_name name ,c.delete_rule FROM all_constraints c, all_cons_columns cc, all_constraints r, all_cons_columns rc WHERE c.owner = :owner AND c.table_name = :desc_table_name AND c.constraint_type = 'R' AND cc.owner = c.owner AND cc.constraint_name = c.constraint_name AND r.constraint_name = c.r_constraint_name AND r.owner = c.owner AND rc.owner = r.owner AND rc.constraint_name = r.constraint_name AND rc.position = cc.position ORDER BY name, to_table, column_name, references_column  
D, [2019-12-03T11:51:07.902299 #23130] DEBUG -- :   Foreign Keys (4.4ms)  SELECT r.table_name to_table ,rc.column_name references_column ,cc.column_name ,c.constraint_name name ,c.delete_rule FROM all_constraints c, all_cons_columns cc, all_constraints r, all_cons_columns rc WHERE c.owner = :owner AND c.table_name = :desc_table_name AND c.constraint_type = 'R' AND cc.owner = c.owner AND cc.constraint_name = c.constraint_name AND r.constraint_name = c.r_constraint_name AND r.owner = c.owner AND rc.owner = r.owner AND rc.constraint_name = r.constraint_name AND rc.position = cc.position ORDER BY name, to_table, column_name, references_column  
D, [2019-12-03T11:51:07.909708 #23130] DEBUG -- :   Foreign Keys (5.6ms)  SELECT r.table_name to_table ,rc.column_name references_column ,cc.column_name ,c.constraint_name name ,c.delete_rule FROM all_constraints c, all_cons_columns cc, all_constraints r, all_cons_columns rc WHERE c.owner = :owner AND c.table_name = :desc_table_name AND c.constraint_type = 'R' AND cc.owner = c.owner AND cc.constraint_name = c.constraint_name AND r.constraint_name = c.r_constraint_name AND r.owner = c.owner AND rc.owner = r.owner AND rc.constraint_name = r.constraint_name AND rc.position = cc.position ORDER BY name, to_table, column_name, references_column  
D, [2019-12-03T11:51:07.916429 #23130] DEBUG -- :   Foreign Keys (4.4ms)  SELECT r.table_name to_table ,rc.column_name references_column ,cc.column_name ,c.constraint_name name ,c.delete_rule FROM all_constraints c, all_cons_columns cc, all_constraints r, all_cons_columns rc WHERE c.owner = :owner AND c.table_name = :desc_table_name AND c.constraint_type = 'R' AND cc.owner = c.owner AND cc.constraint_name = c.constraint_name AND r.constraint_name = c.r_constraint_name AND r.owner = c.owner AND rc.owner = r.owner AND rc.constraint_name = r.constraint_name AND rc.position = cc.position ORDER BY name, to_table, column_name, references_column  
D, [2019-12-03T11:51:07.922451 #23130] DEBUG -- :   Foreign Keys (4.2ms)  SELECT r.table_name to_table ,rc.column_name references_column ,cc.column_name ,c.constraint_name name ,c.delete_rule FROM all_constraints c, all_cons_columns cc, all_constraints r, all_cons_columns rc WHERE c.owner = :owner AND c.table_name = :desc_table_name AND c.constraint_type = 'R' AND cc.owner = c.owner AND cc.constraint_name = c.constraint_name AND r.constraint_name = c.r_constraint_name AND r.owner = c.owner AND rc.owner = r.owner AND rc.constraint_name = r.constraint_name AND rc.position = cc.position ORDER BY name, to_table, column_name, references_column  
D, [2019-12-03T11:51:07.928488 #23130] DEBUG -- :   Foreign Keys (4.2ms)  SELECT r.table_name to_table ,rc.column_name references_column ,cc.column_name ,c.constraint_name name ,c.delete_rule FROM all_constraints c, all_cons_columns cc, all_constraints r, all_cons_columns rc WHERE c.owner = :owner AND c.table_name = :desc_table_name AND c.constraint_type = 'R' AND cc.owner = c.owner AND cc.constraint_name = c.constraint_name AND r.constraint_name = c.r_constraint_name AND r.owner = c.owner AND rc.owner = r.owner AND rc.constraint_name = r.constraint_name AND rc.position = cc.position ORDER BY name, to_table, column_name, references_column  
D, [2019-12-03T11:51:07.935357 #23130] DEBUG -- :   Foreign Keys (4.7ms)  SELECT r.table_name to_table ,rc.column_name references_column ,cc.column_name ,c.constraint_name name ,c.delete_rule FROM all_constraints c, all_cons_columns cc, all_constraints r, all_cons_columns rc WHERE c.owner = :owner AND c.table_name = :desc_table_name AND c.constraint_type = 'R' AND cc.owner = c.owner AND cc.constraint_name = c.constraint_name AND r.constraint_name = c.r_constraint_name AND r.owner = c.owner AND rc.owner = r.owner AND rc.constraint_name = r.constraint_name AND rc.position = cc.position ORDER BY name, to_table, column_name, references_column  
D, [2019-12-03T11:51:07.941469 #23130] DEBUG -- :   Foreign Keys (4.4ms)  SELECT r.table_name to_table ,rc.column_name references_column ,cc.column_name ,c.constraint_name name ,c.delete_rule FROM all_constraints c, all_cons_columns cc, all_constraints r, all_cons_columns rc WHERE c.owner = :owner AND c.table_name = :desc_table_name AND c.constraint_type = 'R' AND cc.owner = c.owner AND cc.constraint_name = c.constraint_name AND r.constraint_name = c.r_constraint_name AND r.owner = c.owner AND rc.owner = r.owner AND rc.constraint_name = r.constraint_name AND rc.position = cc.position ORDER BY name, to_table, column_name, references_column  
D, [2019-12-03T11:51:07.947645 #23130] DEBUG -- :   Foreign Keys (4.3ms)  SELECT r.table_name to_table ,rc.column_name references_column ,cc.column_name ,c.constraint_name name ,c.delete_rule FROM all_constraints c, all_cons_columns cc, all_constraints r, all_cons_columns rc WHERE c.owner = :owner AND c.table_name = :desc_table_name AND c.constraint_type = 'R' AND cc.owner = c.owner AND cc.constraint_name = c.constraint_name AND r.constraint_name = c.r_constraint_name AND r.owner = c.owner AND rc.owner = r.owner AND rc.constraint_name = r.constraint_name AND rc.position = cc.position ORDER BY name, to_table, column_name, references_column  
D, [2019-12-03T11:51:08.112125 #23130] DEBUG -- :   synonyms (164.3ms)  SELECT synonym_name, table_owner, table_name, db_link FROM all_synonyms where owner = SYS_CONTEXT('userenv', 'session_user')

@jlahtinen
Copy link
Contributor

Hopefully this could help you to narrow reason for slowness:
We have

  • jruby-9.2.7.0
  • fresh install of 19.3 oracle
  • activerecord-oracle_enhanced-adapter 5.2.6
  • rails 5.2.3

Our 94 table dump takes 45s. All indexes queries takes about 70ms

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests