Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: parquet cache #349

Closed
wants to merge 3 commits into from
Closed

Conversation

jiacai2050
Copy link
Contributor

@jiacai2050 jiacai2050 commented Oct 27, 2022

Which issue does this PR close?

Closes

Rationale for this change

After #14, parquet reader will read all bytes out, ignoring whether it's already cached.

What changes are included in this PR?

Are there any user-facing changes?

No

How does this change test

UT

@jiacai2050
Copy link
Contributor Author

Close since this PR has flaw which cannot be avoided in current design.

thread 'tests::read_write_test::test_table_write_read_reverse_after_flush_rocks' panicked at 'Cannot start a runtime from within a runtime. This happens because a function (like `block_on`) attempted to block the current thread while the thread is being used to drive asynchronous tasks.', /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.1/src/runtime/thread_pool/mod.rs:89:25
stack backtrace:
   0: std::panicking::begin_panic
             at /rustc/d394408fb38c4de61f765a3ed5189d2731a1da91/library/std/src/panicking.rs:616:12
   1: tokio::runtime::enter::enter
             at /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.1/src/runtime/enter.rs:40:9
   2: tokio::runtime::thread_pool::ThreadPool::block_on
             at /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.1/src/runtime/thread_pool/mod.rs:89:25
   3: tokio::runtime::Runtime::block_on
             at /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.1/src/runtime/mod.rs:484:43
   4: common_util::runtime::Runtime::block_on
             at /home/runner/work/ceresdb/ceresdb/common_util/src/runtime/mod.rs:82:9
   5: analytic_engine::sst::parquet::reader::GetResultReader::init_if_necessary
             at ./src/sst/parquet/reader.rs:68:9
   6: <analytic_engine::sst::parquet::reader::GetResultReader as parquet::file::reader::ChunkReader>::get_read
             at ./src/sst/parquet/reader.rs:99:9
   7: parquet::file::footer::parse_metadata
             at /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/parquet-23.0.0/src/file/footer.rs:48:5
   8: parquet_ext::serialized_reader::CacheableSerializedFileReader<R>::new
             at /home/runner/work/ceresdb/ceresdb/components/parquet_ext/src/serialized_reader.rs:85:42
   9: analytic_engine::sst::parquet::reader::read_sst_meta::{{closure}}
             at ./src/sst/parquet/reader.rs:155:23

@jiacai2050 jiacai2050 closed this Oct 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant