Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Formats maximum count exceeded on TPC-C #3924

Closed
kyukhin opened this issue Jan 9, 2019 · 1 comment
Closed

Formats maximum count exceeded on TPC-C #3924

kyukhin opened this issue Jan 9, 2019 · 1 comment
Assignees
@kyukhin
Labels
bug Something isn't working
Milestone
2.1.2

Comments

@kyukhin
Copy link
Contributor

kyukhin commented Jan 9, 2019

Tarantool version: 2.1

OS version: any

Bug description:
When massive parallel SQL queries are executed by Tarantool, there is
a possibility for tuple formats maximum count to be exceeded.

This is due to the fact, that SQL queries are eager for ephemeral spaces
(to perform JOINs) and count of such spaces which exists simultaneously
might be big.

Even more, when space is about to be deleted,
formats are freed on lazy basis, i.e. they are only freed when
corresponding tuple is freed, which in turn freed only when we're out of space.

Steps to reproduce: Run TPC-C w/ at least 64 connections.

Optional (but very desirable):

  • coredump
  • backtrace
  • netstat
@kyukhin kyukhin added the bug Something isn't working label Jan 9, 2019
@kyukhin kyukhin added this to the 2.1.1 milestone Jan 9, 2019
@kyukhin kyukhin self-assigned this Jan 9, 2019
@kyukhin
Copy link
Contributor Author

kyukhin commented Jan 9, 2019

Current proposal:
in tuple_format_new() calculate a hash value of fields and look for the hash in a table
of formats. This should be done when format's dictionary is NULL only.

@kyukhin kyukhin reopened this Jan 22, 2019
kyukhin added a commit that referenced this issue Jan 22, 2019
@kyukhin
Allow to reuse tuple_formats for ephemeral spaces
29b9663 
Since under heavy load with SQL queries ephemeral
spaces might be extensively used it is possible to run out
of tuple_formats for such spaces. This occurs because
tuple_format is not immediately deleted when ephemeral space is
dropped. Its removel is postponed instead and triggered only
when tuple memory is exhausted.
As far as there's no way to alter ephemeral space's format,
let's re-use them for multiple epehemral spaces in case
they're identical.

Closes #3924
kyukhin added a commit that referenced this issue Jan 24, 2019
@kyukhin
Allow to reuse tuple_formats for ephemeral spaces
60bf031 
Since under heavy load with SQL queries ephemeral
spaces might be extensively used it is possible to run out
of tuple_formats for such spaces. This occurs because
tuple_format is not immediately deleted when ephemeral space is
dropped. Its removel is postponed instead and triggered only
when tuple memory is exhausted.
As far as there's no way to alter ephemeral space's format,
let's re-use them for multiple epehemral spaces in case
they're identical.

Closes #3924
kyukhin added a commit that referenced this issue Jan 24, 2019
@kyukhin
Allow to reuse tuple_formats for ephemeral spaces
a612285 
Since under heavy load with SQL queries ephemeral
spaces might be extensively used it is possible to run out
of tuple_formats for such spaces. This occurs because
tuple_format is not immediately deleted when ephemeral space is
dropped. Its removel is postponed instead and triggered only
when tuple memory is exhausted.
As far as there's no way to alter ephemeral space's format,
let's re-use them for multiple epehemral spaces in case
they're identical.

Closes #3924
kyukhin added a commit that referenced this issue Jan 25, 2019
@kyukhin
Allow to reuse tuple_formats for ephemeral spaces
3518686 
Since under heavy load with SQL queries ephemeral
spaces might be extensively used it is possible to run out
of tuple_formats for such spaces. This occurs because
tuple_format is not immediately deleted when ephemeral space is
dropped. Its removel is postponed instead and triggered only
when tuple memory is exhausted.
As far as there's no way to alter ephemeral space's format,
let's re-use them for multiple epehemral spaces in case
they're identical.

Closes #3924
@locker locker closed this as completed in dbbd931 Jan 25, 2019
@kyukhin kyukhin added the tmp label Mar 29, 2019
@kshcherbatov kshcherbatov mentioned this issue Apr 5, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kyukhin kyukhin

Labels
bug Something isn't working
Projects
None yet
Milestone
2.1.2
Development

No branches or pull requests
1 participant
@kyukhin