Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

parallelise population of tables blocks and transactions #128

Closed
adamkrellenstein opened this issue Jun 15, 2014 · 3 comments
Closed

parallelise population of tables blocks and transactions #128

adamkrellenstein opened this issue Jun 15, 2014 · 3 comments
Labels
performance

Comments

@adamkrellenstein
Copy link
Member

No description provided.

@adamkrellenstein adamkrellenstein mentioned this issue Aug 14, 2014
Closed
@mcelrath
Copy link

I have a data point from someone else trying to read out the entire blockchain via JSON that parallelizing actually hurt performance (if true there would be another bottleneck inside bitcoind). But I'll give this a stab, I recently wrote a bunch of code using the new python 3.4 asyncio and aiohttp. If someone wants to briefly point some details, I could get started quicker.

@adamkrellenstein
Copy link
Member Author

The slow operation is calling getrawtransaction on every transaction in a given block, for every block since about 278000. Also, a small number of those calls (really only the Counterparty transactions) require a few follow-up getrawtransaction calls on some other related transactions elsewhere in the blockchain. I think it's the first set of calls that's the problem, though.

@adamkrellenstein
Copy link
Member Author

See #261.

jotapea added a commit to CNTRPRTY/counterparty-lib that referenced this issue Feb 14, 2023
@jotapea
the future is now: moving counterparty-cli into counterparty-lib
1e1b997 
Last commit from github.com/CounterpartyXCP/counterparty-cli repo:
"""
Merge pull request CounterpartyXCP#128 from pataegrillo/master

- Adding checkdb as a new parameter that will be used by counterparty to perform a database integrity check
Commit:
0f7a0a2 [0f7a0a2]
"""
jotapea added a commit to CNTRPRTY/counterparty-lib that referenced this issue Feb 14, 2023
@jotapea
the future is now: moving counterparty-cli into counterparty-lib
b9b793f 
Last commit from github.com/CounterpartyXCP/counterparty-cli repo:
"""
Merge pull request CounterpartyXCP#128 from pataegrillo/master

- Adding checkdb as a new parameter that will be used by counterparty to perform a database integrity check
Commit:
0f7a0a2 [0f7a0a2]
"""
jotapea added a commit to CNTRPRTY/counterparty-lib that referenced this issue Mar 6, 2023
@jotapea
the future is now: moving counterparty-cli into counterparty-lib
3f226ed 
Last commit from github.com/CounterpartyXCP/counterparty-cli repo:
"""
Merge pull request CounterpartyXCP#128 from pataegrillo/master

- Adding checkdb as a new parameter that will be used by counterparty to perform a database integrity check
Commit:
0f7a0a2 [0f7a0a2]
"""

(cherry picked from commit b9b793f)
jotapea added a commit to CNTRPRTY/counterparty-lib that referenced this issue Mar 17, 2023
@jotapea
the future is now: moving counterparty-cli into counterparty-lib
67fd96a 
Last commit from github.com/CounterpartyXCP/counterparty-cli repo:
"""
Merge pull request CounterpartyXCP#128 from pataegrillo/master

- Adding checkdb as a new parameter that will be used by counterparty to perform a database integrity check
Commit:
0f7a0a2 [0f7a0a2]
"""

(cherry picked from commit b9b793f)
(cherry picked from commit 3f226ed)
jotapea added a commit to CNTRPRTY/counterparty-lib that referenced this issue Jun 17, 2023
@jotapea
the future is now: moving counterparty-cli into counterparty-lib
812587d 
Last commit from github.com/CounterpartyXCP/counterparty-cli repo:
"""
Merge pull request CounterpartyXCP#128 from pataegrillo/master

- Adding checkdb as a new parameter that will be used by counterparty to perform a database integrity check
Commit:
0f7a0a2 [0f7a0a2]
"""
jotapea added a commit to CNTRPRTY/counterparty-lib that referenced this issue Nov 30, 2023
@jotapea
the future is now: moving counterparty-cli into counterparty-lib
1103c2a 
Last commit from github.com/CounterpartyXCP/counterparty-cli repo:
"""
Merge pull request CounterpartyXCP#128 from pataegrillo/master

- Adding checkdb as a new parameter that will be used by counterparty to perform a database integrity check
Commit:
0f7a0a2 [0f7a0a2]
"""
jotapea added a commit to CNTRPRTY/counterparty-lib that referenced this issue Dec 1, 2023
@jotapea
the future is now: moving counterparty-cli into counterparty-lib
9c88576 
Last commit from github.com/CounterpartyXCP/counterparty-cli repo:
"""
Merge pull request CounterpartyXCP#128 from pataegrillo/master

- Adding checkdb as a new parameter that will be used by counterparty to perform a database integrity check
Commit:
0f7a0a2 [0f7a0a2]
"""
jotapea added a commit to CNTRPRTY/counterparty-lib that referenced this issue Dec 2, 2023
@jotapea
the future is now: moving counterparty-cli into counterparty-lib
e3ed182 
Last commit from github.com/CounterpartyXCP/counterparty-cli repo:
"""
Merge pull request CounterpartyXCP#128 from pataegrillo/master

- Adding checkdb as a new parameter that will be used by counterparty to perform a database integrity check
Commit:
0f7a0a2 [0f7a0a2]
"""
jotapea added a commit to CNTRPRTY/counterparty-lib that referenced this issue Dec 2, 2023
@jotapea
the future is now: moving counterparty-cli into counterparty-lib
69a68e3 
Last commit from github.com/CounterpartyXCP/counterparty-cli repo:
"""
Merge pull request CounterpartyXCP#128 from pataegrillo/master

- Adding checkdb as a new parameter that will be used by counterparty to perform a database integrity check
Commit:
0f7a0a2 [0f7a0a2]
"""
jotapea added a commit to CNTRPRTY/counterparty-lib that referenced this issue Dec 6, 2023
@jotapea
the future is now: moving counterparty-cli into counterparty-lib
ff74284 
Last commit from github.com/CounterpartyXCP/counterparty-cli repo:
"""
Merge pull request CounterpartyXCP#128 from pataegrillo/master

- Adding checkdb as a new parameter that will be used by counterparty to perform a database integrity check
Commit:
0f7a0a2 [0f7a0a2]
"""
jotapea added a commit to CNTRPRTY/counterparty-lib that referenced this issue Dec 8, 2023
@jotapea
the future is now: moving counterparty-cli into counterparty-lib
630bc4f 
Last commit from github.com/CounterpartyXCP/counterparty-cli repo:
"""
Merge pull request CounterpartyXCP#128 from pataegrillo/master

- Adding checkdb as a new parameter that will be used by counterparty to perform a database integrity check
Commit:
0f7a0a2 [0f7a0a2]
"""
ouziel-slama pushed a commit that referenced this issue Jan 26, 2024
@jdogresorg
Merge pull request #128 from pataegrillo/master
0f7a0a2 
- Adding checkdb as a new parameter that will be used by counterparty to perform a database integrity check
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
performance
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mcelrath @adamkrellenstein