Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove v0 version of the database #5698

Merged
merged 5 commits into from
Jul 13, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@

### Breaking Changes

- Removed support for version 0 of the database as it is no longer used by any active node.

### Additions and Improvements
- EvmTool now executes the `execution-spec-tests` via the `t8n` and `b11r`. See the [README](ethereum/evmtool/README.md) in EvmTool for more instructions.
- Improve lifecycle management of the transaction pool [#5634](https://github.com/hyperledger/besu/pull/5634)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -80,16 +80,6 @@ public BackupRoundTripAcceptanceTest(
public static Object[][] getParameters() {
return new Object[][] {
// First 10 blocks of ropsten
new Object[] {
"Before versioning was enabled",
"version0",
0xA,
singletonList(
new AccountData(
"0xd1aeb42885a43b72b518182ef893125814811048",
BigInteger.valueOf(0xA),
Wei.fromHexString("0x2B5E3AF16B1880000"))),
},
new Object[] {
"After versioning was enabled and using multiple RocksDB columns",
"version1",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,16 +56,6 @@ public DatabaseMigrationAcceptanceTest(
public static Object[][] getParameters() {
return new Object[][] {
// First 10 blocks of ropsten
new Object[] {
"Before versioning was enabled",
"version0",
0xA,
singletonList(
new AccountData(
"0xd1aeb42885a43b72b518182ef893125814811048",
BigInteger.valueOf(0xA),
Wei.fromHexString("0x2B5E3AF16B1880000"))),
},
new Object[] {
"After versioning was enabled and using multiple RocksDB columns",
"version1",
Expand Down
Binary file not shown.
Original file line number Diff line number Diff line change
Expand Up @@ -24,16 +24,19 @@
import org.hyperledger.besu.ethereum.core.Difficulty;
import org.hyperledger.besu.ethereum.core.ExecutionContextTestFixture;
import org.hyperledger.besu.ethereum.core.MessageFrameTestFixture;
import org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueSegmentIdentifier;
import org.hyperledger.besu.evm.frame.MessageFrame;
import org.hyperledger.besu.metrics.noop.NoOpMetricsSystem;
import org.hyperledger.besu.plugin.services.storage.KeyValueStorage;
import org.hyperledger.besu.plugin.services.storage.rocksdb.RocksDBMetricsFactory;
import org.hyperledger.besu.plugin.services.storage.rocksdb.configuration.RocksDBConfigurationBuilder;
import org.hyperledger.besu.plugin.services.storage.rocksdb.unsegmented.RocksDBKeyValueStorage;
import org.hyperledger.besu.plugin.services.storage.rocksdb.segmented.OptimisticRocksDBColumnarKeyValueStorage;
import org.hyperledger.besu.services.kvstore.SnappableSegmentedKeyValueStorageAdapter;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.List;

import com.google.common.io.MoreFiles;
import com.google.common.io.RecursiveDeleteOption;
Expand All @@ -58,12 +61,18 @@ private OperationBenchmarkHelper(

public static OperationBenchmarkHelper create() throws IOException {
final Path storageDirectory = Files.createTempDirectory("benchmark");
final KeyValueStorage keyValueStorage =
new RocksDBKeyValueStorage(
final OptimisticRocksDBColumnarKeyValueStorage optimisticRocksDBColumnarKeyValueStorage =
new OptimisticRocksDBColumnarKeyValueStorage(
Comment on lines +64 to +65
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Struggling to see the relevance of this change...KeyValueStorage still exists doesn't it? Or maybe we should delete KeyValueStorage too?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like the change is it's now a two step process - KeyValueStorage (which is the interface used by plugins) is created from this storage

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes we want to start a refactor of this part to be only segmented as we are only using that now. this one is the first PR and we will have more PRs after this one. We found a lot of code smell in this part when we wanted to change the code in order to have only one transaction to commit the worldstate

new RocksDBConfigurationBuilder().databaseDir(storageDirectory).build(),
List.of(KeyValueSegmentIdentifier.BLOCKCHAIN),
emptyList(),
new NoOpMetricsSystem(),
RocksDBMetricsFactory.PUBLIC_ROCKS_DB_METRICS);

final KeyValueStorage keyValueStorage =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

KeyValueStorage still used here @siladu

new SnappableSegmentedKeyValueStorageAdapter<>(
KeyValueSegmentIdentifier.BLOCKCHAIN, optimisticRocksDBColumnarKeyValueStorage);

final ExecutionContextTestFixture executionContext =
ExecutionContextTestFixture.builder().blockchainKeyValueStorage(keyValueStorage).build();
final MutableBlockchain blockchain = executionContext.getBlockchain();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ public class RocksDBKeyValuePrivacyStorageFactory implements PrivacyKeyValueStor
private static final Logger LOG =
LoggerFactory.getLogger(RocksDBKeyValuePrivacyStorageFactory.class);
private static final int DEFAULT_VERSION = 1;
private static final Set<Integer> SUPPORTED_VERSIONS = Set.of(0, 1);
private static final Set<Integer> SUPPORTED_VERSIONS = Set.of(1);

private static final String PRIVATE_DATABASE_PATH = "private";
private final RocksDBKeyValueStorageFactory publicFactory;
Expand Down Expand Up @@ -96,7 +96,7 @@ private int readDatabaseVersion(final BesuConfiguration commonConfiguration) thr
commonConfiguration.getStoragePath().resolve(PRIVATE_DATABASE_PATH).toFile().exists();
final int privacyDatabaseVersion;
if (privacyDatabaseExists) {
privacyDatabaseVersion = DatabaseMetadata.lookUpFrom(dataDir).maybePrivacyVersion().orElse(0);
privacyDatabaseVersion = DatabaseMetadata.lookUpFrom(dataDir).maybePrivacyVersion().orElse(1);
LOG.info(
"Existing private database detected at {}. Version {}", dataDir, privacyDatabaseVersion);
} else {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@
import org.hyperledger.besu.plugin.services.storage.rocksdb.segmented.OptimisticRocksDBColumnarKeyValueStorage;
import org.hyperledger.besu.plugin.services.storage.rocksdb.segmented.RocksDBColumnarKeyValueStorage;
import org.hyperledger.besu.plugin.services.storage.rocksdb.segmented.TransactionDBRocksDBColumnarKeyValueStorage;
import org.hyperledger.besu.plugin.services.storage.rocksdb.unsegmented.RocksDBKeyValueStorage;
import org.hyperledger.besu.services.kvstore.SegmentedKeyValueStorageAdapter;
import org.hyperledger.besu.services.kvstore.SnappableSegmentedKeyValueStorageAdapter;

Expand All @@ -50,15 +49,14 @@ public class RocksDBKeyValueStorageFactory implements KeyValueStorageFactory {

private static final Logger LOG = LoggerFactory.getLogger(RocksDBKeyValueStorageFactory.class);
private static final int DEFAULT_VERSION = 1;
private static final Set<Integer> SUPPORTED_VERSIONS = Set.of(0, 1, 2);
private static final Set<Integer> SUPPORTED_VERSIONS = Set.of(1, 2);
private static final String NAME = "rocksdb";
private final RocksDBMetricsFactory rocksDBMetricsFactory;

private final int defaultVersion;
private Integer databaseVersion;
private Boolean isSegmentIsolationSupported;
private RocksDBColumnarKeyValueStorage segmentedStorage;
private KeyValueStorage unsegmentedStorage;
private RocksDBConfiguration rocksDBConfiguration;

private final Supplier<RocksDBFactoryConfiguration> configuration;
Expand Down Expand Up @@ -163,17 +161,7 @@ public KeyValueStorage create(
// version. Introducing intermediate booleans that represent database properties and dispatching
// creation logic based on them is error-prone.
switch (databaseVersion) {
case 0 -> {
segmentedStorage = null;
if (unsegmentedStorage == null) {
unsegmentedStorage =
new RocksDBKeyValueStorage(
rocksDBConfiguration, metricsSystem, rocksDBMetricsFactory);
}
return unsegmentedStorage;
}
case 1, 2 -> {
unsegmentedStorage = null;
if (segmentedStorage == null) {
final List<SegmentIdentifier> segmentsForVersion =
segments.stream()
Expand Down Expand Up @@ -249,7 +237,7 @@ private void init(final BesuConfiguration commonConfiguration) {
}

private boolean requiresInit() {
return segmentedStorage == null && unsegmentedStorage == null;
return segmentedStorage == null;
}

private int readDatabaseVersion(final BesuConfiguration commonConfiguration) throws IOException {
Expand Down Expand Up @@ -283,9 +271,6 @@ private int readDatabaseVersion(final BesuConfiguration commonConfiguration) thr

@Override
public void close() throws IOException {
if (unsegmentedStorage != null) {
unsegmentedStorage.close();
}
if (segmentedStorage != null) {
segmentedStorage.close();
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ private static DatabaseMetadata resolveDatabaseMetadata(final File metadataFile)
try {
databaseMetadata = MAPPER.readValue(metadataFile, DatabaseMetadata.class);
} catch (FileNotFoundException fnfe) {
databaseMetadata = new DatabaseMetadata(0, 0);
databaseMetadata = new DatabaseMetadata(1, 1);
} catch (JsonProcessingException jpe) {
throw new IllegalStateException(
String.format("Invalid metadata file %s", metadataFile.getAbsolutePath()), jpe);
Expand Down

This file was deleted.