Skip to content

Commit

Permalink
Make export timeout configurable via --export-timeout <secs>
Browse files Browse the repository at this point in the history
  • Loading branch information
@chadlwilson
chadlwilson committed Jul 17, 2023
1 parent 5743472 commit 7ce3921
Show file tree
Hide file tree
Showing 3 changed files with 23 additions and 18 deletions.
31 changes: 16 additions & 15 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -43,21 +43,22 @@ Follow detailed instructions in [Upgrading to GoCD 20.5.0](https://docs.gocd.org

# Command Arguments:

| Argument &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | Description |
|:---------------------------- |:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `source-db-url` | The source database url. Specify the existing GoCD database url. <br/> If none specified, it will default to `cruise` (this is the H2 database file without the `.h2.db` extension) in the current directory. See [Example database connection URLs](#example-database-connection-urls). |
| `source-db-driver-class` | The source database driver class. <br/> If none specified, it will choose the appropriate driver class based on the specified `--source-db-url`. See [Default database driver class](#default-database-driver-class). |
| `source-db-user` | The username of the source database. |
| `source-db-password` | The password of the source database. |
| `target-db-url` | The target database url. Specify the newly created database url where the data will be copied. See [Example database connection URLs](#example-database-connection-urls). |
| `target-db-driver-class` | The target database driver class. <br/> If none specified, it will choose the appropriate driver class based on the specified `--target-db-url`. See [Default database driver class](#default-database-driver-class). |
| `target-db-user` | The username of the target database. |
| `target-db-password` | The password of the target database. |
| `batch-size` | The number of records to `SELECT` from the source database to `INSERT` into the target database in each batch. <br/> **Default:** 100000 |
| `output` | The output SQL file. Specify `.gz` extension to enable gzip compression. |
| `insert` | Perform `INSERT` into target database. <br/> **Default:** false |
| `progress` | Show the progress of the export operation. <br/> **Default:** false |
| `threads` | Number of import threads. <br/> **Default:** the number of processor cores (up to 8) |
| Argument | Description |
|:-------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `source-db-url` | The source database url. Specify the existing GoCD database url. <br/> If none specified, it will default to `cruise` (this is the H2 database file without the `.h2.db` extension) in the current directory. See [Example database connection URLs](#example-database-connection-urls). |
| `source-db-driver-class` | The source database driver class. <br/> If none specified, it will choose the appropriate driver class based on the specified `--source-db-url`. See [Default database driver class](#default-database-driver-class). |
| `source-db-user` | The username of the source database. |
| `source-db-password` | The password of the source database. |
| `target-db-url` | The target database url. Specify the newly created database url where the data will be copied. See [Example database connection URLs](#example-database-connection-urls). |
| `target-db-driver-class` | The target database driver class. <br/> If none specified, it will choose the appropriate driver class based on the specified `--target-db-url`. See [Default database driver class](#default-database-driver-class). |
| `target-db-user` | The username of the target database. |
| `target-db-password` | The password of the target database. |
| `batch-size` | The number of records to `SELECT` from the source database to `INSERT` into the target database in each batch. <br/> **Default:** 100000 |
| `output` | The output SQL file. Specify `.gz` extension to enable gzip compression. |
| `insert` | Perform `INSERT` into target database. <br/> **Default:** false |
| `progress` | Show the progress of the export operation. <br/> **Default:** false |
| `threads` | Number of import threads. <br/> **Default:** the number of processor cores (up to 8) |
| `export-timeout` | Number of seconds to allow data to be exported from source to target database before timing out. <br/> **Default:** 1800 secs (30 minutes) |


## Example database connection URLs:
Expand Down
5 changes: 2 additions & 3 deletions src/main/java/com/thoughtworks/go/dbsync/DbSync.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,6 @@ public class DbSync {
System.setProperty("org.jooq.no-tips", "true");
}
static final Logger LOG = LoggerFactory.getLogger(DbSync.class);
private static final long EXPORT_TIMEOUT_SECONDS = TimeUnit.MINUTES.toSeconds(90);

private final Args args;

Expand Down Expand Up @@ -206,8 +205,8 @@ private void doExport(BasicDataSource sourceDataSource, DataSource targetDataSou
}));
LOG.debug("Shutting down thread pool executor");
executor.shutdown();
if (!executor.awaitTermination(EXPORT_TIMEOUT_SECONDS, TimeUnit.SECONDS)) {
throw new InterruptedException(String.format("Timed out after [%s] seconds waiting for DB migration to complete", EXPORT_TIMEOUT_SECONDS));
if (!executor.awaitTermination(args.exportTimeoutSeconds, TimeUnit.SECONDS)) {
throw new InterruptedException(String.format("Timed out after [%s] seconds waiting for DB migration to complete. You may want to consider increasing --export-timeout or allocating more resources.", args.exportTimeoutSeconds));
}
} catch (RuntimeException e) {
LOG.error(null, e);
Expand Down
5 changes: 5 additions & 0 deletions src/main/java/com/thoughtworks/go/dbsync/cli/Args.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@
import com.beust.jcommander.Parameter;
import com.beust.jcommander.Parameters;

import java.util.concurrent.TimeUnit;

@Parameters(separators = "=")
public class Args {
@Parameter(names = {"--help", "-h"}, description = "Prints help.", help = true, order = 100)
Expand Down Expand Up @@ -62,4 +64,7 @@ public class Args {

@Parameter(names = {"--threads", "-t"}, description = "Number of import threads. Defaults to number of processors (max of 8).", order = 1400)
public int threads = Math.max(8, Runtime.getRuntime().availableProcessors());

@Parameter(names = {"--export-timeout"}, description = "Number of seconds to allow data to be exported from source to target database before timing out.", order = 1500)
public long exportTimeoutSeconds = TimeUnit.MINUTES.toSeconds(30);
}

0 comments on commit 7ce3921

Please sign in to comment.