Skip to content

Commit

Permalink
Allow disabling space partitioning
Browse files Browse the repository at this point in the history
TSBS uses space partitioning with only 1 partitioning which is a
configuration that provides no benefits at all. It also doesn't
allow disabling space partitioning which is unfortunate as not all
timescaledb optimization work with space partitioning.

This patch adds support for creating hypertables without space
partitions by setttings `--partitions` to `0`. This patch also
changes the default for partitions to `0`.
  • Loading branch information
svenklemm authored and atanasovskib committed Feb 22, 2021
1 parent 3cc83b2 commit 259ec99
Show file tree
Hide file tree
Showing 5 changed files with 15 additions and 11 deletions.
2 changes: 1 addition & 1 deletion README.md
Expand Up @@ -269,7 +269,7 @@ required, with a gzipped data set as created in the instructions above:
cat /tmp/timescaledb-data.gz | gunzip | tsbs_load_timescaledb \
--postgres="sslmode=require" --host="my.tsdb.host" --port=5432 --pass="password" \
--user="benchmarkuser" --admin-db-name=defaultdb --workers=8 \
--in-table-partition-tag=true --partitions=1 --chunk-time=8h --write-profile= \
--in-table-partition-tag=true --chunk-time=8h --write-profile= \
--field-index-count=1 --do-create-db=true --force-text-format=false \
--do-abort-on-exist=false
```
Expand Down
8 changes: 3 additions & 5 deletions docs/timescaledb.md
Expand Up @@ -95,11 +95,9 @@ time.Duration string, meaning a number followed by a unit abbreviation
(s = seconds, m = minutes, h = hours), e.g., the default `12h` is 12 hours.
This should be adjusted based on the dataset size.

#### `-partitions` (type: `int`, default: `1`)
Number of space partitions for the primary tag. Increasing this from 1 may
be useful for larger number of devices, but further testing is still
needed.

#### `-partitions` (type: `int`, default: `0`)
Number of space partitions for the primary tag. Setting this to `0` will
disable space partitioning.

### Index related

Expand Down
12 changes: 9 additions & 3 deletions pkg/targets/timescaledb/creator.go
Expand Up @@ -171,9 +171,15 @@ func (d *dbCreator) createTableAndIndexes(dbBench *sql.DB, tableName string, fie

if d.opts.UseHypertable {
MustExec(dbBench, "CREATE EXTENSION IF NOT EXISTS timescaledb CASCADE")
MustExec(dbBench,
fmt.Sprintf("SELECT create_hypertable('%s'::regclass, 'time'::name, partitioning_column => '%s'::name, number_partitions => %v::smallint, chunk_time_interval => %d, create_default_indexes=>FALSE)",
tableName, "tags_id", d.opts.NumberPartitions, d.opts.ChunkTime.Nanoseconds()/1000))
if d.opts.NumberPartitions > 0 {
MustExec(dbBench,
fmt.Sprintf("SELECT create_hypertable('%s'::regclass, 'time'::name, partitioning_column => '%s'::name, number_partitions => %v::smallint, chunk_time_interval => %d, create_default_indexes=>FALSE)",
tableName, "tags_id", d.opts.NumberPartitions, d.opts.ChunkTime.Nanoseconds()/1000))
} else {
MustExec(dbBench,
fmt.Sprintf("SELECT create_hypertable('%s'::regclass, 'time'::name, chunk_time_interval => %d, create_default_indexes=>FALSE)",
tableName, d.opts.ChunkTime.Nanoseconds()/1000))
}
}
}

Expand Down
2 changes: 1 addition & 1 deletion pkg/targets/timescaledb/implemented_target.go
Expand Up @@ -51,7 +51,7 @@ func (t *timescaleTarget) TargetSpecificFlags(flagPrefix string, flagSet *pflag.
flagSet.Bool(flagPrefix+"use-jsonb-tags", false, "Whether tags should be stored as JSONB (instead of a separate table with schema)")
flagSet.Bool(flagPrefix+"in-table-partition-tag", false, "Whether the partition key (e.g. hostname) should also be in the metrics hypertable")

flagSet.Int(flagPrefix+"partitions", 1, "Number of partitions")
flagSet.Int(flagPrefix+"partitions", 0, "Number of partitions")
flagSet.Duration(flagPrefix+"chunk-time", 12*time.Hour, "Duration that each chunk should represent, e.g., 12h")

flagSet.Bool(flagPrefix+"time-index", true, "Whether to build an index on the time dimension")
Expand Down
2 changes: 1 addition & 1 deletion scripts/load/load_timescaledb.sh
Expand Up @@ -13,7 +13,7 @@ DATABASE_USER=${DATABASE_USER:-postgres}

# Load parameters - personal
CHUNK_TIME=${CHUNK_TIME:-8h}
PARTITIONS=${PARTITIONS:-1}
PARTITIONS=${PARTITIONS:-0}
HASH_WORKERS=${HASH_WORKERS:-false}
TIME_PARTITION_INDEX=${TIME_PARTITION_INDEX:-false}
PERF_OUTPUT=${PERF_OUTPUT:-}
Expand Down

0 comments on commit 259ec99

Please sign in to comment.