Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

influxd restore fails for a database name containing periods #16753

Open
SpComb opened this issue Feb 6, 2020 · 3 comments
Open

influxd restore fails for a database name containing periods #16753

SpComb opened this issue Feb 6, 2020 · 3 comments
Labels

Comments

@SpComb
Copy link

SpComb commented Feb 6, 2020

If you have a database name containing periods, then it is possible to backup the database, but it is not possible to restore the backup.

Steps to reproduce:

Create a database with a . in the name, take a backup, and attempt to restore it

  1. CREATE DATABASE "hcloud-infra.telegraf"
  2. influxd backup -host infra-influxdb1:8088 -database hcloud-infra.telegraf hcloud-infra.telegraf
  3. influxd restore -online -host infra-influxdb1:8088 -db hcloud-infra.telegraf -newdb 'hcloud-infra.telegraf-restore' hcloud-infra.telegraf

Note that the backup shard files will contain an extra ., because the database name is included at the start of the filename:

bash-4.4$ influxd backup -host infra-influxdb1:8088 -database hcloud-infra.telegraf hcloud-infra.telegraf
2020/02/06 13:43:24 backing up metastore to hcloud-infra.telegraf/meta.00
2020/02/06 13:43:24 backing up db=hcloud-infra.telegraf
2020/02/06 13:43:24 backing up db=hcloud-infra.telegraf rp=default shard=359 to hcloud-infra.telegraf/hcloud-infra.telegraf.default.00359.00 since 0001-01-01T00:00:00Z
2020/02/06 13:43:25 backing up db=hcloud-infra.telegraf rp=default shard=362 to hcloud-infra.telegraf/hcloud-infra.telegraf.default.00362.00 since 0001-01-01T00:00:00Z
2020/02/06 13:43:25 backing up db=hcloud-infra.telegraf rp=default shard=365 to hcloud-infra.telegraf/hcloud-infra.telegraf.default.00365.00 since 0001-01-01T00:00:00Z

Expected behavior:
The database shards should be restored to a new hcloud-infra.telegraf-restore database.

Actual behavior:
The metastore snapshot is succesfully restored and the new hcloud-infra.telegraf-restore database is created, but the shard restore fails and crashes with an error:

bash-4.4$ influxd restore -online -host infra-influxdb1:8088 -db hcloud-infra.telegraf -newdb 'hcloud-infra.telegraf-restore' hcloud-infra.telegraf
2020/02/06 13:48:10 Using metastore snapshot: hcloud-infra.telegraf/meta.00
2020/02/06 13:48:11 Restoring live from backup hcloud-infra.telegraf/hcloud-infra.telegraf.*
2020/02/06 13:48:11 Skipping mis-named backup file: hcloud-infra.telegraf/hcloud-infra.telegraf.default.00359.00
2020/02/06 13:48:11 error updating shards: strconv.ParseUint: parsing "default": invalid syntax
restore: strconv.ParseUint: parsing "default": invalid syntax

Environment info:

  • System info: Linux 3.10.0-1062.12.1.el7.x86_64 x86_64
  • InfluxDB version: InfluxDB v1.7.8 (git: 1.7 ff383cdc0420217e3460dabe17db54f8557d95b6)
  • Other relevant environment details: Using the Docker influxdb:1.7.8 image
@SpComb
Copy link
Author

SpComb commented Feb 6, 2020

It's pretty clear from the influxd restore source code why this is happening, with strings.Split(filepath.Base(fn), "."):

parts := strings.Split(filepath.Base(fn), ".")
if len(parts) != 4 {
cmd.StderrLogger.Printf("Skipping mis-named backup file: %s", fn)
}
shardID, err := strconv.ParseUint(parts[2], 10, 64)
if err != nil {
return err
}

Because the uploadShardsLegacy only happens for a single database at a time, this could perhaps be fixed by removing the cmd.sourceDatabase prefix from the filename, and then only split the remaining parts?

@SpComb
Copy link
Author

SpComb commented Feb 6, 2020

There's probably a second bug in the error-handling here, the "Skipping mis-named backup file ..." case is missing a continue, so it's crashing hard on the strconv.ParseUint instead of skipping the backup file as intended.

@dgnorton dgnorton removed their assignment Feb 14, 2020
@Elbehery Elbehery self-assigned this Feb 18, 2020
@rodrigorega
Copy link

I have this problem too restoring Influx 1.8 databases with periods:

# /usr/bin/influxd restore -db telegraf_redacted01.ab.redacted02.com -datadir /var/lib/influxdb/data /mnt/backups/20220329-130428/data/telegraf_redacted01.ab.redacted02.com
2022/03/30 11:52:35 Restoring offline from backup /mnt/backups/20220329-130428/data/telegraf_redacted01.ab.redacted02.com/telegraf_redacted01.ab.redacted02.com.*
restore: backup tarfile name incorrect format

I need to restore a full backup into a new server, "-portable" also not works for me because is not backing up users. Is there any workaround?

Regards

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants