error compacting TSM files: open /***/000000064-000000004.tsm: no such file or directory engine=tsm

[hpbieker]

Bug report

System info:

Influx 1.3.7

Steps to reproduce:

Just wait for a compaction to run, and in some cases the error is shown in the log.

Expected behavior:

Influx should not try to open .tsm files that was never created.

Actual behavior:

From time to time I get error messages like:

error compacting TSM files: open /media/db_volume/data/XXX/autogen/5667/000000064-000000004.tsm: no such file or directory engine=tsm

Additional Info:
From the log below (everything matching this shard/generation) you can see that the file was actually created at January 10th 2018, 14:45:11.000 .

January 10th 2018, 14:46:40.000	compacting full group (0) /media/db_volume/data/XXX/autogen/5667/000000064-000000004.tsm (#0) engine=tsm1
January 10th 2018, 14:46:40.000	error compacting TSM files: open /media/db_volume/data/XXX/autogen/5667/000000064-000000004.tsm: no such file or directory engine=tsm1
January 10th 2018, 14:46:10.000	compacting full group (0) /media/db_volume/data/XXX/autogen/5667/000000064-000000004.tsm (#0) engine=tsm1
January 10th 2018, 14:45:11.000	compacted level 3 group (0) into /media/db_volume/data/XXX/autogen/5667/000000064-000000004.tsm.tmp (#0) engine=tsm1
January 10th 2018, 14:45:01.000	compacting level 3 group (0) /media/db_volume/data/XXX/autogen/5667/000000064-000000003.tsm (#3) engine=tsm1
January 10th 2018, 14:45:00.000	compacted level 2 group (0) into /media/db_volume/data/XXX/autogen/5667/000000064-000000003.tsm.tmp (#0) engine=tsm1
January 10th 2018, 14:44:58.000	compacted level 1 group (0) into /media/db_volume/data/XXX/autogen/5667/000000064-000000002.tsm.tmp (#0) engine=tsm1
January 10th 2018, 14:44:58.000	compacting level 2 group (0) /media/db_volume/data/XXX/autogen/5667/000000064-000000002.tsm (#1) engine=tsm1
January 10th 2018, 14:44:52.000	compacting level 1 group (0) /media/db_volume/data/XXX/autogen/5667/000000064-000000001.tsm (#1) engine=tsm1

[hpbieker]

Searching for each of the filenames with the problem above in my logs gives the following hits:

/media/db_volume/data/XXX/autogen/5667/000000064-000000004.tsm

January 10th 2018, 14:46:40.000	compacting full group (0) /media/db_volume/data/XXX/autogen/5667/000000064-000000004.tsm (#0) engine=tsm1
January 10th 2018, 14:46:40.000	error compacting TSM files: open /media/db_volume/data/XXX/autogen/5667/000000064-000000004.tsm: no such file or directory engine=tsm1
January 10th 2018, 14:46:10.000	compacting full group (0) /media/db_volume/data/XXX/autogen/5667/000000064-000000004.tsm (#0) engine=tsm1

/media/db_volume/data/XXX/autogen/5532/000000704-000000005.tsm

January 9th 2018, 00:22:07.066	compacting full group (0) /media/db_volume/data/XXX/autogen/5532/000000704-000000005.tsm (#0) engine=tsm1
January 9th 2018, 00:22:07.066	error compacting TSM files: open /media/db_volume/data/XXX/autogen/5532/000000704-000000005.tsm: no such file or directory engine=tsm1
January 9th 2018, 00:21:05.004	compacting full group (0) /media/db_volume/data/XXX/autogen/5532/000000704-000000005.tsm (#0) engine=tsm1

I notice that "compacting full group" for each file is listed twice. Is the problem that, due to some race condition, the compation of this file is done by twice -- in parallel?

[hpbieker]

This looks similar to #8664, but it was closed by the author.

[jwilder]

This should be fixed by #8886. There was a bug in 1.3 where the same file could be returned in two different plans.