Follow up task after https://github.com/apache/beam/pull/4235 merge. This task is about ensuring we test that. Imported from Jira [BEAM-3479](https://issues.apache.org/jira/browse/BEAM-3479). Original Jira may contain additional context. Reported by: romain.manni-bucau.