Cloudflare stops running after 168 hours

[paullryan]

Describe the bug
With the new cloudflare scraper after 168 hours it stops running as it tries to fill in log holes for older times.

The error below prints for a stream that has this issue and then it stops trying to get anything again from that zone:
5 errors: HTTP status 400: bad query: error parsing time: invalid time range: too early: logs older than 168h0m0s are not available ; HTTP status 400: bad query: error parsing time: invalid time range: too early: logs older than 168h0m0s are not available ; HTTP status 400: bad query: error parsing time: invalid time range: too early: logs older than 168h0m0s are not available ; HTTP status 400: bad query: error parsing time: invalid time range: too early: logs older than 168h0m0s are not available ; HTTP status 400: bad query: error parsing time: invalid time range: too early: logs older than 168h0m0s are not available

I will also note that If I clear loki it will start running again but this is not a complete solution either and retention settings to less than 168h or reject_old_samples of less don't seem to help.

To Reproduce
Steps to reproduce the behavior:

1. Started Loki
2. Started Promtail main-4f2f088 with cloudflare scrape targets (mutiple of them)

          - job_name: cloudflare/cloudflare_site-test
            cloudflare:
              pull_range: 1m
              fields_type: all
              api_token: ${CLOUDFLARE_TOKEN}
              zone_id: <site.test zoneId>
              labels:
                job: cloudflare
                cloudflare_zone_id: <site.test zoneId>
                cloudflare_site_name: site.test

3. Query: {job="cloudflare"}

Expected behavior
Logs to be streaming in, which does work for the first few days.

Environment:

• Infrastructure: Kubernetes - azure storage backed loki
• Deployment tool: helm