Export all should export all zones

[hebbet]

What would you like?
when runninng docker run -it --rm -e CF_API_KEY="$CF_API_KEY" -e CF_API_EMAIL="$CF_API_EMAIL" lfaoro/flares --all it should export all zones and not only 20

Describe the solution you'd like
all zones are exported

Describe alternatives you've considered (if any)

Additional context (if you want)

[lfaoro]

Yes, all zones should be exported. I don't have 20 zones to test this, so I'll need your help with the testing.

Let's say you have 30 zones, you run the command and only 20 zones get exported and it finishes?

I think maybe CloudFlare is limiting the export to 20 zones, I will check the API I'm calling and give you more info.

[hebbet]

Yes, all zones should be exported. I don't have 20 zones to test this, so I'll need your help with the testing.

yes i can help testing. we have around 400 domains in CF

Let's say you have 30 zones, you run the command and only 20 zones get exported and it finishes?

yes.

I think maybe CloudFlare is limiting the export to 20 zones,

List Zones default value is 20 per page.

https://api.cloudflare.com/#zone-list-zones

  "result_info": {
    "page": 1,
    "per_page": 20,
    "count": 1,
    "total_count": 2000
  }

maybe a loop through the "result_info" could easy solve it

[lfaoro]

maybe a loop through the "result_info" could easy solve it

yes, that's one options, maybe another option is to not limit the result page if CF api supports it.

will issue a fix by tomorrow; thank you for the time invested reporting this @hebbet

[hebbet]

any news on that?

[lfaoro]

hello @hebbet, I'm very sorry for keeping you waiting on this issue this long -- totally forgot about it.

I pushed a fix right now, I have no way to test it -- lemme know if it solves your problem.

as you suggested I loop through the result_info and calculate how many requests do we need to send in order to obtain all data total_count / per_page

then we just keep extracting the data for every page with a new request

[hebbet]

i pulled the image and tried again:
docker run -it --rm -e CF_API_KEY="$CF_API_KEY" -e CF_API_EMAIL="$CF_API_EMAIL" lfaoro/flares --all
DEBUG: [map[code:6003 error_chain:[map[code:6105 message:Invalid Content-Type header, valid values are application/json,multipart/form-data]] message:Invalid request headers]]
Error: 🔥: cloudflare: domain not found

[lfaoro]

the docker build is still in progress:

you will have to pull the new image once it's done in order to have the new code running

I suggest you pull every now and then, once you receive a new image, you know to have the new app now

[lfaoro]

I created the new image locally to confirm the new code works as expected: http://share.vlct.io/0f9d28

Just need you to confirm it works for 400 domains now :)

[lfaoro]

@hebbet -- does it work as expected?

[hebbet]

no :-(

docker run -it --rm -e CF_API_KEY="$CF_API_KEY" -e CF_API_EMAIL="$CF_API_EMAIL" lfaoro/flares -a
Unable to find image 'lfaoro/flares:latest' locally
latest: Pulling from lfaoro/flares
bdf0201b3a05: Pull complete
f402d13f8c6a: Pull complete
1697c109f50e: Pull complete
Digest: sha256:fe241ec3e641f58d3164f1f77c589c500301648e653b3aa11c89ad317efd7060
Status: Downloaded newer image for lfaoro/flares:latest
DEBUG: [map[code:6003 error_chain:[map[code:6105 message:Invalid Content-Type header, valid values are application/json,multipart/form-data]] message:Invalid request headers]]
Error: 🔥: cloudflare: domain not found

[lfaoro]

Try again, I removed that faulty header.

$ docker pull lfaoro/flares
$ docker run -it --rm -e CF_API_KEY="$CF_API_KEY" -e CF_API_EMAIL="$CF_API_EMAIL" lfaoro/flares --all

[hebbet]

we are getting closer, but with a new error:

DEBUG: [map[code:6008 error_chain:[map[code:11001 message:page variable must be a number] map[code:11002 message:page variable must be a number]] message:Invalid request parameters]]

[lfaoro]

Good! Progress :D

I'm coding in the blind, I don't have a way to test pagination -- gotta imagine what's going on.

No worries, will try again, we'll get this issue solved one way or another.

[lfaoro]

@hebbet -- I've refactored the whole tool, now you should be able to handle as many domains as you want in one go.

$ flares --all --export