Instagram scraping warning despite monitoring only one account

[hitanshtiwari]

Hey everyone, I am pretty new to using this project and I have been trying to monitor just one Instagram account that I already follow.

I have been running into repeated scraping and rate limit warnings even though I am only monitoring one account. At first I was using the default configuration with no follower or following batch limits, and I started getting errors like challenge_required and 401 Unauthorized with Instagram telling me to wait a few minutes.

I reimported my Firefox session after clearing the Instagram challenge and made sure I was not using Instagram in Firefox while the monitor was running. I also enabled follower churn detection,human mode,jittering and started adding delays and batches to reduce the request rate.

The account I am monitoring has around 366 followers and 441 following. In one successful run, the monitor was able to complete the scan, although it could only save the names of 358 followers and 433 following. The scan itself finished successfully without any errors.

After that, I stopped the monitor. I did use Instagram normally for a little while afterward and watched some Reels, but I did not run the scraper again. The next morning when i woke up i noticed i had been logged and had that warning again out of nowhere?

At this point I am a bit confused about what is actually triggering Instagram's restrictions. I am only monitoring one account and I have already added batching, delays and jitter. I am currently using batches of 50 accounts with a 30 second delay between batches.

I would really appreciate some advice from people who have used this project or Instaloader for this kind of monitoring. Is there a known reasonable request rate or batch size for a single account? Is the warning likely caused by the amount of data being fetched, the authenticated session, or something else?

I am mainly trying to find a setup that can reliably check one account every now and then without repeatedly triggering Instagram's anti scraping systems.

[tomballgithub]

@hitanshtiwari
I wrote the feature that splits up follower/following fetching to avoid getting accounts flagged for scraping. I have done extensive testing on this problem, and one thing is that once you trip the detection logic, it often doesn't cause a failure until the next day (or 24 hours). This could be what you are experiencing. Your fetching all looked good, account still works, but then the next day the account has an issue.

For your case specifically, I suggest you try the following:

FOLLOWERS_PER_BATCH = 20
FOLLOWEES_PER_BATCH = 20
FOLLOWER_DELAY_PER_BATCH = 120
FOLLOWEE_DELAY_PER_BATCH = 120

BE_HUMAN = False

Note: Run once per day to start. If that makes it more than 2 days, that's a good sign & you can experiment further.
Note: I've seen BE_HUMAN cause accounts to get flagged while in the middle of BE_HUMAN, so I keep this disabled
Note: I recommend disabling FOLLOWERS_CHURN_DETECTION to avoid extra fetching, but to force a fetch every day as a test, you can enable it. Once you get this working and want to let it run continuously, disable it.

I definitely noticed across 100's of tests that exceeding those limits caused problems when trying to fetch 4000+ followings, but I was never able to keep an account active for more than a few days when fetching 4000+. For example, <=60 DELAY_PER_BATCH never worked (90s maybe works). For example, 30 PER_BATCH was a problem (25 maybe works)

I am definitely interested in your results.