Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to limit the Result #197

Closed
Ammar8cs opened this issue May 5, 2021 · 8 comments
Closed

How to limit the Result #197

Ammar8cs opened this issue May 5, 2021 · 8 comments

Comments

@Ammar8cs
Copy link

Ammar8cs commented May 5, 2021

There are about 10k followers and I want to extract the first 3k emails. Because after 12k it gives exception.
How can i limit the number of iterations.

@Datalux
Copy link
Owner

Datalux commented May 6, 2021

The tool has no output limit, so you can edit the code adding a limit to the output. I will consider to add this feature to the project.

@Ammar8cs
Copy link
Author

Ammar8cs commented May 6, 2021

Thanks for replying and please let me where I can insert the limit.

@superman7886
Copy link

let me know if you make one

@Datalux
Copy link
Owner

Datalux commented May 11, 2021

Thanks for replying and please let me where I can insert the limit.

You can add a counter inside the methods called by commands.

@FayzulSaimun
Copy link

like fwersemail(counter = 1000) ?

@powellthepog
Copy link

powellthepog commented Aug 4, 2021

Is there a way to slow the scrape speed as opposed to the counter limit overall? I am thinking of adding

import time

then adding: sleep.time(3)
somewhere within the fwingsemail definition.

Maybe after:
for follow in followings:

#Osintgram.py

def get_fwingsemail(self):
if self.check_private_profile():
return

    followings = []

    try:

        pc.printout("Searching for emails of users followed by target... this can take a few minutes\n")

        rank_token = AppClient.generate_uuid()
        data = self.api.user_following(str(self.target_id), rank_token=rank_token)

        for user in data.get('users', []):
            u = {
                'id': user['pk'],
                'username': user['username'],
                'full_name': user['full_name']
            }
            followings.append(u)

        next_max_id = data.get('next_max_id')
        
        while next_max_id:
            results = self.api.user_following(str(self.target_id), rank_token=rank_token, max_id=next_max_id)

            for user in results.get('users', []):
                u = {
                    'id': user['pk'],
                    'username': user['username'],
                    'full_name': user['full_name']
                }
                followings.append(u)

            next_max_id = results.get('next_max_id')
    
        results = []
        
        pc.printout("Do you want to get all emails? y/n: ", pc.YELLOW)
        value = input()
        
        if value == str("y") or value == str("yes") or value == str("Yes") or value == str("YES"):
            value = len(followings)
        elif value == str(""):
            print("\n")
            return
        elif value == str("n") or value == str("no") or value == str("No") or value == str("NO"):
            while True:
                try:
                    pc.printout("How many emails do you want to get? ", pc.YELLOW)
                    new_value = int(input())
                    value = new_value - 1
                    break
                except ValueError:
                    pc.printout("Error! Please enter a valid integer!", pc.RED)
                    print("\n")
                    return
        else:
            pc.printout("Error! Please enter y/n :-)", pc.RED)
            print("\n")
            return

        for follow in followings:
            sys.stdout.write("\rCatched %i followings email" % len(results))
            sys.stdout.flush()
            user = self.api.user_info(str(follow['id']))
            if 'public_email' in user['user'] and user['user']['public_email']:
                follow['email'] = user['user']['public_email']
                if len(results) > value:
                    break
                results.append(follow)
    
    except ClientThrottledError as e:
        pc.printout("\nError: Instagram blocked the requests. Please wait a few minutes before you try again.", pc.RED)
        pc.printout("\n")
    
    print("\n")

    if len(results) > 0:
        t = PrettyTable(['ID', 'Username', 'Full Name', 'Email'])
        t.align["ID"] = "l"
        t.align["Username"] = "l"
        t.align["Full Name"] = "l"
        t.align["Email"] = "l"

        json_data = {}

        for node in results:
            t.add_row([str(node['id']), node['username'], node['full_name'], node['email']])

        if self.writeFile:
            file_name = self.output_dir + "/" + self.target + "_fwingsemail.txt"
            file = open(file_name, "w")
            file.write(str(t))
            file.close()

        if self.jsonDump:
            json_data['followings_email'] = results
            json_file_name = self.output_dir + "/" + self.target + "_fwingsemail.json"
            with open(json_file_name, 'w') as f:
                json.dump(json_data, f)

        print(t)
    else:
        pc.printout("Sorry! No results found :-(\n", pc.RED)

@yaryatchii
Copy link

Hi,

Have you found a solution to limit the results so as not to be blocked by Instagram bots?

@Datalux
Copy link
Owner

Datalux commented Oct 18, 2021

This was fixed in version 1.3 (#150)

@Datalux Datalux closed this as completed Oct 18, 2021
@Datalux Datalux unpinned this issue Oct 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants