Defaults to Chrome Headless #25
Conversation
|
Is there a reason to use chrome at all? Can accounts be created and applications be sent via API requests? |
That's a great question. This program has used selenium as a driver and chrome as a base since I've contributed. I know firefox and selenium pair together well as well. I'm sure you could accomplish something similar using GET/POST requests. I haven't really explored that quite yet (I'm in the process of taking finals now). You might get a better response from another contributor if you throw it in issues? |
|
Assuming it won't simply allows post requests. An measure as simple as that is typically already implemented |
There was a problem hiding this comment.
might be nice to have a variable that can be passed in from the main driver (if name == "main") something like "debugMode" that if false will make it headless.
This way during development, you can set it to true and still see what is happening.
Plus in the future if other features are added similar to this, they can also look at this variable to decide how to run
|
to answer @big-labor 's question... usually production-level applications require some sort of authentication when sending API requests, such as a token. meaning without a valid token these requests would likely fail with a 401 status code |
|
Can do. Moved to draft for the time being. Should the application default to being headless or headed? I can see pros/cons to both. |
Headless is a lot more lightweight and allows the logs to be visible. Other than for debugging/development purposes, what reasons would you prefer headed @pws1453 ? |
|
Honestly, the only other thing I could see is if someone wanted to record the applications being submitted (which while a use case, is served by the --debug option). Additionally, I've added the parsing for the |
Before, in fill_out_application_and_submit(), it would just wait for a certain amount of time before continuing. Sometimes it would continue before the page was fully loaded (SeanDaBlack#31). I've instead added three new selenium imports and used WebDriverWait().until to wait for the page to finish loading.
- Replaced all occurences of `print` with `printf` to remove syntax error. - Replaced both `match`es with ifs/elifs.
|
Alright, here is where it gets fun, recaptcha is trying to detect whether or not the system is automated. So, we somehow need to beat it. I'll attach an image from the html file (I can't upload it here sadly, but I will include it in the next commit) I was able to pull from driver.page_source while debugging. |
…tml to display issue
|
There's a patched version of the chromedriver that is designed to be undetectable to things like this. You could try to implement that. |
|
Have tried also using firefox headless with geckodriver - no difference. |
|
Interesting to note, the error occurs when you try to download the recaptcha's audio file, so if we don't find a way to get past this block we could just fall back to making the user do the normal recaptcha if the audio fails to download. |
Added the undetected_chromedriver module as a dependency & some other changes
Salary is now a range i.e. 20-25 instead of just 20. Salary now in multiples of 5.
Make salary more convincing
- Removed the random_email function and constants/email.py Emails are now gotten by: - Sending a 'get_email_address' request to guerrilla, this returns our temporary email and session id. I use json.loads to parse json for each of the requests. - I added 'sid' to 'fake_identity', it just makes things easier. - After it clicks the 'CREATE_ACCOUNT_BUTTON' we start waiting for an email by checking with the 'check_email' guerrilla request. - We check if the response contains an email, if so we fetch it. - I didn't want to have to import a whole html parsing library just for this so I used regex to search for the passcode. - Type the passcode into 'VERIFY_EMAIL_INPUT' then click 'VERIFY_EMAIL_BUTTON' and it's done.
|
In the recent change they made (adding email verification) they seem to have removed the recaptcha, so once we get around the email verification this problem should be solved. |
Hopefully get around the email verification
- I now use a modified version of the `random_email` function for generating emails for mail.tm
Changes to `random_email`:
- I removed the `lambda` expressions with dots in them - mail.tm strips dots from it's emails.
- Doesn't pick domain from a preprogrammed list anymore - it gets the domain from mail.tm via it's api
- Added a command line option, `--mailtm`. Using this will use mail.tm by default
If you don't use that option, it will automatically fallback to it if guerrilla
doesn't respond within 180 seconds (let me know if it's a bit much) and vice versa
mail.tm pipeline
- In `random_email`, we get the domain to use from the `/domains` endpoint
- Create the account with the `/accounts` endpoint
- Get the token used to access the account from the `/token` endpoint
...
sign up
...
- Checks inbox with the `/messages?page=1` endpoint every 1.5 seconds until the email arrives
- Fetches the email from `/messages/{id}`
Added support for mail.tm
Email verification
|
Newest commit should resolve above conversations, will await response before resolving |
|
Looks good to me! |
ghost
merged commit 
As title says, chrome runs headlessly now.