Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Codeforces login from terminal is broken and download samples requires login #73

Closed
Saurav-Paul opened this issue Jul 13, 2020 · 12 comments · Fixed by #80
Closed

Codeforces login from terminal is broken and download samples requires login #73

Saurav-Paul opened this issue Jul 13, 2020 · 12 comments · Fixed by #80
Labels
bug Something isn't working difficulty:high

Comments

@Saurav-Paul
Copy link

Saurav-Paul commented Jul 13, 2020 •
edited by kmyk
Loading

Description

A few user from my project ai-virtual-assistant inform me they tried to login into codeforces that is not working. Then i looked it into this it seems like oj-api login for codeforces not working. i tried to login manually even tried to login using oj-tools still it is not working. I hope you will check it.

Error log

INFO:root:online-judge-api-client 10.2.1
INFO:root:sleep 1.000000 sec
INFO:onlinejudge._implementation.logging:[-] unknown problem: https://codeforces.com/
INFO:onlinejudge._implementation.logging:[-] unknown contest: https://codeforces.com/
INFO:onlinejudge._implementation.logging:[x] service recognized: CodeforcesService.from_url('https://codeforces.com/'): https://codeforces.com/
INFO:onlinejudge._implementation.logging:[x] load cookie from: /home/saurav/.local/share/online-judge-tools/cookie.jar
INFO:onlinejudge._implementation.logging:[x] GET: https://codeforces.com/enter
INFO:onlinejudge._implementation.logging:[x] 200 OK
ERROR:root:
Traceback (most recent call last):
  File "/home/saurav/.local/lib/python3.8/site-packages/onlinejudge_api/main.py", line 279, in main
    result = login_service.main(service, username=parsed.username, password=parsed.password, check_only=parsed.check, session=session)
  File "/home/saurav/.local/lib/python3.8/site-packages/onlinejudge_api/login_service.py", line 40, in main
    service.login(get_credentials=get_credentials, session=session)
  File "/home/saurav/.local/lib/python3.8/site-packages/onlinejudge/service/codeforces.py", line 44, in login
    form = utils.FormSender(form, url=resp.url)
  File "/home/saurav/.local/lib/python3.8/site-packages/onlinejudge/_implementation/utils.py", line 61, in __init__
    assert isinstance(form, bs4.Tag)
AssertionError
@kmyk
Copy link
Member

kmyk commented Jul 13, 2020

Thank you for reporting! It seems that there is an update of Codeforces and this breaks our code. I'll fix soon.

@kmyk
Copy link
Member

kmyk commented Jul 13, 2020

Now the /enter redirects to ?f0a28=1 with JavaScript. why?

$ curl 'https://codeforces.com/enter'  
<html><body>Redirecting... Please, wait.<script type="text/javascript" src="/aes.min.js" ></script><script>function toNumbers(d){var e=[];d.replace(/(..)/g,function(d){e.push(parseInt(d,16))});return e}function toHex(){for(var d=[],d=1==arguments.length&&arguments[0].constructor==Array?arguments[0]:arguments,e="",f=0;f<d.length;f++)e+=(16>d[f]?"0":"")+d[f].toString(16);return e.toLowerCase()}var a=toNumbers("e9ee4b03c1d0822987185d27bca23378"),b=toNumbers("188fafdbe0f87ef0fc2810d5b3e34705"),c=toNumbers("ebe0e82fefc615e78c3acba688bf16b1");document.cookie="RCPC="+toHex(slowAES.decrypt(c,2,a,b))+"; expires=Thu, 31-Dec-37 23:55:55 GMT; path=/";document.location.href="https://codeforces.com/enter?f0a28=1";</script></body></html>

@kmyk
Copy link
Member

kmyk commented Jul 13, 2020

I found blog posts about this
https://codeforces.com/blog/entry/80065
https://codeforces.com/blog/entry/80070

@kmyk
Copy link
Member

kmyk commented Jul 13, 2020

related issue xalanq/cf-tool#112

@Saurav-Paul
Copy link
Author

Yeah it seems like recent change. That's why this problem suddenly start appearing.
Please let me know if you solve that issue.

@Saurav-Paul
Copy link
Author

I checked the blogs. Lot's of people facing the bug

@kmyk
Copy link
Member

kmyk commented Jul 13, 2020

This issue is difficult and I cannot fix immediately because this, the feature of RCPC tokens, is the feature to kill scraping scripts.
Of course the main target is scripts with terrible manners, but affects almost all tools including online-judge-tools, cf-tool, etc.

(BTW, the bug of RCPC tokens doesn't affects us.)

@kmyk kmyk added bug Something isn't working difficulty:high labels Jul 13, 2020
@Saurav-Paul
Copy link
Author

If i could help you to solve this problem, it would awesome. But i don't have that great knowledge.
Yeah it's really very difficult. I hope this problem will be solved.

RCPC token will not affect us that's awesome to hear.

kmyk added a commit that referenced this issue Jul 13, 2020
@kmyk
Disable the login feature for Codeforces
62eb1ab 
see #73
@kmyk kmyk mentioned this issue Jul 13, 2020
Merged
@kmyk kmyk changed the title Codeforces login is broken many features for Codeforces are broken Jul 13, 2020
@kmyk
Copy link
Member

kmyk commented Jul 13, 2020

I found this problem also breaks the feature to download sample cases.

@kmyk kmyk changed the title many features for Codeforces are broken Codeforces login from terminal is broken and download samples requires login Jul 13, 2020
@kmyk kmyk mentioned this issue Jul 13, 2020
Open
@Saurav-Paul
Copy link
Author

Shit man, they makes it so complex... Now may be need to use oj login.... Today i will change my code and turn towards using webdriver.

@i9kin
Copy link

i9kin commented Jul 14, 2020

I decided

kmyk added a commit that referenced this issue Jul 14, 2020
@kmyk
Compute RCPC value
ff77e29 
This commit allow us to obtain sample cases without logging in.
However, we cannot login without something like Selenium even with this commit, due to the tokens `ftaa` and `bfaa`.

-   see #73
-   thanks to @9kin for his comment xalanq/cf-tool#112 (comment)
-   related #75
@kmyk kmyk mentioned this issue Jul 14, 2020
Merged
@qryxip
Copy link

qryxip commented Jul 15, 2020 •
edited
Loading

It seems that Codeforces reverted the change. (I found this while trying to parse the <script> and compute RCPC value)

codeforces

(edit)

$ oj --version
[!] update available for online-judge-tools: 10.0.4 -> 10.0.5
[*] run: $ pip3 install -U online-judge-tools
[!] update available for online-judge-api-client: 10.2.1 -> 10.2.3
[*] run: $ pip3 install -U online-judge-api-client
online-judge-tools 10.0.4 (+ online-judge-api-client 10.2.1)
$ oj d https://codeforces.com/contest/1/problem/A
[!] update available for online-judge-tools: 10.0.4 -> 10.0.5
[*] run: $ pip3 install -U online-judge-tools
[!] update available for online-judge-api-client: 10.2.1 -> 10.2.3
[*] run: $ pip3 install -U online-judge-api-client
[*] online-judge-tools 10.0.4 (+ online-judge-api-client 10.2.1)
[x] clear the downloading history for this directory: /home/ryo/.cache/online-judge-tools/download-history.jsonl
[x] append the downloading history: /home/ryo/.cache/online-judge-tools/download-history.jsonl

[*] sample 0
[x] input: sample-1
6 6 4

[+] saved to: test/sample-1.in
[x] output: sample-1
4

[+] saved to: test/sample-1.out

@kmyk kmyk mentioned this issue Jul 16, 2020
Merged
@kmyk kmyk closed this as completed in #80 Jul 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working difficulty:high
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Revert the patch for RCPC token of Codeforces online-judge-tools/api-client
4 participants
@kmyk @qryxip @Saurav-Paul @i9kin