Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

invalid syntax error #3

Closed
sinancetinkaya opened this issue Dec 7, 2020 · 33 comments
Closed

invalid syntax error #3

sinancetinkaya opened this issue Dec 7, 2020 · 33 comments
Assignees
Labels
bug Something isn't working

Comments

@sinancetinkaya
Copy link

sinancetinkaya commented Dec 7, 2020

image

java.lang.Throwable: No connection (command:  113 )
	at com.intellij.openapi.diagnostic.Logger.error(Logger.java:159)
	at com.jetbrains.python.debugger.pydev.AbstractCommand.execute(AbstractCommand.java:159)
	at com.jetbrains.python.debugger.pydev.RemoteDebugger.evaluate(RemoteDebugger.java:158)
	at com.jetbrains.python.debugger.pydev.MultiProcessDebugger.evaluate(MultiProcessDebugger.java:167)
	at com.jetbrains.python.debugger.PyDebugProcess.evaluate(PyDebugProcess.java:752)
	at com.jetbrains.python.debugger.PyDebugProcess.evaluate(PyDebugProcess.java:748)
	at com.uriyyo.evaluate_async_code.AsyncPyDebugRunner$createDebugProcess$1.evaluate(AsyncPyDebugRunner.kt:40)
	at com.jetbrains.python.debugger.PyDebuggerEvaluator.lambda$doEvaluate$0(PyDebuggerEvaluator.java:59)
	at com.intellij.util.RunnableCallable.call(RunnableCallable.java:20)
	at com.intellij.util.RunnableCallable.call(RunnableCallable.java:11)
	at com.intellij.openapi.application.impl.ApplicationImpl$1.call(ApplicationImpl.java:270)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:668)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:665)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1.run(Executors.java:665)
	at java.base/java.lang.Thread.run(Thread.java:834)

I'm using pyppeteer

@uriyyo uriyyo self-assigned this Dec 8, 2020
@uriyyo uriyyo added the bug Something isn't working label Dec 8, 2020
@uriyyo
Copy link
Owner

uriyyo commented Dec 8, 2020

Hi @sinancetinkaya

This issue is fixed in version 1.6. The new version of the plugin will be available within two days at PyCharm.

I will close this issue, feel free to reopen it in case when issue still occurs in a new version.

@uriyyo uriyyo closed this as completed Dec 8, 2020
@sinancetinkaya
Copy link
Author

sinancetinkaya commented Dec 12, 2020

using v1.6

java.lang.Throwable: Timeout waiting for response on 113
	at com.intellij.openapi.diagnostic.Logger.error(Logger.java:159)
	at com.jetbrains.python.debugger.pydev.AbstractCommand.execute(AbstractCommand.java:159)
	at com.jetbrains.python.debugger.pydev.RemoteDebugger.evaluate(RemoteDebugger.java:158)
	at com.jetbrains.python.debugger.pydev.MultiProcessDebugger.evaluate(MultiProcessDebugger.java:167)
	at com.jetbrains.python.debugger.PyDebugProcess.evaluate(PyDebugProcess.java:752)
	at com.jetbrains.python.debugger.PyDebugProcess.evaluate(PyDebugProcess.java:748)
	at com.uriyyo.evaluate_async_code.AsyncPyDebugRunner$createDebugProcess$1.evaluate(AsyncPyDebugRunner.kt:37)
	at com.jetbrains.python.debugger.PyDebuggerEvaluator.lambda$doEvaluate$0(PyDebuggerEvaluator.java:59)
	at com.intellij.util.RunnableCallable.call(RunnableCallable.java:20)
	at com.intellij.util.RunnableCallable.call(RunnableCallable.java:11)
	at com.intellij.openapi.application.impl.ApplicationImpl$1.call(ApplicationImpl.java:270)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:668)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:665)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1.run(Executors.java:665)
	at java.base/java.lang.Thread.run(Thread.java:834)

@uriyyo
Copy link
Owner

uriyyo commented Dec 12, 2020

@sinancetinkaya Can you please provide more information?
Information that I needed is a version of a PyCharm and code snippet that you tried to evaluate.

@uriyyo uriyyo reopened this Dec 12, 2020
@sinancetinkaya
Copy link
Author

image

import asyncio
import os
import aiohttp
import pyppeteer

loop = asyncio.get_event_loop()


async def main():
    os.system("start chrome --remote-debugging-port=9222")

    async with aiohttp.ClientSession() as session:
        try:
            async with session.get("http://127.0.0.1:9222/json/version") as response:
                chrome = await response.json()
        except aiohttp.ClientConnectorError:
            return

    browser = await pyppeteer.launcher.connect(
        loop=loop,
        browserWSEndpoint=chrome['webSocketDebuggerUrl']
    )
    page = await browser.newPage()
    await page.goto('https://tr.tradingview.com')
    print(await page.evaluate(""" navigator.webdriver  """))
    await browser.close()

loop.run_until_complete(main())

for instance when I try to evaluate await page.evaluate(""" navigator.webdriver """) the plugin crashes

@uriyyo
Copy link
Owner

uriyyo commented Dec 21, 2020

Hi @sinancetinkaya,
Sorry for the long response, I hadn't a free time to look at this issue.

This issue has been fixed at version 1.7., it will be available within 3 days.

I will close this issue, in case if it appears again please reopen this issue.

@uriyyo uriyyo closed this as completed Dec 21, 2020
@sinancetinkaya
Copy link
Author

sinancetinkaya commented Dec 21, 2020

Updated to 1.7, now I get this error when I start to debug a script. Btw my python version is 3.9. Pycharm works fine when your plugin is disabled
image

@uriyyo uriyyo reopened this Dec 21, 2020
@uriyyo
Copy link
Owner

uriyyo commented Dec 21, 2020

I see. You using Windows OS and I didn't test it on it.

Basically, the plugin creates a script that patches pydevd functions and runs the main debugger script.
Look like there is some permissions issue with the created file, I will look and try to fix it.

I am so sorry that you are receiving errors, I am trying to do my best, but there are no good documentation and tutorials regarding plugin development😔

@sinancetinkaya
Copy link
Author

You don't need to pardon for a volunteer work. I'm just trying to help you on your plugin

@uriyyo
Copy link
Owner

uriyyo commented Dec 21, 2020

Looks like I have fixed this issue.
Can you please download new version of plugin and verify that everything works as expected?
You should use install plugin from disk option at plugins window to install downloaded plugin
Link to new version https://srv-store3.gofile.io/download/M8RZWJ/evaluate-async-code-1.8.zip
This is development version, it isn't published now. I will really appreciate if you can help me with it)

@espdev
Copy link

espdev commented Dec 23, 2020

I'm using Windows too. I think the plugin cannot write to protected (system) "Program Files" directory. The program needs administrator rights for this. In my case I use another directory for installed programs (C:\Programs). This directory is not protected and not a system directory, therefore I'm not getting an error. I think the plugin should not write any data in the installation directory.

@uriyyo
Copy link
Owner

uriyyo commented Dec 24, 2020

@espdev Thanks for your time and especially for finding the root cause of the problem.

The solutions which I see:

  1. Create tmp file instead of creating a file in the PyCharm helpers directory.
  2. Check if plugin has permissions to write to helpers, if there no permission then fallback to 1 option.

@espdev
Copy link

espdev commented Dec 24, 2020

@uriyyo

Can the plugin write the helper files to PyCharm settings directory?

%APPDATA%\JetBrains\<product><version>

For example:

C:\Users\Eugene\AppData\Local\JetBrains\PyCharm2020.3\

Maybe we can use some directory in the settings directory to store the plugin data and helper python code?

@uriyyo
Copy link
Owner

uriyyo commented Dec 24, 2020

@espdev Thanks, it's a great idea, we definitely should use this directory.

I will update code to use this approach.

@uriyyo
Copy link
Owner

uriyyo commented Dec 24, 2020

@espdev @sinancetinkaya

This issue must be fixed in version 1.8 (this version will be available within 3 workdays)

Thank you for your time and efforts 💫

@sinancetinkaya I won't close this issue until you verify that everything works on your computer 😊

@sinancetinkaya
Copy link
Author

sinancetinkaya commented Dec 24, 2020

@uriyyo

D:\Program Files\JetBrains\PyCharm Community Edition 2020.3.1\plugins\python-ce\helpers\pydev\_pydevd_bundle\pydevd_comm.py:1531: RuntimeWarning: coroutine '__async_exec_func__' was never awaited
  dbg.writer.add_command(cmd)
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

image

According to @espdev 's suggestion, reinstalled PyCharm to another drive.

@uriyyo
Copy link
Owner

uriyyo commented Dec 24, 2020

@sinancetinkaya Can you please provide more context?

I have tried to make async request using aiohttp and everything works as expected
image

@sinancetinkaya
Copy link
Author

sinancetinkaya commented Dec 24, 2020

Try this. Btw this code won't work until you disable your plugin. You don't have a running event loop in your sample. It should not work at all.

import aiohttp
import asyncio

loop = asyncio.get_event_loop()

async def main():
    async with aiohttp.ClientSession() as session:
        async with session.get("https://jsonplaceholder.typicode.com/todos/1") as response:
            print(await response.json())

loop.run_until_complete(main())
loop.close()

@uriyyo
Copy link
Owner

uriyyo commented Dec 24, 2020

I have updated code to disable SSL verification:

import aiohttp
import asyncio

loop = asyncio.get_event_loop()


async def main():
    async with aiohttp.ClientSession() as session:
        async with session.get("https://jsonplaceholder.typicode.com/todos/1", verify_ssl=False) as response:
            print(await response.json())


loop.run_until_complete(main())
loop.close()

And it works as expected.

image

image

You don't have a running event loop in your sample. It should not work at all.

Actually, it will work because plugin patches asyncio using nest-asyncio library.

Basically, when you are trying to evaluate async code, it actually will be translated from this:

await foo()

to smth like this

import asyncio

async def _async_func():
    return await foo()

asyncio.get_event_loop().run_until_completed(_async_func())

So in case, where there no event loop it will be created, it allows to use async/await when debugging at module level or when you use Python Console.

Just to clarify, have you updated to the latest available version or you are using dev version of plugin which I have posted several messages above?

@sinancetinkaya
Copy link
Author

That code should work without verify_ssl=False flag. Try this website https://www.eksisozluk.com/
verify_ssl is required only if you or the website have invalid certificate. This should never happen in normal cases.
Yes the version is 1.8

@uriyyo
Copy link
Owner

uriyyo commented Dec 24, 2020

Everything works okay on my PC, but I am using Mac OS.

Is there are any problems with simple cases on your PC?

Let for instance try to check this case
image

Code that I used:

from asyncio import run


async def foo():
    return 10


async def main():
    await foo()


if __name__ == '__main__':
    run(main())

@sinancetinkaya
Copy link
Author

that simple code works fine 👍

@uriyyo
Copy link
Owner

uriyyo commented Dec 24, 2020

Hmm, let me think.

As far as I know, you are using python 3.9 (I am using the same version).

Can you please provide:

  1. Output of pip freeze
  2. Code that reproduces the issue
  3. Line where I should put a breakpoint

And I will try to reproduce this isssue

@sinancetinkaya
Copy link
Author

My actual code is a lot more complex and I can't post it here. Try the code I gave you. verify_ssl should not be required.
Try to use the same version of the packages below to see if it's installed packages related.

aiohttp==3.6.3
appdirs==1.4.4
asgiref==3.3.1
async-timeout==3.0.1
attrs==20.3.0
blinker==1.4
Brotli @ file:///D:/Downloads/Brotli-1.0.9-cp39-cp39-win_amd64.whl
certifi==2020.12.5
cffi==1.14.4
chardet==4.0.0
click==7.1.2
cryptography==3.3.1
discord.py==1.5.1
dnspython==2.0.0
feedgen==0.9.0
Flask==1.1.2
h11==0.11.0
h2==4.0.0
hpack==4.0.0
hyperframe==6.0.0
idna==2.10
itsdangerous==1.1.0
Jinja2==2.11.2
kaitaistruct==0.9
ldap3==2.8.1
lxml==4.6.2
MarkupSafe==1.1.1
mitmproxy==6.0.2
motor==2.3.0
msgpack==1.0.2
multidict==4.7.6
passlib==1.7.4
protobuf==3.14.0
psutil @ file:///D:/Downloads/psutil-5.7.3-cp39-cp39-win_amd64.whl
publicsuffix2==2.20191221
pyasn1==0.4.8
pycparser==2.20
pydivert==2.1.0
pyee==8.1.0
pymongo==3.11.2
pyOpenSSL==20.0.1
pyparsing==2.4.7
pyperclip==1.8.1
pyppeteer==0.2.2
python-dateutil==2.8.1
requests==2.25.1
ruamel.yaml==0.16.12
six==1.15.0
sortedcontainers==2.3.0
tornado==6.1
tqdm==4.54.1
typing-extensions==3.7.4.3
urllib3==1.26.2
urwid==2.1.2
websocket-client==0.57.0
websockets==8.1
Werkzeug==1.0.1
wsproto==1.0.0
yarl==1.5.1
zstandard==0.14.1

@uriyyo
Copy link
Owner

uriyyo commented Dec 24, 2020

Thanks, I will try to reproduce this issue. Also, I will try to get Windows OS to check if it's a platform issue.

@uriyyo
Copy link
Owner

uriyyo commented Dec 25, 2020

@espdev Can you please check if you can reproduce this issue?

Because I can't reproduce this issue on my computer.

@espdev
Copy link

espdev commented Dec 25, 2020

@uriyyo I have tried to run the example with aiohttp. There are two issues on Windows.

  1. When I use WindowsSelectorEventLoopPolicy I get "RuntimeError: This event loop is already running"
  2. When I use default event loop policy (WindowsProactorEventLoopPolicy) the code hangs in debug console and in "evaluate expression" window.

On Windows we need to use WindowsSelectorEventLoopPolicy with aiohttp. See this issue.

The code sample:

import asyncio
import aiohttp

async def main():
    async with aiohttp.ClientSession() as session:  # <-- breakpoint here
        async with session.get("https://jsonplaceholder.typicode.com/todos/1") as response:
            print(await response.json())

if __name__ == '__main__':
    asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
    asyncio.run(main())

2020-12-25_12-35-22

@uriyyo
Copy link
Owner

uriyyo commented Dec 25, 2020

@espdev Thank you very much. Now somehow, I must get Windows OS and fix this issue)

@uriyyo
Copy link
Owner

uriyyo commented Dec 26, 2020

@sinancetinkaya This issue is fixed in version 1.9 (today is a weekend so I believe it will be available on Monday).

I have tested it on Windows OS with test cases that you provided and now everything works as expected.

I am waiting for your feedback🙂

@espdev Thank you for your efforts, and especially regarding the usage of WindowsSelectorEventLoopPolicy it saved me a lot of hours of debugging)

@uriyyo
Copy link
Owner

uriyyo commented Dec 28, 2020

Hi @sinancetinkaya, new version 1.9 is available.

Could you please verify that this issue is no longer present?

@sinancetinkaya
Copy link
Author

@uriyyo
image

at least I'm not getting any errors in the evaluation window but I don't see response variable in the debug window.

@uriyyo
Copy link
Owner

uriyyo commented Dec 28, 2020

Can you try to assign it to another variable name? The problem might be that this name is used in async with statement.

@sinancetinkaya
Copy link
Author

Can you try to assign it to another variable name? The problem might be that this name is used in async with statement.

Aha, you are right! Interesting... That's not the normal behavior but it's trivial

@uriyyo
Copy link
Owner

uriyyo commented Dec 28, 2020

I totally agree, but let's create a separate issue regarding this minor problem and finally close this one)

@uriyyo uriyyo closed this as completed Dec 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants