Synchronous XMLHttpRequest deprecation fix coming?

[aroberge]

When loading http://brython.info/ and looking at the console, I see the following warning:

Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to     the end user's experience. For more help, check http://xhr.spec.whatwg.org/.

Is there a planned fix for this? Google has been known to suddenly stop supporting features after giving warnings for a while.

[PierreQuentel]

Synchrounous requests are used for imports and for the built-in function open(). I don't know how these could be done with asynchronous requests, so I hope the feature won't be removed !

[aroberge]

As mentioned in a post on the discusssion group, perhaps the approach used by Skulpt with "suspensions" could be used. https://github.com/skulpt/skulpt/search?utf8=%E2%9C%93&q=suspension

[jsbueno]

Interesting thing -

As I wrote a couple of weeks ago, I could manage to get a similar mechanism
working in Brython
just by using "yield" - the very same way different async startegies are
being implemented
in the Pyhton language itself.

However there are two issues with that:

• as I said there, I am stuck on an old version of Brython (yuck - dead
  slow str.format)
• one do have to write some special syntax to have the suspensions occur.
  Like, a special decorator,
  and use "yield async_call(...)" at the point you want the suspension to
  occur.

From skimming the skulpt document, they make teh same thing, but without the
need to have to write an "yield" statement in the main function body.
I don't think that is a big matter - since it does work with the yield, it
is something that can work
with the currently implemented pure Pytho semantics, and that is good.

Factoring my code out of the old Brython I am usingm and setting it free to
other projects could help with (1), and we'd get the thing basically "for
free" (actually, its cost waspaid for, when yield was implemented in
Brython )

So here is the basic idea:
such suspendable co-routines are decorated with a class -decorator, using a
call method. Whe they are called, the call wraper calls the orignal
function with the passed parameters, and issues a "next()" call on it -
therefore, the co-routine runs until a yield expression is used. What he
"yield" returns have to be a special function on its own - it should be an
ordinary function, that returns values that can be interpreted inside the
call method. In my case, for example, I have it just like this:

def get_json(url, data):
return url, data

The basic idea method works like thus (give or take some try/except, and
"while" to allow "yield" to be in a loop, and such)

class Sapore:
def init(self, func):
self.func = func

def call(self, _args, *_kw):
self.running = self.func(_args, *_kw)
url, data = next(self.running)
# 3-4 linesd to setup a regular Brython ajax call,
# with url, data and calling back self.complete

def complete(self, response):
# load response data
self.running.send(response)

@sapore
def my_coroutine():

do things before async request

server_data = yield get_json("/get_data", {"mydata": "whatever"})

do things with retrieved ajax data.

--And it is that simple.
Sapore stands for
"Seamless Asynchronous Python Object REquest" -
(and is italian for "flavor" although in a sense somewhat more concrete
than the English word,
recalling promptly of a "tasty food")

On 15 May 2015 at 07:12, André Roberge notifications@github.com wrote:

As mentioned in a post on the discusssion group, perhaps the approach used
by Skulpt with "suspensions" could be used.
https://github.com/skulpt/skulpt/search?utf8=%E2%9C%93&q=suspension

—
Reply to this email directly or view it on GitHub
#188 (comment)
.

[olemis]

I'll be taking a look into this ticket since I'd like to work towards async module imports .

@jsbueno afaict the scope of this feature should be bound to brython core . What you propose seems to have some impact on python code , which would not be suitable e.g. for module imports . In any case your contribution will be highly appreciated .

[olemis]

At present , and considering my previous experience , it seems to me that the key factors to consider when implementing async ops are :

• performance
• recovery on failure (e.g. accurate stack traces , ...)
• prevent writing spaghetti code commonly found in event handler driven code
  • so I'm personally advocating for introducing composable promises / futures .

p.s. I've been following in python-dev the current draft of async / await PEP and I'm hoping a similar approach will allow us to implement that as well .

[mscuthbert]

Would implementing the loading of modules via the CommonJS/AMD/requirejs standard yield a solution? Inner code could be wrapped into a function that awaits the loading of all previous modules before executing. For instance, code like, main.py:

import re
import midi

ch = midi.channel()

import types

would get refactored to something like (pseudo-code):

require(['python_import-re', 'python_import-midi', 'python_import-types'], 
                function(python_re, python_midi, python_types) {
         python_compile('ch = midi.channel()', imports=[python_re, python_midi, python_types])
    })

nested imports could be handled similarly with nested require() functions.

The tricky part would be if you had something like this:

# fastImport.py
do_something_fast()
title = window.location.hash

# slowImport.py
do_something_slow()
window.location.hash = "hi!"

# main.py
import slowImport
import fastImport
do_main_code()

In this case, the asynchronous importing of slowImport and fastImport could mean that fastImport.py finishes before slowImport and thus that title in fastImport is incorrect (not "hi!"). It could be solved by using nested requires instead of the flat list:

require(['python_import-slowImport'], function(python_slowImport) {
    require(['python_import-fastImport'], function(python_slowImport, python_fastImport) {
            python_compile('do_main_code()', imports=[python_slowImport, python_fastImport);
          });
    });

to make brython fully cPython compatible, something like this (nested wrappers) should be the default option (and it would raise the proper "NameError" if types is used before it is defined), but one could allow something like:

from __future__ import asynchronous_imports

or

if sys.brython is True:
     brython.options['asynchronous_imports'] = True

in any case a wrapper around the import statements so that they return the inner block as a function after require(), or an equivalent async import framework, seems like the only way to get around the inevitable removal of synchronous XHTTP requests.

The part that I haven't read the brython code enough to be sure how to do is to make sure that variables defined before the import statement are still available in the inner function. It's not something I've dealt with in my CommonJS coding.

[olemis]

@mscuthbert Add suport for AMD modules in Brython core is the ultimate destination of the work I'm making to improve the import system . Nevertheless it's not a trivial refactoring .

As such we have to improve many things and finish other tickets first e.g. this ticket #188 and maybe #142 and #210 .

The main goal of the project in the short term should be running the CPython test suite and improve upon current compatibility as well as regression testing (see #234) .

@PierreQuentel cmiiw

[olemis]

@mscuthbert about your code snippets , these problems are found in Python code as well and they are solved by following the principle better explicit than implicit .

[mscuthbert]

@olemis sorry if I was read as implying you didn't have a great handle on the problems. I'm sure that with the great work you and others have done on the project that you have a roadmap towards solving it. I was just worried that synchronous support could be dropped any day (a major project of my own was hit when Chrome dropped plugin support in April with hundreds of users depending on it) and I didn't see anywhere a public declaration of the way the work planned to head. I do think that what is explicit to one person can be implicit to another and vice-versa; I'm not sure what is implicit above. But I'm new to trying to contribute so I should have probably listened more before sending in potential directions. Thanks for the kind responses.

[olemis]

@mscuthbert I apologize in advance if I was rude or something . I just assumed you were following the mailing list and previously read "Port of jQuery UI" thread Jan 14th, 2015 but now I notice this might not be the case . In there I mention that either AMD or UMD might be useful . The former is widely adopted but the later is supposed to work on both browser as well as node.js environments .

[olemis]

... and FWIW , the purpose of my comment about your code snippets is to emphasize upon the fact that even if async imports are implemented the import statement will still be synchronous , because that's the semantics of Python . On the one hand cases like import w, x, y, z might run faster if imports (or at least parts of the process e.g. downloading module code) happen in parallel . In cases where the Python instructions may be executed in parallel the other mechanisms should explicitly be used e.g. threads , coroutines , async / await ...

[mscuthbert]

agreed. I've now joined the mailing list and hope to be able to contribute soon. I'm just so amazingly excited by the progress of this project -- I've been wanting to work on Skulpt or pypy.js for some time, but the amount of work still to be done (esp. in speedups) was so daunting; here it seems like enough has been done that adding other components will be a joy to do. Thank you.

[BrenBarn]

I also find this pretty worrisome. I don't see any real reason to suppose that synchronous XMLHTTPRequest will not eventually be removed, and it seems like if that were to happen Brython would completely break. The approach described by olemis seems pretty promising as far as I understand it. But has it just been abandoned?

[mscuthbert]

Since these discussions took place, the "async" and "await" syntax has become fully a part of JS and supported by most transpilers. It's should be easy to have python syntax such as:

def main():
    from a import b
    b.x()

be compiled into code like:

async function main() {
    const b = await py_import('a', 'b');
    return b.x();
} 

So long as the original calling function is Promise-aware and brython's JS engine passes Promises around, the worst problems of Synchronous Async can be worked around without much hassle.

[PierreQuentel]

If it was simple, I have no doubt that in 6 years someone would have already implemented it.

But it is not so simple : all imports are not performed by a statement import X at the module level, they can be inside a function, or inside a method of a class, possibly in an imported module.

Suppose that a module X defines the class A:

class A:

    def foo(self, x):
        import math
        return math.sin(x)

    def bar(self, x):
        return 2 * x

If the main module has :

import X

a = X.A()
print(a.foo(1))
print(a.bar(2))

the translation of the last 2 lines would have to be something like

print(await a.foo(1)) // because of the async "import math" in foo
print(a.bar(2)) // nothing to await

How would the Brython engine know, by static code analysis, which calls should use await and which ones shouldn't ? I would like to be wrong, but I don't think it can.

Even more difficult - and totally weird I admit, but it must work, otherwise it's not real Python:

Module Y :

x = "impo"
y = "rt foobar"

Main module:

from Y import *
exec(x + y)

[mscuthbert]

I did not mean to imply that it was simple -- apologies if I did. I just meant to say that I don't think it is the "death knell for the project" if it happened that it might have been if it had happened in 2015. It would be a major pain and difficult agreed and probably need to make every statement become async-await and normal statements return an immediately-resolving Promise, which would be pretty awful.

[PierreQuentel]

Another thing that I forgot to mention is that loading modules with Ajax calls can be avoided if the program uses the Virtual File System in brython_stdlib.js or brython_modules.