Local (nested) classes only one level deep

[axgkl]

Hey Jacques,

its alive ;-)

After finishing developing our container orchestration I will now go work further on an application framework for the GUI, clearly using the one and only Transcrypt!

Its amazing how much progress Transcrypt made, absolutely slick meanwhile. "Works as expected" is really an understatement and I'm happy to see you find time for esoteric stuff like type mgmt ;-)

---

One thing I found though, regarding local (nested) class handling, and I think there is a somewhat cheap fix:

class T1:                                                                       
    class T2:                                                                   
        class T3:                                                               
            class T4: pass                     

is compiled into

        var T1 = __class__ ('T1', [object], {
        });
        T1.T2 = __class__ ('T2', [object], {
        });
        T2.T3 = __class__ ('T3', [object], {
        });
        T3.T4 = __class__ ('T4', [object], {
        });

works for T2, i.e. one nested level but not deeper.

E.g. T2.T3 = ..., should be e.g. T1.T2.T3 = ...

i.e. instead of just taking the parent here...

if type (self.getScope () .node) == ast.ClassDef:
    self.emit ('{}.{}'.format (self.filterId (self.getScope () .node.name), self.filterId (node.name)))

... we should go 'up/outside' and stop only when container is module. Maybe by remembering the container nodes and their names in a cache or so (?)

I'm a bit in time pressure so I can't create a PR for that, sorry, but maybe you also find it straight forward(?)

[JdeH]

Hi Gunther, good to hear from you again!

And thanks for signaling this problem and a probable solution. I'll take a look into this, probing the direction you propose.

You make me curious as to what exactly (or less exactly) 'our container orchestration' is. And I am also curious about the GUI framework you're working on.

About the static typing: I've used it a bit and I must say it finds an amazing amount of errors that would otherwise surface only late in a project. I guess Microsoft came to the same conclusion with Typescript.

In order to gradually gain trust for professional use, I invested some time in a new website that's more appealing to decision makers. Going mainstream will take time, though. There is so much movement in the web development area, it's not always easy for developers to pick out the things that can really help them. So far for recent developments...

[axgkl]

Hey.

First of all: Did not mention - but the website rocks!! Believe me I know the crazy effort in such a thing...

---

The 'our' in 'our container orchestration' is referring to my company ;-) Its about deploying, upgrading and maintaining many (1000s) based Linux containers on given server clusters - all from a 100% declarative spec file, which takes care for the generation of the filesystems within those containers as well - so you have a really deterministic setup and an API knowing 'everything' on the cluster - apps, packages, services, topology(...) . And for this I need a powerful GUI...

The GUI tool I'll put on github, maybe interesting for you, plus I'll send you screenshots of it when in production, for your gallery - but it'll take some time still...

---

Static typing, guess you are right, guess its the same feeling when you finally took to effort to write a test suite and realize that crazy amount of errors you would have never thought of being possible ;-)

But, unfortunately, my religious hatred of Python3 forbids that I look into it ;-)

Ok, seriously:

For the GUI App framework to be really modular and 'bendable' in projects I thought it makes a lot of sense to create inheritable classes encapsulating GUI functions which are run on client and server. In the js world I think they call this concept 'isomorphic'. But using Transcrypt this can be made far more convenient than anything seen, imho, with auto-syncronized state between client and server (I never lack big visions - ok, not that big than yours though ;-) ):

The components are connected via always on channels (using socketio), so state changes on the server can be seen on the client and vice versa. In any case, server and client functionality is kept at the same place, in those component modules.
Really cool is that server only functions and attributes I just overload or create in a descendent class with the same name in the same module, prevented from compiling into the client via pragmas. While the server does not care about what javascript calls are done in client only functions, python on the server does not look into those when not called.

Like this (after putting a no-op pragma into the builtins on the server):

class MyGrid(Grid):
    filter = '' # default state value
    def get_data(self):
          # dispatch 'get_data' function async, on server via websocket. 

    def on_data(self, data):
          self.cur_data_page_id = data['id']  # on server and client
          if on_client:
              self.render(data)  


__pragma__ ('ifdef', 'on_server')   
class MyGrid(MyGrid):   # the one seen by importers on the server
    def get_data(self):
        # load data from e.g. SQL and return on the MyGrid channel
        # having the current value of the filter on the client at hand automatically,
        # since the filter state change was dispatched to the server 
        # guaranteed before this get_data call.                

and the state on server AND client is a redux / redux.py store, parametrized by components names, which are (lazy) syncronized between client and server. Here the redux concept of global transactional state is really key, to avoid race conditions.

And in the project, one could simple overload the MyGrid class, so that it now fetches data from another store.
(...)

Lets see if it all turns out well but I have the feeling that large parts of the gui, at least all the state and data processing, should be testable on the server alone, using redux.py....

Why I'm bugging you with all of this?

Because you see that I'm dependent on the components being compiled on server and in Transcrypt - and on the server I'm running Python2. And, finally, thats the reason why type annotations are not an option for me at this time.

There are though, recent developments in Py3 where they seem to begin to understand that they made a huge error with their unicode sandwich idea.... asserting that any text operation shall only be possible if the bytes are decoded into unicode, an assertion which is absolutely crazy for an I/O company like mine. They seem to be willing to go into an asserted UTF-8 defaultencoding mode, like the rest of the world (Go, Rust, Elixir...) We invited Armin Ronacher next week, to discuss those positive developments and will see how we can speed up that process.

Maybe we go to Py3 then rather sooner than never - and then I'll for sure fall in love with your 'esoteric' type annot. stuff. And then I'll invite you for a few consulting days to Munich, since you have so much time ;-)

[pierrejean-coudert]

@axgkl I'm very interested in your GUI tool. When do you intend to put it on Github please ?

[JdeH]

Fixed & commited.
Will be in next bugfix release.

[axgkl]

@pierrejean-coudert : hey. you=interested in the transcrypt gui framework OR in the actual (container orchestration) use case implemented on top of it? First one is naturally earlier than the other (and already somewhat functional here) but as always, while doing the first heavy use case there might be changes required on the framework and I don't want people to get angry because of that while not having time to implement compat shims.

In any case, about timeline of the components framwork, I'll talk a little bit about the framework stack on pycon.de Munich end of October, then I latest should have sth useable on github.

[pierrejean-coudert]

@axgkl I'm really interested in working on a Redux or Elm like architecture with transcrypt.
I'm currently working on a VirtualDom implementation in python.

Defining a standardized/simplified way of sharing/synchronizing state between client and server could be really helpfull to advocate Transcrypt.

[axgkl]

Hi @pierrejean-coudert: Yes, I also somehow feel that a working undo function/timeslider proves that you have done sth "right" in your state management.

VDom in python: You mean runnable on client and server? I mean it would make not much sense (I guess) to transfer any state change to the server just to get the dom diffs but for testing the app on the server alone it would rock...

I must say I was up to now pretty conent with simple templating on the client, did not need the full 'any string' to diffable vdom (see riotjs versus react) - but I guess there are many use cases.

---

I'm currently doing this and this will be tldr; but maybe you'd like to comment, since I have nobody challenging me on design choices (in the company all are still using angular 1 and older tech... :x ) - so I'd appreciate nasty questions ;-)

(Have to explain first why - looking back a few months)

First Design (no transcrypt)

Before I met Transcrypt I was configuring my app on the server via nested classes, inheriting from 'Component' or from 'Property'.

Simplified:

from components import Page, Grid
from properties.form import InputBox
class Customers(Page):
       simplevar = 123
       class Filter(InputBox):
                value = ''
                label = ...
       class Result(Grid):
                 ....        

At startup that class tree was instantiated into an objects tree which got their state from a session store (see below)

• Also at start up from the server sided nested class structure I created nested riot components.
• All the simpletype values (with default from the class defs) I put into a flat redux store
  There was no websockets but on demand server state updates from the client (i.e. no server push but I always planned to include a backchannel anyway):

The classes could have event handlers (def on_change(...)) which caused the client to immediately call the server method at any state change, via a kefir based stream processing chain, just before dispatch into the reducer - and the answer form the server (with the data) was dispatching action SERVER_UPDATE as well.

On the server the object tree was instantiated just once, not for every user - but at state updates from the client the request got (and updated) the user's personal state by overloaded getattr and settr, which did look into a session store and not into the object's settings (which always just kept the initial default values).
So on the server dot operations on components delivered the correct state on the client, which made writing the event handlers real fun.

But:
Components and Properties did have a template property (which is ok but also a js property with all the client sided riot code - which after meeting transcrypt turned out to be hated by myself ;-))

Because with Transcrypt it was clear that I can forget riot js stuff, since this is just too canonical:

Current Design (with Transcrypt)

class MyGrid(Grid):
       statevar1 = <defaultvalue>
       class MyFilter(Filter):
                 ....
__pragma__(ifdef, 'on_server')
class MyGrid(MyGrid):
       def get_data(self):
                 <look into the database>

i.e. having really same shared code for client and server and just keep some data access functions on the server variant.

---

And that all really really asks for a two way state sync - i.e. for websockets or long polling, which socketio abstracts nicely [but I also currently check (again) wamp, which seems to provide for tons of more infrastructure (message routing) when its not just one server communicating with a client].

In any case, having committed on an always on infrastructure in place then the setattr/getattr hack can be dropped since the state is on the server, in the ram of the process which has established the first comm. socket.

My current thinking and implementation principles:

• Primary state of any session is in the browser's redux. Was looking into pyredux for a server store but I think that makes no sense to have two primary stores and reducers. There is just one store, running in the browser, and one reducer, managing any state change.

Syncing of client -> server

• I think not every action dispatch on the client must be sent to the server
• I think the design with event handlers on the server part of the component is nice as a decider if a state change must be hot synced or just remembered, transferred with the next hot event. The remembering can be done in a simple mutated dict on the client, forgetting all intermediate states (slider back and forth then click results in a transfer of only the current value of the slider).
  So histories would be smaller on the server (should you keep them).

Syncing of server -> client

• All server events need to be dispatched into the client's reducer when they happen.
• Client state rules: If a component instance is not present anymore on the client for which the server sent data (or sent a general state change) the client not only discards the udate but also runs a hot sync, so that the server stops sending not interesting traffic by also

---

Also:

• 'data' is not part of the store, just a data_id is updated in the store for any chunk of data.
• to not overload the server I use a (kefir backed) FRP infrastructure on the client - i.e. I can throttle the update rates even if I have many hot events (like in an autocomplete input box).
• The store update -> component updates are also handled via this, using kefir-redux (store is an observable).

---

Currently in a bit of project pressure but I hope I can send you some first implemented use case - in case I'll set sth up for the talk end of October.