New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simpler mechanisms for asynchronous processing (thoughts) #1380

Open
dhalbert opened this Issue Dec 6, 2018 · 10 comments

Comments

Projects
None yet
7 participants
@dhalbert
Copy link
Collaborator

dhalbert commented Dec 6, 2018

These are some strawman thoughts about how to provide handling of asynchronous events in a simple way in CircuitPython. This was also discussed at some length in our weekly audio chat on Nov 12, 2018, starting at 1:05:36: https://youtu.be/FPqeLzMAFvA?t=3936.

Every time I look at the existing solutions I despair:

  • asyncio: it's complicated, has confusing syntax, and pretty low level. Event loops are not inherent in the syntax, but are part of the API.
  • interrupt handlers: MicroPython has them, but they have severe restrictions: should be quick, can't create objects.
  • callbacks: A generalization of interrupt handlers, and would have similar restrictions.
  • threads: Really hard to reason about.

I don't think any of these are simple enough to expose to our target customers.

But I think there's a higher-level mechanism that would suit our needs and could be easily comprehensible to most users, and that's

Message Queues
A message queue is just a sequence of objects, usually first-in-first-out. (There could be fancier variations, like priority queues.)

When an asynchronous event happens, the event handler (written in C) adds a message to a message queue when. The Python main program, which could be an event loop, processes these as it has time. It can check one or more queues for new messages, and pop messages off to process them. NO Python code ever runs asynchronously.

Examples:

  • Pin interrupt handler: Add a timestamp to a queue of timestamps, recording when the interrupt happened.
  • Button presses: Add a bitmask of currently pressed buttons to the queue.
  • UART input: Add a byte to the queue.
  • I2CSlave: Post an I2C message to a queue of messages.
  • Ethernet interface: Adds a received packet to a queue of packets.

When you want to process asynchronous events from some builtin object, you attach it to a message queue. That's all you have to do.

There are even already some Queue classes in regular Python that could serve as models: https://docs.python.org/3/library/queue.html

Some example strawman code is below. The method names are descriptive -- we'd have to do more thinking about the API and its names.

timestamp_queue = MessageQueue()        // This is actually too simple: see below.
d_in = digitalio.DigitalIn(board.D0)
d_in.send_interrupts_to_queue(timestamp_queue, trigger=RISE)

while True:
    timestamp = timestamp_queue.get(block=False, timeout=None) // Or could check for empty (see UART below)
     if timestamp:    // Strawman API: regular Python Queues actually throw an exception if nothing is read.
        // Got an interrupt, do something.
        continue
        // Do something else.
```

Or, for network packets:
```
packet_queue = MessageQueue()
eth = network.Ethernet()
eth.send_packets_to_queue(packet_queue)
...
```

For UART input:
```
uart_queue = MessageQueue()
uart = busio.UART(...)
uart.send_bytes_to_queue(uart_queue)
while True:
    if not uart_queue.is_empty:
        char = uart_queue.pop()
```

Unpleasant details about queues and storage allocation:

It would be great if queues could just be potentially unbounded queues of arbitrary objects. But right now the MicroPython heap allocator is not re-entrant, so an interrupt handler or packet receiver, or some other async thing can't allocate the object it want to push on the queue. (That's why MicroPython has those restrictions on interrupt handlers.) The way around that is pre-allocate the queue storage, which also makes it bounded. Making it bounded also prevents queue overflow: if too many events happen before they're processed, events just get dropped (say either oldest or newest). So the queue creation would really be something like:
```
// Use a list as a queue (or an array.array?)
timestamp_queue = MessageQueue([0, 0, 0, 0])
// Use a bytearray as a queue
uart_queue = MessageQueue(bytearray(64))

// Queue up to three network packets.
packet_queue = MessageQueue([bytearray(1500) for _ in range(3)], discard_policy=DISCARD_NEWEST)
```

-----------------------------

The whole idea here is that event processing takes place synchronously, in regular Python code, probably in some kind of event loop. But the queues take care of a lot of the event-loop bookkeeping.

If and when we have some kind of multiprocessing (threads or whatever), then we can have multiple event loops.

@dhalbert dhalbert added the enhancement label Dec 6, 2018

@dhalbert dhalbert added this to the Long term milestone Dec 6, 2018

@dhalbert

This comment has been minimized.

Copy link
Collaborator Author

dhalbert commented Dec 6, 2018

For a different and interesting approach to asynchronous processing, see @bboser's https://github.com/bboser/eventio for a highly constrained way of using async / await, especially the README and https://github.com/bboser/eventio/tree/master/doc. Perhaps some combination of these makes sense.

@brennen

This comment has been minimized.

Copy link

brennen commented Dec 6, 2018

I'm unqualified at this point to talk about implementation, but from an end user perspective I like the idea of this abstraction quite a bit. It feels both like a way to shortcut some ad hoc polling loop logic that I suspect people duplicate a lot (and often badly), and also something that could be relatively friendly to people who came up on high-level languages in other contexts.

People aren't going to stop wanting interrupts / parallelism, but this answers a lot of practical use cases.

@deshipu

This comment has been minimized.

Copy link

deshipu commented Dec 6, 2018

I like event queues and I agree they are quite easy to understand and use, however, I'd like to point out a couple of down sides for them, so that we have more to discuss.

  1. Queues need memory to store the events. Depending on how large the event objects are, how often they get added and how often you check them, this can be a lot of memory. Since events are getting created and added from a callback, that has to be pre-allocated memory. And I don't know of a good way of signalling and handling overflows in this case — depending on use case, you might want to have an error, drop old events, drop new events, etc. To save memory you might want to have an elaborate filtering scheme, and that gets complex really fast.
  2. Queues encourage a way of writing code that introduces unpredictable latency. The way your code usually would flow, you would do your work for the given frame, then you would go through the content of all the event queues and act on them, then you would wait for the next frame. In many cases that is perfectly fine, but in some you would rather want to react to the event as soon as possible.
  3. Every new kind of queue will need a new class and its own C code for the callback and handling of the data. So if you have a sensor that signals availability of new data with an interrupt pin, you will need custom C code that will get called, read the sensor readings and put them on the queue. That means that all async drivers would need to be built-in.
  4. Sometimes a decision needs to be done while the event is being created, and can't wait. For example, in the case of the I2C slave, you need to ACK or NACK the data, and you have to hold the bus in a clock-stretch until you do.

That's all I can think of at the moment.

@framlin

This comment has been minimized.

Copy link

framlin commented Dec 19, 2018

Hm, I do not fully understand, why such a MessageQueue model should be easier to understand than callbacks. Maybe it's, because I am used to callbacks ;-)
What is so special with your target customers, that you think, they do not understand callbacks?

I think, you have to invest much more brain in managing a couple of MessageQueues for different types of events (ethernet, i2c, timer, exceptions, spi, .....) or one MessageQueue, where you have to distinguish between different types of events, than in implementing one callback for each type of event ant pass it to a built in callback-handler.

def byte_reader(byte):
deal_with_the(byte)

uart = busio.UART(board.TX, board.RX, baudrate=115200, byte_reader)

@siddacious

This comment has been minimized.

Copy link
Collaborator

siddacious commented Dec 19, 2018

I like the idea of message queues but I'm not convinced that they're any easier to understand than interrupt handers. Rather I think that conceptually interrupt handlers/callbacks are relatively easy to understand but understanding how to work with their constraints is where it gets a bit more challenging. Message queues are a good way of implementing the "get the operable data out of the hander and work on it in the main loop" solution to the constraints of interrupt handlers but as @deshipu pointed out, there are still good reasons to need to put some logic in the handler. Maybe both?

Similarly I like how eventio works but I think it's even more confusing than understanding and learning to work with the constraints of interrupt handlers. That in mind, it's tackling concurrency in a way that I think might be more relatable to someone who came to concurrency from the "why can't I blink two leds at once" angle.

One thing I was wondering about is what a bouncy button would do to a message queue. Ironically I think overflow might actually be somewhat useful in this case as if the queue was short enough you'd possibly lose the events for a number of bounces (but not all of them unless your queue was len=1. I'll have to ponder this one further). With a longer queue you could easily write a debouncer by looking for a time delta between events above a threshold.

No matter how you slice it, concurrency is a step beyond the basics of programming and I don't think any particular approach is going to allow us to avoid that. It seems to me that we're being a bit focused choosing a solution to a set of requirements that we don't have a firm grasp on yet. I think it's worth taking the time to understand who the users of this solution are and what their requirements are.

@notro

This comment has been minimized.

Copy link
Collaborator

notro commented Dec 20, 2018

See #1415 for an async/await example.

@deshipu

This comment has been minimized.

Copy link

deshipu commented Dec 21, 2018

What is so special with your target customers, that you think, they do not understand callbacks?

The problem is not with the callback mechanism itself, but in the constraint that MicroPython has that you can't allocate memory inside a callback. This is made much more complex than necessary by the fact that Python is a high level language with automatic memory management, that lets you forget about memory allocation most of the time, so it's not really obvious what operations can be used in a callback, and how to work around the ones that can't.

@notro

This comment has been minimized.

Copy link
Collaborator

notro commented Dec 22, 2018

One solution would be to enable MICROPY_ENABLE_SCHEDULER and only allow soft IRQ's, running the callback inline with the VM. This would prevent people from shooting themselves in the foot.

Refs:

@bboser

This comment has been minimized.

Copy link

bboser commented Dec 22, 2018

@dhalbert

This comment has been minimized.

Copy link
Collaborator Author

dhalbert commented Dec 26, 2018

Thank you all for your thoughts and trials on this. I'll follow up in the near future but am deep in Bluetooth at the moment. The soft interrupts idea and the simplified event loop / async / await stuff is very interesting. I think we can make some progress on this.

@dhalbert dhalbert changed the title Using message queues for asynchronous processing (thoughts) SImpler mechanisms for asynchronous processing (thoughts) Jan 8, 2019

@dhalbert dhalbert changed the title SImpler mechanisms for asynchronous processing (thoughts) Simpler mechanisms for asynchronous processing (thoughts) Jan 8, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment