Future Improved IO Plans/Thoughts

[mossblaser]

• Blocking interfaces should always be present and should continue to work just like they do now
• Plenty of performance is lost to waiting for roundtrips on SCP commands
  • Must ask ST what restrictions apply to current implementations/his thoughts on having multiple open SCP requests
  • In many cases, blocking functions make lots of potentially-parallel SCP calls (e.g. get_machine), thus many functions with blocking interfaces would end up being faster for free.
• In the future, opportunities will exist for using multiple endpoints simultaneously for SCP
  • Should ask ST about side-effects of communicating with the system from multiple ethernet (or otherwise) links. I would guess none.
  • It might not be necessary/beneficial to limit comms to just use of the local ethernet connection
• Alternative interface should allow the queuing/dispatching of many SCP commands at once
  • Should, behind the scenes, schedule these commands appropriately (i.e. limit maximum outstanding commands)
  • Some plan needs to be implemented to handle deferred responses
    • Callbacks on completion (messy)
    • Use a "real" framework for this (e.g. Twisted)
    • Methods could be "two-phase" generators
      • Calling the method yields after the command is either queued or sent
      • Doing a next() on the generator would then produce the received response, blocking until the command has both been sent and the response received. The generator could then yield the response or throw an appropriate exception (e.g. timeout)
      • Potentially, enabling this mode could simply be an argument of all the existing commands -- better still, it would make it possible to use a decorator for this function. Also, as an argument it can be set by the context -- schweet.
      • Additionally provide some flush-style function which blocks until all commands have been sent and responded to.
      • Clearly JH has taken a liking to this idea

[mundya]

• Use a "real" framework for this (e.g. Twisted)

Twisted would require rethinking most of the API, for the record I'm against using it.

[mossblaser]

Just had a chat with ST:

• There is a hard-limit on the number of outstanding connections going through a ethernet-connected monitor imposed by the transient IPTag pool size (a transient IP tag is created for every SCP cmd). This can be queried from the machine. This can also be cheaply extended if it proves valuable.
• If the bandwidth/latency ratio for loading a core is sufficiently good (unverified but highly plausible) then the only situations where parallel requests are useful is when many cores are being loaded/polled simultaneously.
• Having multiple outstanding SCP requests where each request is to a different core doesn't require any new flow-control mechanisms in SCP.
• ST recommends only using the ethernet connection local to a given chip for comms.

May not be too bad!

[mossblaser]

So first off, here's a sketch of how implementing/using the non-blocking stuff could look:

class MachineController(object):

    ...

    def my_non_blocking_thing():
        # Queues/sends the command
        token = self.send_scp(...)
        yield

        # Either returns the already-recieved response or blocks until response
        # has been received. This function contains a mainloop which
        # sends/receives queued commands.
        packet = self.wait_for_response(token)
        yield packet.data

# Make a load of requests
outstanding_requests = []
for _ in range(10):
    gen = mc.my_non_blocking_thing()
    next(gen)  # Triggers the sending of the command
    outstanding_requests.append(gen)

# Now make use of the replies
for gen in outstanding_requests:
    print(next(gen))

Of course, blocking interfaces are easy and actually are all people want half the time. By adding a simple decorator to the above API you can make it look blocking. Even better, that decorator can make the async-version accessible via an option:

def look_blocking(f):
    """Decorator."""
    def f_(*args, **kwargs, async=False):
        g = f(*args, **kwargs)
        if async:
            return g
        else:
            next(g)
            return next(g)
    return f_

class MachineController(object):

    ...

    @look_blocking
    def my_non_blocking_thing():
        token = self.send_scp(...)
        yield
        packet = self.wait_for_response(token)
        yield packet.data

# Blocking mode (default)
print(mc.my_non_blocking_thing(...))

# Non-blocking mode
gen = mc.my_non_blocking_thing(..., async=True)
next(gen)
response = next(gen)

/me is really quite excited by this interface and based on the excuse that "the test suite/get_machine is zlow over the web" that's enough motivation for me to start working on this...

[mundya]

I'm interested to see how this turns out.

[mossblaser]

Further chatting with ST:

• Existing SCAMP/SARK may have slight issues with being sent n_transient_iptags_full_size_scp packets at once: new coming-soon SCAMP/SARK have *much_ more buffering and won't blink at this.
• The transient tags have a timeout after while they are re-cycled if no SCP reply comes back (e.g. due to network congestion/code deadness).
  • Since retransmitting a packet requires a new transient tag, retransmit timeouts must be long enough to allow this timeout to expire to free up the transient tag (if out of transient tags, the machine just silently drops the SCP packet).
  • Current default transient tag time is 2-or-so seconds. According to ST, this should be very safe to change to 10ms since no SCP command should ever take that long to run and the network should never be that slow, even accounting for network delays within SpiNNaker
  • Transient tag timeouts can be read/set using the IPTAG_TTO command. Timeouts are encoded as 10ms * (1 << (v - 1)). This command also gives the number of transient tags.

[mossblaser]

Regrettably I'm mothballing this implementation on account of its lacklustre performance. Follow #77 for its successor!