Sample Broker seems to get lost when it gets out of sync

[jecollins]

I ran a sim last night with both server and sample broker on my desktop machine. Around timeslot 414, the server took nearly 5 seconds, and the sample broker's orders were still arriving when the market clearing began. Some of the orders ended up in the next market clearing, and the broker appears to have choked on one or more of the resulting market transactions. Eventually the server choked on a NPE:

Exception in thread "Thread-10" java.lang.NullPointerException
    at org.powertac.server.BrokerProxyService.localSendMessage(BrokerProxyService.java:94)
    at org.powertac.server.BrokerProxyService.broadcastMessage(BrokerProxyService.java:149)
    at org.powertac.server.WeatherService.activate(WeatherService.java:152)
    at org.powertac.server.CompetitionControlService.step(CompetitionControlService.java:557)

[jecollins]

Actually, the backtrace shown above is unrelated to the sample broker issue, and it seems that the sample broker is not causing the sim to crash. What we see in the backtrace is apparently what happens when the sim happens to end while the weather service is trying to retrieve data, which happens once every 24 timeslots. Presumably this error would go away with a solution to #348.

[jecollins]

I believe there is a problem with the threading model in the sample broker. The broker starts up, sets up the JMS channels, logs in, then waits for a signal that the sim is complete. All the actual processing happens in the threads managed by the jms infrastructure. Depending on how the JMS system manages its threads, this could easily lead to significant lag between the server's clock and the broker's clock, and therefore to broker messages being received too late in the server. I suggest we do all the processing other than receiving messages in the original thread, and cut off processing when the timeslot advances (when we receive the timeslot-update message).

[jecollins]

Added new TimeslotComplete message as last in timeslot, check for correct timeslot during processing. Change threading model to move broker processing out of jms threads, handle all message types to avoid error messages.