Softwareserial doesn't work at higher baudrates

[matthijskooijman]

Trying to use SoftwareSerial at 115200 on an Arduino Uno only gave me garbage output. After trying to fix the timings, things improved, but I've still not managed to fix everything.

To be able to reliably and repeatedly test the SoftwareSerial class, I
wrote a small python script. It talks to the Uno's USB serial and to
a separate USB serial convertor (I used this one and a
cp210x one, but any one should do).

The serial convertor is connected to pin 2 and 3 of the Uno, which
creates a SoftwareSerial object on those pins. The sketch on the Uno
then forwards data between SoftwareSerial and the regular Serial object.

Additionally, using some binary commands that are not forwarded, the
python script can tell the sketch to open and close the SoftwareSerial
port and specify the baudrate to use.

I tested all rates listed in the SoftwareSerial timing table and found
that reception of the faster rates (and somehow, also 300 baud) failed.
For transmission, it's the reverse: Only the higest rates work, the
lower ones fail. Using gcc 4.8, things are even worse - none of the TX
rates works.

I tested all the rates listed in the SoftwareSerial timing table and
found that TX works every time (at 16Mhz) and fails at 115200 (at 8Mhz), but RX
fails at 115200 baud (at 16Mhz) and 38400 and up (at 8Mhz).

31250 baud also failed, but this appears to be a problem in the cp210x I
was using, using the Arduino USBSerial board 31250 worked (but 300 baud
failed...). When using gcc 4.8, RX baudrates from 28800 and up fail.

I tested the 1.5 branch with both the bundled gcc 4.3 version and a
locally compiled gcc 4.8 version and the master branch with just the
bundled 4.3 version. This was tested on a Uno, running at 16Mhz and at
8Mhz (using the prescaler register in the sketch below and setting the
cpu speed in boards.txt).

I'll see if I can fix this, but wanted to provide some documentation
about this issue already.

Here's the output from 1.5 with gcc 4.3 at 8Mhz.

Waiting for Arduino to start up
Testing at 300 baud
RX: Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
TX: Lorem ipsum dolor sit amet, consectetuer adipiscing elit.

Testing at 600 baud
RX: Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
TX: Lorem ipsum dolor sit amet, consectetuer adipiscing elit.

Testing at 1200 baud
RX: Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
TX: Lorem ipsum dolor sit amet, consectetuer adipiscing elit.

Testing at 2400 baud
RX: Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
TX: Lorem ipsum dolor sit amet, consectetuer adipiscing elit.

Testing at 4800 baud
RX: Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
TX: Lorem ipsum dolor sit amet, consectetuer adipiscing elit.

Testing at 9600 baud
RX: Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
TX: Lorem ipsum dolor sit amet, consectetuer adipiscing elit.

Testing at 14400 baud
RX: Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
TX: Lorem ipsum dolor sit amet, consectetuer adipiscing elit.

Testing at 19200 baud
RX: Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
TX: Lorem ipsum dolor sit amet, consectetuer adipiscing elit.

Testing at 28800 baud
RX: Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
TX: Lorem ipsum dolor sit amet, consectetuer adipiscing elit.

Testing at 31250 baud
RX: &\xa5K\xb5\xa8\x16g\xbd-\x a}hW\x17\xb5S\xbc\xa8\xb6Ui\x94\xa8\xf6eg\xb5\x96K\xbcoJA\xb1\xa9\x16S\xbbV\xe3O\x90-\x15i\x96
TX: \x18\xff\xdf\xff\xe4\xff\xcb\xff\xdb\xff\xc0\xd3\xff\xe0\xff\xe7\xff\xeb\xff\xdb\xff\xc0\xc8\xff\xdf\xff\xd8\xff\xdf\xff\xe4\xff\xc0\xe7\xff\xd3\xff\xe8\xff\xc0\xc3\xff\xdb\xff\xcb\xff\xe8\xff\xd8\xc0\xc7\xff\xdf\xff\xdc\xff\xe7\xff\xcb\xff\xc7\xff\xe8\xff\xcb\xff\xe8\xff\xeb\xff\xcb\xff\xe4\xff\xc0\xc3\xff\xc8\xff\xd3\xff\xe0\xff\xd3\xff\xe7\xff\xc7\xff\xd3\xff\xdc\xff\xcf\xff\xc0\xcb\xff\xd8\xff\xd3\xff\xe8\xff\xdc

Testing at 38400 baud
RX: Lore\xdd \xc9psum@d\xdflor \xe3i\xe4 ame\xec,@conse\xc3t\xc5tuer@adip\xd1s\xc3ing \xcdl\xc9t.
TX: Lorem ipsum dolor sit amet, consectetuer adipiscing elit.

Testing at 57600 baud
RX: L\xdfye\xed \xe9p\xf3u\xed \xe4o\xeco\xf2 \xf3i\xf4 \xc1m\xc5tL \xc3o\xces\xc5c\xe4e\xe4u\xc5r@a\xc4i\xe0i\xe3c\xd1n\xcf\x10dl\xd1t^
TX: Lorem ipsum dolor sit amet, consectetuer adipiscing elit.

Testing at 115200 baud
RX: NyW\x17Q\x13\xf3\xf5\xcd@\xe4\xef\xec\xef\xf2`\xf3\xe8\xf4 \xe0\x9b\xd8\xdc\xcb\xc8\x96y\x9d%\x d\xb15\xb1\x95\x15\x89\x 0\x 4\x90\xe8\xf0\xe8\xf3\xe3\xe8\xef\x 1-aM\xa1y\xff
TX: \x98\xde\xe4\xca\xda@\xd2\xe0\xe6\xea\xda@\xc8\xde\xd8\xde\xe4@\xe6\xd2\xe8@\xc2\xda\xca\xe8X@\xc6\xde\xdc\xe6\xca\xc6\xe8\xca\xe8\xea\xca\xe4@\xc2\xc8\xd2\xe0\xd2\xe6\xc6\xd2\xdc\xce@\xca\xd8\xd2\xe8\

Here's the python script I was using (requires python 3.3):

#!/usr/bin/python3

import sys
import time
import struct
import serial
import threading

tester = serial.Serial()
tester.port = "/dev/serial/fonera"

dut = serial.Serial()
dut.port = "/dev/ttyACM0"
dut.baudrate = 57600

text = "Lorem ipsum dolor sit amet, consectetuer adipiscing elit."

def read(prefix, ser, timeout):
  sys.stdout.write(prefix)
  sys.stdout.flush()

  ser.timeout = .1
  end = time.monotonic() + timeout
  while (time.monotonic() < end):
    def printable(c):
      return c >= 0x20 and c < 0x7f

    data = ''.join(chr(c) if printable(c) else "\\x{:2x}".format(c) for c in ser.read())
    sys.stdout.write(data)
    sys.stdout.flush()

  sys.stdout.write("\n")

def testBaud(rate):
  sys.stdout.write("Testing at {} baud\n".format(rate))

  # First open the tester, then enable the dut, to prevent garbage on the line
  # from opening the tester being reported.
  tester.baudrate = rate
  tester.open()
  time.sleep(0.1)

  # Switch device-under-test to the new baudrate, by sending a byte with value
  #1 followed by the baudrate as a big-endian 32-bit number
  dut.write(struct.pack(">BI", 1, rate))
  time.sleep(0.1)

  # Test RX
  tester.write(text.encode('ascii'))
  read("RX: ", dut, len(text) / rate * 16 + 1)

  # Test TX
  dut.write(text.encode('ascii'))
  read("TX: ", tester, len(text) / rate * 16 + 1)
  sys.stdout.write("\n")

  # Switch the dut reception off first, and then the close the tester
  dut.write(struct.pack(">B", 2))
  time.sleep(0.1)

  tester.close()

# Open the device-under-test port, this causes the Arduino to reset
dut.open()

sys.stdout.write("Waiting for Arduino to start up\n")
time.sleep(3)

testBaud(300)
testBaud(600)
testBaud(1200)
testBaud(2400)
testBaud(4800)
testBaud(9600)
testBaud(14400)
testBaud(19200)
testBaud(28800)
testBaud(31250)
testBaud(38400)
testBaud(57600)
testBaud(115200)


# vim: set sw=2 sts=2 expandtab:

And the Arduino sketch:

#include <SoftwareSerial.h>

SoftwareSerial ss(2, 3);

void setup() {
  if (F_CPU == 8000000L) {
    CLKPR = (1 << CLKPCE);
    CLKPR = 1;
  }

Serial.begin(57600);
}

bool listening = false;

void loop() {
  int c = Serial.peek();
  if (c == 1) {
    if (Serial.available() >= 5) {
      Serial.read();
      uint32_t baud = ((uint32_t)Serial.read() & 0xff) << 24 | 
                      ((uint32_t)Serial.read() & 0xff) << 16 |
                      ((uint32_t)Serial.read() & 0xff) << 8 |
                      ((uint32_t)Serial.read() & 0xff);

      ss.begin(baud);
      listening = true;
    }
  } else if (c == 2) {
    Serial.read();
    ss.end();
    listening = false;
  } else if (c != -1) {
    static int avail = 0;
    delay(10);
    // Forward any other bytes to SoftwareSerial. However, take care to
    // only do this when we haven't received any bytes in the past 10ms,
    // since SoftwareSerial::write at lower baudrates will break regular
    // Serial reception due to disabled interrupts.
    if (Serial.available() == avail) {
      Serial.read();
      ss.write(c);
      --avail;
    } else {
      avail = Serial.available();
    }
  }

  while (listening && ss.available())
    Serial.write(ss.read());
}

[matthijskooijman]

Oh, the 300 baud problems seem to be unrelated to SoftwareSerial. I just tried using a CP2102 USB->Serial convertor, and RX at 300 baud works perfectly. I suspect that the Arduino Serial convertor I was using before, was suffering from this Arduino bug as well.

[matthijskooijman]

And it seems most of the TX problems were caused by an oversight in my test setup: When SoftwareSerial is transmitting at low baudrates, it disables interrupts for long periods of time, which cause incoming bytes at the HardwareSerial side to be dropped. I modified the sketch and test output in the first post to first receive all bytes and only then start transmitting them with SoftwareSerial, which fixes TX at lower rates (its still broken at 115200 at 8Mhz, though).

[PaulStoffregen]

That's exactly why I wrote AltSoftSerial...

[matthijskooijman]

@PaulStoffregen Yeah, I think AltSoftSerial works more reliably at higher rates, but it also works on specific pins only and needs timers AFAIK, so it's not usable in all situations (so both libraries have their place and can co-exist, I think).

[admblake]

Does AltSoftSerial run at the MIDI baudrate of 31250? I'm trying to swap SoftwareSerial out for AltSoftSerial (setup with the appropriate pins of 9 and 10) and seeing my data stream while using SoftwareSerial but not seeing anything when using AltSoftSerial.

Edit: My apologies. Pilot error. The Arduino needed pins 9 and 8 (or 8 and 9). Either way, swapping SoftwareSerial for AltSoftSerial works like a champ while using the MIDI Library 4.1 Thanks!

[matthijskooijman]

Not sure about AltSoftSerial, but this is definately not the right place to ask. Doesn't the library have it's own support forum/list/issuetracker?

[PaulStoffregen]

Since writing AltSoftSerial years ago, I've switched most of my development focus from AVR to ARM chips, where 3+ hardware serial ports are available. There's no issue tracker or other active development on AltSoftSerial, but it still works well (far better than SoftwareSerial, if you can spare the timer hardware).

[sabues]

Just wanted to share, that I've solved this problem @38400 receiving from a serial GPS (which baud rate cannot be changed), by tweaking the delay_table. I knew it was a sync issue since the GPS worked well with physical serial.
First print your F_CPU to make sure you modify the right table. Then start to increase / decrease rxintra and rxstop, find their upper/lower working limits and choose a value in the middle.
Once I made it work for the MEGA, It just worked with the same values for the Yun.

Just for reference the original values for
{ 38400, 25, 57, 57, 54, },
I ended up with
{ 38400, 25, 53, 53, 54, },

Hope it helps.

[matthijskooijman]

The delay table is entirely removed by the commit fixing this issue, so a recent 1.5.x and all 1.6.x versions should no longer have the delay table (and work with 38400 out of the box). I take it you're still using an older version, or perhaps have an old version of the SoftwareSerial library lying around?

[sabues]

Oh man, I've been fighting with this for two days, you're absolutely right, I had a SoftwareSerial folder under libraries. I store my sketchbook location on a cloud based storage, and it must be left behind. The current library worked untouched. Thanks!