Four I2C sensors - I2C bus deadlock

[chrfriese123]

Dear all,

For quite some time I used two I2C sensors (one air quality and one temperature/humidity sensors) with a ESP8266 device with Tasmota 9.x based firmware. I compile my firmware using gitpod. This works quite efficient without any issues.

For a second project I duplicated the sensors to be able to have a relative measurement inside and outside a room. So now I have two air quality sensors and two temperature humidity sensors running.

After some time the sensors stop working and I2CScan provides an output Error 4 at 0x01. Most likely that the I2C bus is in deadlock. The root cause is not clear to me yet. But in case someone has similar issues with I2C communication, I'd like to propose a fix based on https://github.com/maarten-pennings/I2Cbus

In support.ino I added the following function

bool I2CClear(char *devs, unsigned int devs_len) 
{
  // Make SDA pin (data) and SCL pin (clock) inputs with pull-up.
  pinMode(Pin(GPIO_I2C_SDA), INPUT_PULLUP); 
  pinMode(Pin(GPIO_I2C_SCL), INPUT_PULLUP);
  delay(2500);

  // Check status of SCL pin
  int scl_low = digitalRead(Pin(GPIO_I2C_SCL))==LOW; 
  if( scl_low ) { 
    // Can not correct SCL line being low
    snprintf_P(devs, devs_len, PSTR("{\"" D_CMND_I2CCLEAR "\":\"" D_JSON_I2CCLEAR_SCL_LOW "\"}"));
    return false; 
  }
 
  // Check status of SDA
  int sda_low = digitalRead(Pin(GPIO_I2C_SDA))==LOW;
  if( !scl_low && !sda_low ) {
    // Bus is ok (SDA and SCL high)
    Wire.begin(Pin(GPIO_I2C_SDA), Pin(GPIO_I2C_SCL));
    snprintf_P(devs, devs_len, PSTR("{\"" D_CMND_I2CCLEAR "\":\"" D_JSON_I2CCLEAR_OK "\"}"));
    return true; 
  }
 
  // Give SCL pulses, see https://www.nxp.com/docs/en/user-guide/UM10204.pdf#page=20
  int clockCount = 20; // > 2x9 clock
  while( sda_low && (clockCount>0) ) { 
    clockCount--;
    // Note: I2C bus is open collector so do NOT drive scl or sda high.
    pinMode(Pin(GPIO_I2C_SCL), INPUT);        // release scl pullup so that when made output it will be LOW
    pinMode(Pin(GPIO_I2C_SCL), OUTPUT);       // then clock scl Low
    delayMicroseconds(10);      // for >5uS
    pinMode(Pin(GPIO_I2C_SCL), INPUT);        // release scl LOW
    pinMode(Pin(GPIO_I2C_SCL), INPUT_PULLUP); // turn on pullup resistors again
    // Do not force high as slave may be holding it low for clock stretching.
    delayMicroseconds(10);      //  for >5uS
    // The >5uS is so that even the slowest I2C devices are handled.
    scl_low = digitalRead(Pin(GPIO_I2C_SCL))==LOW; // Check if scl is Low.
    int counter = 20;
    while (scl_low && (counter > 0)) {  // loop waiting for scl to become High only wait 2sec.
      counter--;
      delay(100);
      scl_low = digitalRead(Pin(GPIO_I2C_SCL))==LOW;
    }
    if (scl_low) { // still low after 2 sec error
      // I2C bus error. Could not clear. SCL clock line held low by slave clock stretch for >2sec
      snprintf_P(devs, devs_len, PSTR("{\"" D_CMND_I2CCLEAR "\":\"" D_JSON_I2CCLEAR_SCL_2SEC "\"}"));
      return false; 
    }
    sda_low = digitalRead(Pin(GPIO_I2C_SDA))==LOW; 
  }
  
  // Recheck SDA
  if( sda_low ) { 
    // I2C bus error. Could not clear. SDA data line held low
    snprintf_P(devs, devs_len, PSTR("{\"" D_CMND_I2CCLEAR "\":\"" D_JSON_I2CCLEAR_SDA_LOW "\"}"));
    return false; 
  }
 
  // Pull sda line low for Start or Repeated Start
  pinMode(Pin(GPIO_I2C_SDA), INPUT); // remove pullup.
  pinMode(Pin(GPIO_I2C_SDA), OUTPUT);  // and then make it LOW i.e. send an I2C Start or Repeated start control.
  // When there is only one I2C master a Start or Repeat Start has the same function as a Stop and clears the bus.
  delayMicroseconds(10); // wait >5uS
  pinMode(Pin(GPIO_I2C_SDA), INPUT); // remove output low
  pinMode(Pin(GPIO_I2C_SDA), INPUT_PULLUP); // and make sda high i.e. send I2C STOP control.
  delayMicroseconds(10); // wait >5uS
  pinMode(Pin(GPIO_I2C_SDA), INPUT); // and reset pins as tri-state inputs which is the default state on reset
  pinMode(Pin(GPIO_I2C_SCL), INPUT);
  
  // Bus repaired from low SDA
  Wire.begin(Pin(GPIO_I2C_SDA), Pin(GPIO_I2C_SCL));
  snprintf_P(devs, devs_len, PSTR("{\"" D_CMND_I2CCLEAR "\":\"" D_JSON_I2CCLEAR_OK_SDA "\"}"));
  return true; 
}

I call this function twice with I2CClear(TasmotaGlobal.mqtt_data, sizeof(TasmotaGlobal.mqtt_data));. First time when the sensor detect function is executed to release the I2C bus after any startup / restart / power loss etc. And the second time as part of the update function in in case the sensor read-out/measurement fails.

With this setup it works for myself for quite some time now.

If someone has a clue / idea what could cause the I2C deadlock and how to prevent it, please let me know.
Otherwise it might be an interesting feature to add. Feedback welcome.

Additional changes:
i18n.h

#define D_JSON_I2CCLEAR_SCL_LOW "Can not correct SCL being low"
#define D_JSON_I2CCLEAR_OK "Ok (SDA and SCL are high)"
#define D_JSON_I2CCLEAR_SCL_2SEC "SCL held low by slave clock stretch for >2 sec"
#define D_JSON_I2CCLEAR_SDA_LOW "SDA still low after bus clear"
#define D_JSON_I2CCLEAR_OK_SDA "Bus repaired from low SDA"
#define D_CMND_I2CCLEAR "I2CClear"

support_command.ino - line 32

D_CMND_I2CSCAN "|" D_CMND_I2CDRIVER "|" D_CMND_I2CCLEAR "|"

support_command.ino - line 59

  &CmndI2cScan, CmndI2cDriver, CmndI2cClear,

support_command.ino - after void CmndI2cDriver(void) {...}

void CmndI2cClear(void)
{
  //Response_P(PSTR("{\"" D_CMND_I2CCLEAR "\":"));
  I2CClear(TasmotaGlobal.mqtt_data, sizeof(TasmotaGlobal.mqtt_data));
  ResponseJsonEnd();
}

[sfromis]

The root cause of Error 4 at 0x01 is usually bad wiring/connections, with fixing to focus on making that correct. Sometimes ensuring appropriate pullup resistors on the I2C bus can help, depending on devices.

And most Tasmota I2C drivers only supports one device of the relevant type (a few drivers do support two or even more).

[chrfriese123]

Thanks for the hint, the connections itself (3V3, GND, SDA & SCL) is correct. The wiring is also acceptable, not too long. Proper pullup resistors (10k) are added to SDA and SCL. I'll check the signal shape with an oscilloscope later.

After power-on the sensors & communication works well. The deadlock happens only after some minutes/hours randomly. It could only be recovered with a full power disconnect. Or now with this I2CClear command.

Indeed, the sensors I'm using are not existing in the official Tasmota releases. So I created my own drivers that do support two devices (with different I2C addresses of course).

[Jason2866]

As sfromis wrote wiring is very important. I2C bus is made for short connecting. 50 cm is already a very long wiring. If you have such long wires replace the pull up resistors (usually 10k) with 4k7 or even 1k ones. Take care the bus is correctly terminated.
About the I2C bus clear. There was heavily development in Arduino core since this https://github.com/maarten-pennings/I2Cbus In short it is not needed anymore, since there is a solution in Arduino ESP8266 core. For ESP32 i dont know. We did no heavily tests nor we checked the code. So ESP32 experience can differ from ESP8266
In general i2c driver writing can be a challenge finding the reason why it "hangs the bus"
If there are no problems with the standard drivers Tasmota supports and problems are arriving with the new driver the dive in the rabbit hole starts.
To double the fun, there are i2c devices which do not care about the i2c protocol.
You can simply not use this with other ones on the bus.
A logic analyzer and "some" research about i2c and you are "in"
Maybe interesting https://github.com/esp8266/Arduino/issues?q=Clock+stretch

[Jason2866]

Many Arduino libs available for sensors are badly written. In Tasmota most of them are optimized.