Two troubles with new 1v97 v on EspruinoWiFi

[espruino-discuss]

Posted at 2018-05-09 by Vladimir

Hi, with new 1v97 I've got this terrific trouble with “New interpreter error: FIFO_FULL”
I use EspruinoWiFi, which is managed through TCP client connection, to handle water valves, read water-flow sensors, read humidity sensors and so on. Therefore, with new 1v97 I have to disable all pins, that read any data! As soon as I initialize any of them WiFi is down!
And more troubles with 1v97. When initializing tcp client connection, double event-listeners are created (two onerror, two ondata and so on).

[espruino-discuss]

Posted at 2018-05-10 by @gfwilliams

Ahh, sorry - the double event listeners on TCP issue is something that we spotted yesterday and that literally just got fixed 15 minutes ago. I'll be doing a new release (1v98) with it in soon, but new firmwares from https://www.espruino.com/binaries/travis/master/ have a fix in.

However could you provide any more info on how the FIFO_FULL message comes about? Is it during upload? It's caused when there are too many characters/input events coming in to Espruino WiFi for it to handle.

What was the older firmware that worked for you? The FIFO_FULL checking and message has been around for a while, but we recently added flow control to Espruino WiFi to avoid losing characters from the ESP8266, it will now stop the ESP8266 from sending data if the fifo is looking full.

Could it be:

• You have sensors on B6/B7 (the default UART pins), but you haven't put USB.setConsole(1) in onInit? If so they could be causing lots of garbage data to be coming in to Serial1
• You're using setWatch on a pin that changes state very quickly, most of the time (eg. 1kHz or so)

If you're able to share your code (or a subset of it that exhibits the problem - ideally without requiring external hardware to be connected) I can take a look and try and figure out what the problem is.

---

Posted at 2018-05-10 by Vladimir

FIFO_FULL message comes on load.
1v96 worked fine. At least I've never recieved this mistake. But I could not be 100% sure with 1.96 as I'm in the procees of building my system and always update the code adding new fitures

I provide modified sample of the code which caused the problem. The solution to connect WiFi was to comment the code with flow-sensor and also do not put ANY pin in to the input mode. Flow-sensor is library written by Amperka and it woked absolutely fine with 1.96 and 1.95. I have not modified any part of this library

var HOSTS = ["192.168.1.92", "192.168.1.93", "192.168.1.95"];
var WIFI = [
  ["Alhimik" , { password : "" } ],  //0
  ["Alhimik_N", { password : "" }],  //1
  ["A-Alhimik ", { password : "" }]  //2
];
var INIT_OPTIONS = {"HOSTS": [0], 'WIFI': 0, 'ISSTART': 0, "Name": 'EspWater'};
var wifi = require("EspruinoWiFi");

var L = {
  'LED1': require("ledHandling")(LED1),
  'LED2': require("ledHandling")(LED2),
  'A5': require("ledHandling")(A5, 1),
  'A6': require("ledHandling")(A6, 1),
  'A7': require("ledHandling")(A7),
  'B1': require("ledHandling")(B1)
};

E.on('init', function() {
  USB.setConsole();
  console.log('started');
  L.LED1.On();
  //INIT_OPTIONS.ISSTART = 1;
  setTimeout(wifiOn, 1000);
});

var wifiOn = function(){
  console.log('starting WiFi');
  console.log(WIFI[INIT_OPTIONS.WIFI][0] + ' - ' + WIFI[INIT_OPTIONS.WIFI][1]);
  wifi.connect(WIFI[INIT_OPTIONS.WIFI][0], WIFI[INIT_OPTIONS.WIFI][1], function(err) {
    if (err) {
      console.log("Connection error: "+err);
      L.LED1.blink(50);
      setTimeout(wifiOn, 60000);
      return;
    }
    console.log("WiFi connected!");
    L.LED1.On(0); L.LED2.blink(100);
    for (var i in INIT_OPTIONS.HOSTS) {
      setTimeout(ccon, 5000 * i, INIT_OPTIONS.HOSTS[i]);
    }
  });
  return;
};

/*********
 FLOW-Sensor
 **********/
 var flowSensor = [];
 flowSensor[0]  = require('water-flow').connect(A0, {measurePeriod: 5});
 flowSensor[1]  = require('water-flow').connect(A1, {measurePeriod: 5});

---

Posted at 2018-05-10 by @gfwilliams

I just inlined the Amperka module (and made a version of ledHandling) and uploaded, but it uploads just fine for me with the latest EspruinoWiFi firmware:

Modules.addCached("water-flow",function() {
  
var WaterFlow = function(pin, opts) {
  this._pin = pin;

  this._pin.mode('input_pulldown');

  this._litres = 0;
  this._pulses = 0;

  this._pulseTimerID = null;

  this._speed = 0;

  opts = opts || {};

  this._avg = opts.averageLength || 10;
  this._pulsesPerLitre = opts.pulsesPerLitre || 450;
  this._minimumSpeed = opts.minimumSpeed || 1;

  this._litresPerPulse = 1 / this._pulsesPerLitre;
  this._speedNumerator = this._litresPerPulse * this._avg;
  this._updatePeriod = (60 * 1000) / (this._minimumSpeed * this._pulsesPerLitre);

  this._avgArray = new Array(this._avg); // [litres per second]
  this._avgIterator = 0;

  this.reset();

  this._watch();
};

WaterFlow.prototype._watch = function() {
  setWatch(this._onChange.bind(this), this._pin, {
    repeat: true,
    edge: 'rising',
    debounce: 1
  });
};

WaterFlow.prototype._average = function() {

  this._avgArray[this._avgIterator] = getTime();

  var last;
  if (this._avgIterator === this._avg - 1) {
    last = this._avgArray[0];
  } else {
    last = this._avgArray[this._avgIterator + 1];
  }

  var speed = this._speedNumerator / (this._avgArray[this._avgIterator] - last);

  if (++this._avgIterator === this._avg) {
    this._avgIterator = 0;
  }

  return speed;
};

WaterFlow.prototype._onChange = function() {
  this._pulses++;
  this._litres += this._litresPerPulse;

  if (this._pulseTimerID !== null) {
    clearTimeout(this._pulseTimerID);
    this._pulseTimerID = null;
    this._speed = this._average();
  }

  var self = this;
  this._pulseTimerID = setTimeout(function() {
    self._pulseTimerID = null;
    self._speed = 0;
    self.emit('drain');
  }, this._updatePeriod);

  this.emit('pulse');
};

WaterFlow.prototype.volume = function(units) {
  switch (units) {
    case 'l': return this._litres;
    case 'cm^3': return this._litres * 1000;
    case 'm^3': return this._litres / 1000;
    default: return this._litres;
  }
};

WaterFlow.prototype.reset = function() {
  var time = getTime();
  for (var i = 0; i < this._avg; ++i) {
    this._avgArray[i] = time;
  }
  this._litres = 0;
  this._pulses = 0;
};

WaterFlow.prototype.speed = function(units) {
  switch (units) {
    case 'l/min': return this._speed * 60;
    case 'cm^3/min': return this._speed * 60 * 1000;
    case 'm^3/min': return this._speed * 60 / 1000;
    default: return this._speed * 60;
  }
};

exports.connect = function(pin, opts) {
  return new WaterFlow(pin, opts);
};
});
Modules.addCached("ledHandling",function() {
  exports = function(pin) {
    return {
      blink:function(){ digitalPulse(pin,1,100); },
      On:function(){ pin.set();},
      Off:function(){ pin.reset(); },
    };
  };
});


var HOSTS = ["192.168.1.92", "192.168.1.93", "192.168.1.95"];
var WIFI = [
  ["Alhimik" , { password : "" } ],  //0
  ["Alhimik_N", { password : "" }],  //1
  ["A-Alhimik ", { password : "" }]  //2
];
var INIT_OPTIONS = {"HOSTS": [0], 'WIFI': 0, 'ISSTART': 0, "Name": 'EspWater'};
var wifi = require("EspruinoWiFi");
var L = {
  'LED1': require("ledHandling")(LED1),
  'LED2': require("ledHandling")(LED2),
  'A5': require("ledHandling")(A5, 1),
  'A6': require("ledHandling")(A6, 1),
  'A7': require("ledHandling")(A7),
  'B1': require("ledHandling")(B1)
};
E.on('init', function() {
  USB.setConsole();
  console.log('started');
  L.LED1.On();
  //INIT_OPTIONS.ISSTART = 1;
  setTimeout(wifiOn, 1000);
});
var wifiOn = function(){
  console.log('starting WiFi');
  console.log(WIFI[INIT_OPTIONS.WIFI][0] + ' - ' + WIFI[INIT_OPTIONS.WIFI][1]);
  wifi.connect(WIFI[INIT_OPTIONS.WIFI][0], WIFI[INIT_OPTIONS.WIFI][1], function(err) {
    if (err) {
      console.log("Connection error: "+err);
      L.LED1.blink(50);
      setTimeout(wifiOn, 60000);
      return;
    }
    console.log("WiFi connected!");
    L.LED1.On(0); L.LED2.blink(100);
    for (var i in INIT_OPTIONS.HOSTS) {
      setTimeout(ccon, 5000 * i, INIT_OPTIONS.HOSTS[i]);
    }
  });
  return;
};
/*********
 FLOW-Sensor
 **********/
 var flowSensor = [];
 flowSensor[0]  = require('water-flow').connect(A0, {measurePeriod: 5});
 flowSensor[1]  = require('water-flow').connect(A1, {measurePeriod: 5});

I can't see anything in there that'd cause problems either - unless there was loads of water (100ml/sec) flowing through your water flow sensors.

Can you try calling require("Storage").eraseAll()? It's possible that you had some code saved (with 'save on send, even after reset') that's doing something in the background that is causing problems.

---

Posted at 2018-05-10 by Vladimir

Big thanks for your respond. I will have a look on the problem on weekends and will try to determine the problem more precisely.

---

Posted at 2018-05-20 by Vladimir

I just inlined

Hi Mr. Gordon,
I don't know but your code is not uploading to my module (Web IDE 0.68.6, firmware 1.97) because inlined module could not be found. I tryied different minification settings but I have not found the way to upload the code.
Regarding errors:
1.98 solved the problem with TCP client
1.98 (as 1.97) I still can not use pins for reading data. I start water-flow module with delay of 50000 ms to let WiFi establish connection. But when I create water-flow sensors in couple of seconds I receive FIFO_FULL message.
I'm not profecial programmer, where should I use this: require("Storage").eraseAll()?

---

Posted at 2018-05-20 by Vladimir

Hi again,
Update, probably the problem with FIFO full solved.
What I did.

1. I added require("Storage").eraseAll() to the E.on('init', function()...
2. I changed sensor settings to

flowSensor[0]  = require('water-flow').connect(A0, {measurePeriod: 50});

I guess the problem that to many data come to the module. Now I clear the cash and recieve less bytes per second.
The module started to work propely! Great!

---

Posted at 2018-05-21 by @gfwilliams

Great! Sounds like there was already some code in memory from something else that might have been keeping Espruino too busy.

where should I use this: require("Storage").eraseAll()?

All you need to do is copy&paste it on the left-hand side. No need to add it to onInit as that'll remove any saved program code on boot (so if you power off & on the WiFi the code will be gone).

---

Posted at 2018-05-22 by Vladimir

So, finally I did a lot of different tests with Amperka library.
I can not fully avert "FIFO_full" error. But I managed to adapt my code (exclude some options) to avoid crash of WiFi connections.

1. I start initialisating of sensors with delay after the module has established connection to the server
2. I do noy use on "pulse" listener. I suppose this function consumes lots of memory

flowSensor[0].on('pulse', function () {})

---

Posted at 2018-05-22 by @gfwilliams

I would maybe avoid calling flowSensor[0].on('pulse', function () {} at all, even if the function has no code.

It looks like Amperka's library isn't expecting the number of pulses per second that you are getting from your sensors. In fact looking at it the whole thing is really inefficient if you have even 100 pulses/second.

You could just try using something like this instead:

function waterFlow(pin) {
  var lastPulses = 0, pulses = 0;
  var total = 0;
  pin.mode('input_pulldown');
  setWatch(function(){pulses++},pin, {repeat: true, edge: 'rising'});
  setInterval(function() {
    total += pulses;
    lastPulses = pulses;
    pulses = 0;
  }, 10000);
  return {
    volume : function() { return total; },
    speed : function() { return lastPulses; }
  };
};

Although you'd have to apply some scale factors to volume and speed.

---

Posted at 2018-05-22 by Vladimir

You could just try using something like this instead

Thanks, It's quite simple and more clear for me.
I implemented the code, but after some time I again recieve "New interpreter error: FIFO_FULL"
It's important to mention that no sensors are connected to the pins (I do testing at home without any sensors) A0 and A1

---

Posted at 2018-05-23 by @gfwilliams

It's important to mention that no sensors are connected to the pins

Wow, ok. I must have missed that - that is very surprising then.

So you're saying that if uploading the code from here: http://forum.espruino.com/conversations/320598/#14228511

Just leaving the board alone for a few minutes will cause a FIFO_FULL message? Or is there some other code I could upload to a bare Espruino WiFi to see if I can reproduce it?

---

Posted at 2018-05-23 by Vladimir

Just leaving the board alone for a few minutes will cause a FIFO_FULL message?

Well, After I have looked at this topic http://forum.espruino.com/conversations/311969/ I found that there is also some logic for FIFO_FULL message to appear in my case. What I do?
My server sends TCP request to Espruino to get consumption statistic from water-flow sensor.
After I read data from the variable (flowSensor[0].volume()), I send data to console.log and back to the server in JSON-format through TCP connection.
So I suppose the problem arise when simultaneously three functions execute their commands: setWatch, console.log, socket.write
I need more tests to locolize the problem, but for now it works for 20 hours with a real waterflow sensor connected to the pin A0 with this change in the code: I execute socket.write("JSON formated data") with a delay of 200 ms.!
If I occasionaly send simultaneously 2 TCP requests in a row, I will get FIFO_FULL error because to many instances of functions will occupy the input buffer

---

Posted at 2018-05-24 by @gfwilliams

I execute socket.write("JSON formated data") with a delay of 200 ms.!

How much data is there in the JSON?

And you are using one of the recent firmwares like 1v97 or later? Those should include flow control on the WiFi chip, so the Wifi itself shouldn't be able to trigger a FIFO_FULL unless something else is pushing lots of data in (but you say nothing is connected to the flow sensor pins?).

Actually, you could be hitting issues because of your console.log commands. It's fine when you're connected to a PC with a terminal application running, but when you're logging but no application is reading then Espruino will stall while waiting for the information to be sent.

Also, when you're running when disconnected from USB, the console will switch over to Serial on B6/B7, which is running at 9600 baud so would potentially slow things down. You could fix it using USB.setConsole(1) in onInit (without USB connected the data will then just be thrown away).

Is there any chance you could come up with a self-contained example you could send me that actually exhibits the problem?

---

Posted at 2018-05-24 by Vladimir

How much data is there in the JSON?

[["A0",49.06],["A1",0]]

And you are using one of the recent firmwares like 1v97 or later?

1v98 and 1v99

but you say nothing is connected to the flow sensor pins?

A0 pin connected to nowere produced to many "noise" - probably that triggered the problem. Now I connect the real sensor to the pin A0 to make situation close to the real life usage

Is there any chance you could come up with a self-contained example you could send me that actually exhibits the problem?

Well, I'm motivated to identify the problem if it still exists. But for now after your advices probably the solution is found. At least for now the test module works correctly for 1.5 day and I'm planning to use it in real life

---

Posted at 2018-05-24 by @gfwilliams

A0 pin connected to nowere produced to many "noise"

The pin mode should have been set to input_pulldown by the code? Something seems very wrong if you're getting inputs from it still. What if you upload just:

pinMode(A0,'input_pulldown');
setWatch(print,A0,{repeat:true});

Do you get anything on the console?

---

Posted at 2018-06-04 by Vladimir

It works for two weeks w/o errors.
What I did:

1. I use your code to work with water-flow sensor. I avoid calling
   flowSensor[0].on('pulse', function () {}
2. I do not call console.log!
   And, enclosed some pics how it works

---

Attachments:

• 5269cf46-a70f-4b58-b533-0f0bfc4380d7.jpg
  
• 9495dc8d-7822-438c-bcbb-80d059ea6f78.jpg
  
• a0d8a8e4-d71f-4bbb-9459-4cf172799ce9.jpg

---

Posted at 2018-06-05 by @gfwilliams

That's great! Thanks for letting me know!

Is it logging water flow, or does it also control solenoid valves as well?

---

Posted at 2018-06-05 by Vladimir

Is it logging water flow, or does it also control solenoid valves as well?

Of course it controls solenoids. You can find 5V 2-Channel Relay Module inside the plastic box. Solenoids are powered by 12v. I use Adafruit DC-DC converter to power the Espruino. So the hole sistem is powered by one DC trasnformer.
And I plan to add Soil Humidity Sensor Modules for better control and more statistic.