Question About pid_d and Derivative Calculation in PID

[echavet]

Hi,

First of all, thank you for this great project and for making it available to the community! I've been using the PID functionality provided by the smart_thermostat module, and while analyzing the debug data, I noticed something that might be worth discussing.

From my understanding of the code, the derivative term (pid_d) is calculated based on the temperature change (di) between the last two samples and the time elapsed (dt), as shown in the formula:

if self.dt != 0:
    self.D = -(self._Kd * self._input_diff) / self.dt
else:
    self.D = 0.0

However, I observed that the impact of pid_d remains very low, regardless of how steep the temperature slope is over a longer period. This seems to occur because the derivative is computed only between two consecutive samples, rather than over a longer duration that might better represent the system's dynamics (e.g., the configured PWM period).

Would you consider this a limitation of the current implementation? Or is this behavior intentional (for stability reasons)? If the goal is to improve the sensitivity of pid_d, would it make sense to calculate the derivative based on the slope over a longer period, such as the PWM cycle duration, rather than only between two successive measurements?

Thank you for your time and for your amazing work on this project!

Looking forward to hearing your thoughts.

[echavet]

I reply to myself:
use sampling_period!

[ScratMan]

Hi, that's something I've been thinking about, but I had no time to implement it, and I lost my development environment with the trials I made; so now I would have to restart from scratch.
The idea would be to have a FIFO (with configurable size) storing samples (values and timestamps) to compute derivative between samples and perform rolling averaging.

But no time left at the moment.

[echavet]

Exactly!
I thought I could get a better pid_d by setting a sampling_period to 3 or 5 minutes. But it doesn't change the low values of the final derivative factor.

[echavet]

What would you think of such an approach?
I only change the way the derivative is processed when a PWM is used.

def __init__(
        self,
        kp,
        ki,
        kd,
        ke=0,
        out_min=float("-inf"),
        out_max=float("+inf"),
        sampling_period=0,
        cold_tolerance=0.3,
        hot_tolerance=0.3,
    ):
        #... 
        self._temp_history = deque()
        self._time_history = deque()

def calc(
        self,
        input_val,
        set_point,
        input_time=None,
        last_input_time=None,
        ext_temp=None,
        pwm=None,
    ):
    # ...

    # if pwm is defined, we use this period for derivative calculation
    if pwm != None:
        # Add current values to history
        self._temp_history.append(input_val)
    
        if input_time is None:
            input_time = time()
        else:
            self._time_history.append(input_time)
    # ...    

    if pwm == None:
                if self._dt != 0:
                    self._derivative = -(self._Kd * self._input_diff) / self._dt
                else:
                    self._derivative = 0.0
            else:
                self._derivative = self.process_derivative()


def _clean_history(self, now, pwm):
        """Remove old entries outside PWM window"""
        while self._time_history and (now - self._time_history[0]) > pwm:
            self._time_history.popleft()
            self._temp_history.popleft()

def process_derivative(self):
        # Calculate the average derivative over the PWM window
        if len(self._time_history) > 1:
            total_derivative = 0
            total_weight = 0

            # Iterate through all successive pairs of points in the history
            for i in range(1, len(self._time_history)):
                time_diff = self._time_history[i] - self._time_history[i - 1]
                temp_diff = self._temp_history[i] - self._temp_history[i - 1]

                if time_diff > 0:  # Avoid division by zero
                    weight = time_diff  # Weight by the duration of the interval
                    total_derivative += (temp_diff / time_diff) * weight
                    total_weight += weight

            # Compute the weighted average derivative
            if total_weight > 0:
                return -(self._Kd * total_derivative) / total_weight
            else:
                return 0.0
        else:
            return 0.0

[splichy]

Hi, maybe just having a separate sampling_period for Kd would be enough.
I'm trying to solve the same issue, in my case, heating is a bit oversized(5.5kW electric ceiling heating for 22m2 room = 250W/m2) its reaction time is quite quick - like 4-6min to first 0.1° change, then increase by 0.1 every 2 minutes,, when turned off temperature still increasing for next 8-10 minutes by 0.5-0.6 degrees, running it for 20min increase room temp by 1.8 degrees easily, with low sampling_period(0 or 1min) Kp and Ki works very well, but Kd becomes basically useless, On other hand having longer sampling period makes Kp & Ki less responsive.