extruder: Implement non-linear extrusion

[Tobba]

This allows klipper to compensate for slippage in the extruder at high back-pressures.
When enabled through SET_NONLINEAR_EXTRUSION, or nonlinear_a/b in the configuration,
the actual extrusion velocity will be modified to v(t) + Av(t)^2 + Bv(t)^3
where v(t) is the requested velocity, and A/B the provided parameters.

This is similar to RepRap's M592 command, except correctly applied to accelerating moves
rather than simply scaling the distance of the entire move based on its average velocity.

This PR isn't really ready yet, but I need comments from someone more familiar with the kinetics code. I'm not totally sure if the way the additional extrusion from previous moves is applied onto future moves is okay, or if there's a better way to do it.

Also, unlike M592, this is currently being applied to extruder-only moves, which will screw up retractions (but makes testing easier)

[KevinOConnor]

Interesting.

This PR isn't really ready yet, but I need comments from someone more familiar with the kinetics code. I'm not totally sure if the way the additional extrusion from previous moves is applied onto future moves is okay, or if there's a better way to do it.

Well, currently the high-level code and the low-level code both track an absolute stepper motor position. The "trapq moves" encode that absolute position and then the stepper step times are generated from the trapq. What's your high-level plan for tracking the absolute stepper motor position?

-Kevin

[Tobba]

Currently, the added travel from past moves is tracked inside the extruder_kinematics struct, modified through a new post_move callback from itersolve_generate_steps after it finishes processing each move, and this value is added onto the position in calc_position. The high-level code is completely unaware of the added distance and simply feeds in its desired filament movement.

This means that calc_position will only return correct values for the current move, placing the constraint on itersolve to only evaluate one move at a time (which seems to hold in the current code, but could pose a problem in the future?). The "previous move" handling and the fact that itersolve_generate_steps doesn't actually remove processed steps from the queue itself also seemed somewhat concerning w.r.t the correctness of the post_move callback.

A possible alternative to this would be modify the starting position of moves as they enter the trapq, but this would be visible to and affect pressure advance in presumably undesirable ways. Or doing it in the RepRap-firmware style in the high-level movement code by simply scaling the move distance based on velocity, but this obviously wouldn't correctly scale accelerating moves.

Also, by "not ready yet" I meant that it's only a barebones implementation of the feature and needs more work before it's ready to merge (i.e changing the parameters while extruder moves are queued causes a mysterious crash, needs some cleanup, no docs, etc).

[KevinOConnor]

Okay, thanks. Alas, I haven't had a chance to look at this in-depth. What's the next steps from your perspective? Are you looking for users to test it and report results, or are you looking for some further feedback on the implementation?

Thanks again,
-Kevin

[gluap]

edited to reflect @Tobba's suggestions below

@Tobba I had measured underextrusion at different extrusion rates previously. Therefore I had all the data to test your feature ready. A simple test extrusion (20mm of filament extruded at 10mm³/s) was underextruding by about 17% on the bare printer. With the same print settings but using your nonlinear extrusion feature the same extrusion came in within 1-2% of the expected weight (I'm weighing the test extrusions with a scale that resolves .01g, so for the .60g of filament I'm test extruding I won't get more precise than that).

I'll shortly summarize the steps I followed to use your code, in case others want to test:

1. Measure extrusion at different extrusion rates, for instance using this GCODE. To measure the extruded weight a decent precision scale (resolving town to 1/100 of a gram or better) comes in handy because it saves on filament extruded. Alternative methods would be measuring the length of filament pulled into the extruder (which can be done using calipers alone).
2. Arrive at a table like this one

Feed rate [mm/min]	F=25	F=75	F=125	F=249
extruded filament [g]	0.6	0.57	0.55	0.51

3. To calculate your coefficients run the following in a python interpreter with numpy installed. If you don't have one ready here's an in-browser one that you can copy-paste the code into: https://pyodide.org/en/0.17.0/console.html (won't display the plots though). Quick sanity check: Values for A that significantly larger than 0.1 would seem suspicious to me.

import numpy

# use your measured values here
feeds=numpy.array([25,75,125,259])
extrusion=numpy.array([0.6,0.57,0.55,0.51])

# set expected extrusion to lowest measured value
expected_extrusion=max(extrusion)
# alternatively calculate your expected extrusion from desired extrusion length, filament density and filament radius
# this is left as an excercise to the reader

# polynomial fit, division of feed by 60 to arrive at velocity in mm/sec, the units used in klipper
# edit: adapted with @Tobba's fix
coefs=numpy.polynomial.polynomial.polyfit(extrusion/expected_extrusion*feeds/60., feeds/60.,3)
print("A={:0.5f} B={:0.5f}".format(coefs[2],coefs[3]))
# Old broken fit
# coefs=numpy.polynomial.polynomial.polyfit(feeds/60.,expected_extrusion/extrusion-1.,2)
# print("A={:0.5f} B={:0.5f}".format(coefs[1],coefs[2]))

If you also have matplotlib you can plot your results, too:

# in case matplotlib is installed: you can plot your results:
import matplotlib.pyplot as plt
plt.plot(extrusion/expected_extrusion*feeds/60,feeds/60,"ro")
smooth_x=numpy.arange(0,6,.1)
plt.plot(smooth_x,numpy.polynomial.polynomial.polyval(smooth_x,coefs))
plt.xlabel("feed*extrusion/expected[mm/s]")
plt.ylabel("feed[mm/s]")

4. Enable the feature in g-code using the obtained values

SET_NONLINEAR_EXTRUSION EXTRUDER=extruder A=0.0441 B=-0.00036883

PS: Why did I have the data ready? I was working on implementing nonlinear extrusion for klipper, but hadn't come as far as you yet - firstly I was planning on using an interpolation to represent the underextrusion curve which is inferior to your analytic approach because it is more costly nuerically to calculate the integral. Secondly I hadn't found out about where klipper keeps track about the absolute extruder position and thus my code was crashing.

[Tobba]

Okay, thanks. Alas, I haven't had a chance to look at this in-depth. What's the next steps from your perspective? Are you looking for users to test it and report results, or are you looking for some further feedback on the implementation?

Thanks again,
-Kevin

Both, it'd be nice to gain more information about the utility of the feature itself, i.e how widespread the problem of speed-dependent extrusion is and the effects on print quality, to motivate including it in mainline Klipper.

Implementation-wise, I'm still not 100% sure if the changes to itersolve are correct, or if there's just a better way to accomplish the tracking of the added movement, so a look by someone totally familiar with it would be appreciated. The interaction with gen_steps_pre_active is what I'm most uncertain about.

The check for whether another move should be processed in the !active_stepper is suspicious and looks like it could lead to a move being processed twice (i.e two calls to post_move_cb), since the next call begins processing from last_flush_time. It also seems rather pointless to add sk->gen_steps_pre_active here to begin with.

if (flush_time + sk->gen_steps_pre_active <= move_end)
    return 0;

Also, while writing this I noticed that post_move_cb will be skipped if the end time of a move happens to be exactly equal to flush_time. So the code is definitely not entirely free of bugs.

[Tobba]

@gluap

Your fitting code is a bit broken, expected_extrusion should be the maximum measured weight, not the minimum. Somehow it still generates close-to-correct results regardless though.

A better way to calculate the parameters is to fit a third-degree polynomial over the relationship between actual and requested feedrate, i.e:

coefs=numpy.polynomial.polynomial.polyfit(extrusion/expected_extrusion*feeds/60., feeds/60.,3)
print("A={:0.5f} B={:0.5f}".format(coefs[2],coefs[3]))

Which gives A=0.05111 B=-0.00068 for your data.

edit: Interestingly your printer seems to have a very linear loss of extrusion, while I arrived at a nearly quadratic loss (ASA@250C through a Volcano heat block and 0.4mm nozzle).

[gluap]

@Tobba: I edited the fitting code in my comment above to match your corrections. Originally I was using the calculated extrusion amount from feedrate and filament density and had wedged in the min() the last second to spare people from calculating it. Of course this has to be a max(). I also modified it to reflect your way of calculating the coefficients. Even though the multiplication by feedrate seems counter-intuitive to me but I'll revisit it to understand it better.edit: understood now

edit: Interestingly your printer seems to have a very linear loss of extrusion, while I arrived at a nearly quadratic loss (ASA@250C through a Volcano heat block and 0.4mm nozzle).

Mine is an ender3 stock extruder, I was running PLA@227C. I guess if I measure beyond the 10mm³/s the extrusion efficiency will fall off nonlinearly. CNC Kitchens has plots of extrusion efficiencies in a post about bimetal heatbreaks (full plot with all hotend combinations near the bottom of the post). Most of the combinations he tested were quasi-linear up to a point and then fall off rather steeply.

[dewi-ny-je]

@gluap
Mine is an ender3 stock extruder, I was running PLA@227C. I guess if I measure beyond the 10mm³/s the extrusion efficiency will fall off nonlinearly. CNC Kitchens has plots of extrusion efficiencies in a post about bimetal heatbreaks (full plot with all hotend combinations near the bottom of the post). Most of the combinations he tested were quasi-linear up to a point and then fall off rather steeply.

In fact, rather than using coefficients of a curve, why not using (measured) data points as configuration, with a simple linear interpolation between them? It's very simple (it's basically a series of "if then" with straight Interpolation) and allows reproducing the behaviour of any extruder.
The fact that such a calibration has sharp changes should not matter because it would still not involve any discontinuities in the correction itself, only in the derivative.

[dewi-ny-je]

CNC kitchen shows that even with the same extruder, depending on the hotend and on the use of socks, the correction may need to be straight or curved.
Are we sure it would be a quadratic? I don't think so.

A polyline gives total freedom and will be easier to set but also more accurate

[gluap]

@dewi-ny-je At first glance I also thought that something able to accomodate arbitrarily-shaped extrusion loss would be best. But now I don't think so anymore. If you look at CNC kitchens data closely, little is to be gained from printing in the regime where the curve becomes notably nonlinear for most of the hotends. Why? Have a look at this last plot of his post I hope the link won't decay.

For all of the hotends, the regime that looks nonlinear in his plots looks like "we could compensate this in software" at first glance. But when one does the math and calculates the amount actually extruded (instead of the percentage), one realizes that in this regime actually higher feedrate even reduces the extruded amount.

The extrusion rates as shown in his last plot (I calculated amount extruded by multiplying the percentage with the target feedrate, I highlighted the values where the extruded amount reduces with increased feedrate):

Target feed rate [mm³/s]	10	12.5	15	17.5	20
Bowden w. gap extrusion [mm³/s]	7	6.75	6	5.77	5
Bowden w/o gap extrusion	8.3	9.375	8.25	6.65	6.4
Copperhead heatbreak extrusion	9	10.625	11.85	10.85	11.6
Dual metal heatbreak extrusion	9	11.25	12.75	13.65	13.6

So with that background I'd say a low-degeree polynomial should be a safe bet.

[dewi-ny-je]

Cool, thanks for the calculation! I suggest you post that table as image on Twitter to suggest CNC kitchen to show the effect in a next video, since it's much more informative that way, instead of percent over requested.

Back to klipper, I think then that a maximum extrusion speed should be defined for safety, since crossing it would cause grinding and dirtying of the gears.

And also, all these parameters adjustable via gcode, since they are heavily filament dependent.

[KevinOConnor]

Implementation-wise, I'm still not 100% sure if the changes to itersolve are correct, or if there's just a better way to accomplish the tracking of the added movement, so a look by someone totally familiar with it would be appreciated.

Okay, I can try to give a few pointers. However, my feedback is largely going to be "high-level". I don't think I'll be able to dedicate sufficient time to perform detailed feedback on this feature.

If I'm understanding the concept correctly, the goal is to track both "commanded filament position", and "stepper position" - with the idea that the "stepper position" will be different from "commanded filament position" because filament grip can "slip" in predictable ways.

It seems you're tracking the difference between "commanded filament position" and "stepper position" entirely in the low-level step generation code. And have added a post_move_cb to track the value as it is needed when processing each move.

FWIW, I think it may be easier to have the high-level code track that "slip distance" than have the low-level code track it. It's awkward to maintain state between moves within the iterative solver itself, and the high-level code will need the "slip distance" anyway to correctly implement its find_past_position().

Unless I'm missing something, the PrinterExtruder class could calculate the "slip distance" in the move() method, track it between subsequent moves, and store that tracked value along with each move in the trapq. The low-level code still needs to implement the correct timing of the "slip distance", but it wouldn't have to worry about the tracking between moves.

Separately, I'm not sure I understand the math you've chosen for calculating the "slip distance", but I haven't looked at it closely.

it'd be nice to gain more information about the utility of the feature itself

FWIW, it may help if you put together a guide on how to calibrate the feature and what tests can be used to demonstrate its quality improvement. (Think of something like docs/Pressure_Advance.md and docs/Resonance_Compensation.md.) I'd guess there are many users interested in the feature and willing to test it, but don't know about it or don't know where to begin.

Cheers,
-Kevin

[KevinOConnor]

The check for whether another move should be processed in the !active_stepper is suspicious
[...]
Also, while writing this I noticed that post_move_cb will be skipped if the end time of a move happens to be exactly equal to flush_time.

I'm not sure if you're reporting an issue with the current code on the master branch or the code on this PR. If you think there is a problem with the master code branch, could you describe the issue in more detail?

Thanks,
-Kevin

[KevinOConnor]

Unless I'm missing something, the PrinterExtruder class could calculate the "slip distance" in the move() method, track it between subsequent moves, and store that tracked value along with each move in the trapq.

As a follow up, note that if you want to only implement the math in the C code, it's also possible to define a new extruder_trapq_append() and change extruder.py's move() method to call that instead of trapq_append().

-Kevin

[Tobba]

Unless I'm missing something, the PrinterExtruder class could calculate the "slip distance" in the move() method, track it between subsequent moves, and store that tracked value along with each move in the trapq.

That's a good idea. I thought about adding the slippage onto the start position to avoid needing post_move_cb (which would break PA), but didn't think of just storing it separately and adding it on later. I'll work on updating the implementation to work this way.

I'm not sure if you're reporting an issue with the current code on the master branch or the code on this PR. If you think there is a problem with the master code branch, could you describe the issue in more detail?

It's only a problem with the post_move_cb implementation.

[StrikeEagleCC]

This looks great, and it's at least part (if not the whole) answer to a problem I've observed on just about every print with relevant geometry: the tendency of large areas of infill to be more heavily extruded than small areas. I believe there may be another physical factor at play for which compensation/correction might be integrated into this solution: the time dependency of the back pressure and slippage on velocity changes.

With more discussion below, my main questionis this. Is it feasible to change your current approach, which as I understand it is based on instantaneous velocities, to one which is based on a average velocity, averaged over some time, either a user-definable constant, or some function of a user-definable constant.

v(t) = v(t) * (1 + Av_avg(t) + Bv_avg(t)^2)
where v_avg is previous velocities averaged over some time.

My hypothesis is this: at any constant extrusion velocity, the hotend system reaches some equilibrium of temperature, and pressure. But the time required to reach that equilibrium might be relevant. Consider a prolonged period of low velocity extrusion, during which this equilibrium is reached. Then transition to a high velocity extrusion. Two factors change, at least one of which I think is important:

First, the first few mm of filament extruded have already been in the hotend and have absorbed more heat during slow extrusion than the filament behind it will during the fast extrusion. Therefore, the pressure required to extrude these first few mm may be lower than the following filament, as the following filament won't be spending as much time in the hotend, and won't have as much time to melt. If this factor is dominant, the appropriate time over which to average velocities would be pretty dependent on the velocity. Perhaps a constant would be good enough though.

Second, and maybe less impactful is that once equilibrium is reached for the fast extrusion condition, the temperature of the interior surfaces will be lower than during slow extrusion, further increasing the required pressure and extruder slippage. I think that for this factor, the time over which to average would less dependent on the velocity.

The combined affect of these two factors is that the required additional velocity to compensate for slippage isn't instantaneous, but continues to change for some time after the velocity changes.

I'm in the process of building some gear to measure and plot commanded velocity vs actual velocity over short time intervals and highish resolutions (1-5ms/0.004mm). It will be interesting to see if my hypothesis is supported by the data, and I will share here if there is interest.

Also, Is this an appropriate place to have this discussion?

[KevinOConnor]

I'm not sure what the state of this PR is.

-Kevin

[StrikeEagleCC]

I am still very interested in this functionality, but have not had a chance to test it. The data I've collected thus far from the testing I mentioned above was not as clear as I would like.

[KevinOConnor]

Okay. I will leave the "pending feedback" label on this PR then.

-Kevin

[github-actions]

It looks like this GitHub Pull Request has become inactive. If there are any further updates, you can add a comment here or open a new ticket.

Best regards,
~ Your friendly GitIssueBot

PS: I'm just an automated script, not a human being.