Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integration with inventory management #57

Open
alanpuccinelli opened this issue Apr 28, 2022 · 17 comments
Open

Integration with inventory management #57

alanpuccinelli opened this issue Apr 28, 2022 · 17 comments
Labels
enhancement New feature or request

Comments

@alanpuccinelli
Copy link

While one can always set an incredibly high number of copies might be nice to have a repeat till cancelled option. Maybe just setting a "-1" value for copies or a checkmark option?
@smartin015
Copy link
Owner

Hey Alan - thanks for the feature request. Can you elaborate on when you'd want to print something indefinitely? I'm hesitant to add this as a feature as it has the potential to be quite wasteful.

I also suspect an "infinite loop" setting might be a coarse way to specify something that could be better done with a specific signal (e.g. "print enough plastic beads to fill a WxHxD container" may be a better intent than "print plastic beads until the heat death of the universe").

@alanpuccinelli
Copy link
Author

Probably just what I'm most used to. We run thousands of parts for weeks at a time on many of our farm printers and I've just been using M808 for that up until now. Its obviously a nice to have but not critical. Frankly I don't see how it would be any more wasteful than just setting a really high number of copies.

@smartin015
Copy link
Owner

smartin015 commented Apr 29, 2022 •
edited
Loading

Ah, gotcha. What are some reasons for stopping your print run, after you've started? Is timing a factor, or is it more of a general sense of having enough inventory?

(I'm also kind of curious - what kind of things are you printing thousands of?)

@alanpuccinelli
Copy link
Author

alanpuccinelli commented Apr 29, 2022 via email

@smartin015
Copy link
Owner

Ah, I was wondering why your name sounded familiar - I was just watching the 3DPN youtube demo of your CR30 upgrades a couple days ago :) cool stuff.

Now I'm imagining an integration with some (pre-existing) inventory management service that allows you to issue commands like "print until I have X amount in stock", with additional logic that adjusts the print priority based on whatever's currently the least in-stock. WDYT?

I have some work ongoing that allows for multiple queues with different queue behaviors, which would open up the ability to have a "manual" linear/job-order queue and a separate "smart" queue such as above, where the printer can pull work from any of them based on some predetermined rules (e.g. "do stock replenishment, but preempt that with anything in the manual queue"). Also planning support for multiple printers managing these multiple queues (#35) in a many-to-many sort of way. This could be a potential extension/adaptation of that work.

@alanpuccinelli
Copy link
Author

alanpuccinelli commented Apr 29, 2022 via email

@smartin015 smartin015 changed the title Add ability to set a number of prints to infinite (repeat till cancelled) Integration with inventory management Apr 30, 2022
@smartin015 smartin015 added the enhancement New feature or request label Apr 30, 2022
@smartin015
Copy link
Owner

smartin015 commented Apr 30, 2022 •
edited
Loading

The first and best way for you to help is with the specifics - it appears you're using Shopify for the e-commerce part of your site, which I haven't used in the past. Do they have built-in inventory management? Is it sufficient, or do you think a third-party IMS would work better? (IMO it'd be great to have an option not tied to a specific sales service, but convenience trumps that)

The overall design can be somewhat generic: a way to match SKUs to CPQ jobs, plus an extensible way to sync inventory state via polling/pushing with the IMS, in addition to the queue logic. But it's helpful to have an target integration in mind.

It's probably going to be another month or so before I've worked through what's already planned for CPQ and start working on this. If you're interested in faster delivery or in helping further, we can chat about contribution / incentives.

@alanpuccinelli
Copy link
Author

alanpuccinelli commented May 3, 2022 via email

@alanpuccinelli
Copy link
Author

alanpuccinelli commented May 24, 2022 via email

@smartin015
Copy link
Owner

Hey, sorry for not handling your question earlier! I somehow lost it in the flow of things.

That does sound interesting, sort of a legacy/compat option. Please do file a feature request and we can chat about what makes sense to implement there :)

@smartin015
Copy link
Owner

Hey @alanpuccinelli - I haven't forgotten about our conversation here. I sent a request via your contact form - we can follow up using the email I submitted there.

@smartin015 smartin015 mentioned this issue Oct 21, 2022
Merged
@smartin015
Copy link
Owner

High-level feedback

  • Uses spreadsheets track inventory
  • QC is required before inventory counts can increase as prints may fail

Ideas:

  • Trigger a print whenever a part sells (regardless of QA/reprinting, definitely need an extra one to make up inventory)
  • Scale up the number of reprints based on first pass yield data acquired over time
  • computer vision based initial QC pass to ensure parts at least look similar
  • Support writing aggregate of history to a google sheet - maybe by having totals in the history tab and allowing connection to a sheet?

This was referenced Mar 8, 2023
Closed
Open
@nickgabbard
Copy link

Weighing in as requested.

We have 18 printers running 2 shifts a day. We use shopify for sales and orders. Shopify inventory management kind of sucks, so we use an in-house PostgreSQL database to handle inventory. Our tech stack also has an automation platform called N8N running in the shop. It's similar to a self-hosted IFTTT that does webhooks and http requests. It's currently used for file management when creating new listings, but we are trying to come up with a way to integrate that into the printing process as well.

For clarity in the rest of the way I'm going to explain things, here's some more background. We refer to a sales unit as an "object." If it's for sale on the website, it's an object. There are also packages of multiple objects for sale. We treat that as a completely new object. Objects are made up of parts. An object can have 1 or more parts. Each part is 1 print file (either a .gcode if we print it with an FDM printer, or a .stl file if we print it with a resin printer).

We don't track inventory on objects (which is why shopify's inventory management doesn't work for us). We do track inventory on parts.

Our catalog is huge. We usually only have less than 10% of parts in stock at any given time.

So, we are essentially a print-on-demand shop, but we need a check step to see if we happen to have one laying around first.

Current thought process on a workflow is:

  1. Get notified of order from shopify trigger to n8n.
  2. Based on "skus" transmitted from shopify (these match up with objects) determine what "'parts" need to be shipped. (probably can be done with Postgres trigger by creating and writing to an "orders" table or something)
  3. Based on the sum of current parts in inventory AND the sum of parts in currently unfulfilled orders AND the sum of parts that are currently in the "to be printed list" (not the continuous print queue, btw) , generate a list of parts that need to be printed and add them to a "to be printed list" (some postgres table)
  4. The "to be printed list" should have a flag that indicates if it has been added to a CP queue or not, and some sort of indicator that the print is actually complete/good.
  5. Iterate through the "to be printed list" and identify items that haven't been sent to CP and use n8n and the CP API to add them to a print queue. It would be very handy at this point to be able to move them directly to a network queue, assuming that the automatic printer profile detection is working properly. Additionally, since we are shop that only runs one specific model of FDM printer, a global configuration option in CP that defaults all jobs to a specific printer profile would be awesome.
  6. and this is where my train of thought breaks down......

At this point, physical objects start printing. We would need to come up with a way to link those physical objects back to items on the "to be printed list" to indicate that they are finished, but we have to have a human QC point in there. Everything that comes out of the print room should be added to inventory, so orders would always be filled from inventory, instead of straight from production.

Honestly, I don't know if any of this adds to the conversation or just muddies the water.

@smartin015 smartin015 mentioned this issue Mar 11, 2023
Open
@smartin015
Copy link
Owner

Thanks, this is exactly what I was hoping for! Also hadn't heard of N8N before - that's a great alternative to IFTTT, Zapier etc.

Objects are made up of parts. An object can have 1 or more parts. Each part is 1 print file... We don't track inventory on objects (which is why shopify's inventory management doesn't work for us). We do track inventory on parts.

This lines up perfectly with what I've heard about other inventory systems - that extra indirection between sale items and parts means a custom solution is needed every time. It'd be great if there was a "common" custom solution, which is where I get interested in the details :)

Current thought process on a workflow is:

  1. Get notified of order from shopify trigger to n8n.
  2. Based on "skus" transmitted from shopify (these match up with objects) determine what "'parts" need to be shipped. (probably can be done with Postgres trigger by creating and writing to an "orders" table or something)
  3. Based on the sum of current parts in inventory AND the sum of parts in currently unfulfilled orders AND the sum of parts that are currently in the "to be printed list" (not the continuous print queue, btw) , generate a list of parts that need to be printed and add them to a "to be printed list" (some postgres table)
  4. The "to be printed list" should have a flag that indicates if it has been added to a CP queue or not, and some sort of indicator that the print is actually complete/good.

Although I'd initially thought about hosting inventory data in CPQ, I like the idea of an "authoritative" external DB that feeds CPQ. That means fewer integration points needed on the plugin side and less "util columns" for holding opaque user data.

  1. Iterate through the "to be printed list" and identify items that haven't been sent to CP and use n8n and the CP API to add them to a print queue. It would be very handy at this point to be able to move them directly to a network queue, assuming that the automatic printer profile detection is working properly.

#215 sounds like what you're looking for. I'd definitely want the same validation checks as with moving from local to network queues. Which slicer are you using? At this point, Kiri:Moto, Prusa slicer, and Simplify3D are supported. If your gcode files already have printer metadata, a new slicer can pretty easily be added in.

On the flip side, you could always pass STL files around and slice them on the fly. That doesn't require profile inference, but does require a slicing service to be set up. Maybe not worth it if the files being printed don't change regularly, and there's no caching as of yet (see #225).

Additionally, since we are shop that only runs one specific model of FDM printer, a global configuration option in CP that defaults all jobs to a specific printer profile would be awesome.

I've heard this request before, but I'm afraid to "defeat" the gcode validation step like that if it's not actively causing any problems. That being said... I don't think it's been done before, but /set/add should allow passing the printer profile in the request body. If you want to assert your files are for a certain printer, it seems fine to do it that way.

We would need to come up with a way to link those physical objects back to items on the "to be printed list" to indicate that they are finished, but we have to have a human QC point in there.

Does jneilliii/OctoPrint-BedReady#16 seem useful to you at all? Not saying it would eliminate all QC, but how much work do you think could be eliminated if you could auto-reject based on image differences?

Re: linking back to the "to be printed list", I'm imagining a couple options:

  1. Create a custom at-command that allows making a web request to a given URL with info about the currently printed file, state of CPQ etc. This would be run at script execution time, so response data couldn't easily be used.
  2. Add a function to the preprocessor's symbol table that permits sending a web request. Benefit over (1) is that the response could be used to affect the next executed GCODE (e.g. send "it's done" to a server, receive "push it this way or that way off the bed", and move accordingly). This could risk a jailbreak from ASTEval if an exploit is discovered, but IMO the risk is pretty low (would have to be an exploit to do with assigning a string to a variable, and the most obvious one - buffer overflow - can be pretty easily mitigated).
  3. Formal integration of "send a web request on completion" to CPQ's printing state machine. This gives the most opportunity for changes in CPQ behavior, but locks changes into the release cycle and complicates the system overall.
  4. External service polls for changes in the queue and diffs against prior state. Requires no changes to CPQ, but does seem kind of wasteful and overly custom.

WDYT?

@nickgabbard
Copy link

nickgabbard commented Mar 12, 2023 •
edited
Loading

Objects are made up of parts. An object can have 1 or more parts. Each part is 1 print file... We don't track inventory on objects (which is why shopify's inventory management doesn't work for us). We do track inventory on parts.

This lines up perfectly with what I've heard about other inventory systems - that extra indirection between sale items and parts means a custom solution is needed every time. It'd be great if there was a "common" custom solution, which is where I get interested in the details :)

If there is a standard I've not found it.

Although I'd initially thought about hosting inventory data in CPQ, I like the idea of an "authoritative" external DB that feeds CPQ. That means fewer integration points needed on the plugin side and less "util columns" for holding opaque user data.

Having it in CPQ I feel like that could get messy really fast, and would make it harder to integrate into anything else.

  1. Iterate through the "to be printed list" and identify items that haven't been sent to CP and use n8n and the CP API to add them to a print queue. It would be very handy at this point to be able to move them directly to a network queue, assuming that the automatic printer profile detection is working properly.

#215 sounds like what you're looking for. I'd definitely want the same validation checks as with moving from local to network queues. Which slicer are you using? At this point, Kiri:Moto, Prusa slicer, and Simplify3D are supported. If your gcode files already have printer metadata, a new slicer can pretty easily be added in.

We use Prusa Slicer, but automatic detection doesn't work for a majority of our older gcode files. I've not really dug into why yet, because it looks from the code like it should.

On the flip side, you could always pass STL files around and slice them on the fly. That doesn't require profile inference, but does require a slicing service to be set up. Maybe not worth it if the files being printed don't change regularly, and there's no caching as of yet (see #225).
The file being printed does not change once we get a good working print. A lot of these things are pretty fiddly and need custom supports. There's also a few thousand different gcodes in the library this point, so that's also another reason I don't want to re-slice them all to attempt to get profile inference to work.

Additionally, since we are shop that only runs one specific model of FDM printer, a global configuration option in CP that defaults all jobs to a specific printer profile would be awesome.

I've heard this request before, but I'm afraid to "defeat" the gcode validation step like that if it's not actively causing any problems. That being said... I don't think it's been done before, but /set/add should allow passing the printer profile in the request body. If you want to assert your files are for a certain printer, it seems fine to do it that way.

I full intend to try passing it thru /set/add later today.

We would need to come up with a way to link those physical objects back to items on the "to be printed list" to indicate that they are finished, but we have to have a human QC point in there.

Does jneilliii/OctoPrint-BedReady#16 seem useful to you at all? Not saying it would eliminate all QC, but how much work do you think could be eliminated if you could auto-reject based on image differences?

It might, but we don't run webcams. Actually, we don't run individual hardware for each printer. All 18 printers are connected via USB to an HP server and we run 19 different Octoprint docker containers. We leave one of them not connected to a printer and use it sort of like a master control to add new sets/files/etc. Adding USB webcams to that mix would tank the USB throughput and cause a host of other issues. Maybe when rpis aren't selling at black market prices.

Re: linking back to the "to be printed list", I'm imagining a couple options:

  1. Create a custom at-command that allows making a web request to a given URL with info about the currently printed file, state of CPQ etc. This would be run at script execution time, so response data couldn't easily be used.
  2. Add a function to the preprocessor's symbol table that permits sending a web request. Benefit over (1) is that the response could be used to affect the next executed GCODE (e.g. send "it's done" to a server, receive "push it this way or that way off the bed", and move accordingly). This could risk a jailbreak from ASTEval if an exploit is discovered, but IMO the risk is pretty low (would have to be an exploit to do with assigning a string to a variable, and the most obvious one - buffer overflow - can be pretty easily mitigated).
  3. Formal integration of "send a web request on completion" to CPQ's printing state machine. This gives the most opportunity for changes in CPQ behavior, but locks changes into the release cycle and complicates the system overall.
  4. External service polls for changes in the queue and diffs against prior state. Requires no changes to CPQ, but does seem kind of wasteful and overly custom.

WDYT?

Option 1 or 2 seems like the most immediately useful, however N8N is probably capable of doing Option 4 now with no changes.

@smartin015
Copy link
Owner

smartin015 commented Mar 15, 2023 •
edited
Loading

Having it in CPQ I feel like that could get messy really fast, and would make it harder to integrate into anything else.

Agreed.

We use Prusa Slicer, but automatic detection doesn't work for a majority of our older gcode files. I've not really dug into why yet, because it looks from the code like it should.

If you want to toss an example file my way, I could take a look and see where the soft matching breaks down :)

The file being printed does not change once we get a good working print. A lot of these things are pretty fiddly and need custom supports. There's also a few thousand different gcodes in the library this point, so that's also another reason I don't want to re-slice them all to attempt to get profile inference to work.

Makes sense.

It might, but we don't run webcams. Actually, we don't run individual hardware for each printer. All 18 printers are connected via USB to an HP server and we run 19 different Octoprint docker containers. We leave one of them not connected to a printer and use it sort of like a master control to add new sets/files/etc. Adding USB webcams to that mix would tank the USB throughput and cause a host of other issues. Maybe when rpis aren't selling at black market prices.

Yeah, rpi's are quite expensive right now. I started thinking about a lower-cost option involving cheaper embedded devices. That's likely to be several months away.

Do you have any special hardware on your HP server to handle 18 independent serial connections? I had another user on #213 experiencing bandwidth issues, and I wonder if there's enough info around that it'd be worth writing a guide in the docs for a single-host, many-printer hardware setup.

Option 1 or 2 seems like the most immediately useful, however N8N is probably capable of doing Option 4 now with no changes.

Agreed - between the two, I might tackle option 2 in the future as it has the most potential. If you find a good solution with option 4 in the meantime, I'd be interested in documenting it so others have a starting point for their integrations.

@smartin015 smartin015 mentioned this issue Apr 10, 2023
Open
@smartin015
Copy link
Owner

I've split off the computer vision feature discussion to #229. May also work on option 2 as part of it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@smartin015 @alanpuccinelli @nickgabbard