[FR] Implement tracking of storage containers (part bins/bags/boxes/storage cases/toolboxes)

[laurus-lx]

Is your feature request the result of a bug?
No

Problem

It's currently difficult to track location of inventory inside movable containers - (part bins/bags/boxes/storage cases/toolboxes). It's hard to know what's inside various storage boxes and keep track of where they are. Bags with parts tend to get lost.

Locations module presumes fixed location, and is very limited in functionality (no barcode/serial/asset tag, does not support tracking of container location history, does not have customizable attributes such as type , size/weight, material, specs, pic).

Suggested solution

Enhance locations module to track location of inventory inside various containers. Bulk check-in, barcode scan, track relocation history, have extended attributes (pic, size/weight, address, fixed or movable location)

Another alternative - allow assigning inventory items to other inventory items for the purpose of location tracking. So for example a box with resistors would be an inventory item, and hold stock of resistors inside of it.

Describe alternatives you've considered
Currently using Google Sheets with custom views for storage management. This is not ideal

Examples of other systems
SnipeIT supports assigning assets to assets. So containers can be tracked as assets.

Do you want to develop this?
Possibly. I'm familiar with Python, but not with Web Development.

[SchrodingersGat]

Hi @laurus-lx thanks for the comments

Stock locations currently do support barcodes, and actions such as "bulk check in" are supported if you have a barcode scanner (or use the Android / iOS app).

What is missing now is the ability to move an entire location (into another location) via barcode scanning. This would be a good addition, potentially some locations could be marked as "fixed location" to disable this action.

Additional attributes for a StockLocation would be a good idea, and add some value.

Location history could be a little trickier to implement.

[laurus-lx]

Hi @laurus-lx thanks for the comments

Stock locations currently do support barcodes, and actions such as "bulk check in" are supported if you have a barcode scanner (or use the Android / iOS app).

What is missing now is the ability to move an entire location (into another location) via barcode scanning. This would be a good addition, potentially some locations could be marked as "fixed location" to disable this action.

Additional attributes for a StockLocation would be a good idea, and add some value.

Location history could be a little trickier to implement.

Hi SchrodingersGat,
Thank you for the feedback. Yes, need ability to move and track history of locations with some easy to use interface.

Example would be - small bags with electronics components and kits being checked-in to a bin, then later moved to a different bin or box when resorting inventory.

Or perhaps a better way to handle this would be to make each parts bag or box a stock item, and allow assigning to it stock items that it holds (via bulk entry check-in, like for locations check-in).

[SchrodingersGat]

@laurus-lx I've assigned this to the 0.6.0 release milestone

[matmair]

@laurus-lx do you need any assistance with this? We are happy to help!

[exp]

@laurus-lx have you made any progress on this? I would very much love this feature to keep track of parts installed into a pick and place machine. I have a thought about how to implement it and am happy to contribute but do not want to step on any work you have done.

[exp]

@matmair @SchrodingersGat @eeintech

In the absence of a response from the submitter, can we discuss the options for implementation? I have toyed with several different approaches to implementing sub-locations to keep the patch small and simple, but I have consistently found that this isn't sufficient.

In fact, what is needed is to take full advantage of the existing stock system, and have the ability to mark Parts as being able to store Stock. As the original submitter suggested, this is quite an elegant solution to allow complex storage schemes, such as:

• Pick and Place machine has Positions
  • which store Feeders which have Serials, Parameters, Tests and can be purchased
  • which store Reels which have Serials and can be purchased
  • which store Parts that may have Tests, Serials, Parameters or just Batch Codes.
• Test Equipment has Module Slots
  • which store Modules which have Serials, Parameters, Tests and can be purchased
  • sometimes these even nest, there are SFPs in Modules in Test Equipment
• CNC Mill has a Tool Changer which has Positions
  • which store Tool Holders which have Parameters and Serials
  • which store Tools which have Parameters and Serials

The critical point is that this is a system of installation which does not mark the contents as being consumed. It would be possible to mark a Module as Installed in Test Equipment, but this module would then count as consumed. I suggest the first approach here should be to allow any stock item to be installed in any other stock item but provide an additional boolean field, Permanently Installed to capture this difference. This would be set by Builds for example.

In order for this to not be incredibly spammy, and to represent reality well, Parts would need an additional flag such as Storage to indicate they can store other items within them. Thanks to them being real Stock, plugins may also check parameters such as "Reel Width" to find which feeders a part may be installed in.

These two changes then effectively eliminate the need for the 'Stock Location' system, as it is a parallel hierarchy with fewer features than the Stock hierarchy. I would like to eliminate this entirely and by introducing the Storage attribute at the same time, safe forward and reverse migrations can be created to allow this to happen seamlessly (assuming reserved characters are the same).

In order to do this, when a user creates their first Stock Location they would actually be creating a Stock Item, but it requires a Part to link to. I think adding a default Part is a reasonable option, but a user would not want to see this in their list (nor many categories of Location, but not all), therefore I propose one additional change.

Parts which have only the Virtual and Storage flags should be hidden by default from the Parts and Stock lists (via a filter or implicitly). This model elegantly encompasses the concept of "Rows" and "Columns" as of course the Storage Rack is a non-virtual Storage item, and the containing Rows/Columns are virtual subdivisions.

One more wrinkle is the need to support existing location labels which have IDs which cannot be reused in the Stock table (UUID only labels future please? ☺) so another attribute Location ID will need to be added, or a translation table in order to handle these edge cases.

In summary, I would like to propose a patch with the following changes

• Add a boolean field to Stock, Permanently Installed, set to False by default, set to True for all installed items in forward migration
• Add a boolean field to Parts, Storage, set to False by default
• Add an integer field to Stock, Location ID, to store a numeric ID so that old stock labels work
  • This is quite hacky but numerical IDs always suffer from these problems, this has to be used for all location stuff otherwise Stock ID and Location ID could be confused. This can be handled in migration and set sequentially in future when a Stock item is created from a Storage Part
• Create a 'Location' Part (with a random suffix if a Part named Location already exists) with Virtual and Storage flags set
• Remove the current StockLocation implementation and migrate existing Locations to be Virtual Storage Stock
• Miscellaneous fixes to selection boxes etc

I appreciate this is a relatively major change and it will touch quite a few areas of InvenTree but hopefully I have illustrated how the existing Stock hierarchy is more powerful and should be taken advantage of fully. As far as I can tell from my investigation so far this can be implemented completely safely and will provide significant power to plugins in future (such as letting me run my entire P+P setup!)

No rush for a response, I won't be fully implementing this next week most likely.
Cheers!
exp

[matmair]

This is a major change that could break a lot of things if not done carefully. I will have to think about this proposal a bit before I give my final opinion but here is a non-hirachical collection:

• You will need to keep the locations API etc. in place as a lot of stuff in the wild uses it
• This sounds to be a nightmare to explain to newbies. We aim for hobby, SME and labs - so please keep that in mind. The default UX must be intuitive
• I already proposed UUID ids - the consensus was that positive-integers are preferred
• You can propose adding stuff but switching out the base identifier is a heavy step: A lot of python-module and API users have built up their stuff on the existence of the id / pk being an integer that always exists - so we most likely will need to keep that up which means two unique identifiers - what is the benefit of that switch?
• Filtering should be transparent so you could propose adding a default filtering. Filtersets are saved in a cookie on device. Should be just override those or think of a mechanism for that?

@exp There is more to come - I think a UML with the proposed changes would help. In general I think this sounds like a 1.0.0

[SchrodingersGat]

@exp what you are proposing here is a very major change to the way that InvenTree fundamentally works - and for what seems to amount to an "edge case" of how it is being used.

The existing "simple" setup of StockLocation and StockItem has been developed to be as user friendly as possible. I know from experience that any extra complexities in the system reduce the likelihood that the operators will use it properly, as it is too cumbersome. I would like to keep it as simple as possible.

However your suggested use case (tool holders, PNP cassettes, etc) does interest me. At my work we also have CNC / PNP machines and have been using InvenTree to track parts going through our pick and place machine. So I would like to explore this idea further.

I believe we can make it work within the existing framework, without the major overhaul you have suggested. The original suggestions by @laurus-lx would be sensible here - extending the fields on the StockLocation model to provide enhanced behaviours.

• UUID field could be added
• Easier methods for "transferring" stock locations around (e.g. boxes / bins)
• location history tracking
• image attribute

Adding extra features to the existing models would provide compatibility with existing installs, new features (for those users who want them) and retain the simple workflow which is already in place.

[exp]

Thank you both for your replies. Instead of going through point by point I'm going to address the common themes if that's ok.

• UX Changes
  I don't plan to make any UX changes at all, the Virtual Storage suggestion is just the best current option I have for not changing the UX. I will review this suggestion and see if I can find something even lower impact

• UUID IDs
  It seems there are different opinions on these, and it may be that it's 'too late' to switch to them from integer IDs. I strongly believe that UUIDs are the correct approach for location/stock/part/build IDs as they permit refactoring, databases merges etc. This is not a core part of this proposal so I will not mention it again.

• Extra complexities
  I feel that adding extra fields to the StockLocation model is not keeping things simple but actually adding in the extra complexities you wish to avoid. There is no doubt that the physical reality of storage is that Stock is stored inside other Stock. Parts are kept on shelves which have serials and are purchased, they are stored inside parts with parameters, parts with tests, parts with every attribute the Stock system currently handles.

Eventually these features will need to be added for reasons illustrated above, such as allocating reels to feeders based on tape width. This means effectively re-creating the Stock system but in the StockLocation subsystem.

I don't see the point of maintaining this parallel hierarchy given that Stock already fully encompasses what's required, and in fact the concept of an Installed item already exists and very nearly matches the real semantics. The only reason to do so in my mind are the risks of using Stock as a StockLocation and the unintended complexity that could result, however I feel this is quite well bounded. There is a simple top down hierarchy and the only additional check would be to ensure you can't Install an item in a circular fashion. I haven't checked if this code already exists but circular references are the only pitfall I can see.

Compatibility is something I have thought long and hard about and my primary concern. The changes that I propose could be safely carried out on existing databases in both a forward and reverse manner with no information loss whatsoever. That I would expect is something no PR would be accepted without.

I could produce a UML diagram to continue the discussion here, but after reviewing my proposal above, the core of the patch I think is much smaller than I expected. I have some other work to complete at the start of this coming week, but I would like to produce a quick demo install (with patches) to illustrate my proposals much more clearly, if that's ok?

Cheers,
exp

PS. Apologies for the succinct response, I am happy to illustrate any part of the proposal in more depth, and I thank you for even entertaining my ramblings ;) Hope you all have a lovely weekend.

[matmair]

@SchrodingersGat There is a PR in the same direction already present #2482

[rshalaev]

I've been looking for something similar to locate items in storage. My use case - I put items/parts in to zip lock bags or small boxes, and they then go in to storage boxes. I then need to find where the item is (or been last seen), and where to find the storage box.

My main problems have been:

1. Parts and boxes get shuffled/moved around a lot, so I can't rely on fixed locations hierarchy, and need to be able to update it quickly.
2. Often there is no time to keep inventory up to date, so information can get stale. That's acceptable, as most inventory is non-critical. With that said - I need to know how stale is the tracking/inventory data for the following reasons:
   1. To know which stale inventory to update when there is time
   2. To know when NOT to rely on tracking/inventory info because it's of date
   3. To know where an item (or it's container) was seen last, to help with the search when inventory gets lost
3. Data entry and reporting needs to be very quick and lightweight. Time is highly valuable, while stock items are diverse, numerous and inexpensive.

For schema - I really need 2 tables:
ITEMS - TAG, DESCRIPTION, ATTRIBUTES
HISTORY - ITEM_TAG, PARENT_TAG, DATE, ACTION, ATTRIBUTES

So far I've implemented this in SqLite, and been using DBeaver (GUI/Front-End) for data-entry and viewing reports. New stock goes in to items table, and I note items location in history with date, action (Install/Remove/Purchase/Sell/Test/Dispose etc.), and parent tag.

For reporting I mostly rely on Recursive Location that can report hierarchical location for a single item OR report what's stored at a given item/location (in a hierarchical/tree like format). I also combine Recursive Location report with tracking history report to search for lost items. DBeave/SQLite solution has been working well for the last year.

Hoping to move my custom solution to Inventree at some point to be able to view item's pictures, attributes and make it more user friendly for non technical users (so my wife can use it as well).

Wanted to ask: How feasible would it be to add items Tracking History, implement dynamic Hierarchical Location based on Tracking History, and report on Tracking History in Inventree? - I'd most likely contribute with the development. Main concerns are that Inventree would be trying to enforce some additional rules on inventory (I mostly need to know what's where, get tracking history, and be able to enter a lot of items very quickly); and if there is interest in this use case (so it would get merged).

[matmair]

@rshalaev have you tried the demo / read the docs?
I think your use case should be mostly covered by what is there already. Locations are hierarchical and stock has a history tracking model attached by default. Manipulation via QR codes might make inventory faster.

[rshalaev]

@rshalaev have you tried the demo / read the docs? I think your use case should be mostly covered by what is there already. Locations are hierarchical and stock has a history tracking model attached by default. Manipulation via QR codes might make inventory faster.

@matmair Yes. I've setup Inventree, and been trying to get it to work for this use case (minimal data entry/lightweight system for tracking installed and stored items and components).

Here's an example of tracking movement of an internal PC component (for example SATA/m.2 adapter):

Day 1: Room_1 - Item_A -> Item_B -> Item_C (Item C is installed in component B in device A)
Day 2: Storage_1 - Box_1 -> Bag _1 -> Item_C (Item C is removed/ placed in to storage)
Day 3: Storage_2 - Box_1 -> Bag _1 -> Item_C (Box moved to a new storage location)
Day 4: Storage_2 - Box_2 -> Bag _1 -> Item_C (Bag moved to a different box)
Day 5: Storage_2 - Box_1 -> Bag _2 -> Item_C (Item moved to a different bag)
Day 6: Room_1 - Item_D -> Item_E -> Item_C (Item installed in to component E in device D)

Problems, as I recall:

• Locations don't have bulk check-in of other locations - moving boxes between storage - need to find and update each box (as location) individually. Time-consuming
• Locations don't have history - can't lookup tracking / can't undo an erroneous entry when looking for lost items
• Process of assigning items to other items was a multi-step process. Required updating comparability model / generating and executing a build order

Short term solution was to track things in a simple custom database. Just a barebone/raw DB interface.

Longer term - looking to see if Inventree can be enhanced to support this use case (lightweight tracking with minimal data entry, hierarchical locations/components, tracking history).

[matmair]

@rshalaev there were several ppl. interested in this but no PRs materialized until now so feel free to submit something.

Locations don't have history but the stock has right now as the whole system is oriented around material flow (in a manufacturing sense), not warehousing. I think that explains why some things are set up that way. But this is def. not set in stone.

[SchrodingersGat]

@rshalaev adding "history" to the stock locations themselves would be a welcome addition, and one that we would certainly be happy to consider!

[exp]

I apologise for the massive delay in producing a PR, I have been completely swamped with PCB and device failures, firmware, bootloaders, holidays and birthdays!

I will be on this issue full time from some time today if all goes to plan.

[matmair]

@exp no worries, we are all doing this on free time! Ping us if you need assistance.

[exp]

Turns out things did not go to plan, but I have small amounts of progress made. I'll aim to give you something workable this week as I need to get this feature working in some form that's acceptable to all.

Cheers

[matmair]

@exp no worries. Feel free to just dump what you have into a WIP PR so we can take a look and give you feedback. The things ppl struggle with often turn out to be just a quirk with the way Django/our structure is set up.

[adamwolf]

@exp any chance you'd want to post what you've got? This is likely a blocker for switching to inventree so I could put some hours in...

[SchrodingersGat]

@laurus-lx a lot of pieces of this puzzle are now in place.

#3640 was a major refactor of the barcode scanning feature, you can now assign arbitrary barcodes to stock locations, as you could already do with stock item.

#3887 provides some additional barcode functionality to the web interface.

Still missing is historical tracking information for stock locations.

[HammyHavoc]

Just chiming in to say that whilst I'm a hobbyist, being able to track what's in the PnP feeders would be very useful, as would a lot of other suggestions mentioned above.

Happy holidays everyone!

[spalex78]

Hi, I think this feature is the SSCC containers implemented by other solutions. Very simple and it is used both for storage of pallets and boxes, and for shipping.

[adamwolf]

Hiya folks! I'm trying to figure out if there's anything left or what's left to enable historical tracking of locations? i.e. if I have a box with things in it, can I track where that box has been?

[SchrodingersGat]

@adamwolf no, currently we only have historical tracking of individual stock items - not the locations.