New notebook + 1 video + 1 image file #1700
Conversation
There was a problem hiding this comment.
video = cv2.VideoCapture("/Users/anurag/github/openai-cookbook/openai-cookbook/examples/data/manufacturing/warehouse_operations.mp4")
Use the relative path to the video that you uploaded on github if you want them to use that directly e.g. "data/manufacturing/warehouse_operations.mp4" or just generalize this with a placeholder e.g. "<PATH_TO_YOUR_VIDEO>
There was a problem hiding this comment.
In your cost estimation, also provide more description upfront including your assumptions e.g. "Assuming that we take 1 image per minute, every hour of the day, for 365 days in a year..." etc...
It also looks like your printed output is duplicated:
Total annual cost: $1451.97
Token cost per image: 1105
Annual token cost (1 image per minute): 1451.97
There was a problem hiding this comment.
In your analysis, I would highlight what works well e.g. we correctly identify the 5 workers in the first two frames with high confidence... in the last frame we miss a worker, but we also have lower confidence... gpt-4o is really good at respecting bounding boxes and it never counts workers outside the bounding box etc...
Would be good to see some commentary on the results
There was a problem hiding this comment.
"Implementing advanced function calling to streamline interactions between YOLO detections and GPT-4o analysis." - What are YOLO detections?
"Exploring real-time Vision APIs, currently under development, to achieve true real-time insights and faster decision-making." - Not sure what this is referring to? Note this is a public resource!
Summary
Briefly describe the changes and the goal of this PR. Make sure the PR title summarizes the changes effectively.
This PR introduces a detailed notebook demonstrating how to leverage GPT-4o's vision capabilities for analyzing video frames to extract structured operational insights in a manufacturing warehouse. It provides step-by-step instructions, best practices for bounding boxes, structured data extraction, confidence scoring, and cost considerations to effectively implement an AI-driven monitoring system.
Motivation
Why are these changes necessary? How do they improve the cookbook?
Warehouse managers often lack real-time visibility into their operations, relying instead on delayed or manual reporting methods, leading to reactive rather than proactive decision-making. This contribution addresses these issues by using GPT-4o's vision capabilities to analyze video footage, enabling rapid identification of safety concerns, monitoring space utilization, and detecting operational inefficiencies in near-real-time. This significantly improves decision-making speed, enhances safety compliance, and reduces operational inefficiencies.
For new content
When contributing new content, read through our contribution guidelines, and mark the following action items as completed:
We will rate each of these areas on a scale from 1 to 4, and will only accept contributions that score 3 or higher on all areas. Refer to our contribution guidelines for more details.