-
-
Notifications
You must be signed in to change notification settings - Fork 1.1k
[DMP 2025]: Color sensor for Music Blocks for photos and real-time video #4537
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
i understand what you are trying to say. I think i can implement this. Can you assign this to me ? |
This is reserved for a DMP mentee. Please look into how to apply for DMP: https://www.codeforgovtech.in/ (which seems to be down atm, so use https://web.archive.org/web/20250216103313/https://codeforgovtech.in/) and https://discord.gg/aFWG6CwP In the meantime, I recommend that you join the conversation on https://matrix.to/#/#sugar:matrix.org and https://matrix.to/#/#musicblocksdev:matrix.org |
got it . Thanks |
Hello @pikurasa I came across this exciting initiative to integrate an AI-powered debugger into Music Blocks, and I’d love to contribute. With my experience in AI, NLP, and full-stack development, I have worked on fine-tuning open-source LLMs, deploying AI-powered applications, and integrating AI solutions into platforms. Could you please specify the further steps...? |
I would like to work on this project, what are the key steps for this, to get selected in this project. |
Hi! I’d love to work on this issue for DMP 2025. I’ve already explored how to detect pixel colors using canvas for both uploaded images and live webcam feeds using JavaScript. |
Hi @pikurasa |
Hello @pikurasa Unlocking Music Blocks' Full Potential I'm Chandni Gupta, and I'm thrilled to throw my hat into the ring for the Music Blocks project. With my passion for accessibility and ed-tech, I believe I can bring a unique perspective to this project. My Superpowers:
The Goal:
Why I'm the Perfect Fit: I've got the skills, experience, and passion to make this project shine. I'm confident that my expertise in JavaScript and React will enable me to deliver high-quality results. Plus, I'm excited about the prospect of working on a project that can make a real difference in people's lives. |
Hello @pikurasa I'm very excited about this feature request — it aligns perfectly with both my technical skills and my enthusiasm for building inclusive educational tools. The idea of integrating a color sensor system into Music Blocks, especially with the goal of helping visually impaired users interact with and create music through tangible tools like LEGO bricks, is truly inspiring. From what I understand, the implementation would involve: Moreover, I really appreciate Music Blocks’ mission of making music education engaging and accessible through a visual programming language. Enhancing this with sensory input like color detection feels like a natural next step, and I’d love to be a part of that journey. Kindly consider assigning this to me — I’d be thrilled to explore ideas, collaborate, and start building |
Hello @pikurasa From what I understand, the goal is to enable colour detection from both webcam streams and uploaded images, so that the turtle/mouse glyph in Music Blocks can read pixel colours—not just from the drawing canvas but also from external visual sources. This would support use cases like Lego music notation for the blind. I’ve experimented with detecting pixel colour from webcam feeds using the canvas API (getImageData) in vanilla JS, and I’d love to explore integrating it into Music Blocks. |
Hello @pikurasa |
Hello @pikurasa, I'm Tarun and I'm a good fit for working on this project because of my skills in python, gen ai and working with LLMs. I have worked on several projects with similar implementation, you can check our in my resume or my LinkedIn profile. This will be a very good opportunity for me and i will be very helpful asset to the project. Please reach our to me if you have any doubts or want clarification on me or my projects. Looking forward to contributing to the project. Linkedin: https://linkedin.com/in/tarunkumar8278 |
Subject: Interest in Contributing to Music Blocks' Color Detection Feature Hello @walterbender and @pikurasa , I hope this message finds you well. My name is Ashi Jain, and I am enthusiastic about contributing to the Music Blocks project under the Code4GovTech initiative. I am particularly inspired by your initiative to enhance accessibility in Music Blocks by enabling color detection from uploaded images and live webcam feeds. This feature, aimed at supporting LEGO music notation for blind users, is a commendable step towards inclusive education. The prospect of translating visual elements into auditory experiences resonates deeply with my passion for leveraging technology to bridge accessibility gaps. With a strong foundation in JavaScript and experience in client-side development, I am confident in my ability to contribute effectively to this project. I have previously worked on projects involving real-time video processing and color detection, which aligns well with the technical requirements of this enhancement. I am eager to collaborate with your team to develop and integrate the extended color detection functionality. I would appreciate the opportunity to discuss how I can best support this project and align my efforts with the team's goals. Thank you for considering my application. I look forward to the possibility of contributing to Music Blocks and learning from your guidance. |
@pikurasa This is my expertise.. Could you please assign this project to me? |
Hi there, I'd like to contribute by working on a solution to extend Music Blocks’ color detection capabilities. I have a background in JavaScript and would be happy to help with this task. Let me know if it's okay to proceed. Thanks! |
Hello @walterbender and @pikurasa I am Abhiram from IIT Guwahati .I gone through project Issue and I made a research for color detection from 1.first thing is to extract the RGB values from uploaded images or live webcam feeds. I am eagerly waiting to collaborate with your team to work on this project and get guidance under excellent tech professionals Using this opportunity I will build my skills in JavaScript a bit more and meet new highly professionals. Eagerly waiting to work in this project hope it happen for sure . |
could you please assign this project to me it will be grateful to showcase my skills |
Hi @walterbender @devinulibarri, I hope you're doing well! My name is Mudit, and I’m incredibly excited about the possibility of contributing to the Music Blocks – Pixel Color Detection project as part of DMP 2025. This project’s vision—enhancing accessibility by enabling pixel color detection from uploaded images and live webcam feeds—deeply resonates with me. I believe this functionality has great potential to make Lego Music Notation for the blind even more impactful and inclusive. I have a solid foundation in JavaScript, and I’ve explored the Music Blocks platform to get hands-on familiarity with its interface and programming style. I’m particularly interested in developing the client-side color detection logic, and I’d be thrilled to help integrate this into the turtle/mouse glyph system for both uploaded and live video content. As a suggestion, to improve real-time color detection and minimize flickering or false reads, we could consider implementing a sampling threshold mechanism or an averaging window around the cursor to stabilize the readings from the live feed—especially in low-light or noisy conditions. Would it be possible to connect briefly and understand the current status of this feature or any available prototypes? I’d love to align with your roadmap and begin contributing in a meaningful way. Looking forward to collaborating with you and the team! Warm regards, |
Hello @pikurasa Unlocking Music Blocks' Full Potential I'm bhumika kashyap , and I'm thrilled to throw my hat into the ring for the Music Blocks project. With my passion for accessibility , I believe I can bring a unique perspective to this project. My Superpowers: Html css JavaScript React node js mongo db my sql andmake api calls and expertiseTeam player with a knack for learning quicklyPassionate about making tech more accessible The Goal: Why I'm the Perfect Fit: I've got the skills, experience, and passion to make this project shine. I'm confident that my expertise in html css JavaScript and mongo db node js will enable me to deliver high-quality results. Plus, I'm excited about the prospect of working on a project that can make a real difference in people's lives. Looking forward to collaborating with you and the team! |
Hello @pikurasa Unlocking Music Blocks' Full Potential I'm bhumika kashyap , and I'm thrilled to throw my hat into the ring for the Music Blocks project. With my passion for accessibility , I believe I can bring a unique perspective to this project. My Superpowers: Html css JavaScript React node js mongo db my sql andmake api calls and expertiseTeam player with a knack for learning quicklyPassionate about making tech more accessible The Goal: Enhance Color Detection: Support for Uploaded Images and Webcam Feed This pull request adds functionality to enhance the color detection system in Music Blocks by:
Tested with:
Everything works smoothly as expected. Looking forward to your feedback! |
This is a special issue. It's for https://codeforgovtech.in/dedicated_mentoring_program/. We encourage interested contributors to apply there. Also, we recommend that you join the conversion on https://matrix.to/#/#musicblocksdev:matrix.org |
Could you please assign this project to me it will be grateful to showcase my skills in this project |
About Me : Why Me : Familiarity with pixel-level color detection, image processing, and webcam stream integration. Deep understanding of client-side development, ensuring smooth performance without server dependencies. Focus on building accessible and user-friendly tools for educational purposes. Excited to contribute to open-source educational technology and work collaboratively with classroom partners for real-world testing and feedback. Thank you for considering my application for this meaningful project. |
Hi @walterbender @DevinUlibarri 👋, I'm really excited about the opportunity to contribute to the Music Blocks project—especially this feature that enhances accessibility through live pixel color detection. With my experience in JavaScript and image processing, I’d love to help implement color detection for both uploaded images and webcam feeds, and integrate it seamlessly into the existing pixel color block system. This project resonates with me because of its creative blend of music, education, and accessibility, and I’m eager to collaborate and iterate based on classroom testing results. Please consider assigning me to this issue—I’d be thrilled to get started! Thanks, |
Hi Walter and Devin, I saw the Music Blocks project about detecting pixel color from a live feed, and it really caught my interest — especially how it can help create accessible tools like Lego music notation for the blind. I’d love to be part of it. I’m still new to JavaScript, but I’ve started learning it and am ready to put in the work to get up to speed quickly. Would it be okay if I apply while continuing to build my skills along the way? Thanks for your time, and I’d really appreciate the chance to learn under your guidance! Best, |
Hi @walterbender @DevinUlibarri , Thank You, |
I’m really interested in your project to add color detection from uploaded images and webcam feeds in Music Blocks. I think it’s a great idea, especially for making things like Lego music notation for the blind more interactive and useful. I have some experience with working on images and webcam inputs, and I’d love to help build this feature so the turtle/mouse can detect colors from anywhere on the canvas—not just what it draws. I'd be excited to be part of this project and help improve Music Blocks in a meaningful way. |
Hi! I’d love to work on this issue for DMP 2025. I’ve already explored how to detect pixel colors using canvas for both uploaded images and live webcam feeds using JavaScript. |
Subject: Interest in Contributing to [Project Name] Hi @pikurasa , I’m Ghanshyam, a final-year B.Tech CSE student from Stani Memorial College of Engineering and Technology. I’m excited about the opportunity to contribute to this project, particularly because of its focus on Color sensor for Music Blocks. The real-world impact it could have—especially in underserved communities—aligns closely with my passion for building meaningful applications. I’ve explored the problem space and have begun brainstorming solutions, including leveraging MERN stack for real-time progress tracking or integrating interactive UI/UX to boost engagement. With experience in full-stack development (MERN), I’ve built projects like: Wandarlust: An Airbnb-inspired platform with user auth, dynamic search, and booking systems. E-commerce App: A scalable product with payment integration and admin dashboards. Simon Says Game: A lightweight interactive game showcasing frontend logic. I’m confident I can contribute to making this platform robust and user-centric. Is there a Slack/Discord space for collaborators? I’d love to join discussions and refine my approach. Looking forward to your thoughts! Best, |
Hi @walterbender @DevinUlibarri , Thank You, |
Hi @walterbender @DevinUlibarri , |
Hi @walterbender @DevinUlibarri 👋, I'm genuinely excited about the opportunity to contribute to the Music Blocks project, especially this feature that brings together accessibility, creativity, and music. The idea of enabling color detection from webcam feeds and uploaded images to support Lego music notation for the blind really resonates with me—it’s meaningful, impactful, and something I’d be proud to contribute to. With my background in JavaScript and frontend development, I believe I can be helpful in implementing and refining this feature. I'm enthusiastic about working with the team, learning through the process, and ensuring the feature is as accessible and user-friendly as possible. I’d love to be considered for this project and am eager to get started if selected. Thank you for the opportunity! Best regards, |
Hi @walterbender @pikurasa I have experience with JavaScript, and I want to build a feature that allows the turtle/mouse to detect colors from uploaded images and live webcam streams. |
I would like to work on this project, Currently i am pursuing Engineering degree in IT, I am a final year student and recently i have done my project which is Design the chat Application using Mern Stack, and i am very excited to work on this project so what are the key steps and how can i selected for this project. |
Hi! 👋 Thanks and regards, |
Ticket Contents
Description
Music Blocks has a feature to detect the color of pixels generated
from drawing within the program, but it cannot detect the color of
pixels from images that are either uploaded or from a webcam. By
adding a feature to detect color from both uploaded images and a live
webcam stream, users would be able to implement Lego music notation
for the blind and similarly interactive programs.
The goal of the project is to develop extended functionality to our
existing tools of turtle/mouse glyph movement and limited color
detection to sense color from uploaded images, as well as the
real-time feed from a webcam. Upon successful implementation, the
turtle/mouse glyph will be able to detect the color of pixels
underneath it, regardless of whether those pixels were drawn by the
turtle/mouse itself, part of an uploaded image stamped to the canvas,
or part of a live webcam video feed into Music Blocks. One test of
success is to run our Lego music notation for the blind project with
a live feed. The result should be able to playback and record the
abstract brick notation based on its contrasting colors.
Goals & Mid-Point Milestone
Goals
Setup/Installation
No response
Expected Outcome
Upon successful implementation, users should be able to use a live feed within Music Blocks and use the "pixel color" block to detect the color of pixels from a live webcam feed. An important use case for us is to be able to use this feature to encode musical data via LEGO bricks, which promises to make Music Blocks accessible to the blind.
Acceptance Criteria
No response
Implementation Details
Ideally, the pixel color feature should be implemented in client-side Javascript, the same language Music Blocks is written. Implementations done via server-side Javascript or Python are acceptable as well, but less ideal.
Mockups/Wireframes
References:
Product Name
Music Blocks
Organisation Name
Sugar Labs
Domain
Education
Tech Skills Needed
JavaScript
Mentor(s)
Coding Mentors
Walter Bender
Assisting Mentors
Devin Ulibarri
Category
Accessibility
The text was updated successfully, but these errors were encountered: