[DMP 2025]: Improve synth and sample features in Music Blocks

[pikurasa]

Ticket Contents

Description

Users have two main methods within Music Blocks to play with sound:
synths and samples. For our synth, we use tone.js. For samples, we use
.wav binaries and transpose the sound to different pitches. While
these features work "well enough," there is still more that can been
to make them useful. For this project, a contributor would work
closely with their mentors to 1) update the sampler widget, 2) port a
list of free/libre/open samples into Music Blocks, and 3) add to the
Set Instrument feature and Sampler Widget the ability to assign
multiple samples for the same instrument with criteria (e.g. high and
low, short and long) for a more natural sound.

Updating the sampler widget will involve updating tone.js to its
current version, debugging any issues that updates may cause, and
making improvements to the UI/UX of the widget itself. Improvements
include adding a tuner feature in Sampler Widget, just like
https://www.musicca.com/tuner and adding a way to do micro
adjustments in cents.

Porting samples into Music Blocks will require following the
directions specified in the Music Blocks documentation to convert a
curated list of samples. After completing this, the user-facing menus
showing the samples will need to be updated and organized based on
instrument type. There is some room to get creative with the
presentation of the instruments, perhaps adding icons for each
instrument.

The final part of the project is perhaps the most challenging. It will
require adding additional functionality so that a user can either
upload or record multiple samples of an instrument or voice to be
assigned to a custom instrument in Music Blocks. Doing this will make
the overall tone of the instruments more persuasive. For example, if
the Music Blocks project has short, staccato sounds, the playback can
use the short sample created by a recorded instrument.

Goals & Mid-Point Milestone

Goals

• Implement (possibly refactor) how Music Blocks samples works so that multiple sounds can be assigned to the same instrument.
• Test implementation based on developer tests and improve performance
• Test optimized deployment in classrooms and receive feedback
• Make any final improvements based on classroom feedback
• Document progress, outcomes, and technical guides for future contributors

Setup/Installation

No response

Expected Outcome

No response

Acceptance Criteria

No response

Implementation Details

Javascript and Tone.js

Mockups/Wireframes

References:

• https://github.com/sugarlabs/musicblocks/tree/master/sounds/samples
• "Processing for pitched (non-percussion) samples" section of https://wiki.sugarlabs.org/go/Music_Blocks/2025-02-09-meeting
• A professionals guide to creating "virtual instruments": https://www.nicolastiteux.com/en/blog/making-a-virtual-instrument-a-guide-to-sampling/
• Possible samples: https://philharmonia.co.uk/resources/sound-samples/
• Possible samples: https://github.com/sonic-pi-net/sonic-pi/tree/dev/etc/samples
• Possible samples: https://freesound.org/people/MTG/

Product Name

Music Blocks

Organisation Name

Sugar Labs

Domain

⁠Education

Tech Skills Needed

Feature

Mentor(s)

Coding Mentors

Walter Bender

Assisting Mentors

Devin Ulibarri

Category

Frontend

[AnvitaPrasad]

Hi @pikurasa,

This project looks really interesting! I have a background in music and have learned up to Grade 4 in Trinity music theory. I think that knowledge might be useful here, and I’d love to contribute. Looking forward to learning more!

Thanks!

[Vanaja-Nannepaga]

Hi! @pikurasa, I'm really excited about this project and eager to contribute. I have a solid foundation in JavaScript and have started exploring Tone.js and the Music Blocks codebase. I'm particularly interested in improving the sampler widget and enhancing the user experience with more expressive instrument playback. Looking forward to learning more and collaborating with the team!

[kamat-abhi]

Hey @pikurasa, please assign to me . i'm very much exited to contribute on this isuue.

[praveen-86176]

Hey @pikurasa, please assign to me . i'm very much exited to contribute on this isuue. I'm very enthusiastic in contributing in open source and will love to work on this project.

Thank You!

[pikurasa]

Wonderful to meet you all. This is a special issue for DMP. Please learn more about DMP: https://codeforgovtech.in/dedicated_mentoring_program/

We discuss Music Blocks here: https://matrix.to/#/#musicblocksdev:matrix.org

[Aditya-269]

Hi @pikurasa ,
Aditya Kumar Sahoo here. I’m excited about the opportunity to contribute to this issue.
I have strong experience with JavaScript and would love to collaborate and contribute further to the project.

[marvillage]

I hope you're doing well! I have reviewed the project details for Music Blocks V4 and I am very excited about the opportunity to contribute to this module. I have experience with TypeScript, React, and UI/UX design, and I am confident that I can help move the project forward by implementing the necessary components and utilities for Music Blocks programs.

Could you please assign this project to me? I would love to collaborate with you and the team to contribute to the success of this module.

Looking forward to your response!

[pikurasa]

Could you please assign this project to me? I would love to collaborate with you and the team to contribute to the success of this module.

This is a special issue. It's for https://codeforgovtech.in/dedicated_mentoring_program/. We encourage interested contributors to apply there.

[hemantmm]

Hi @pikurasa ,

This project looks really interesting! I have a background in music, and I think that knowledge might be useful here. I’d love to contribute, and I'm looking forward to learning more!

Thanks!

[psuryasnata]

Hi @pikurasa @walterbender ,
I’m excited to contribute to improving the synth and sample features in Music Blocks. With my full-stack development experience and strong JavaScript skills, I’m eager to enhance the Sampler Widget, integrate multiple samples, and refine the UI/UX for a more dynamic user experience.
I’m also keen to work with Tone.js and streamline the process of adding custom samples. I’m drafting my proposal and would appreciate any insights on the technical direction, particularly regarding the Sampler Widget.
Looking forward to collaborating!

[vishaljadhav7]

Hi @walterbender and @devinulibarri,

I'm interested in contributing to the Music Blocks synth and sample improvement project. My experience includes:

• JavaScript and web audio development
• Working with Tone.js for audio synthesis
• UI/UX design for audio applications
• Audio processing and sample management

I'd be particularly excited to work on:

1. Updating the sampler widget with the latest Tone.js
2. Implementing the tuner feature with cent-level adjustments
3. Developing the multiple sample system for more natural instrument sounds

I've reviewed the existing samples directory and the meeting notes regarding pitched sample processing. I can comfortably work with virtual instrument creation principles.

[UjjwalSaini07]

Hey hi @walterbender, @pikurasa

Contributor: Ujjwal Saini
GitHub: https://github.com/UjjwalSaini07
Education: 3rd-year B.Tech, Information Technology, GGSIPU
My Skills:

• Languages: Python, JavaScript, C, C++
• Technologies/Frameworks: ReactJs, ExpressJs, NodeJs, jQuery, Sass/Scss, Tailwind, Mongoose, Nextjs
• Databases: MySql, MongoDB, PostgreSQL, Redis.
• Developer Tools: Git, GitHub, VS Code, vercel, Docker, PostMan, Firebase, Replit, Google Cloud, Figma, ShadCn.

I'm very interested in contributing to the Music Blocks synth and sample improvement project.

My Role & Contributions:

Core Skills and Experience:

• Great proficiency in JavaScript and web audio development, with extensive experience in Tone.js for audio synthesis and audio processing.
• Expertise in UI/UX design tailored for audio applications, ensuring intuitive interfaces and seamless user experiences.

Key Achievements and Technical Contributions:

• Spearheaded the development and optimization of full-stack features using Next.js, React.js, and Node.js, delivering seamless and robust frontend-backend integration.
• Achieved significant performance enhancements, reducing page load times by 25% and API response times by 35%, demonstrating strong problem-solving and optimization skills.
• Produced clean, maintainable, and scalable code, complemented by comprehensive documentation to ensure team collaboration and long-term project maintainability.

Exciting Opportunities and Projects:

• Updating the sampler widget with the latest features and improvements in Tone.js.
• Designing and developing a multi-sample system to produce more natural and realistic instrument sounds, aligning technical advancements with user-centric goals.

This blend of audio development expertise, full-stack optimization, and user-focused design ensures a well-rounded approach to creating innovative and high-performing applications.

Goal: To contribute my full-stack development skills and collaborative mindset to deliver a high-quality, scalable, and secure solution, while continuously learning and adapting to the project’s technical requirements.

• Directly Contact me at - Mail, Linkedin, Github

[Commanderk3]

@AnvitaPrasad This is the branch.
As you know, in music blocks, there are only one sample for one duration per instrument. I have implemented a way to add multiple duration samples for each instrument and multiple pitched samples as well. I would recommend looking at the synthutils.js
(The code is untidy btw, I just wanted to get it done initially and then proceed with optimization and formatting).