Music Lab: initial prototype work #47808

breville · 2022-08-24T22:48:21Z

This is the initial prototype of Music Lab.

This standalone app is designed to allow the creation of music with code.

Initially available standalone at https://studio.code.org/musiclab.
The current API model allows the scheduling of a sound play at a specific time, provided as a measure number; non-integer values may be specified.
The timeline displays all audio plays.
The code under the "when run" block is executed whenever the blocks are changed. The timeline is updated in realtime to reflect these code changes, whether the song is currently playing or not.
When the music is playing, the user may trigger events using the "beat pad" which executes additional code which can schedule additional audio plays. The timeline is updated as soon as these are scheduled.
The available sounds are specified with a JSON file stored on S3, along with the relevant sounds.
The instructions are also specified with a JSON file stored on S3, along with the relevant images.
Keyboard shortcuts:
- i: show/hide instructions
- t: toggle timeline location, above/below the blocks
- d: sample block mode = play block + dropdown
- v: sample block mode = values
- p: sample block mode = play block + values
- space: play/stop song
- 1, 2, 3, 4, 5, 6: trigger buttons
Not yet integrated with existing game/script/level system.
Sharing is not yet implemented.

Some changes of interest:

The work involved in integrating with the game/script/level system: Music prototype: standalone #48456
Discussion of the audio scheduling model: Music prototype: adjust sound events management #48538

Needs proper variable handling still, though it does manage to loop sound placement.

It only checks against a hardcoded variable called measure.

No need to show a horizontal scrollbar.

…layout Music prototype: fix SCSS layout

…tructions-scrollbars Music prototype: only show vertical scrollbar for instructions

A second (and now simpler) attempt at reducing the blockly text to 14px.

…ockly-text-3 Music prototype: shrink blockly text

We don't support localization for now.

…cale-support Music prototype: Update locale support

…renderers Music prototype: separate blockly renderers

…tra-styling Music prototype: remove extra styling

It was just removed for the convenience of early development.

…eacher-panel Music prototype: restore teacher panel

The user can click the small instructions image to see a big version, and click that to go back to the small one. Also, the instructions text scrollbar is only shown when necessary.

…uctions-image Music prototype: big instructions image

molly-moen

A few random questions but looks good to me! I'm new to blockly so it would be good to get an approval from someone who knows more about it :)

apps/src/music/MusicView.jsx

molly-moen · 2022-10-18T21:17:38Z

apps/src/music/Instructions.jsx

+import PropTypes from 'prop-types';
+import React from 'react';
+
+export default class Instructions extends React.Component {


I'd be interested to hear if you all are hoping to replace instructions with a pattern like this or if this is intended to be only for music lab

This is just a temporary component that can be replaced by a real instructions component once we integrate with our level system.

molly-moen · 2022-10-18T21:31:31Z

apps/src/blockly/addons/cdoPathObjectZelos.js

@@ -0,0 +1,15 @@
+import GoogleBlockly from 'blockly/core';
+
+export default class CdoPathObject extends GoogleBlockly.zelos.PathObject {


for my learning: what is zelos?

Learned this recently, they're different blockly renderers. I think their docs are kind of broken at the moment so I can't see the main zelos renderer page. No idea why they're using greek names 🤦

molly-moen · 2022-10-18T21:32:21Z

apps/src/music/MusicView.jsx

+var hooks = {};
+
+class UnconnectedMusicView extends React.Component {
+  static propTypes = {


is this a class component because of the need for redux?

Yeah, this just started out as a class component but with the need for Redux and the fact that it currently houses a lot of the blockly logic, it made sense to keep it a class. As we move stuff out of here it would be ideal to convert this (and the other views) into a functional component.

bencodeorg · 2022-10-18T21:41:26Z

apps/src/blockly/googleBlocklyWrapper.js

@@ -187,6 +189,12 @@ function initializeBlocklyWrapper(blocklyInstance) {
    CdoRenderer,
    true /* opt_allowOverrides */
  );
+  blocklyWrapper.blockly_.registry.register(


Preeety sure this shouldn't affect anything given we explicitly set the renderer elsewhere in this file, but maybe worth a quick check that this doesn't affect rendering of any of our google blockly labs (eg, poetry)?

Glad you called this out, as it was a specific concern. I've been manually checking https://studio.code.org/flappy/10 to make sure we don't change behavior in an existing Google Blockly level that uses the other renderer.

apps/src/blockly/addons/cdoRendererZelos.js

bencodeorg · 2022-10-18T21:51:45Z

dashboard/app/views/shared/_extra_links.html.haml

@@ -1,6 +1,6 @@
 .extra-links
  %h4
-    Extra Links:
+    %span{onclick: 'return extraLinksDismissClick()'} Extra Links:


What is this change doing?

Hah, this was a little convenience that I'll remove from this PR. It allowed the viewer to dismiss the admin menu in the bottom-left corner by clicking on its "Extra links:" text. :)

…ew-fixes Music prototype: code review fixes

bencodeorg

I skimmed a lot of the React and tried to focus on the pieces that touch other stuff, but I think that part looks good! Looks like Molly took a closer look at some of the React, so I think you have some decent coverage of most of what's here. Exciting stuff!

bencodeorg · 2022-10-18T23:56:16Z

apps/src/music/player/sound.js

@@ -0,0 +1,118 @@
+import WebAudio from './soundSub';


One other q -- is all of this new relative to our existing Sound.js file? Or could be refactored/combined at some point? Or do we intentionally want to keep this separate?

This is all new. I looked at the existing code but found it a bit confusing, especially since it supported both Web Audio and HTML Audio in the one file. (A great candidate for a factory that provides one or the other implementation, as it happens.). I had this code from previous hobby projects and it fit the needs well here. Certainly open to a unification, and would love if we had a common library that served all needs, but that feels like a nontrivial effort by itself.

I also find the existing code supporting both audio types in one file pretty confusing, wonder if we could split it out at some point.

Music prototype: update media URL

Music: initial prototype work

807e6c4

breville requested a review from a team as a code owner August 24, 2022 22:48

breville marked this pull request as draft August 24, 2022 22:48

breville added 27 commits August 25, 2022 09:44

Can now play sounds with blocks under 'when run'

b2b7070

Clear timeline events before running

de6c2e9

User gets 1-based measures

ed6700b

Can use button to trigger sound on next measure, with blocks

17f6247

Now plays sounds from selected pack

01c9b32

Small improvements

4a641b9

Realtime timeline preview & varied waveform images

463970f

Resize Blockly when layout changes

2cbc65e

Just hide teacher panel

0cfd6f5

Some styling

9581c78

Stop existing song playback before starting new one

56c5953

Sample package sound previews

2ce6756

Try zelos renderer

2aee306

Hide Blockly border

97b75ed

Update locale name

3b6cbfd

Add locale to a test

955c2b2

Add some country

fd82bfa

Each bar is 4 seconds for now

cf106d9

Prettier

ca619d1

Merge branch 'staging' into music-prototype

51cf594

Show measure numbers and separate waveforms vertically

12be226

Highlight current measure in timeline

6031fd3

Sample packs are now data driven & have experimental highlight image

8ce8b35

Hacky attempt at a loop

bafce68

Needs proper variable handling still, though it does manage to loop sound placement.

Hacky if_even_then block

6f190b1

It only checks against a hardcoded variable called measure.

Restore play_sound & add play_sound_with_variable

96ac1c9

Loop block gets two lines

dc078eb

breville and others added 18 commits October 15, 2022 06:06

Music prototype: fix SCSS layout

34291cd

Music prototype: only show vertical scrollbar for instructions

5ed62e2

No need to show a horizontal scrollbar.

Merge pull request #48654 from code-dot-org/music-prototype-fix-scss-…

7fa0252

…layout Music prototype: fix SCSS layout

Merge pull request #48655 from code-dot-org/music-prototype-fewer-ins…

176c256

…tructions-scrollbars Music prototype: only show vertical scrollbar for instructions

Music prototype: shrink blockly text

5af9413

A second (and now simpler) attempt at reducing the blockly text to 14px.

Merge pull request #48656 from code-dot-org/music-prototype-shrink-bl…

65ce134

…ockly-text-3 Music prototype: shrink blockly text

Music prototype: Update locale support

a9940eb

We don't support localization for now.

Merge pull request #48662 from code-dot-org/music-prototype-update-lo…

73554f2

…cale-support Music prototype: Update locale support

Music prototype: separate blockly renderers

1f3ef14

Merge pull request #48663 from code-dot-org/music-prototype-separate-…

fbd972f

…renderers Music prototype: separate blockly renderers

Music prototype: remove extra styling

9611713

Merge pull request #48664 from code-dot-org/music-prototype-remove-ex…

096631e

…tra-styling Music prototype: remove extra styling

Merge branch 'staging' into music-prototype

b5e8a44

Music prototype: restore teacher panel

2d5b80c

It was just removed for the convenience of early development.

Merge pull request #48689 from code-dot-org/music-prototype-restore-t…

17abfba

…eacher-panel Music prototype: restore teacher panel

Music prototype: big instructions image

e4afb15

The user can click the small instructions image to see a big version, and click that to go back to the small one. Also, the instructions text scrollbar is only shown when necessary.

Add border radius to big instructions image

6a91995

Merge pull request #48692 from code-dot-org/music-prototype-big-instr…

185ac3e

…uctions-image Music prototype: big instructions image

breville marked this pull request as ready for review October 18, 2022 16:03

molly-moen approved these changes Oct 18, 2022

View reviewed changes

bencodeorg reviewed Oct 18, 2022

View reviewed changes

breville and others added 2 commits October 18, 2022 23:16

Music prototype: code review fixes

e917d4d

Merge pull request #48710 from code-dot-org/music-prototype-code-revi…

b974769

…ew-fixes Music prototype: code review fixes

bencodeorg approved these changes Oct 18, 2022

View reviewed changes

bencodeorg reviewed Oct 18, 2022

View reviewed changes

breville and others added 2 commits October 19, 2022 04:55

Music prototype: update media path

a1c116d

Merge pull request #48716 from code-dot-org/music-prototype-media-path

08dd4cc

Music prototype: update media URL

breville merged commit d6d71e2 into staging Oct 19, 2022

breville deleted the music-prototype branch October 19, 2022 19:03

breville mentioned this pull request Mar 11, 2024

Music: revive the advanced model #57166

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Music Lab: initial prototype work #47808

Music Lab: initial prototype work #47808

breville commented Aug 24, 2022 •

edited

molly-moen left a comment

molly-moen Oct 18, 2022

breville Oct 18, 2022

molly-moen Oct 18, 2022

bencodeorg Oct 18, 2022

molly-moen Oct 18, 2022

sanchitmalhotra126 Oct 18, 2022

bencodeorg Oct 18, 2022

breville Oct 18, 2022

bencodeorg Oct 18, 2022

breville Oct 18, 2022

bencodeorg left a comment

bencodeorg Oct 18, 2022

breville Oct 19, 2022

bencodeorg Oct 19, 2022

		@@ -0,0 +1,15 @@
		import GoogleBlockly from 'blockly/core';

		export default class CdoPathObject extends GoogleBlockly.zelos.PathObject {

Music Lab: initial prototype work #47808

Music Lab: initial prototype work #47808

Conversation

breville commented Aug 24, 2022 • edited

molly-moen left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bencodeorg left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

breville commented Aug 24, 2022 •

edited