Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement stereo sound. #138

Merged
merged 2 commits into from
May 30, 2022
Merged

Implement stereo sound. #138

merged 2 commits into from
May 30, 2022

Conversation

waptaff
Copy link
Contributor

@waptaff waptaff commented May 21, 2022

Original Space Cadet has mono sound. To achieve stereo, the two
following steps were accomplished:

  • Extend all calls to sound-playing functions so that they include the
    position of the sound source. For instance, when a collision occurs
    on a bumper, the {x, y} position of the hit is sent along with the
    sound-playing arguments.

    For some cases, like light rollovers, the sound position is taken from
    the ball itself.

    For other cases, like holes, flags and targets, the sound position is
    taken from the geometrical center of the corresponding graphic sprite.

    For some special cases like ramp activation, sound is positioned to
    the nearest light position that makes sense.

    For all game-progress sounds, like mission completion sounds or ball
    drain sounds, they are positioned at the top center of the table,
    where speakers on a pinball machine normally are.

  • Make the Sound::PlaySound() method position the sound in the stereo
    field.

    This project uses SDL_mixer and there is a function called
    Mix_SetPosition() that allows placing a sound in the stereo field, by
    giving it a position and an angle.

    We arbitrarily place the player's ears at the bottom of the table; we
    set the ears' height to half a table's length. Intensity of the
    stereo effect is directly related to this value; the farther the
    player's ears from the table, the narrowest the stereo picture gets,
    and vice-versa.

    From there we have all we need to calculate distance and angle; we do
    just that and position all the sounds.

Notes:

  1. It's understood that this patch deviates from the original game. The
    decision to integrate such a change into the official project is up
    to the original reverse engineer and the patch could be rejected on
    that basis alone; that would be a perfectly valid reason.

  2. Even though the position of most sounds could be pre-calculated to
    avoid pointless CPU usage, the current approach dynamically
    calculates most positions. That can be changed if necessary.

  3. A mono sound is more forgiving when the number of channels is low; as
    each individual sound is sent to a free channel, the current code
    logic can abort an already-playing sound when all channels are in
    use, and those sounds suddenly stopping can become quite obvious in
    stereo; increasing the number of channels in the game configuration
    helps with that side-effect.

  4. There still is an extra argument on sound-playing function, that
    indicates the sound source. That was invaluable for ensuring the
    sound positions were correct. This extra argument will be removed if
    the patch is accepted.

  5. It seems OpenAL can do a much better job with stereo positioning than
    Mix_SetPosition(). It subjectively appears the current
    implementation is good enough but OpenAL could be investigated.

@RiedleroD
Copy link

I'd recommend to make this an optional feature - on by default, but disableable (I need a thesaurus) in the audio settings.

waptaff added a commit to waptaff/SpaceCadetPinball that referenced this pull request May 21, 2022
@k4zmu2a
Copy link
Owner

k4zmu2a commented May 23, 2022

An interesting idea, not something I planned for at all.
My first reaction was – out of scope; but then it kind of made sense, we can make it work.
This just about passes on my arbitrary complexity/benefit scale.

On coordinate systems:
Ideally, collision coordinate system should be used for positional audio (ball position is in this system).
The downside: not all sound emitters have their positions defined in this system, and I am not sure if reverse-projecting from raster coordinates makes sense here.
The downside of raster coordinates: they are perspective projected in relation to the true collision coordinate system, making all angle and distance calculations wrong.
Raster coordinates are fine as the easiest to achieve solution, it is not likely that someone could notice the approximation.

On sound API, I envision it as:
Loader::play_sound(int soundIndex, TPinballComponent* soundSource const char* soundId)
TPinballComponent has a virtual method that returns sound position.
For most components this can be precomputed at construction from main RenderSprite.
For the ball, this can return projected position.
Loader can resolve TPinballComponent->coordinates, leaving Sound as is.
If nullptr=’default position’ is not enough, add play_sound overload with explicit position argument.

Channel count: We can bump default channel count (to 16 for example) if channel exhaustion becomes a major problem for stereo.
Default of 8 channels was set to mimic the original.

Sound Id argument: It is ok to keep the extra sound argument for testing this feature in the future.

On OpenAL: I’d rather not change/add audio backend at this point, SDL_Mixer is good enough for the base game.

Original Space Cadet has mono sound.  To achieve stereo, the following
steps were accomplished:

- Add a game option to turn on/off stereo sound.  Default is on.

- TPinballComponent objects were extended with a method called
  get_coordinates() that returns a single 2D point, approximating the
  on-screen position of the object, re-mapped between 0 and 1 vertically
  and horizontally, {0, 0} being at the top-left.

    - For static objects like bumpers and lights, the coordinate refers
      to the geometric center of the corresponding graphic sprite, and
      is precalculated at initialization.

    - For ball objects, the coordinate refers to the geometric center of
      the ball, calculated during play when requested.

- Extend all calls to sound-playing methods so that they include a
  TPinballComponent* argument that refers to the sound source, e.g.
  where the sound comes from.  For instance, when a flipper is
  activated, its method call to emit a sound now includes a reference to
  the flipper object; when a ball goes under a SkillShotGate, its method
  call to emit a sound now includes a reference to the corresponding
  light; and so on.

  For some cases, like light rollovers, the sound source is taken from
  the ball that triggered the light rollover.

  For other cases, like holes, flags and targets, the sound source is
  taken from the object itself.

  For some special cases like ramp activation, sound source is
  taken from the nearest light position that makes sense.

  For all game-progress sounds, like mission completion sounds or ball
  drain sounds, the sound source is undefined (set to nullptr), and the
  Sound::PlaySound() method takes care of positioning them at a default
  location, where speakers on a pinball machine normally are.

- Make the Sound::PlaySound() method accept a new argument, a
  TPinballComponent reference, as described above.

  If the stereo option is turned on, the Sound::PlaySound() method calls
  the get_coordinates() method of the TPinballComponent reference to get
  the sound position.

  This project uses SDL_mixer and there is a function called
  Mix_SetPosition() that allows placing a sound in the stereo field, by
  giving it a distance and an angle.

  We arbitrarily place the player's ears at the bottom of the table; we
  set the ears' height to half a table's length.  Intensity of the
  stereo effect is directly related to this value; the farther the
  player's ears from the table, the narrowest the stereo picture gets,
  and vice-versa.

  From there we have all we need to calculate distance and angle; we do
  just that and position all the sounds.
@waptaff
Copy link
Contributor Author

waptaff commented May 27, 2022

Refactored the stereo sound patch to include suggestions from @k4zmu2a — thank you for taking the time to think about this and come up with this better approach. Patch is hence cleaner.

Please let me know if there are further changes you'd like me to work on.

@k4zmu2a k4zmu2a merged commit a4c6165 into k4zmu2a:master May 30, 2022
@k4zmu2a
Copy link
Owner

k4zmu2a commented May 30, 2022

You are welcome.
And thank you in turn for your interest in the project.

The change is now ready for merge as v1.
I am going to clean it up and change a few things, most notably – this feature will be off by default.
That is because it noticeably changes how the game sounds as compared to the original.

Things I am going to try for v2 (that you could also independently try):
Uniform coordinate normalization taking into account table aspect ratio.
Sound positions in collision coordinates.
Stereo effect strength slider in the options.

@waptaff
Copy link
Contributor Author

waptaff commented May 30, 2022

Great news. I'm with you when it comes to stereo sound off by default, after all it's not the original game behavior.

Ideas for V2 are interesting, indeed V1 projections are crude approximations; they work but could be improved. Effect strength slider would be neat.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants