MacOS Support

[OPNA2608]

I was approached by a macOS user on Discord recently about getting BambooTracker to run natively on macOS. While it supposedly does run well in Wine, afew things still cause crashes and a native .dmg image for installation is just more appealing than fiddling with Wine.

I think it therefore makes sense to invest some time into getting it running on macOS and providing a precompiled binary via CI, as manual compilation is an unreasonable expectation for alot of end-users of this platform (downloading Xcode, the massive Qt5 framework, working in the Terminal, …) and, as of right now, shouldn't even succeed.

---

Points of Interest / Problems:

• macOS (or Xcode?) comes with the clang compiler instead of gcc, we should check if the argument and strictness differences between these two is significant enough to abort the compilation prematurely
  (I tried compiling BambooTracker with clang on Linux some time ago and afaik it only failed on the very last compilation step with unknown option: -stdlib=libstdc++, might be a system or qmake problem though)
  clang appears to work just fine, it just issues afew more warnings about unused varibales
• macOS does not support ALSA, instead it offers the CoreAudio framework for audio and MIDI as far as I understand
  this appears to be handled in the RtMidi code, see next comment for problems
• we'll need to setup Travis CI to build the code on macOS as well
• and the READMEs need to be expanded with a macOS section
• and, of course, someone'd need to get a test system to develop and test all of these changes on. all of the Apple systems i own are way too old to get a modern-enough macOS version for Qt5 running (PowerMac G5, Late 2006 XServe), but i can see if i can eventually get a spare, more modern macOS system at work
  I can get my hands on a macOS 10.11.3 machine at irregular intervals

[OPNA2608]

I got a system running macOS 10.11.3, installed Xcode 7.3.1 and Qt 5.9.7 and attempted a compilation. Surprisingly, it got pretty far but eventually did end up failing.

I think something's not right with RtMidi on macOS. Maybe I'm missing something @jpcima? Take a look:

BambooTracker.log

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -stdlib=libc++ -headerpad_max_install_names -arch x86_64 -Wl,-syslibroot,/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk -mmacosx-version-min=10.10 -Wl,-rpath,@executable_path/Frameworks -Wl,-rpath,/Users/########/Qt/5.9.7/clang_64/lib -o BambooTracker.app/Contents/MacOS/BambooTracker main.o mainwindow.o chip.o opna.o resampler.o 2608intf.o emu2149.o fm.o ymdeltat.o bamboo_tracker.o audio_stream.o audio_stream_mixier.o jam_manager.o pitch_converter.o instruments_manager.o command_manager.o add_instrument_command.o remove_instrument_command.o add_instrument_qt_command.o remove_instrument_qt_command.o instrument_editor_fm_form.o fm_operator_table.o labeled_vertical_slider.o labeled_horizontal_slider.o slider_style.o change_instrument_name_qt_command.o change_instrument_name_command.o opna_controller.o instrument.o envelope_fm.o event_guard.o tick_counter.o module.o song.o pattern.o track.o step.o order_list_panel.o order_list_editor.o pattern_editor_panel.o pattern_editor.o instrument_editor_ssg_form.o set_key_off_to_step_qt_command.o set_key_off_to_step_command.o set_key_on_to_step_command.o set_key_on_to_step_qt_command.o set_instrument_to_step_qt_command.o set_instrument_to_step_command.o erase_instrument_in_step_qt_command.o erase_instrument_in_step_command.o set_volume_to_step_qt_command.o set_volume_to_step_command.o erase_volume_in_step_qt_command.o erase_volume_in_step_command.o set_effect_id_to_step_command.o set_effect_id_to_step_qt_command.o erase_effect_in_step_command.o erase_effect_in_step_qt_command.o set_effect_value_to_step_command.o set_effect_value_to_step_qt_command.o erase_effect_value_in_step_command.o erase_effect_value_in_step_qt_command.o insert_step_command.o delete_previous_step_command.o insert_step_qt_command.o delete_previous_step_qt_command.o erase_step_qt_command.o erase_step_command.o deep_clone_instrument_qt_command.o deep_clone_instrument_command.o clone_instrument_command.o clone_instrument_qt_command.o set_pattern_to_order_command.o set_pattern_to_order_qt_command.o insert_order_below_command.o delete_order_command.o insert_order_below_qt_command.o delete_order_qt_command.o paste_copied_data_to_pattern_command.o paste_copied_data_to_pattern_qt_command.o erase_cells_in_pattern_command.o erase_cells_in_pattern_qt_command.o paste_copied_data_to_order_command.o paste_copied_data_to_order_qt_command.o instrument_form_manager.o lfo_fm.o visualized_instrument_macro_editor.o command_sequence.o effect_iterator.o paste_mix_copied_data_to_pattern_command.o paste_mix_copied_data_to_pattern_qt_command.o decrease_note_key_in_pattern_qt_command.o increase_note_key_in_pattern_qt_command.o increase_note_octave_in_pattern_qt_command.o decrease_note_octave_in_pattern_qt_command.o increase_note_key_in_pattern_command.o decrease_note_key_in_pattern_command.o increase_note_octave_in_pattern_command.o decrease_note_octave_in_pattern_command.o module_properties_dialog.o groove.o groove_settings_dialog.o configuration_dialog.o expand_pattern_command.o expand_pattern_qt_command.o shrink_pattern_command.o shrink_pattern_qt_command.o abstract_instrument_property.o duplicate_order_command.o move_order_command.o clone_patterns_command.o clone_order_command.o duplicate_order_qt_command.o move_order_qt_command.o clone_patterns_qt_command.o clone_order_qt_command.o set_echo_buffer_access_qt_command.o set_echo_buffer_access_command.o comment_edit_dialog.o file_io.o binary_container.o interpolate_pattern_qt_command.o interpolate_pattern_command.o reverse_pattern_qt_command.o reverse_pattern_command.o replace_instrument_in_pattern_qt_command.o replace_instrument_in_pattern_command.o export_container.o vgm_export_settings_dialog.o wave_export_settings_dialog.o configuration.o configuration_handler.o color_palette.o paste_overwrite_copied_data_to_pattern_qt_command.o paste_overwrite_copied_data_to_pattern_command.o file_io_error.o wopn_file.o bank.o instrument_selection_dialog.o s98_export_settings_dialog.o timer.o module_io.o export_handler.o instrument_io.o bank_io.o fm_envelope_set_edit_dialog.o file_history_handler.o file_history.o midi.o RtMidi.o qrc_bamboo_tracker.o moc_mainwindow.o moc_audio_stream.o moc_audio_stream_mixier.o moc_instrument_editor_fm_form.o moc_fm_operator_table.o moc_labeled_vertical_slider.o moc_labeled_horizontal_slider.o moc_order_list_panel.o moc_order_list_editor.o moc_pattern_editor_panel.o moc_pattern_editor.o moc_instrument_editor_ssg_form.o moc_instrument_form_manager.o moc_visualized_instrument_macro_editor.o moc_module_properties_dialog.o moc_groove_settings_dialog.o moc_configuration_dialog.o moc_comment_edit_dialog.o moc_vgm_export_settings_dialog.o moc_wave_export_settings_dialog.o moc_s98_export_settings_dialog.o moc_fm_envelope_set_edit_dialog.o -F/Users/########/Qt/5.9.7/clang_64/lib -framework CoreMIDI -framework QtMultimedia -framework QtNetwork -framework QtCore -framework DiskArbitration -framework IOKit -framework QtGui -framework QtWidgets -framework OpenGL -framework AGL

Undefined symbols for architecture x86_64:
"_AudioConvertHostTimeToNanos", referenced from:
midiInputCallback(MIDIPacketList const*, void*, void*) in RtMidi.o
"_AudioGetCurrentHostTime", referenced from:
midiInputCallback(MIDIPacketList const*, void*, void*) in RtMidi.o
MidiOutCore::sendMessage(unsigned char const*, unsigned long) in RtMidi.o
"_CFDataGetBytePtr", referenced from:
ConnectedEndpointName(unsigned int) in RtMidi.o
"_CFDataGetLength", referenced from:
ConnectedEndpointName(unsigned int) in RtMidi.o
"_CFRelease", referenced from:
MidiInCore::initialize(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&) in RtMidi.o
MidiInCore::openPort(unsigned int, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&) in RtMidi.o
MidiInCore::openVirtualPort(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&) in RtMidi.o
EndpointName(unsigned int, bool) in RtMidi.o
MidiInCore::getPortName(unsigned int) in RtMidi.o
ConnectedEndpointName(unsigned int) in RtMidi.o
MidiOutCore::initialize(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&) in RtMidi.o
...
"_CFRunLoopRunInMode", referenced from:
MidiInCore::openPort(unsigned int, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&) in RtMidi.o
MidiInCore::getPortCount() in RtMidi.o
MidiInCore::getPortName(unsigned int) in RtMidi.o
MidiOutCore::getPortCount() in RtMidi.o
MidiOutCore::getPortName(unsigned int) in RtMidi.o
MidiOutCore::openPort(unsigned int, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&) in RtMidi.o
"_CFStringAppend", referenced from:
EndpointName(unsigned int, bool) in RtMidi.o
ConnectedEndpointName(unsigned int) in RtMidi.o
"_CFStringCompareWithOptions", referenced from:
EndpointName(unsigned int, bool) in RtMidi.o
"_CFStringCreateMutable", referenced from:
EndpointName(unsigned int, bool) in RtMidi.o
ConnectedEndpointName(unsigned int) in RtMidi.o
"_CFStringCreateWithCString", referenced from:
MidiInCore::initialize(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&) in RtMidi.o
MidiInCore::openPort(unsigned int, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&) in RtMidi.o
MidiInCore::openVirtualPort(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&) in RtMidi.o
MidiOutCore::initialize(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&) in RtMidi.o
MidiOutCore::openPort(unsigned int, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&) in RtMidi.o
MidiOutCore::openVirtualPort(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&) in RtMidi.o
"_CFStringGetCString", referenced from:
MidiInCore::getPortName(unsigned int) in RtMidi.o
MidiOutCore::getPortName(unsigned int) in RtMidi.o
"_CFStringGetLength", referenced from:
EndpointName(unsigned int, bool) in RtMidi.o
"_CFStringInsert", referenced from:
EndpointName(unsigned int, bool) in RtMidi.o
"___CFConstantStringClassReference", referenced from:
CFString in RtMidi.o
CFString in RtMidi.o
"_kCFRunLoopDefaultMode", referenced from:
MidiInCore::openPort(unsigned int, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&) in RtMidi.o
MidiInCore::getPortCount() in RtMidi.o
MidiInCore::getPortName(unsigned int) in RtMidi.o
MidiOutCore::getPortCount() in RtMidi.o
MidiOutCore::getPortName(unsigned int) in RtMidi.o
MidiOutCore::openPort(unsigned int, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&) in RtMidi.o

ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [BambooTracker.app/Contents/MacOS/BambooTracker] Error 1

[jpcima]

I think something's not right with RtMidi on macOS. Maybe I'm missing something @jpcima? Take a look:

This is -framework CoreAudio missing.
CF* functions are of CoreFoundation so try linking that also.

[OPNA2608]

Manually appending -framework CoreAudio -framework CoreFoundation to the Makefile's LIBS did the job for now, but

1. Starting the application produces a very loud noise.
2. Playback and the pattern following (alike to Stuttery drawing of Pattern Editor #49) is very stuttery, no matter the buffer size.
3. The bottom of the operators/LFO/arps/… editor window is too large on my display (1280x800). No settings appears to be cut off by the launchbar so it's merely a minor problem I guess.

[OPNA2608]

I think that at least 2. is an extreme example of the referenced issue, which causes alot of stuttering under Linux with >3ms. Reducing the vertical window size as low as it can go produces usable sound output, but that's obviously not a usable setup.

I reviewed code and noticed that the current step position was updated when the chip register was written tick data in setting samples to sound buffer.

BambooTracker/BambooTracker/stream/audio_stream_mixier.cpp

Lines 80 to 93 in d9772cd

 while (requiredCount) { 
 	if (!intrCountRest_) {	// Interruption 
 		intrCountRest_ = intrCount_;    // Set counts to next interruption 
 		emit streamInterrupted(); 
 	} 
  
 	count = std::min(intrCountRest_, requiredCount); 
 	requiredCount -= count; 
 	intrCountRest_ -= count; 
  
 	emit bufferPrepared(destPtr, count); 
  
 	destPtr += (count << 1);	// Move head 
 } 

In L83 emit streamInterrupted(); read tick data from pattern and write these to the chip (by calling BambooTracker::streamCountUp). This also updates the current step position.
In L90 emit bufferPrepared(destPtr, count); set mixed samples to sound buffer destPtr (by calling BambooTracker::getStreamSamples).
As repeat them until sound buffer is filled, it looks like that the position suddenly jump.

Hence we should maybe consider @rerrahkr's idea to make a separate buffer and somehow sync up the chip output, playback and pattern drawing.

I thought making a new buffer. It receives samples from the chip when reading tick, and sends it when the sound buffer requires. It will avoids stuttery drawing to divide tick update and sound buffering tasks, but I'm not sure how to synchronize pattern drawing and playback.

The current solution is to set the buffer length to 3 ms or less if there is no problem with playback.

[jpcima]

The general solution for latency is that you drop buffer-based playback in favor of callback-based. (QtMultimedia, as far as I know, supports only the buffered processing model)

The audio callback should handle all interaction with chips and generating; UI should communicate by use of some lock-free channel. (commonly ring-buffer used as FIFO queue)

[OPNA2608]

About CI:

We can add a Travis CI builder to test if BT will build on (various versions of) macOS, but that one can only natively upload artifacts to an AWS.

We can't add an AppVeyor builder because AppVeyor doesn't offer any macOS images to build on, and cross-compiling Qt to macOS supposedly hasn't been possible for awhile now.

So I guess we'd have to stick to build instructions on macOS, unless someone has any creative ideas. 😕

Edit: there are more deployment options than AWS S3 storage, but nothing that appears to be like a "drop-in and forget" replacement to AppVeyor.
https://docs.travis-ci.com/user/deployment/

[jpcima]

Since the idea of the Power Mac has been mentioned in the OP, I'll mention a problem specific to big-endian, therefore PowerPC.

The file formats are handled in the assumption of a little-endian computer.
So, they cannot work correctly. See example:
https://github.com/rerrahkr/BambooTracker/blob/c6d0b00b6bc02764bf4135c0e2fd2407c8b4162d/BambooTracker/io/export_handler.cpp#L18

[OPNA2608]

@jpcima I don't think there's any hope of getting BambooTracker to run on a PowerMac with OS X anymore these days anyway, even the oldest supported Qt5 versions are not available for PowerPC anymore. 😜
PowerMac with Linux (like Lubuntu 16.04) might still work, but the last available Qt5 version there (Ubuntu packages) is Qt5.5.1 which is officially unsupported by the Qt team since March 16th, 2018.

[jpcima]

Depending on how the task of PPC is important to you, there can be a compatibility if the program is made compatible at the same time with Qt4 and Qt5.
At OPL3BankEditor, we manage this. In fact, (don't you laugh 😄) Qt4 manages us compatibility down to Windows 98.

[OPNA2608]

I think PowerPC architecture support would definitely be neat to have - the more architectures the better - but I'm sure my not-quite-portable heater G5 wouldn't get much use out of that anymore. I'd rather use it in text-mode than boot up my Lubuntu installation and crawl my way through LXDE with outdated packages nowadays :v

[OPNA2608]

I think that at least 2. is an extreme example of the referenced issue, which causes alot of stuttering under Linux with >3ms.

I'm unsure now if this is actually the case. On my test system

• both the drawing and audio output are affected by this (but buffer size seemingly doesn't matter)
• the effect can be made worse by moving the mouse over the pattern editor fields alot
• the effect can be lessened by reducing the window (and therefore the pattern editor) size

This all sounds more like a rendering (or callback?) load problem to me now. 🤔

[jpcima]

@OPNA2608 I don't believe at all the stuttering to be a problem specific to MacOS.

For example, on another OS, when you set a relatively low buffer (eg 10ms), start a playback, and then open the configuration dialog, you will likely be able to hear the sound skipping as it opens.

What this means is: the sound generation is interdependent with the event loop of the GUI.
In the event loop, a single thread does all kinds of handling, including drawing.
So the good timing of sound generation is at the mercy of how fast the GUI doing its work, which will happen in indefinite time and you are unable to preempt it. Currently it's a soft-realtime situation, mitigated by a more-or-less large buffer size setting.

In the ideal, I think the sound producer would be engineered in such a way that it will operate by itself, in its distinct thread, never interrupted. You would need, I believe, a communication channel separate of the Qt event loop, and adapted to realtime, which is why I mentioned about the FIFO queue.

(it's a kind of reengineering I made when I adapted our BankEditor program for MIDI and low-latency)

[OPNA2608]

I didn't mean to imply that it's not related to the sound generation and buffering, merely that, on macOS, this seems to affect and is affected by more/different factors than on Linux.

On Linux, I need to use a buffer of ~<3ms to get a usable pattern editor rendering rate, and nothing else will impact how fast or slow the pattern editor and the oscilloscope renders, but actual playback is flawless.

On macOS, every buffer size setting causes severe rendering and playback lag (you can barely call it "playback" at that point, more like a seemingly never-ending struggle 😆), but lowering the window size makes the rendering and playback butter-smooth again.

I'm sure it's all symptoms emerging from the same problem, but the ways of triggering these appear to be platform-specific.

[jpcima]

It's as I just told, the symptom could be more or less pronounced by platform; when you'll be dependent on a black-box GUI-related code to terminate in due time, so audio can take over, there is not a timing guarantee.

The absolute way that you solve the problem, is when audio processing and GUI processing are fully independent. Then the window size, or how much of a time slice it takes to draw a graphics part with CoreGraphics or whatever that is, can not have impact on the audio process.

Right now, the behavior lets me observe that GUI and audio are co-dependent.
It's needed to examine and determine the reason, for now I haven't really researched.

[OPNA2608]

With the release of v0.2.0 and the mention of the Travis CI tests for macOS, I was reminded that Travis CI can upload its build result to Github Releases if the commit it's building is tagged, but this needs an encrypted OAuth API key in the .travis.yml file by @rerrahkr and some extra steps after testing the compilation to properly deploy it.
(macdeployqt BambooTracker.app -verbose=2, with -dmg to make an installable application image I think)

Untested, but I think this yaml snippet could work for setting up the uploading part, minus the token:

deploy:
  provider: releases
  api_key: "GITHUB OAUTH TOKEN"
  file: "BambooTracker.app"
  skip_cleanup: true
  on:
    repo: rerrahkr/BambooTracker
    tags: true
    condition: $TRAVIS_OS_NAME = osx

I'll test uploading and what to do to get a runnable application image deployed on a fork when I can access that macOS box again.

[rerrahkr]

I changed .travis.yml to deploy the zip file at macos branch, confirmed it works well on another repository by changing OAuth key. Its deployment section is commented out because current binary has some problems.

Since I can run macOS 10.13.6 from time to time, I tried to build the tracker. I also confirmed the problems @OPNA2608 mentioned:

1. Starting the application produces a very loud noise.
2. Playback and the pattern following (alike to Stuttery drawing of Pattern Editor #49) is very stuttery, no matter the buffer size.
3. The bottom of the operators/LFO/arps/… editor window is too large on my display (1280x800). No settings appears to be cut off by the launchbar so it's merely a minor problem I guess.

1. It was the reason that the audio buffer was not zero-initialized. I fixed this at a370e40.
2. Like @jpcima said, we need to separate thread into audio and GUI.
3. Although I changed FM editor scrollable at 5971057, it is not reflected on mac 😕, so I have to fix the gui again.

[bmosley]

awesome! +1 for macOS support. long ago i remember building it and running fine in macOS but that was pre-Qt days i think. lots have change (and for the better!) but can't wait for macOS support!

[nyanpasu64]

I am starting to develop my own tracker, and was discussing with the OpenMPT team about how to prevent audio stuttering. https://forum.openmpt.org/index.php?topic=6211.0 The posts by their developers may be useful here. I don't know if BambooTracker still relies on the GUI event loop to generate or playback audio (which is probably a bad idea).

[rerrahkr]

In #152, the audio task is separated from Qt main event loop by using RtAudio. Now sample generation no longer interferes with GUI drawing.

And as I mentioned, there is another problem about drawing: paint event in some views (especially the pattern editor) take too cost to update its appearances. I checked costs of QPainter method used drawing the pattern editor, and found that QPainter::drawText was the heaviest.

After studying the source of 0CC-FamiTracker, I realized that it was not necessary to draw the entire area of pattern editor each time the paint event was called. For example, the hover cursor is changed, it is possible to update its appearance to stay characters and redraw only the background of the pattern cells.
Therefore I try to speed up drawing by reusing some parts depending on the situation.

Another solution is to make redrawing periodically. BambooTracker calls paint event whenever an event such as moving cursor happens, and there is a problem of a large number of calls due to playback following and cursor hovering.
The number of updates is simply reduced by redrawing every 50 Hz or 60 Hz and skipping if not necessary.

[rerrahkr]

After merging #155, I modified .travis.yml at 845986d to upload the binary to the release page. I tested it on another repository, so it will work fine.

Since serious problems were solved, I will release BambooTracker for macos from the next version.

[rerrahkr]

As you know, since v0.3.0, BambooTracker has supported macOS. Thanks for all help! 😄