[audio_to_spectrogram] Add AudioAmplitudePlot node to visualize audio amplitude

[iory]

What is this?

This PR adds a script to publish audio amplitude plot image.
The following video shows the audio amplitude visualized at this node while clapping.
This PR is based on #2654 .

audio_amplitude_plot--SLASH--output--SLASH--viz.mp4

[708yamaguchi]

Thank you very much for the nice plotter and documentation!

This is because my code was ad-hoc, but can you make some of the code common?

[708yamaguchi]

These lines are the settings to convert audio stream and used in audio_to_spectrum.py, too.

jsk_recognition/audio_to_spectrogram/scripts/audio_to_spectrum.py

Lines 18 to 30 in 6ff031d

	# Audio topic config
	# The number of channels in audio data
	self.n_channel = rospy.get_param('~n_channel', 1)
	# Sampling rate of microphone (namely audio topic).
	mic_sampling_rate = rospy.get_param('~mic_sampling_rate', 16000)
	# Period[s] to sample audio data for one fft
	fft_sampling_period = rospy.get_param('~fft_sampling_period', 0.3)
	# Bits per one audio data
	bitdepth = rospy.get_param('~bitdepth', 16)
	if bitdepth == 16:
	self.dtype = 'int16'
	else:
	rospy.logerr("'~bitdepth' {} is unsupported.".format(bitdepth))

I think these codes should be shared.

Could you create python class like AudioBuffer, which receives audio_common_msgs/AudioData and create self.audio_buffer buffer.

[iory]

Thanks. Created AudioBuffer class.

[708yamaguchi]

As I mentioned in another comment, could you create python class like AudioBuffer?

jsk_recognition/audio_to_spectrogram/scripts/audio_to_spectrum.py

Lines 64 to 74 in 6ff031d

	def audio_cb(self, msg):
	# Convert audio buffer to int array
	data = msg.data
	audio_buffer = np.frombuffer(data, dtype=self.dtype)
	# Retreive one channel data
	audio_buffer = audio_buffer[0::self.n_channel]
	# Save audio msg to audio_buffer
	self.audio_buffer = np.append(
	self.audio_buffer, audio_buffer)
	self.audio_buffer = self.audio_buffer[
	-self.audio_buffer_len:]

[iory]

Thanks. Created AudioBuffer class.

[708yamaguchi]

Could you use ConnectionBasedTransport?

In addition, if possible, could you create python class like AudioPlot, which configures matplotlib and can be used in both audio_amplitude_plot.py and spectrum_plot.py

[k-okada]

why did you create custom plot node instead of existing plotting tools (rqt / plotjuggler)

[iory]

@k-okada

how you created custom plot node instead of existing plotting tools (rqt / plotjuggler)

I'm sorry I don't understand well.
Do you suggest that we should make a plugin such as rqt_plot?

[k-okada]

@iory I want to know the reason why you created the nose just to visualize audio amplitude

[iory]

Since sound is invisible, it is nice for the user to have it visualized.
The sample rate, number of channels, and other parameters are given manually by the user.
Sometimes, users give these values wrong.
The simplicity of this node, which visualizes the amplitude of sound, makes it easy to notice these mistakes.
In other words, it is useful for debugging purposes.

[k-okada]

why you can not use rqt? or other plotting tools?

[knorth55]

FYI:
we have audioinfo message to pass meta data of audio data.
ros-drivers/audio_common#152

[iory]

OK. I'll take another way.

[iory]

@k-okada

why you can not use rqt? or other plotting tools?

I'm sorry, I'd like to ask you one point.
How do you think visualizing audio_common_msgs/AudioData is a good way to do it?
Since audio_common_msgs/AudioData is array data, I think it is necessary to insert another node in order to visualize it with rqt_plot.

$ rosmsg show audio_common_msgs/AudioData
uint8[] data

Also, when it comes out as image data, we are happy to be able to correspond with another image because it has a timestamp.

[k-okada]

Since audio_common_msgs/AudioData is array data, I think it is necessary to insert another node in order to visualize it with rqt_plot.

@iory I see, LGTM

[k-okada]

@iory merged in #2755 , but I have manually fixed conflict so it might contain wrong code, please check that.