How to convert waterfall saved in .npy (kiwiwfrecorder) to image ?

[kovalroma]

So I have some data in .npy recorded with kiwiwfrecorder.py
How convert it to image file ?
Any tools or python code or example ?

Thanks in advanced,

[hcab14]

The relevant lines are

kiwiclient/kiwiwfrecorder.py

Lines 106 to 120 in 4f9490e

	if self._wf_data['seq'] in self._store:
	now = time.gmtime()
	if self._start_ts is None:
	self._start_ts = now
	with open(self._get_output_filename(), 'wb') as f:
	np.save(f, self._wf_data['freq_bins'])
	## GNSS timestamp for seq obtained from the SND WS stream
	ts = self._store.pop(self._wf_data['seq'])
	logging.info('found seq %d %f %d (%d|%d,%d)'
	% (self._wf_data['seq'], ts, len(self._wf_data['wf_samples']),
	len(self._store), self._wf_queue.qsize(), self._snd_queue.qsize()))
	with open(self._get_output_filename(), 'ab') as f:
	np.save(f, np.array((ts, self._wf_data['wf_samples']),
	dtype=[('ts', np.float64), ('wf', ('B', 1024))]))

The .npy files contains:

• once at the start: 1024 frequencies: one for each bin
• for each WF line: a pair of timestamp and the WF data: dtype=[('ts', np.float64), ('wf', ('B', 1024))]

[fucksophie]

Hey! How do we decode the WF data into a simple waterfall? Very confused here, need to figure out what's actually in the WF data. Doesn't seem to be documented anywhere either.

[jks-prv]

There is no kiwiclient code to directly convert waterfall data streamed from a Kiwi to an image file. Note that you can do this with the Kiwi web interface itself: "save waterfall as jpg" in the right-click menu. But I imagine you're probably interested in automating the process using kiwiclient.

Most of the pieces of Python code to do it are already in the kiwiclient package. Someone just needs to put it together.
It probably need to go in kiwirecorder.py, not kiwiwfrecorder.py (there is already a comment in kiwiwfrecorder that it needs to be merged with kiwirecorder).

There are three issues with generating waterfall image files:

• Understand the waterfall data format sent from the Kiwi.
• Apply a colormap to this data to convert dBm to [r,g,b] values.
• Save as an incrementally appending image file (as is done with the audio wav files).

None of the Kiwi API is formally documented. But the kiwi/client.py and kiwirecorder.py code shows how most of it is dealt with. The KiwiWaterfallRecorder:_process_waterfall_samples() routine, which just prints WF statistics now, needs to be modified to handle image generation. The samples parameter contains unsigned byte [0,255] values. nbins is always 1024. You convert to dBm by taking 255 - value so the values [0,255] correspond to [-255,0] dBm.

Colormap mapping is a complicated topic. The Kiwi code has all sorts of different schemes. On top of that are the various waterfall averaging options. All of this needs to be considered because what kiwirecorder sees is the raw WF data with none of this processing. The Kiwi code does all WF processing on the client-side in Javascript. So you have to decide how much of that you want to replicate. At the very least you need to copy one of the static colormaps from the openwebrx.js file in the Kiwi code and apply max/min dBm values that the colormap will be stretched to cover.

The kiwifax.py and png.py code shows how to construct a png image file from [r,g,b] data.

[jks-prv]

Fixed, finally!

Have a look at the new kiwirecorder waterfall options: --wf-png --wf-auto --mindb --maxdb (and the old --z --speed)
Also look at the example targets in the Makefile: snd-wf drm-wf drm-wf-auto