extract audio via url

[developer-farhan]

is it possible to extract audio and generate wave from that url I want to use it in chatting app

[pasrot943]

I am looking for the url function as well ?

[ryanheise]

For now, you can download the URL to a file and then use the file with just_waveform.

[loic-hamdi]

For now, you can download the URL to a file and then use the file with just_waveform.

is this still the way to do today?

[ryanheise]

Yes it is.

[ryanheise]

Note you can also check the changelog to see if there have been updates.

[andynvt]

Any updates?

[ryanheise]

Just to update everyone, the current way is still do download the file first and then pass the file into the API.

There is no urgency to do it another way at the moment, and usually it is better to download the file first because if you lose the connection half way through the download, it won't cause problems for the decoder.

If anyone has a use case for which this won't work, please describe it below, otherwise I advise downloading the file first.

[ahetawal-p]

One use case which we have is, we have multiple audio clips on a given page. And user can click and play any one of them. Would ideally not want to download the file for each clip and play as it adds latency on our UI plus maintaining the file system sizing if we always download a file for playing the waveform, while the actual player can just work from a url instead of downloading the file every time.
From flutter_sound or just_audio compatibility perspective this plugin supporting a url would be super helpful.

[Faaatman]

One use case which we have is, we have multiple audio clips on a given page. And user can click and play any one of them. Would ideally not want to download the file for each clip and play as it adds latency on our UI plus maintaining the file system sizing if we always download a file for playing the waveform, while the actual player can just work from a url instead of downloading the file every time. From flutter_sound or just_audio compatibility perspective this plugin supporting a url would be super helpful.

I second this and would love an integration at least with just_audio.

[ryanheise]

I do not see how that use case won't work, and how extracting audio via a URL will help your use case at all. Decoding directly from a URL doesn't mean you avoid downloading, because you must in all cases download something to read the data and decode it. All decoding directly from a URL really means is that you are downloading "simultaneously" while decoding. The downloading is still there. The only difference here is whether you do those two steps in sequence, or simultaneously. You can and should of course delete downloaded data once you've extracted whatever you needed to extract from it, and you should of course cache the extracted data since it was very expensive to extract in the first place.

But by far the biggest latency is the actual decoding, not the downloading, and so with the feature you're asking for, you still must wait for the decoding to finish before you can display the waveform, and the decoding takes at least 10x longer than the downloading. You are not going to help your latency problem this way. In all cases, you must download and decode.

If you want something that eliminates latency, then what you should be asking for is streaming the decoded waveform data while it's processing, rather than having to wait until the entire job is done.

[loic-hamdi]

Hi @ryanheise,
would it be possible to extract the waveform as String of Json in order to store it and then use it to display the waveform instead of downloading + decoding the file?
It would allow to go through the download + decoding process only once.

[ahetawal-p]

@ryanheise

All decoding directly from a URL really means is that you are downloading "simultaneously" while decoding. The downloading is still there. The only difference here is whether you do those two steps in sequence, or simultaneously

Yes that's what I am looking for the simulatneous part. This no different than how a image widget works, as it downloads and then decodes each image when it's shown.

[ryanheise]

@zzterrozz

would it be possible to extract the waveform as String of Json in order to store it and then use it to display the waveform instead of downloading + decoding the file?

It should be clear from the plugin's API because there are only two methods in the API. The first:

final progressStream = JustWaveform.extract(
  audioInFile: '/path/to/audio.mp3',
  waveOutFile: '/path/to/waveform.wave',
  zoom: const WaveformZoom.pixelsPerSecond(100),
);

The waveform is extracted from the given input file (e.g. MP3) and placed into the output file waveOutFile. In other words, the extracted waveform is "stored" in the given output file. You are basically asking if the waveOutFile parameter exists. Yes it exists. You are also asking me to use a JSON format instead, but the current format is much more efficient than the JSON format you are proposing because it's binary and compact. The format I'm currently using is called "audiowaveform" which is a binary format invented by the BBC for this purpose. I have mentioned this format in the plugin's documentation, please follow the link in the documentation to read more about this format.

Once you have outputted a waveform file in this format, you can read it into your program using the only other method in that class:

final waveform = await JustWaveform.parse(File('/path/to/waveform.wave');

So you use the first method only the first time when you haven't previously decoded the file. Once you've done that, however, you can keep the output file and next time just load this existing file using the second method.

[ryanheise]

@ahetawal-p maybe you can make a separate feature request for that because they are distinct features that can be implemented separately.

[ryanheise]

@ahetawal-p actually on rereading your last comment, you are indeed requesting the feature described in this issue, so not a separate issue.

But my question remains as to why this actually solves your problem. You do realise that this will do almost nothing to address latency? Can you please provide me with a benchmark in your app measuring how much time is spent downloading the file vs how much time is spent extracting the waveform?

[loic-hamdi]

@zzterrozz

would it be possible to extract the waveform as String of Json in order to store it and then use it to display the waveform instead of downloading + decoding the file?

It should be clear from the plugin's API because there are only two methods in the API. The first:

final progressStream = JustWaveform.extract(
  audioInFile: '/path/to/audio.mp3',
  waveOutFile: '/path/to/waveform.wave',
  zoom: const WaveformZoom.pixelsPerSecond(100),
);

The waveform is extracted from the given input file (e.g. MP3) and placed into the output file waveOutFile. In other words, the extracted waveform is "stored" in the given output file. You are basically asking if the waveOutFile parameter exists. Yes it exists. You are also asking me to use a JSON format instead, but the current format is much more efficient than the JSON format you are proposing because it's binary and compact. The format I'm currently using is called "audiowaveform" which is a binary format invented by the BBC for this purpose. I have mentioned this format in the plugin's documentation, please follow the link in the documentation to read more about this format.

Once you have outputted a waveform file in this format, you can read it into your program using the only other method in that class:

final waveform = await JustWaveform.parse(File('/path/to/waveform.wave');

So you use the first method only the first time when you haven't previously decoded the file. Once you've done that, however, you can keep the output file and next time just load this existing file using the second method.

Nice, I didn't realize that!
Is there a async future to extract the waveform instead of the stream in order to await the extraction and then upload the file?

[ryanheise]

@zzterrozz I can't keep answering questions of the form: Does the API have X or Y? These types of questions can be found by studying the API. There are literally only two API methods, and I was trying to make it clear in my previous response that you really should have been able to fully see everything that is int his plugin by looking at the API reference, in almost no time. It takes me much longer to write these responses to you than it would take for you to actually read the documentation, and so it can't be that difficult to take a look and see what is there and what isn't there.

Since a stream does more than a future, it is not a concern of the plugin. You are advised to ask someone on StackOverflow if there is an easy way to get a future from a stream, or else, read the Dart documentation to find the answer yourself.

[ahetawal-p]

Hey @ryanheise I don't think its about latency as much as it's about the usage of this package. All other sound related packages support urls, so it was just less work as a consumer to not worry about downloading the file first and then passing it down to the plugin. Instead let the plugin do it's work with the given url.
I do understand the technical limitation with the decoding latency while streaming ...as you mentioned

[ryanheise]

@ahetawal-p I'm sorry if there was any confusion in my question, but it was this:

If anyone has a use case for which this won't work, please describe it below, otherwise I advise downloading the file first.

It sounds like you'd be satisfied with a convenience method so that you don't have to write an extra line of Dart code in your app. As such, it's not that the current plugin does not work, and the priority of this issue remains as before. For now, I advise adding 1 extra line of code to your app. I am not in a rush to add this to the API because API decisions should be carefully thought out so as to minimise breaking changes in the future.

[Faaatman]

@ahetawal-p
The workaround that I will be using is as follows: since I'm using just_audio I will get the remote files using LockCachingAudioSource and specify the cached file location, wait for the audio player to download the file, and then use just_waveform to get the waveform.

I will get back to you once I implement it and test it.

[jpolstre]

Is there a way to get the data from the service and not from the files. The data is already there it is the sound that is being emitted, why get it again.

I need it for live broadcasts.

[ryanheise]

@jpolstre the only way to prevent a double download is to tap into the decoder of whatever player you're using. just_waveform will never be able to do that since it is a standalone plugin, not integrated with any player. For your specific use case, the closest you can get currently would be to use the visualizer branch of just_audio. It is not the goal of just_waveform.

[jpolstre]

@jpolstre the only way to prevent a double download is to tap into the decoder of whatever player you're using. just_waveform will never be able to do that since it is a standalone plugin, not integrated with any player. For your specific use case, the closest you can get currently would be to use the visualizer branch of just_audio. It is not the goal of just_waveform.

Thank you for your reply, do you plan to implement it also for just_audio_background?

[ryanheise]

Eventually, but not in any reasonable timespan since the visualizer branch is experimental. If you need it urgently, you would need to use audio_service directly. In any case, this is not an issue that pertains to just_waveform.

[Amr-Samy]

@Faaatman that's exactly what came to my mind , I will try to figure it out , please if you already found it share the code gratefully

[Amr-Samy]

I got the file and path but still facing problems with the stream as it cannot be a future or null
File file = await audioSource.cacheFile;
print("${file.path}");

[Faaatman]

@Amr-Samy What I did is as follows: I get the file first and once it's downloaded I get the waveform and listen to it's progress in a stream.

 var directory = await getTemporaryDirectory();
      audioFile = File(directory.path + audioFileURI);
      await audioPlayer.setAudioSource(LockCachingAudioSource(
          Uri.parse(audioFileURI),
          cacheFile: audioFile));

          try {
            var source = (audioPlayer.audioSource as LockCachingAudioSource); 
            source.downloadProgressStream
                  .listen((event) {
                  // when download finishes I start getting the waveform
                    if (event == 1) {
                      extractWaveform(WaveformData(_audioFile!.path, _audioFile!))
                    }
                  });
          } catch (e) {
            emit(AudioPlayerError());
          } 

The extract function is:

Future<WaveformData<Waveform>> extractWaveform(
    WaveformData<File> inputData) async {

  var waveFile = File(inputData.file.path);

  /// if the file exists don't extract the waveform again and just use the old
  /// file if this fails then re-extract it
  if (await waveFile.exists()) {
    try {
      Waveform waveform = await JustWaveform.parse(waveFile);
      return WaveformData(inputData.key, waveform);
    } catch (_) {}
  }
  WaveformProgress waveformProgress = await _getWaveform(
    JustWaveform.extract(
      audioInFile: inputData.file,
      waveOutFile: waveFile,
    ),
  );
  return WaveformData<Waveform>(inputData.key, waveformProgress.waveform!);
}


Future<WaveformProgress> _getWaveform(Stream<WaveformProgress> waveform) {
  waveform.listen((event) {}, onError: (e) {
    throw (e);
  });
  return waveform.firstWhere(
      (element) => (element.progress == 1 && element.waveform != null));
}

Just to clarify. I am using Bloc and service locator to handle multiple players and their states. I tried to simplify my solution and put it in a couple of functions.
Sadly I am no longer working on this project nor on Flutter. I hope this helps you. Best of luck!

edit: added clarification.

[lucasjinreal]

This just make thing more complicated. it would be better if just_waveform can parse from uri automatically.

However, I didn't depend on author to do it, now I forked it can it works perfect.

[ryanheise]

Just to quote my earlier comment:

usually it is better to download the file first because if you lose the connection half way through the download, it won't cause problems for the decoder.

Downloading a file can be done in a single line of code using http or HttpClient or dio. For example:

audioInFile.writeAsBytes((await http.get(uri)).bodyBytes);

For people who want a quick solution, you may copy and paste the above one liner.

You are then free to expand on this to deal with connectivity issues during download, resume incomplete or partial downloads and so on so that they fit into the lifecycle of your app.

I am leaving this issue open since I think it could (maybe) be done internally in the future after some design considerations, but as you can see, there are many things to consider when designing the correct API that will work in various edge cases, whereas it is already possible today for apps to download the URL in a single line of code as given above, and since that's under your control, you get the flexibility to handle different use cases such as how and when to resume interrupted downloads. I will probably look at this again after first implementing the streaming API to return partial waveforms while the audio is being processed.

[lucasjinreal]

@ryanheise thanks for your far-sightedness thoughts and this great lib. You were right, please consider it more mature.

I am using random waveform data to show now.

It would be better can have a connection example to show waveform data with flutter_waveforms lib for visualization