Building on iOS/ARM mobile platforms

[loretoparisi]

Any project to run the whole library on mobile iOS ?

FFWT
TagLib
LibYAML

should be compilable on iOS easily.

An idea could be to make a lightweight release of the library without the Python wrappers.

[wackou]

Essentia compiles just fine as a C++ library, the python bindings are optional (as is TagLib, btw), so this should be no problem.

As for compiling on iOS, it hasn't been tried yet, but the code is standard C++ so there should be no major issue. Any input on the subject is welcome!

[loretoparisi]

I was able to build against armv7 / iOS6.0 - iOS 7.0 stripping down dependencies like

• TagLib (even if there is a taglib.Framework that compile agains armv7/iOS5.0)
• libavcodec / FFmpeg and the io/ read & write classes
• Gaia
• Yaml
• libsamplerate

Now the issue is related to the TNT library and it is described here:

http://stackoverflow.com/questions/19506719/xcode-expected-unqualified-id-tnt-template-numerical-toolkit

[loretoparisi]

I added the Essentia iOS framework here:

https://github.com/loretoparisi/EssentiaTouch

The project need the XCode Universal Framework to build:

https://github.com/kstenerud/iOS-Universal-Framework

So far the framework builds, but it is far to be complete, since too many dependencies are missing.

[loretoparisi]

I was able to compile and run the library on armv7 / armv7s:

A algorithm run is like

void Melody::calculatePredominantMelody(float* _audio, float* output) {

/////// PARAMS //////////////
// don't change these default values as they guarantee that pitch extractor output
// is correct, no tests were done on other values
int framesize = 1024;
int hopsize = 128;
int sr = 44100;

// instantiate algorithms
essentia::standard::AlgorithmFactory& factory = essentia::standard::AlgorithmFactory::instance();


essentia::standard::Algorithm* equalLoudness = factory.create("EqualLoudness");

essentia::standard::Algorithm* predominantMelody = factory.create("PredominantMelody",
                                                                  "frameSize", framesize,
                                                                  "hopSize", hopsize,
                                                                  "sampleRate", sr);

// data storage
//essentia::Pool pool;
std::vector<essentia::Real> audio = ( std::vector<essentia::Real>)*_audio;

/// TODO: here we have to fill the real audio vector

equalLoudness->input("signal").set(audio);

std::vector<essentia::Real> audioEq;
equalLoudness->output("signal").set(audioEq);
predominantMelody->input("signal").set(audioEq);

std::vector<essentia::Real> pitches;
std::vector<essentia::Real> pitchConfidences;

predominantMelody->output("pitch").set(pitches);
predominantMelody->output("pitchConfidence").set(pitchConfidences);

//pool.add("tonal.predominant_melody.pitch",pitches);
//pool.add("tonal.predominant_melody.pitch_confidence",pitchConfidences);

predominantMelody->compute();

printf(".");

float tm = 0.0;
for (int i = 0; i<pitches.size(); i++)
{
    tm = tm + float(hopsize)/float(sr);
    printf("\n%f %f %f",pitches[i], pitchConfidences[i], tm);
    output[i]=pitches[i];
}

/*
const std::vector<essentia::Real>& pitchesR = pool.value<std::vector<essentia::Real> >("tonal.predominant_melody.pitch");

const std::vector<essentia::Real>& confidenceR = pool.value<std::vector<essentia::Real> >("tonal.predominant_melody.pitch_confidence");

printf("number of frames: %lu", pool.value<std::vector<essentia::Real> >("tonal.predominant_melody.pitch").size() );

float tm = 0.0;
for (int i = 0; i<pitchesR.size(); i++)
{
    tm = tm + float(hopsize)/float(sr);
    printf("%f %f %f",pitchesR[i], confidenceR[i], tm);
    output[i]=pitchesR[i];
}*/

// clean up
delete predominantMelody;
delete equalLoudness;

}

So far the result is the the arrays pitchesR and confidenceR have no elements.
Are there errors in the procedure ?

[dbogdanov]

You are adding empty vectors to pool before they were actually updated by compute() method

//pool.add("tonal.predominant_melody.pitch",pitches);
//pool.add("tonal.predominant_melody.pitch_confidence",pitchConfidences);

predominantMelody->compute();

[loretoparisi]

That makes sense. Is there any way to do not use the pool?
What the pool does internally. Is there any accumulator logic?
Thanks

Il giorno martedì 19 novembre 2013, Dmitry Bogdanov ha scritto:

You are adding empty vectors to pool before they were actually updated by
compute() method

//pool.add("tonal.predominant_melody.pitch",pitches);
//pool.add("tonal.predominant_melody.pitch_confidence",pitchConfidences);

predominantMelody->compute();

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/11#issuecomment-28819162
.

Dott. Ing. Loreto Parisi
Parisi Labs

Email: loretoparisi@gmail.com
Blog: http://blog.parisilabs.com
Linkedin: http://www.linkedin.com/in/loretoparisi

[loretoparisi]

I tried to add de commenting lines

pool.add("tonal.predominant_melody.pitch",pitches);
pool.add("tonal.predominant_melody.pitch_confidence",pitchConfidences);

but I have an error

libc++abi.dylib: terminating with uncaught exception of type essentia::EssentiaException: Descriptor name 'tonal.predominant_melody.pitch' of type std::vector not found

The code modified was

   void Melody::calculatePredominantMelody(float* _audio, float* output) {



/////// PARAMS //////////////
// don't change these default values as they guarantee that pitch extractor output
// is correct, no tests were done on other values
int framesize = 2048;
int hopsize = 128;
int sr = 44100;

// instantiate algorithms
essentia::standard::AlgorithmFactory& factory = essentia::standard::AlgorithmFactory::instance();


essentia::standard::Algorithm* equalLoudness = factory.create("EqualLoudness");

essentia::standard::Algorithm* predominantMelody = factory.create("PredominantMelody",
                                                                  "frameSize", framesize,
                                                                  "hopSize", hopsize,
                                                                  "sampleRate", sr);

// data storage
essentia::Pool pool;

std::vector<essentia::Real> audio = ( std::vector<essentia::Real>)*_audio;

/// TODO: here we have to fill the real audio vector

equalLoudness->input("signal").set(audio);

std::vector<essentia::Real> audioEq;
equalLoudness->output("signal").set(audioEq);
predominantMelody->input("signal").set(audioEq);

std::vector<essentia::Real> pitches;
std::vector<essentia::Real> pitchConfidences;

predominantMelody->output("pitch").set(pitches);
predominantMelody->output("pitchConfidence").set(pitchConfidences);

pool.add("tonal.predominant_melody.pitch",pitches);
pool.add("tonal.predominant_melody.pitch_confidence",pitchConfidences);

predominantMelody->compute();

const std::vector<essentia::Real>& pitchesR = pool.value<std::vector<essentia::Real> >("tonal.predominant_melody.pitch");

const std::vector<essentia::Real>& confidenceR = pool.value<std::vector<essentia::Real> >("tonal.predominant_melody.pitch_confidence");

printf("number of frames: %lu", pool.value<std::vector<essentia::Real> >("tonal.predominant_melody.pitch").size() );

float tm = 0.0;
for (int i = 0; i<pitchesR.size(); i++)
{
    tm = tm + float(hopsize)/float(sr);
    printf("%f %f %f",pitchesR[i], confidenceR[i], tm);
    output[i]=pitchesR[i];
}

// clean up
delete predominantMelody;
delete equalLoudness;

}

[dbogdanov]

again, you are adding values to pool before computing them.

predominantMelody->output("pitch").set(pitches);
predominantMelody->output("pitchConfidence").set(pitchConfidences);

pool.add("tonal.predominant_melody.pitch",pitches);
pool.add("tonal.predominant_melody.pitch_confidence",pitchConfidences);

predominantMelody->compute();

should be

predominantMelody->compute();

pool.add("tonal.predominant_melody.pitch",pitches);
pool.add("tonal.predominant_melody.pitch_confidence",pitchConfidences);

[dbogdanov]

The advantage of pool is that it can be used to store different variables of different types just like a dictionary in Python. However, there is little sense to use it in the standard mode in the case if you simply want to compute one output of an algorithm. Your original code without pool was ok for that.