UDF from floats to symbol

[andreabedini]

I have tried to implement a user-defined functor which takes two numbers and returns a symbol. AFAIU, it needs to be a stateful functor to be able to access the symbol table. I tried this:

#include <souffle/RamTypes.h>
#include <souffle/SouffleInterface.h>
#include <sstream>

extern "C" {
souffle::RamDomain test(souffle::SymbolTable *symbolTable,
                        souffle::RecordTable *recordTable,
                        souffle::RamFloat x) {

  assert(symbolTable && "NULL symbol table");
  assert(recordTable && "NULL record table");

  std::ostringstream buf;
  buf << x;

  return symbolTable->encode(buf.str());
}
}

and

.functor test(x:float):symbol stateful

.decl In(x: float)
In(0.5).
In(1.0).
In(1.5).

.decl Out(x:symbol)
.output Out

Out(@test(x)) :- In(x).

but this results in a segfault.

$ souffle test.dl 
Segmentation violation signal in rule:
Out(@test(x)) :- 
   In(x).
in file /home/andrea/work/amt-souffle/test_functor/test.dl [11:1-11:24]

What is the correct way to do this?

EDIT: updated code to a complete example

[b-scholz]

In the interface-tests you have some examples. Can you start from there? That might be easier. Have a look here:

souffle/tests/interface/functors/functors.cpp

Line 82 in c2f1665

souffle::RamDomain mycat(souffle::SymbolTable* symbolTable,

[andreabedini]

Hi @b-scholz, thanks for having a look. Unfortuntately, the examples in tests/interface/functors don't seem to include my usecase.

The functor .functor ioo(number):symbol returns a char * but the string I am returning is dynamically allocated so I cannot return a char * without leaking that pointer later (or making further assumptions on how the function is called).

Of the stateful functors (which have access to the symbol table) none take a native argument. It's either a symbol or an ADT.

[b-scholz]

Can you run gdb / lldb and check where it is crashing?

[andreabedini]

#0  souffle::ConcurrentFlyweight<souffle::SeqConcurrentLanes, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, souffle::details::Factory<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::findOrInsert<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&> (this=0x555555de09c0, H=0)
    at /usr/local/include/souffle/datastructure/ConcurrentFlyweight.h:300
#1  0x00007ffff7fc2fb5 in souffle::SeqFlyweight<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, souffle::details::Factory<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::findOrInsert<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&> (this=0x555555de09c0)
    at /usr/local/include/souffle/datastructure/ConcurrentFlyweight.h:470
#2  0x00007ffff7fc2c8d in souffle::SymbolTable::encode (this=0x555555de09c0, symbol="-nan")
    at /usr/local/include/souffle/SymbolTable.h:92
#3  0x00007ffff7fc2417 in test (symbolTable=0x555555de09c0, recordTable=0x555555de08c0, x=-nan(0x7fa670))

See full backtrace at https://gist.github.com/andreabedini/2b06002a200489820cbcacd773f0175f

Note that the cpp function test gets passed an invalid float (nan), this alone doesn't cause the crash but it's a signal things already went wrong somewhere.

[b-scholz]

Can you compile Souffle in debug mode and run valgrind?

[andreabedini]

https://gist.github.com/andreabedini/c09e23c1175e6e4853c2546b88745191

by the way, looking at the code it looks like we are hardcoding FFI_RamSigned as argument type, am I misreading the code?

souffle/src/interpreter/Engine.cpp

Lines 681 to 685 in baacf15

	for (std::size_t i = 0; i < arity; i++) {
	intVal[i] = execute(shadow.getChild(i), ctxt);
	args[i + 2] = &FFI_RamSigned;
	values[i + 2] = &intVal[i];
	}

[b-scholz]

Did you compile with 32bit or 64bit RAM_DOMAIN_SIZE? If there is a mismatch between the dynamic library and souffle, it will lead to a crash.

My interpretation (but I don't want to pre-empt it) is that the argument-passing fails and an exception is thrown because the floating-point number is not passed properly.

[andreabedini]

There might be multiple effects at play, I managed to trigger the segvfault without floats, see #2069 can you reproduce it?

[b-scholz]

Note the passing is always done as RamSigned (also stored internally as such). Perhaps you use RamSigned for x and do a bitcast for the floating point number. There are macros for the conversion.

[andreabedini]

I will try to use RamSigned and bitcast it. Thanks!

[b-scholz]

The functor examples in tests/interface are doing the argument passing correctly. Perhaps you run them outside the test suite first and see whether you can get them to work outside the test suite.

If it works, construct an identity function for floats. See what will happen...

[andreabedini]

as discussed in #2069 the sample code above is supposed to work and does work if functors.cpp is compiled with -fopenmp.

[b-scholz]

Great - can you get your functor to work as well?

[andreabedini]

hi @b-scholz, alas, no. I have a work around (using SQLite as a data preparation layer) and a more important (and interesting) blocker. I will file separate issues about the functor and the blocker. Thanks.

[b-scholz]

Have you checked memory issues with valgrind?