openml integration in shogun #4628

gf712 · 2019-05-03T13:50:50Z

adds OpenML natively to shogun.

TODO:

C++ specific
write GET
write POST
write OpenML flow class
write OpenML task
Shogun specific
write OpenML to SGObject
write SGObject to OpenML
expose OpenML to all interfaces
write shogun extension for runs with flows
Maintenance stuff
figure out how to split all the classes, i.e. files and directories in shogun

karlnapf · 2019-05-05T09:14:51Z

src/shogun/io/OpenmlFlow.cpp

+	return 0;
+}
+
+const char* OpenMLReader::xml_server = "https://www.openml.org/api/v1/xml";


should these things be hard-coded here?

karlnapf · 2019-05-05T09:15:28Z

src/shogun/io/OpenmlFlow.cpp

+
+	if (!curl_handle)
+	{
+		SG_SERROR("Failed to initialise curl handle.")


maybe some infos on what happened?

I am not sure what the exact error would be... From the docs: "If this function [curl_easy_init] returns NULL, something went wrong and you cannot use the other curl functions."

Any value in using this other global init instead?
https://stackoverflow.com/questions/50987505/how-to-get-error-reason-when-curl-easy-init-fails

ah thanks, yes, that seems helpful!

karlnapf · 2019-05-05T09:16:20Z

sweet, looking forward to see this once it is working :)

gf712 · 2019-05-07T13:12:04Z

@vigsterkr I copied a few things from deadbeef to add RapidJSON as a dependency

vigsterkr · 2019-05-07T13:14:29Z

saw it 😼

gf712 · 2019-05-07T13:23:16Z

src/shogun/CMakeLists.txt

+        CONFIG_FLAG HAVE_XML)
+# RapidJSON
+include(external/RapidJSON)
+SHOGUN_INCLUDE_DIRS(SCOPE PUBLIC ${RAPIDJSON_INCLUDE_DIR})


@vigsterkr is it ok to have RapidJSON as PUBLIC?

i guess it was like this in the other PR, right? i mean currently i include the RapidJSON only in .cpp so private should be fine.... but maybe/probably that caused me troubles with unit tests ...?

it was private in the other PR, but swig needs the RapidJSON header files

~~why?~~ now i understand why :)

vigsterkr · 2019-05-07T13:34:48Z

src/shogun/io/OpenMLFlow.h

+#include <shogun/io/SGIO.h>
+
+#include <curl/curl.h>
+#include <rapidjson/document.h>


i would avoid this by all means :)

vigsterkr · 2019-05-07T13:35:41Z

src/shogun/io/OpenMLFlow.h

+		components_type m_components;
+
+#ifndef SWIG
+		static void check_flow_response(rapidjson::Document& doc);


does it really need to be part of the class, meaning can't you just have this function in the implementation?

no, I guess I can move to free function and avoid exposing RapidJSON to swig :)

vigsterkr · 2019-05-07T13:35:48Z

src/shogun/io/OpenMLFlow.h

+#ifndef SWIG
+		static void check_flow_response(rapidjson::Document& doc);
+
+		static SG_FORCED_INLINE void emplace_string_to_map(


same as above...

vigsterkr · 2019-05-07T13:35:56Z

src/shogun/io/OpenMLFlow.h

+				param_dict.emplace(name, "");
+		}
+
+		static SG_FORCED_INLINE void emplace_string_to_map(


same as above

gf712 · 2019-05-17T09:13:31Z

I have a local version mostly working (with binary classification tasks at least). There are some memory issues but they will be fixed once we switch to smart pointers.

iglesias · 2019-05-21T15:16:24Z

@karlnapf so this works now, a minimal example looks like this:

#include <shogun/base/init.h>
#include <shogun/io/openml/OpenMLFlow.h>
#include <shogun/io/openml/OpenMLTask.h>
#include <shogun/io/openml/OpenMLRun.h>
#include <shogun/io/openml/OpenMLSplit.h>

using namespace shogun;

int main()
{
	init_shogun_with_defaults();
	sg_io->set_loglevel(MSG_GCDEBUG);
	auto flow = OpenMLFlow::download_flow("9602", "");
	auto task = OpenMLTask::get_task("4423", "");
	auto run = OpenMLRun::run_flow_on_task(flow, task);
        // TODO: run.publish()
	exit_shogun();
}

Once the smart pointer PR is merged it'll work with interfaces

What are 9602 and 4423?

gf712 · 2019-05-21T15:20:20Z

What are 9602 and 4423?

Those are the IDs of a flow and a task that I used with the openml-python extension I wrote and i know they work

stale · 2020-08-02T15:00:04Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

karlnapf · 2020-08-06T16:44:22Z

yes, let's keep it :)

initial openml code

e6d5a20

karlnapf reviewed May 5, 2019

View reviewed changes

basic flow interface working with python

7cf1d10

gf712 commented May 7, 2019

View reviewed changes

vigsterkr reviewed May 7, 2019

View reviewed changes

gf712 force-pushed the openml_rest branch from 145b591 to 7c0ab91 Compare May 7, 2019 15:41

moved json dependency to library

b70398d

gf712 force-pushed the openml_rest branch 3 times, most recently from 4a6f93b to 45ac04e Compare May 8, 2019 12:10

initial ShogunOpenML class

0500504

gf712 force-pushed the openml_rest branch 4 times, most recently from df8809d to ea27c0a Compare May 10, 2019 10:08

gf712 added 2 commits May 10, 2019 22:11

started working datasets

bbb493a

started work on splits

4c988bb

gf712 force-pushed the openml_rest branch 2 times, most recently from 394b4bd to 8a98439 Compare May 15, 2019 07:34

gf712 added 2 commits May 15, 2019 09:47

removed curl from header file

859786b

added data getters

beb0acf

gf712 force-pushed the openml_rest branch from 8a98439 to beb0acf Compare May 15, 2019 15:21

fixes for new arff api

b045a8a

gf712 force-pushed the openml_rest branch 2 times, most recently from 7842b61 to 54b2936 Compare May 29, 2019 13:53

added metric evaluation

753f8ba

gf712 force-pushed the openml_rest branch from 54b2936 to 753f8ba Compare May 30, 2019 08:20

gf712 added 17 commits February 3, 2020 21:52

initial openml code

f190940

basic flow interface working with python

9b331a6

moved json dependency to library

21d2a00

initial ShogunOpenML class

134cf2e

started working datasets

07b07d0

started work on splits

3cd470b

removed curl from header file

f1546a6

added data getters

d5ac051

get binary classification working [SKIP CI]

df1a8d6

smart pointer and refcount fixes

c615896

split openml classes in different files

c5c9d93

train and test index getter

f9f9c79

added crossvalidation

37e8c63

some cleanups

f17bbe1

fixes for new arff api

c2cf37b

added metric evaluation

6616ecf

rebase

6d6c254

gf712 force-pushed the openml_rest branch from ada5c42 to 6d6c254 Compare February 4, 2020 14:24

stale bot added the stale label Aug 2, 2020

gf712 added Tag: Development Task and removed stale labels Aug 2, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

openml integration in shogun #4628

openml integration in shogun #4628

gf712 commented May 3, 2019 •

edited

karlnapf May 5, 2019

vigsterkr May 5, 2019

karlnapf May 5, 2019

gf712 May 6, 2019

iglesias May 21, 2019

gf712 May 21, 2019

karlnapf commented May 5, 2019

gf712 commented May 7, 2019

vigsterkr commented May 7, 2019

gf712 May 7, 2019

vigsterkr May 7, 2019

gf712 May 7, 2019

vigsterkr May 7, 2019 •

edited

vigsterkr May 7, 2019

vigsterkr May 7, 2019

gf712 May 7, 2019

vigsterkr May 7, 2019

vigsterkr May 7, 2019

gf712 commented May 17, 2019

iglesias commented May 21, 2019

gf712 commented May 21, 2019

stale bot commented Aug 2, 2020

karlnapf commented Aug 6, 2020

openml integration in shogun #4628

Are you sure you want to change the base?

openml integration in shogun #4628

Conversation

gf712 commented May 3, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

karlnapf commented May 5, 2019

gf712 commented May 7, 2019

vigsterkr commented May 7, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vigsterkr May 7, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gf712 commented May 17, 2019

iglesias commented May 21, 2019

gf712 commented May 21, 2019

stale bot commented Aug 2, 2020

karlnapf commented Aug 6, 2020

gf712 commented May 3, 2019 •

edited

vigsterkr May 7, 2019 •

edited