LMFDB data access from external computing platforms (Sage, Magma, Pari, Mathematica, etc...)

[AndrewVSutherland]

Excerpt of an e-mail conversation with Stephen Wolfram regarding the LMFDB:

"I couldn’t find a way to access this from Wolfram Language. Is there one yet? At the very least, you should make sure it's accessible from the new Wolfram Open Cloud, and from Mathematica. (If you need it, I could connect you to someone who could help architect this, or I could even have a quick think about it myself.)"

[JohnCremona]

How about "not unless or until we have a fully functional interface from
Sage" ?
On 15 May 2016 14:47, "Andrew Sutherland" notifications@github.com wrote:

Excerpt of an e-mail conversation with Stephen Wolfram regarding the LMFDB:

Yes, this is an interesting example. I couldn’t find a way to access this
from Wolfram Language. Is
there one yet? At the very least, you should make sure it's accessible
from the new Wolfram Open
Cloud, and from Mathematica. (If you need it, I could connect you to
someone who could help architect
this, or I could even have a quick think about it myself.)

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#1360

[AndrewVSutherland]

I wasn't advocating that we do this (or making it a priority), I just wanted to record it.

[AndrewVSutherland]

In fact I don't think we need to do anything that is specific to mathematica or sage, if we have an api we are willing to support (and prepared to throttle it to mange any load issues), the burden would be on whoever it is that wants to integrate with the LMFDB (e.g. you can access the OEIS from within Sage, but that is due to functionality in Sage, not the OEIS -- but obviously the LMFDB is much more complicated than the OEIS).

[JohnCremona]

On 15 May 2016 15:14, "Andrew Sutherland" notifications@github.com wrote:

In fact I don't think we need to do anything that is specific to
mathematica or sage, if we have an api we are willing to support (and
prepared to throttle it to mange any load issues), the burden would be on
whoever it is that wants to integrate with the LMFDB (e.g. you can access
the OEIS from within Sage, but that is due to functionality in Sage, not
the OEIS -- obviously the LMFDB is much more complicated).

Agreed, I was wearing my sage developer hat. If some sage people worked
on this and came up with some extra features for the api which would assist
that effort I'm sure we (lmfdb) would cooperate. And Harald wears both
hats!

—
You are receiving this because you commented.

Reply to this email directly or view it on GitHub

[haraldschilly]

And Harald wears both hats!

lol :-)

however, what many don't realize, the /api endpoint is just for the database. maybe it should have been called /db.

I think, a neat addition to everything that's done so far would be, to offer all the pages also in other formats. I.e. I propose a query parameter _format that can be text, html, yaml, and json. It just happens that right now it is just "html" as the default, but rendering the content of the provided dictionaries for the website in other formats would work out of the box (with some help, of course, for the more complex types, but it would work).

that would also lead to much better use of the template system, because pre-rendered html strings aren't any good.

so, this sounds cool, but also implies very long-term tasks of cleaning up the code.

[AndrewVSutherland]

I realize its just for the db. The downside is that this means things are represented at a low level that may not immediately translate to what you see on the screen, but the upside is that

(1) it is (or should be) documented, so that that someone who really cares can figure out how to do what they want with the data.
(2) we are moving toward a model where we store more information in the db and do less massaging/computation on the fly (both for efficiency and to enable searches)
(3) going through the db bypasses the need to change a lot of code (which I agree should be done and I like your proposal, I'm just looking for a quick win here -- not to address mathematica, but to address queries from researchers who want to do bulk loading into sage and magma, which should really be raised as a separate issue).

[haraldschilly]

oh yes, that all are good directions. there have been a lot of discussions, and -- at least my impression is -- that right now the first approximation to the website is, to make it suitable for education and casual users. i.e. that you have data and those knowls, in an overall working website package.

I have many ideas how to address those who want to do more technical computations, and I'm sure you have even more. E.g. mongodb is good for random access, but terrible for crunching numbers.

Last fall, I was a bit at this ICERM for lmfdb. My impression is, that the first stepping stones towards that are to clarify one or two initial use-cases with clear requirements, and then build out this as a new extension to the lmfdb project. It's just hard to know what someone really wants, when it is still all in flux and rather vague.

[JohnCremona]

For the record: work is under way in Warwick to make this possible.