Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add "WebAPI" type #1423

Open
IvanGoncharov opened this Issue Nov 2, 2016 · 87 comments

Comments

Projects
None yet
@IvanGoncharov
Copy link

IvanGoncharov commented Nov 2, 2016

At the moment Schema.org don't have appropriate type to describe Web APIs.
By WebAPI I mean any API which uses HTTP/HTTPS as transport mechanism including REST API, SOAP, GraphQL, etc.
At the moment there is no scalable mechanism to discover publicly available APIs on the internet.
Closest solution so far is API catalogs, and for the last two year, I have been maintaining my own.
But it's hard to maintain and impossible to scale.

On the other hand, almost all public APIs have dedicated HTML pages where they can use Schema.org types.
Ideally it should have fields like:

I want to work on a PR adding WebAPI type to schema.org.
Can you please advice which base type should I derive it from?
And should I submit it as PR or you have some formal procedure?

@jaygray0919

This comment has been minimized.

Copy link

jaygray0919 commented Nov 2, 2016

This would complement work done here:
http://webdatacommons.org/structureddata/

@IvanGoncharov

This comment has been minimized.

Copy link
Author

IvanGoncharov commented Nov 3, 2016

@jaygray0919 It will benefit entire developers community since a huge amount of data can be accessed only through Web APIs. And for many companies describing their existing Web APIs will be the first step toward Linked Data.

@IvanGoncharov

This comment has been minimized.

Copy link
Author

IvanGoncharov commented Nov 17, 2016

@danbri @RichardWallis I'm ready to start working on PR, can you please answer my questions?

Can you please advice which base type should I derive it from?
And should I submit it as PR or you have some formal procedure?

@RichardWallis

This comment has been minimized.

Copy link
Contributor

RichardWallis commented Nov 18, 2016

@IvanGoncharov apologies for being a bit slow in responding.

I can see the basic need for something like this - I would like to see some indication of who would apply it to their sites (other than yourself of course).

Most of the properties you describe could be generally available via any Type in Schema (Title (name), description, URLs, etc.) and via the organization Offering the API (Service?) (Logo, Contacts etc.)

As to a base type to enhance or subtype, my inclination is to look at EntryPoint which is generally describing something similar. Not sure if we just enhance that type with a few more specific properties, or create a more specific subtype for API (do we need to constrain it to WebAPI ?)

In the current structure EntryPoint is accessed via the target property of Action and its subtypes. Maybe a new APIAction subtype would work here. This would enable that a description of the Organization/Service that is providing the API to be linked to the actual API description via potentialAction.

@thadguidry

This comment has been minimized.

Copy link

thadguidry commented Nov 18, 2016

@RichardWallis I've pinged David and Wendell from ProgrammableWeb.com to voice their opinions here as well. Hopefully they respond shortly.

@dberlind

This comment has been minimized.

Copy link

dberlind commented Nov 18, 2016

Hey all, I've received @thadguidry's email and replied. As you can imagine, this is an issue that is core to our mission. It's practically our purpose in life. So, we do spend a disproportionate amount of our time thinking about API discovery and search and developing an understanding of what exactly developers want when they start to look for APIs. So, we certainly appreciate the outreach to better understand what role we can play.

@IvanGoncharov

This comment has been minimized.

Copy link
Author

IvanGoncharov commented Nov 21, 2016

I can see the basic need for something like this - I would like to see some indication of who would apply it to their sites (other than yourself of course).

@RichardWallis I think it's most important question and should be answered before technical ones.
And it should be answered by API owners and DX guys so I publish a post on our blog asking for feedback: https://blog.apis.guru/api-discovery-can-we-do-better-2336706d5407

@tlrobinson

This comment has been minimized.

Copy link

tlrobinson commented Nov 21, 2016

This is a great idea.

Nitpick: I'm not sure "web" is the best name for the type of APIs you're talking about, basically 3rd party APIs. HTTP is certainly the most common transport these days but that might not always be the case. Also, "web" generally implies hypertext (of course that's the "H" in HTTP but almost no APIs actually use hypertext)

@pjz

This comment has been minimized.

Copy link

pjz commented Nov 21, 2016

...so 'net API' instead of 'web API'?

@shashanksingh

This comment has been minimized.

Copy link

shashanksingh commented Nov 21, 2016

Hey Guys,
We at faasos (on-demand-delivery)-would love to have these openly discoverable api schema for our public api's.

URL : www.faasos.com

@mesteche

This comment has been minimized.

Copy link

mesteche commented Nov 21, 2016

I'm not sure "web" is the best name for the type of APIs you're talking about, basically 3rd party APIs. HTTP is certainly the most common transport these days but that might not always be the case. Also, "web" generally implies hypertext (of course that's the "H" in HTTP but almost no APIs actually use hypertext)

With WebSockets and WebRTC, this is becoming less and less true.

@maccman

This comment has been minimized.

Copy link

maccman commented Nov 21, 2016

@clearbit would love to see this.

@sotaan

This comment has been minimized.

Copy link

sotaan commented Nov 21, 2016

Hey!
I really like this idea since it will make API discovery smoother

@L4in

This comment has been minimized.

Copy link

L4in commented Nov 21, 2016

As someone who already lost countless hours trying to find good APIs, I totally back up this idea.
A lot of people not in tech complained about how programming looks complex by lack of documentation or visibility - I tend to disagree because this is less and less true - and this is definitely a change needed to welcome even more people into writing their own stuff.

@asbjornu

This comment has been minimized.

Copy link

asbjornu commented Nov 21, 2016

This sounds a lot like Hydra's ApiDocumentation. Please have a look; perhaps there's something that could be learned one way or another.

@tlrobinson:

Nitpick: I'm not sure "web" is the best name for the type of APIs you're talking about, basically 3rd party APIs. HTTP is certainly the most common transport these days but that might not always be the case. Also, "web" generally implies hypertext (of course that's the "H" in HTTP but almost no APIs actually use hypertext)

But won't you agree that having a document describing your API and perhaps even being so bold as to linking to your initial API endpoint, would make at least the Web API description a hypertext format?

@ncthbrt

This comment has been minimized.

Copy link

ncthbrt commented Nov 21, 2016

We (WhereIsMyTransport) would be very interested.

@cmerrick

This comment has been minimized.

Copy link

cmerrick commented Nov 21, 2016

We (stitchdata.com) extract data from dozens of APIs. We think this is a great idea. +1

@adamretter

This comment has been minimized.

Copy link

adamretter commented Nov 21, 2016

+1

@bert2002

This comment has been minimized.

Copy link

bert2002 commented Nov 21, 2016

Would love to see this happen.

@mateomurphy

This comment has been minimized.

Copy link

mateomurphy commented Nov 21, 2016

EntryPoint would seem to be a good type for individual API calls (which should definitely have their own type). But overall API should probably be a Service. Net and/or Web API could be subtype of API

@dberlind

This comment has been minimized.

Copy link

dberlind commented Nov 21, 2016

On ProgrammableWeb, we are making reference more and more to "networkable"
APIs vs. just Web (HTTP) APIs. The latest data model behind our directories
reflects this and one of the strongest use cases has to do with the
Internet of Things which for some "things" doesn't involve HTTP at all (ie:
MQTT over TCP/IP).

On Mon, Nov 21, 2016 at 11:27 AM, Mateo Murphy notifications@github.com
wrote:

EntryPoint would seem to be a good type for individual API calls (which
should definitely have their own type). But overall API should probably be
a Service. Net and/or Web API could be subtype of API


You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#1423 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGVdulSuAr3bh0q1Pu3B-rTAdHwDmRweks5rAcZmgaJpZM4KnIBY
.

@ivan-gammel

This comment has been minimized.

Copy link

ivan-gammel commented Nov 21, 2016

I like this proposal and my company will eventually support it by publishing some of our REST APIs this way. Some things I'd personally be glad to see:

  1. Indication of REST API maturity level (resources/verbs/HATEOAS)
  2. For Level 3 REST APIs what are the supported formats (HAL, Siren etc)
@dberlind

This comment has been minimized.

Copy link

dberlind commented Nov 21, 2016

Ivan, how does data get curated or policed? Is it essentially crowdsourced
by populating a schema as is the case with other schemas such as recipes,
reviews, etc. The reason I ask is that, in our experience of doing this
sort of thing for over a decade, we know for sure that many API providers
(not all, but enough to poison the well) will provide whatever data makes
them look best as opposed to what data is most truthful. For example, API
providers will say they support hypermedia when they don't. That sort of
thing. Sometimes, it's because a non-technical person (eg: a marketing or
SEO person) is responsible for maintaining that data.

David Berlind
editor in chief
ProgrammableWeb.com

On Mon, Nov 21, 2016 at 12:14 PM, Ivan Gammel notifications@github.com
wrote:

I like this proposal and my company will eventually support it by
publishing some of our REST APIs this way. Some things I'd personally be
glad to see:

  1. Indication of REST API maturity level (resources/verbs/HATEOAS)
  2. For Level 3 REST APIs what are the supported formats (HAL, Siren
    etc)


You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#1423 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGVduo_lEmMWgOsVzE7XChgN9qmivrzCks5rAdFogaJpZM4KnIBY
.

@chrisgandi

This comment has been minimized.

Copy link

chrisgandi commented Nov 21, 2016

This is really needed

@schwartzMS

This comment has been minimized.

Copy link

schwartzMS commented Nov 21, 2016

I would be interested in implementing this on our site.

@berezovskyi

This comment has been minimized.

Copy link

berezovskyi commented Nov 21, 2016

@dberlind I think the only reliable way to verify that the API conforms to its declaration is to have the API provide this information itself (e.g. https://www.w3.org/TR/ldp-primer/#filelookup for LDP-enabled apps, WSDL for SOAP etc.) and verify it programmatically. But this seems not to be the purpose behind this PR: it would be good to annotate the existing (i.e. legacy) APIs, in a noninvasive manner (i.e. via such tags in the API documentation).

@dberlind

This comment has been minimized.

Copy link

dberlind commented Nov 21, 2016

Thanks @andrew. What about subjective information that less easy to verify
and potentially more about gaming search? For example, referring to @ivan's
Medium post, if I'm an unscrupulous SEO (those don't exist, right?) or a
marketing person who is judged on eyeballs, and I know that the search term
"email API" is getting a lot of hits on Google, I might gladly put "email"
into the category field and description field (assuming those fields are
part of the schema) even if my API isn't exactly an email API. This is
exactly the behavior we've seen when crowdsourcing this data set on
ProgrammableWeb and we have wrestled with this problem for years. Also,
would a schema be deemed invalid (iow, not processable) if it was missing
the above suggested declaration? If the answer to that is "no," then
ultimately, you still have the issue of search results being gamed or
corrupted.

On Mon, Nov 21, 2016 at 1:05 PM, Andrew Berezovskyi <
notifications@github.com> wrote:

@dberlind https://github.com/dberlind I think the only reliable way to
verify that the API conforms to its declaration is to have the API provide
this information itself (e.g. https://www.w3.org/TR/ldp-primer/#filelookup
for LDP-enabled apps, WSDL for SOAP etc.) and verify it programmatically.
But this seems not to be the purpose behind this PR: it would be good to
annotate the existing (i.e. legacy) APIs, in a noninvasive manner (i.e. via
such tags in the API documentation).


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1423 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGVdutDE274hHwlC7qEFi-I0txHqpTHWks5rAd1RgaJpZM4KnIBY
.

@digitalmaster

This comment has been minimized.

Copy link

digitalmaster commented Nov 21, 2016

@IvanGoncharov This type of out of the box, community driven thinking is why i love what i do. Would love to see this implemented. 👍

@ivan-gammel

This comment has been minimized.

Copy link

ivan-gammel commented Nov 21, 2016

@dberlind to defend from "dark" SEO, it may be useful to include "dependsOn" attribute to API and web site metadata, that will list all APIs referenced by the described resource. API providers will require the inclusion in dependsOn in their terms of service and search engines may use it to rank search results by popularity. API catalogs may also include the APIs that they list and have validated in their dependsOn attribute as a proof of the metadata correctness.

@dberlind

This comment has been minimized.

Copy link

dberlind commented Nov 21, 2016

thanks @ivan-gammel, I apologize, but your answer shot right over my head. For example, why would an API provider require "dependsOn" in their ToS? Algorithmically, how would "dependsOn" influence rank?

@EricAxel

This comment has been minimized.

Copy link
Contributor

EricAxel commented Jul 13, 2017

My concern is around the new "documentation" property being proposed here (http://pending.schema.org/documentation). I suggest that the domain be expanded to include http://schema.org/Product - allowing for describing things such as owner manuals, assembly instructions, and manufacturer spec sheets.

@jpsherman

This comment has been minimized.

Copy link

jpsherman commented Jul 13, 2017

I would also suggest something like
documentUseCase

  • most documentation content has a purpose, such as installation, migration, assembly

documentChapterTitle

  • most documentation has chapters that describe parts, assembly, troubleshooting

When it comes to software documentation I'd suggest a variation of "version" such as "releaseVersion" which would, when it comes to software, break down like this:

  • product v1 = version

  • product v1.2.3 = releaseVersion

overall, this is a fantastic idea, thank you for this.

@bardonelgen

This comment has been minimized.

Copy link

bardonelgen commented Sep 4, 2017

Maybe it could help to have a look at a practical example for a feasibility check:

Let’s say, a town’s authorities want to provide a digital interface to public services for their city-zens using Open311 and have a local hosting company provide the infrastructure on their behalf.

Now, you (as a loyal city dweller) come across a defective lamp post and would like to see it fixed.

You use the provided Open311 interface (via the API, using either a mobile app or a webpage) to notify the respective department of the incident (they probably receive an e-mail, mobile notification or simply a database entry).

Questions to be asked:

Who actually informs the local department ? Is it you using the SERVICE ?

Or does the service do this by itself (being the DIRECT per former of the action, which is actually out of your control…) ? You, after all, only provided some raw data.

If, indeed, the SERVICE does: Wouldn’t it need to be an http://schema.org/agent ?

But, as of now, these by definition need to be either human or an organisation ?

If the service could be an agent by itself, who will the be the http://schema.org/provider ?

The local administration or the IT shop running the machines ? Are THEY the ones sending the information then ?

Or still YOU yourself, as you – after all – initiated the process ?

And how about the generated output ? Of course the API delivers a direct response to the user; but how about the output to 3rd parties (local authorities with this example) ?

Is describing these outcomes out-of-scope for the API description or should it (or at least its typical outcome) be described with the API description for transparency purposes (a.k.a. “good practice”) ?

@acka47

This comment has been minimized.

Copy link
Contributor

acka47 commented Oct 18, 2017

Thinking about how to add a link to an API and its documentation to the description of a website in the OER World Map (hbz/oerworldmap#17 (comment)), I noticed that we already use http://schema.org/ServiceChannel in a way where we can easily integrate API descriptions. Basically, we see a web site as a Service itself that can be accessed by different means (in the browser and via API). For example GitHub:

{
   "name":"GitHub",
   "@type": [ "WebSite", "Service" ],
   "availableChannel":[
      {
         "@type":"ServiceChannel",
         "serviceUrl":"https://github.com",
         "availableLanguage":"en"
      },
      {
         "@type":"WebAPI",
         "serviceUrl":"https://api.github.com",
         "documentation":"https://developer.github.com/v3/"
      }
   ]
}

With this approach, WebAPI is used as a subclass of ServiceChannel. This probably isn't the way intended by schema.org but nonetheless I think it makes sense.

@rajbiswal

This comment has been minimized.

Copy link

rajbiswal commented Dec 12, 2017

Its a good idea to make APIs discoverable and adding the WebAPI to Schemas.org
We are looking for something like this for searching APIs based on context of an use-case or domain. At the moment there are multiple repositories of APIs but there is no standard to guide the structure for describing the APIs. Many don't even provide APIs to search the repository.

@monty241

This comment has been minimized.

@MikeRalphson

This comment has been minimized.

Copy link

MikeRalphson commented Jan 23, 2018

Is there a document which describes the process by which a type leaves pending?

@danbri

This comment has been minimized.

Copy link
Contributor

danbri commented Apr 23, 2018

@MikeRalphson https://github.com/schemaorg/schemaorg/blob/master/README.md has some notes on proposing schemas, and http://schema.org/docs/howwework.html provides a high level view of the whole thing, but we have not got a highly formalized approach for transferring things out of Pending into Core.

The most important considerations: "is anyone using (consuming) it, or at least likely to?" (if so please share details here), "does it add confusion or complication to schemas have already" (e.g. in this case how does it related to Entrypoint), "are the term names sensible and not prone to misunderstanding, not overly general etc.".

@danbri

This comment has been minimized.

Copy link
Contributor

danbri commented Apr 23, 2018

I would like to see this move along and it seems useful. Here are a couple of things that came up when I was reviewing the topic (besides http://schema.org/EntryPoint from the /Action design):

  1. How does it relate to the idea that a http://schema.org/Dataset has a http://schema.org/distribution that is a DataDownload? I suggest we amend /distribution to also note that WebAPI values are expected.

  2. How does it relate to the idea that a SearchAction is a potentialAction on a WebSite? The idiom we use for this at Google is documented at https://developers.google.com/search/docs/data-types/sitelinks-searchbox - it captures something of the information needed to send a simple query to a site via HTTP/S GET URL encoding, e.g.

{
  "@context": "http://schema.org",
  "@type": "WebSite",
  "url": "https://www.example.com/",
  "potentialAction": {
    "@type": "SearchAction",
    "target": "https://query.example.com/search?q={search_term_string}",
    "query-input": "required name=search_term_string"
  }
}
</script>
@jaygray0919

This comment has been minimized.

Copy link

jaygray0919 commented Apr 24, 2018

IOHO, @EntryPoint would be useful (and differentiating) if it is used to define REST and similar endpoints. One might argue that a @Dataset is content in an unspecified form. In contrast, an @EntryPoint might be content in a specified form (e.g. REST and JSON). Similarly, a SPARQL endpoint serves content, and expects the form (JSON, JSON-LD, RDF, TTL, N3, etc.) to be specified in the query ("potentialAction": ("@type": "SearchAction" ... )

One argument for adding @EntryPoint and differentiating from @Dataset ...

@dret

This comment has been minimized.

Copy link

dret commented Nov 15, 2018

maybe as one additional data point for this issue (which i think would be an excellent idea to pursue, schema.org currently is painfully lacking in terms of API descriptions): https://tools.ietf.org/html/draft-wilde-service-link-rel-06 is on the way to becoming an RFC and defines some link relations to identify resources that relate to web services (web APIs). this is clearly much more limited, but may be useful as input.

@AlasdairGray

This comment has been minimized.

Copy link

AlasdairGray commented Nov 15, 2018

  1. How does it relate to the idea that a http://schema.org/Dataset has a http://schema.org/distribution that is a DataDownload? I suggest we amend /distribution to also note that WebAPI values are expected.

In Bioschemas we would like to recommend a pattern for providing a link to an API for a Dataset. Is @danbri's suggestion here the expected approach?

@dret

This comment has been minimized.

Copy link

dret commented Nov 15, 2018

@danbri

This comment has been minimized.

Copy link
Contributor

danbri commented Jan 30, 2019

Any more comments on https://webapi-discovery.github.io/rfcs/rfc0001.html ? Discussion so far has tended to focus on the datasets connection, which we should continue (and try to stay close to what DCAT are developing).

But https://webapi-discovery.github.io/rfcs/rfc0001.html has a lot more to it. Is anyone ximplementing?

@dret

This comment has been minimized.

Copy link

dret commented Jan 30, 2019

@bardonelgen

This comment has been minimized.

Copy link

bardonelgen commented Jan 30, 2019

@danbri

This comment has been minimized.

Copy link
Contributor

danbri commented Jan 30, 2019

@bardonelgen - my understanding is that the "webapi discovery" document linked above does try to use the same Schema.org Actions mechanism as the one used by Gmail.

@dret - thanks re https://tools.ietf.org/html/draft-wilde-service-link-rel-10 - we ought to express the mapping at least, if there is a clear equivalence.

And noted re "Status". Is there a subset we could start with, perhaps? e.g. webApiDefinitions, webApiActions, ...?

@westurner

This comment has been minimized.

Copy link
Contributor

westurner commented Jan 30, 2019

@dret

This comment has been minimized.

Copy link

dret commented Jan 30, 2019

pvgenuchten added a commit to pvgenuchten/core-geonetwork that referenced this issue Feb 15, 2019

improve scope to schema-org types
The meaning of iso19115 scope values are quite poorly documented, most relevant guess seems on comment at http://home.badc.rl.ac.uk/lawrence/blog/2008/03/19/the_scope_of_iso19115

I'm suggesting to use the extension type WebAPI for services, in stead of the dataCatalog type, this is the most relevant change, because service is used quite frequently in the scope of inspire. Following this discussion schemaorg/schemaorg#1423 we may actually be able to link WebAPI's to datasets soon. But we could also wait with merging this PR until schema-org has developed this as part of core
@danbri

This comment has been minimized.

Copy link
Contributor

danbri commented Feb 19, 2019

@MikeRalphson @IvanGoncharov can you propose a pull request to update the version we have in Schema.org's Pending area, based on your Community Group report (aka 'RFC', though I'm wary of calling it that as it may confuse IETF folk)

The files are

I can help with mechanics. Essentially we need a label and description for each type and property, plus info that situates types in a hierarchy, and that relate types to properties.

@danbri

This comment has been minimized.

Copy link
Contributor

danbri commented Feb 19, 2019

Let me start trying to translate the doc's language into schema.org's machinery here -

  • "versions (OPTIONAL array of Thing -> Property -> softwareVersion). It is RECOMMENDED that APIs be versioned using [semver]"
    • We already have https://schema.org.org/softwareVersion so we just declare softwareVersion has a domainIncludes of WebAPI
    • We might also declare sofwareVersion a subPropertyOf the more general version property
  • "entryPoints (OPTIONAL array of Thing -> Intangible -> EntryPoint)"
    • "extended with: responseContentTypes (OPTIONAL Text)"
    • Could we use encodingFormat instead?
  • "license (OPTIONAL, CreativeWork or URL) - the license for the design/signature of the API"
  • "transport (enumerated Text: HTTP, HTTPS, SMTP, MQTT, WS, WSS etc)"
    • todo
  • "apiProtocol (OPTIONAL, enumerated Text: SOAP, GraphQL, gRPC, Hydra, JSON API, XML-RPC, JSON-RPC etc)"
  • "webApiDefinitions (OPTIONAL array of EntryPoints) containing links to machine-readable API definitions"
  • "webApiActions (OPTIONAL array of potential Actions)"
  • "The Content-Type(s) consumed by the WebAPI MAY be included in the EntryPoint.contentType property. The Content-Type(s) produced by the WebAPI MAY be included in the EntryPoint.responseContentType property.
    • All TODO
@MikeRalphson

This comment has been minimized.

Copy link

MikeRalphson commented Feb 20, 2019

@danbri will do, thanks. Will update the 'rfc' with outstanding issues then create a PR as above.

fxprunayre added a commit to geonetwork/core-geonetwork that referenced this issue Mar 1, 2019

improve scope to schema-org types (#3587)
The meaning of iso19115 scope values are quite poorly documented, most relevant guess seems on comment at http://home.badc.rl.ac.uk/lawrence/blog/2008/03/19/the_scope_of_iso19115

I'm suggesting to use the extension type WebAPI for services, in stead of the dataCatalog type, this is the most relevant change, because service is used quite frequently in the scope of inspire. Following this discussion schemaorg/schemaorg#1423 we may actually be able to link WebAPI's to datasets soon. But we could also wait with merging this PR until schema-org has developed this as part of core
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.