Proposal: Schema for Outfitting

[Marginal]

I'd like to propose an addition to the schema to carry station outfitting data. The proposed change to the schema is below - basically it just adds an optional "modules" property which is an array of integer IDs.

The semantics of the "modules" array is that it represents the availability of outfitting at the station at the time of reporting. Each element in the array is an integer ID which represents a module available in the station's outfitting screen. An empty "modules" array indicates that the station does not provide outfitting. (Publishers that do not provide outfitting info should therefore omit the "modules" property rather than supply an empty array).

Here is a not-quite exhaustive list of mappings from IDs to modules. I'd like to also propose that the EDDN project own and maintain this list.

Some Q&A:

Q: Why use integer IDs rather than strings (like we do for the commodity names)?
A: Using IDs does add an extra level of complexity/indirection, but offers some advantages over strings:

• Less ambiguity - no chance of variant spellings etc.
• Easier for non-English speaking subscribers to translate.
• More robust in the face of updates to E:D - subscribers can process IDs and store them in their databases even if they don't (yet) know how to display them.
• More compact - about a third of the size of the corresponding strings.

Q: For consistency, why not convert the commodities array to also use integer IDs instead of string names?
A: Converting the commodities array to use IDs would be a barrier to adoption. The proposed schema change is narrowly scoped in order to be trivial for subscribers to adopt - subscribers can just handle the proposed schema identically to the v2 schema by ignoring the new "modules" property. They can then choose to exploit the outfitting data encapsulated in the "modules" property later, or not at all.

Q: What about prices?
A: Unlike commodities, module prices don't typically vary from station to station. They may vary according to other factors including the player's allegiance, ranking, discounts etc but these are necessarily player-specific.

Q: What if the station provides outfitting, but doesn't have a commodities market?
A: The publisher should supply an empty "commodities" array.

Q: What about shipyard info (c.f. issue #16) ?
A: A new "ships" property could work in the same way and have similar semantics to the proposed "modules" property. Here is a not-quite exhaustive list of mappings from IDs to ships.

The proposed addition to the schema:

--- schema2.xml 2015-06-02 02:43:35.000000000 +0100
+++ schema3.xml 2015-06-12 00:20:06.000000000 +0100
@@ -1,6 +1,6 @@
 {
   "$schema": "http://json-schema.org/draft-04/schema#",
-  "id": "http://schemas.elite-markets.net/eddn/commodity/2#",
+  "id": "http://schemas.elite-markets.net/eddn/commodity/3#",
   "type": "object",
   "additionalProperties": false,
   "properties": {
@@ -96,6 +96,15 @@
                       }
                       ]   
           }
+        },
+        "modules": {
+          "type": "array",
+          "additionalProperties": false,
+          "uniqueItems": true,
+          "items": {
+            "type": "integer",
+            "additionalProperties": false
+          }
         }
       },
       "required": [

[Marginal]

Here is a sample message in the proposed schema.

[AnthorNet]

Those properties should have their own schemas, not going in the commodities.
As for IDs, yes they are unique by commodity, but subscribers would need the conversion table.
And only API based software have that infos, not the other one. But API tends to have the right commodity name, so it's not really something that will help ^^

I'll draft both schema if needed quickly. They should be coherent with commodity v2, but we might add that id as an additional properties, as it will not block v2 subscribers.
But has I said, I don't think this will really help !

[jamesremuscat]

+1 for "should be its own schema". (If necessary/desirable we can move the station information section into a shared schema somewhere so that it can be reused.)

Extra bonus points for using numeric IDs rather than text to identify the modules - had this information been available when v2 was being drafted it would certainly have been used there.

@AnthorNet makes a good point about the conversion table, though. Ideally, if we lived in a world where Frontier Developments acknowledged that they have a third-party developer community that are eager to add value to their game, they'd publish that themselves. I don't think that should stop us, though.

Not sure about taking ownership of the conversion table ourselves though - I'd much rather push FD into releasing the information, or at least keep it community-sourced.

[Marginal]

OK here are separate schema for outfitting and ships - both using integer IDs. There's a lot of commonality with the v2 commodity schema (basically a cut and paste), but I haven't tried to split that commonality out into a shared sub-schema - I'm not sure it's worth the complication.

In the absence of Frontier documenting a third-party API and/or ID tables, someone else has to host community-sourced tables, otherwise the proposed schema are unusable in practice. I'm suggesting that it is most logical and convenient for publishers and subscribers for this project to maintain a master set of those tables, but it's not essential.

[AnthorNet]

Have you check the proposal on issue #16 ?

[Marginal]

@AnthorNet
Sorry, missed that. Have commented on issue #16.

[Marginal]

Some comments on your draft 33d1b1e:

I suggest removing the "minItems" restriction on modules , so that "modules": [] can indicate the absence of outfitting services.

Since you've made the "name" string mandatory, the "fdId" field is redundant. I suggest removing "fdId".

To make it easier for subscribers to parse the messages I suggest adding category (required), subtype (optional), class (required) and rating (required) fields instead of bundling everything into the name field. See this table for the suggested use of these fields.

(The 'subtype' field in that table is ugly - it holds ship type for armour modules, mount type for weapons, and guidance type for missile weapons. Alternatively I guess one could have different objects in the schema for armour, weapons etc).

[AnthorNet]

Are they really a distinction with subtype ingame ?
Tends to be hard to specify no ?

[Marginal]

Yes. For example 34 Pegasi/Webb Port currently offers Reactive armour for Sidewinder, Type-6, Type-7 and Asp, but not for any other ships. If I'm flying a Vulture and am looking for Reactive Armour then the distinction matters to me.

Edit: If you really don't like a subype field then this info could be bundled into the name field. e.g. for armour:
"Sidewinder Lightweight Alloy"
"Sidewinder Reactive Surface Composite"
"Imperial Clipper Lightweight Alloy"
"Type-6 Transporter Military Grade Composite"
etc, and for weapons:
"Pulse Laser - Fixed"
"Pulse Laser - Gimballed"
"Fragment Cannon - Turreted"
"Missile Rack - Dumbfire"
etc.
But this would make the name field no longer identical to what is shown in-game, and I think would make it harder for subscribers to parse.

[AnthorNet]

Effectively, I was just trying to find some good solution if we have to rely on other source than the API.

But since it's not shown in game... I guess we will have to stick with that.
I'll update both schema on monday.

[Marginal]

Difficult. The user could report what armour is available just for the ship that they're flying.
But the semantics of the "modules" array has to be that it represents the total availability of outfitting at the station - otherwise items would never go out of stock.

[Marginal]

Here is a sample message in the updated outfitting schema that I'm suggesting in pull request #22.

[AnthorNet]

Just have a quick look and sounds good to me as long as everything can be populated from API Json, and ingame.
Will look at it more the next week when I come back.

Thx

[Marginal]

I've updated this table to reflect (I think) a complete list of currently available modules, with fields consistent with the outfitting schema that I'm suggesting in pull request #22.