Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
342 changes: 342 additions & 0 deletions text/0000-embedded-refract.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,342 @@
- Start Date: 2015-08-29
- RFC PR: (leave this empty)
- Refract Issue: (leave this empty)

# Summary

This document outlines a new serialization format for embedding Refract.

# Motivation

1. Our current formats either look like XML, the DOM, or Lisp
1. People already "get" JSON
1. Other serializations we have are verbose

# Detailed design

## Overview

For this serialization, there are two basic concepts:

1. There is a reserved property name of `_refract`
1. There is a Refract Object with properties:
- element
- meta
- attributes
- content
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any guidance on how to handle name collisions? What would be the expected behavior of an implementation that could not produce this serialization because of a pre-existing refract property?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I'm not sure exactly. We could make it less likely to collide, like with _refract?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to propose _refract. I really don't like refract. It feels like we are taking away user's freedom. But using _refract diminishes it a bit.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will change to _refract then. Works for me!


All valid JSON that adheres to reserved property is valid Embedded Refract.

## Basic JSON

We'll start with a basic JSON example, which we'll say is a person and their bowling scores.

```json
{
"first_name": "John",
"last_name": "Doe",
"age": 28,
"scores": [150, 202, 145]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to list out the mappings from JSON types to element names for the auto-conversion that happens here.

}
```

## Refracting Object Elements

In Refract, this is an object with four members. With Embedded Refract, I can annotate by embedding refract using the `refract` property.

```json
{
"_refract": {
"element": "object",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is element required? Seems like it should be optional here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could be optional for all object elements, but it is needed for all other element types otherwise.

"meta": {
"id": "bowler-103"
}
},
"first_name": "John",
"last_name": "Doe",
"age": 28,
"scores": [150, 202, 145]
}
```

This is what this would like in full Refract.

```json
{
"element": "object",
"meta": {
"id": "bowler-103"
},
"content": [
{
"element": "member",
"content": {
"key": {
"element": "string",
"content": "first_name"
},
"value": {
"element": "string",
"content": "John"
}
}
},
{
"element": "member",
"content": {
"key": {
"element": "string",
"content": "first_name"
},
"value": {
"element": "string",
"content": "Doe"
}
}
},
{
"element": "member",
"content": {
"key": {
"element": "string",
"content": "age"
},
"value": {
"element": "number",
"content": 28
}
}
},
{
"element": "member",
"content": {
"key": {
"element": "string",
"content": "scores"
},
"value": {
"element": "array",
"content": [
{
"element": "number",
"content": 150
},
{
"element": "number",
"content": 202
},
{
"element": "number",
"content": 145
}
]
}
}
}
]
}
```

## Refracting Value Elements

If values do not have any meta values or attributes, there is no need to represent them as refracted. If I were to add some annotation to one of the values, it would show up as Refract and would be defined as Refracted by use of the `refract` property keyword.

```json
{
"first_name": "John",
"last_name": "Doe",
"age": {
"_refract": {
"element": "number",
"meta": {
"id": "bowler-103-age"
},
"content": 28
}
},
"scores": [150, 202, 145]
}
```

Above, I've added an ID to the age of the bowler. Notice how I just refracted that value in that specific place. This is also the same as:

```json
{
"_refract": {
"element": "object",
"meta": {
"id": "bowler-103"
},
"content": {
"age": {
"element": "number",
"meta": {
"id": "bowler-103-age"
},
"content": 28
}
}
},
"first_name": "John",
"last_name": "Doe",
"scores": [150, 202, 145]
}
```

This will also support full member elements, so even this below is equivalent (note that it's a slight modification from the full Refract):

```json
{
"_refract": {
"element": "object",
"meta": {
"id": "bowler-103"
},
"content": [
{
"element": "member",
"content": {
"key": "age",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have a concrete example, but this is raising a red flag for me since it may lead to ambiguous cases where we cannot determine if a value is refracted. The first element of the first content array has no refract key and so an implementer must determine whether it is refracted by the element and content keys instead. I'm worried about this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, inside the refract object, it's a totally different world :) For instance, the content of an object element can either be an object itself or an array of member elements (which is how it currently is).

BUT we can change up how we manage the member's content to not have to refract everything there. For instance, the key value can be a JSON primitive or it could be a special refract object itself. Oh the recursion!!

"value": {
"_refract": {
"element": "number",
"meta": {
"id": "bowler-103-age"
},
"content": 28
}
}
}
}
]
},
"first_name": "John",
"last_name": "Doe",
"scores": [150, 202, 145]
}
```

In full Refract, you would have to also provide the element for the key "age" in the previous example, but since we can rely on the `refract` keyword, we can leave it unrefracted. We could Refract it more later if necessary.

## Refracting Array and Array Items

Array items can be embedded without refracting the entire array.

```json
{
"first_name": "John",
"last_name": "Doe",
"age": 28,
"scores": [
{
"_refract": {
"element": "number",
"meta": {
"id": "bowler-103-game-1"
},
"content": 150
}
},
202,
145]
}
```

Full arrays can also be refracted.

```json
{
"first_name": "John",
"last_name": "Doe",
"age": 28,
"scores": {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not asking for any changes here but I am really worried about the parser or whatever which deals with this. scores is an array but if it is refracted, it is an object.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually had a branch that handled this, and it wasn't bad. The only thing you do is check every object for _refract. If it's there, you convert to element differently. So:

  • If object, check for _refract
  • If _refract is there, use special instructions for converting the Refract Object to element instance
  • If not there, treat as normal JSON and convert to literal element instance

"_refract": {
"element": "array",
"meta": {
"id": "bowler-103-scores"
},
"content": [150, 202, 145]
}
}
}
```

## Refracting Attributes

This makes it nice for refracting attributes, because now you know what attributes have been refracted. This could become:

```json
{
"_refract": {
"element": "object",
"meta": {
"id": "bowler-103"
},
"attributes": {
"foo": "baz"
}
},
"first_name": "John",
"last_name": "Doe",
"age": 28,
"scores": [150, 202, 145]
}
```

```json
{
"_refract": {
"element": "object",
"meta": {
"id": "bowler-103"
},
"attributes": {
"foo": {
"_refract": {
"element": "string",
"content": "baz"
}
}
}
},
"first_name": "John",
"last_name": "Doe",
"age": 28,
"scores": [150, 202, 145]
}
```

## Edge cases

### Properties in Value Elements

What would this mean, where we have the `foo` property?

```json
{
"first_name": "John",
"last_name": "Doe",
"age": {
"_refract": {
"element": "number",
"content": 28
},
"foo": "bar"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may want to disallow this if it seems confusing enough and just treat it as an error. There's already a lot of magic going on 😉

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm good with that. All properties would then be ignored :)

},
"scores": [150, 202, 145]
}
```

We SHOULD ignore these properties.

# Drawbacks

"Don't we have more important things to do right now?" you may say. Sure! But it's the weekend and I'm doing some free-time thinking.

# Alternatives

We have alternatives already, so the idea would be that we don't do this if the current ones are enough.

# Unresolved questions

None at this point.