NIP-XX Use Nostr as storage for chart data

[ShinoharaTa]

Use Nostr as storage for chart data

The method I propose is to extend NIP-78 (KIND:30078) and define it as a chart.
That mechanism in Nostr is very resistant to communication failures and also overcomes cloud vendor failures.
We believe that this will be of great use in Japan, where there are many disasters, and we believe that by placing preliminary data, etc. on a high-speed distributed network, open data can be delivered reliably.

For your reference, I propose a time series of data that we Japanese see every time a disaster strikes.

• Water level information of rivers
• Tide level data of coastlines
• Precipitation at a certain point
• Hourly wind strength and direction

Of course, you can use a variety of observation data, not just weather data.
Here are some familiar examples.

• Balcony temperature
• Temperature and humidity in the living room
• RAM usage of my relay
• Temperature of the server machine in my home

Although different from a chart, we think it would be useful to store disaster information in a relay.
This should be suggested in another NIP.

• Earthquake epicenter, intensity, energy, and time of occurrence
• Current location of typhoons
• Predicted time of arrival of tsunamis

As an example of using NIP-78, I run the following website and contribute observation data to the relay.

野州田川水系 定点観測所 - * japanese only.

It measures the number of KIND: 1's posted to relays operated by Japanese and stores them in KIND: 30078.
The measured data is used as a chart like the water level of a river.

Open data can be viewed by anyone, and if we can define it as a Chart event, we can provide a system that makes open data acquisition much easier.

I propose a time series data structure, extending the tags of NIP-78 (Kind: 30078) as follows.

Chart structure as I define it

This is a sample and objects CPU usage, RAM usage, and CPU temperature for 12 hours.
KIND: 30078 as well as the d tag and AUTHOR to determine.
You may want to overlay multiple data on a single chart. The number of values can be added as desired so that multiple data can be registered.

{
    "kind": 30076,
    "author": "npub",
    "tags": [
        ["d", "chart_identification_code"],
        ["title", "chart Title"],
        ["category", "Server", "Temperature"],
        ["description", ""],
        ["license", ""],
        ["published_at", "ISO Unix Time String"],
        [
            // dataset, "data set name", "x label", "y label", "data as a JSON string of x,y pairs"
            "dataset", "CPU Usage", "Date", "Percentage", "[{x: 1690862400, y: 12.5},{x:.., y:..}]"
        ],
        [
            "dataset", "RAM Usage", "Date", "Percentage", "[{x: 1690862400, y: 45},{x:.., y:..}]"
        ],
        [
            "dataset", "CPU Temp", "Date", "Temperature (C)", "[{x: 1690862400, y: 65},{x:.., y:..}]"
        ],
    ]
}

We believe that labels and values properly extracted from this data structure can be passed directly to the Chart.js object as an example.
If anyone can provide open data to the site in an easy-to-read format, it will be ready for disasters and cloud failures.

The data defined for this purpose is defined as in NIP-19, nchart1xxxxxx... We believe that it would be more convenient to use if the data defined in this way could be defined as NIP-19 and provided as

Constraint

This proposal places constraints on existing events.
Relay load should also be considered, especially when using large data.
Define constraints.

• The maximum number of values is 10
• maximum number of data per value is 2000
• values * value should be 5000 (2000 * 2.5).

Some data may be retained for a long period of time. In that case, add a date to the d tag as shown below.
The date of interest is controlled by the date tag, which should be written in unixtime. If this is data that is being collected continuously, the date tag is not necessary. Define that the data is always backward from the current time.

{
    "kind": 30076,
    "author": "npub",
    "tags": [
        ["d", "chart_identification_code_yyyymmdd"],
        ["title", "chart Title"],
        ["category", "Server", "Temperature"],
        ["description", ""],
        ["date", "ISO Unix Time String"],
        ["license", ""],
        ["published_at", "ISO Unix Time String"],
        [
            "dataset", "CPU Usage", "Date", "Percentage", "[{x: 1690862400, y: 12.5},{x:.., y:..}]"
        ],
        [
            "dataset", "RAM Usage", "Date", "Percentage", "[{x: 1690862400, y: 45},{x:.., y:..}]"
        ],
        [
            "dataset", "CPU Temp", "Date", "Temperature (C)", "[{x: 1690862400, y: 65},{x:.., y:..}]"
        ],
    ]
}

Data concatenation

When the number of data becomes large, you can manage multiple data by concatenating each POSTS.
This will use the NIP-51 list.

{
    "kind": 3007x,
    "author": "npub",
    "tags": [
        ["d", "chart_identification_code"],
        ["title", "chart Title"],
        ["category", "Server", "Temperature"],
        ["description", ""],
        ["license", ""],
        ["published_at", "ISO Unix Time String"],
        ["a", "30076:fcaec..d590f:abcd"],
        ["a", "30076:fcaec..d590f:efgh"],
        ["a", "30076:fcaec..d590f:ijkl"],
        ...
    ]
}

[vitorpamplona]

No need to change 30078, you can just create a new event type for this application.

[decentropy]

@ShinoharaTa

I dont think you are describing a graph... more like a table, right? graphs have nodes and edges.

im actually working on web apps for both cases.
Im using vis.js to display graph, and handsontable for table.

DM me on nostr if you want to collaborate

[ShinoharaTa]

@decentropy
Indeed, this is not based on graph theory.
The proposal is ngraph... I will change the proposal from "ngraph..." to "chart" and propose it again.

[mattn]

how about IDs for next/previous link?

[mattn]

I suggest that you change this suggestion to make a standard time series data format. Chart is one of the way to display the data. Someone might way to use analyzing.

The elements required for a time series data format are (I think):

• Attributes indicating data title and license to use, etc
• A schema that defines columns and types of
• Huge data possibilities that can span multiple events

[vitorpamplona]

Suggestion for tag structure:

{
    "kind": 30075,
    "author": "npub",
    "tags": [
        ["d", "chart_identification_code"],
        ["title", "chart Title"],
        ["category", "Server", "Temperature"],
        ["description", ""]
        ["published_at", "ISO Unix Time String"], // original date, the updated at is already the `.created_at`
        ["labels",
            "2023-08-01 00:00", "2023-08-01 01:00", "2023-08-01 02:00", "2023-08-01 03:00", 
            "2023-08-01 04:00", "2023-08-01 05:00", "2023-08-01 06:00", "2023-08-01 07:00", 
            "2023-08-01 08:00", "2023-08-01 09:00", "2023-08-01 10:00", "2023-08-01 11:00"
        ],
        [
            "dataset", "CPU Usage", "Percentage" // dataset, name, y-axis label, n-values
            "12.5%", "12.7%", "12.3%", "13.0%",
            "13.5%", "12.9%", "12.8%", "12.6%",
            "13.2%", "13.1%", "12.4%", "12.9%"
        ],
        [
            "dataset", "RAM Usage", "Percentage"
            "45%", "46%", "44%", "45%",
            "47%", "46%", "45%", "44%",
            "46%", "47%", "44%", "45%"
        ],
        [
            "dataset",  "CPU Temp", "Temperature (C)"
            "65°C", "66°C", "64°C", "65°C",
            "66°C", "65°C", "64°C", "64°C",
            "65°C", "66°C", "64°C", "65°C"
        ]
    ]
}

This event kind is ideal for certain charts, where xlabel can be the same (pre-rounded) for all datasets.

Another event kind could be ideal for more granular data that can be rounded at the client side.

{
    "kind": 30076,
    "author": "npub",
    "tags": [
        ["d", "chart_identification_code"],
        ["title", "chart Title"],
        ["category", "Server", "Temperature"],
        ["description", ""]
        ["published_at", "ISO Unix Time String"],   
        [
            // dataset, "data set name", "x label", "y label", "data as a JSON string of x,y pairs"
            "dataset", "CPU Usage", "Date", "Percentage", "[{x: 1690862400, y: 12.5},{x:.., y:..}]"
        ],
        [
            "dataset", "RAM Usage", "Date", "Percentage", "[{x: 1690862400, y: 45},{x:.., y:..}]"
        ],
        [
            "dataset", "CPU Temp", "Date", "Temperature (C)", "[{x: 1690862400, y: 65},{x:.., y:..}]"
        ],
    ]
}

[ShinoharaTa]

@vitorpamplona
Thank you very much.
That suggestion is very simple and straightforward.
I will revise my proposal.

[ShinoharaTa]

@mattn
We have addressed two of your suggestions.

For the life of me, I could not decide on the following.
We think it would be better to concatenate the datasets in KIND, which concatenates multiple Charts.

Can continuous data be defined as follows...

{
    "kind": 3007x,
    "author": "npub",
    "tags": [
        ["d", "chart_identification_code"],
        ["title", "chart Title"],
        ["category", "Server", "Temperature"],
        ["description", ""],
        ["license", ""],
        ["published_at", "ISO Unix Time String"],
        ["chart", "nchart1..."],
        ["chart", "nchart1..."],
        ["chart", "nchart1..."],
        ["chart", "nchart1..."],
        ["chart", "nchart1..."],
        ["chart", "nchart1..."],
        ["chart", "nchart1..."],
        ["chart", "nchart1..."] 
    ]
}

[ShinoharaTa]

MEMO.

chart data kind: 30078 -> 30076,
concat data kind: 30078 -> 3007x,

[AsaiToshiya]

Could you use l, L and a of standardized tags instead of category and chart tag?

The data defined for this purpose is defined as in NIP-19, nchart1xxxxxx... We believe that it would be more convenient to use if the data defined in this way could be defined as NIP-19 and provided as

What is included in TLV of nchart1...? If it is just a marker, I think nevent1... is enough.

{
    "kind": 3007x,
    "author": "npub",
    "tags": [
        ["d", "chart_identification_code"],
        ["title", "chart Title"],
        ["category", "Server", "Temperature"],
        ["description", ""],
        ["license", ""],
        ["published_at", "ISO Unix Time String"],
        ["chart", "nchart1..."],
        ["chart", "nchart1..."],
        ["chart", "nchart1..."],
        ["chart", "nchart1..."],
        ["chart", "nchart1..."],
        ["chart", "nchart1..."],
        ["chart", "nchart1..."],
        ["chart", "nchart1..."] 
    ]
}

It looks like a NIP-51 parameterized replaceable list event.

[ShinoharaTa]

@AsaiToshiya

I would like an example of what the events should look like if we change the tags and use the standardized l, L, a.

The reason for defining this as nchart1 instead of nevent1 is to make it clear that this event is chart data according to the format.
The purpose is to fix the shape of the data for use as a chart.
We believe that providing a standardized chart reference method is highly reusable for websites, applications, and other batches, and is a way to effectively utilize the fault-tolerant performance of Nostr's distributed network.
If we define it as nevent1 and treat it as a NIP-51 list, how can we determine that this is chart data?

[AsaiToshiya]

I would like an example of what the events should look like if we change the tags and use the standardized l, L, a.

Here is an example:

{
    "kind": 3007x,
    "author": "npub",
    "tags": [
        ["d", "chart_identification_code"],
        ["title", "chart Title"],
        ["L", "#t"],
        ["l", "Server", "#t"],
        ["l", "Temperature", "#t"],
        ["description", ""],
        ["license", ""],
        ["published_at", "ISO Unix Time String"],
        ["a", "30076:fcaec..d590f:abcd"],
        ["a", "30076:fcaec..d590f:efgh"],
        ["a", "30076:fcaec..d590f:ijkl"],
        ...
    ]
}

Please see NIP-32 for more information on l and L tag.

The reason for defining this as nchart1 instead of nevent1 is to make it clear that this event is chart data according to the format. The purpose is to fix the shape of the data for use as a chart.

I don't know the benefit of clarifying as nchart1.... Probably user and client know it is chart data.

We believe that providing a standardized chart reference method is highly reusable for websites, applications, and other batches, and is a way to effectively utilize the fault-tolerant performance of Nostr's distributed network.

nevent1... is the same with it.

If we define it as nevent1 and treat it as a NIP-51 list, how can we determine that this is chart data?

You are right, a new kind must be defined even if the event is a NIP-51 list. So this is a matter of where to write.

[vitorpamplona]

Can you also add an example to what's on "a", "30076:fcaec..d590f:abcd" ?

[AsaiToshiya]

The data defined for this purpose is defined as in NIP-19, nchart1xxxxxx... We believe that it would be more convenient to use if the data defined in this way could be defined as NIP-19 and provided as

What is included in TLV of nchart1...? If it is just a marker, I think nevent1... is enough.

Sorry, nevent1... is mistake, it is naddr1....

Can you also add an example to what's on "a", "30076:fcaec..d590f:abcd" ?

An example is about the same as #743 (comment).

{
    "kind": 30076,
    "author": "fcaec..d590f",
    "tags": [
        ["d", "abcd"],
        ["title", "chart Title"],
        ["L", "#t"],
        ["l", "Server", "#t"],
        ["l", "Temperature", "#t"],
        ["description", ""],
        ["license", ""],
        ["published_at", "ISO Unix Time String"],
        [
            // dataset, "data set name", "x label", "y label", "data as a JSON string of x,y pairs"
            "dataset", "CPU Usage", "Date", "Percentage", "[{x: 1690862400, y: 12.5},{x:.., y:..}]"
        ],
        [
            "dataset", "RAM Usage", "Date", "Percentage", "[{x: 1690862400, y: 45},{x:.., y:..}]"
        ],
        [
            "dataset", "CPU Temp", "Date", "Temperature (C)", "[{x: 1690862400, y: 65},{x:.., y:..}]"
        ],
    ]
}