Feature Proposal: Data Validation

[krixano]

Currently validation of data usually happens in the browser to validate additions to the db before it's added to the appropriate data files - for example, ensuring that a title is not blank. But, a person can very easily get around this by directly modifying the files.

So... what if we moved the validation from before the data-file modification to after the updates to the data files are received to the other peers? Currently, ZeroNet already does something like this for filenames and ids via the content.json file (permissions).

So, I propose that we add a new data validation system where all updates to data files are checked on all clients to ensure the data is validated, and if it is not, then the update is not accepted.

And example of how this could be implemented for the data files that would go into the database is something like this, in the dbschema file:

{
    "db_name": "KxoBlock",
    "db_file": "data/KxoBlock.db",
    "version": 2,
    "maps": {
        "users/.+/data.json": {
            "to_table": ["blocklists"]
        },
        "users/.+/content.json": {
            "to_json_table": ["cert_user_id"],
            "file_name": "data.json"
        }
    },
    "tables": {
        "json": {
		    "cols": [
		        ["json_id", "INTEGER PRIMARY KEY AUTOINCREMENT"],
		        ["directory", "TEXT", "directory"],
		        ["file_name", "TEXT", "file"],
		        ["cert_user_id", "TEXT"]
		    ],
		    "indexes": ["CREATE UNIQUE INDEX path ON json(directory, file_name)"],
		    "schema_changed": 6
        },
        "blocklists": {
            "cols": [
                ["blocklist_id", "INTEGER"],
                ["title", "TEXT", "required|max:255"],
                ["description", "TEXT", "required"],
                ["type", "TEXT", "in:zites,users,both"],
                ["file", "TEXT", "ends_with:json|file"],
                ["tags", "TEXT"],
		["zite", "TEXT", "url"],
		["logo", "TEXT", "image"],
                ["date_updated", "INTEGER"],
                ["date_added", "INTEGER"],
                ["json_id", "INTEGER REFERENCES json (json_id)"]
            ],
            "indexes": ["CREATE UNIQUE INDEX blocklists_index ON blocklists(json_id, blocklist_id)", "CREATE INDEX blocklist_id_title ON blocklists(blocklist_id, title)", "CREATE INDEX blocklist_id_file ON blocklists(blocklist_id, file)", "CREATE INDEX blocklist_tags ON blocklists(blocklist_id, tags)"],
            "schema_changed": 6
        }
    }
}

As you can see, each column array accepts a third argument which is a string of all validation rules delimited by |. I based this on how Laravel's system works: https://laravel.com/docs/6.x/validation

You can see a list of all the rules of validation that Laravel supports: https://laravel.com/docs/6.x/validation#available-validation-rules
I've used some rules that laravel doesn't have, like "file" and "directory". Obviously we don't need to completely follow everything that Laravel has, but it offers a pretty good set to base our system off of.

The other problem is if we only implement this in the dbschema file... then we can only validate data files that are connected to the db... but we might want to validate other json files or other files in general.

I got this idea from a person who commented on issue #2204 and from PeerMessage which actually has something exactly like this - it can filter messages based on the content of the message.

[HelloZeroNet]

It could be useful, but I maybe it's better if it's defined separately from the db layer, so the client can reject the invalid data files.