New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to customize presentation of specific columns in HTML view #153

Closed
ftrain opened this Issue Nov 26, 2017 · 14 comments

Comments

Projects
None yet
2 participants
@ftrain

ftrain commented Nov 26, 2017

This ties into #3 in some ways. It would be great to have some adaptability in the HTML views and to specific some columns as displaying in certain ways.

  • 1. Auto-parsing URIs into in-browser links. Why? Lots of public data around cultural commons stuff links to a specific URL. This would be a great utility to turn on at the command line, just parse everything for URLs. Maybe they need to be underlined or represented in a different way than internal URLs.
  • 2. Ability to identify a column as plain/preformatted text. Why? Was trying to import the Enron emails, the body collapses. Hard to read. These fields also tend to screw up the ability to scan a table view. If you knew it was text the system could set an overflow property on the relevant CSS, so you could still scan.
  • 3. Ability to identify a column as HTML. Why? I want to spider some stuff and drop sections into SQLite, and just keep them as HTML.
@simonw

This comment has been minimized.

Show comment
Hide comment
@simonw

simonw Nov 27, 2017

Owner

I've been thinking about 1. a bit - I actually think it would be fine to have a rule that says "if the contents of the cell starts with http:// or https:// and doesn't contain any whitespace, turn that into a link". If you need the non-linked version that will always be available in the JSON.

For the other two... I think #12 may be the way to go here: if you can easily over-ride the row.html and table.html templates for specific databases you can easily set pre-formatted text or similar for certain values - maybe even with CSS that targets a specific table column.

Owner

simonw commented Nov 27, 2017

I've been thinking about 1. a bit - I actually think it would be fine to have a rule that says "if the contents of the cell starts with http:// or https:// and doesn't contain any whitespace, turn that into a link". If you need the non-linked version that will always be available in the JSON.

For the other two... I think #12 may be the way to go here: if you can easily over-ride the row.html and table.html templates for specific databases you can easily set pre-formatted text or similar for certain values - maybe even with CSS that targets a specific table column.

@simonw

This comment has been minimized.

Show comment
Hide comment
@simonw

simonw Nov 27, 2017

Owner

One quick fix could be to add a extra_css_url key to the metadata.json format (which currently hosts title, license_url etc) - if populated, we can inject a link to that stylesheet on every page. We could add a few classes in strategic places that include the database and table names to give people styling hooks.

While we're at it, an extra_js_url key would let people go really nuts!

Owner

simonw commented Nov 27, 2017

One quick fix could be to add a extra_css_url key to the metadata.json format (which currently hosts title, license_url etc) - if populated, we can inject a link to that stylesheet on every page. We could add a few classes in strategic places that include the database and table names to give people styling hooks.

While we're at it, an extra_js_url key would let people go really nuts!

simonw added a commit that referenced this issue Nov 29, 2017

extra_css_urls/extra_js_urls in metadata, refs #153
A mechanism in the metadata.json format for adding custom CSS and JS urls.

Create a metadata.json file that looks like this:

    {
        "extra_css_urls": [
            "https://simonwillison.net/static/css/all.bf8cd891642c.css"
        ],
        "extra_js_urls": [
            "https://code.jquery.com/jquery-3.2.1.slim.min.js"
        ]
    }

Then start datasette like this:

    datasette mydb.db --metadata=metadata.json

The CSS and JavaScript files will be linked in the <head> of every page.

You can also specify a SRI (subresource integrity hash) for these assets:

    {
        "extra_css_urls": [
            {
                "url": "https://simonwillison.net/static/css/all.bf8cd891642c.css",
                "sri": "sha384-9qIZekWUyjCyDIf2YK1FRoKiPJq4PHt6tp/ulnuuyRBvazd0hG7pWbE99zvwSznI"
            }
        ],
        "extra_js_urls": [
            {
                "url": "https://code.jquery.com/jquery-3.2.1.slim.min.js",
                "sri": "sha256-k2WSCIexGzOj3Euiig+TlR8gA0EmPjuc79OEeY5L45g="
            }
        ]
    }

Modern browsers will only execute the stylsheet or JavaScript if the SRI hash
matches the content served. You can generate hashes using www.srihash.org
@simonw

This comment has been minimized.

Show comment
Hide comment
@simonw

simonw Nov 29, 2017

Owner

@ftrain OK I've shipped the first version of this. Here's the initial documentation:

Create a metadata.json file that looks like this:

{
    "extra_css_urls": [
        "https://simonwillison.net/static/css/all.bf8cd891642c.css"
    ],
    "extra_js_urls": [
        "https://code.jquery.com/jquery-3.2.1.slim.min.js"
    ]
}

Then start datasette like this:

datasette mydb.db --metadata=metadata.json

The CSS and JavaScript files will be linked in the <head> of every page.

You can also specify a SRI (subresource integrity hash) for these assets:

{
    "extra_css_urls": [
        {
            "url": "https://simonwillison.net/static/css/all.bf8cd891642c.css",
            "sri": "sha384-9qIZekWUyjCyDIf2YK1FRoKiPJq4PHt6tp/ulnuuyRBvazd0hG7pWbE99zvwSznI"
        }
    ],
    "extra_js_urls": [
        {
            "url": "https://code.jquery.com/jquery-3.2.1.slim.min.js",
            "sri": "sha256-k2WSCIexGzOj3Euiig+TlR8gA0EmPjuc79OEeY5L45g="
        }
    ]
}

Modern browsers will only execute the stylsheet or JavaScript if the SRI hash
matches the content served. You can generate hashes using www.srihash.org

This isn't shipped in a release yet, but you can still access these features in datasette publish like so:

datasette publish now mydb.db --metadata=metadata.json --branch=master

The --branch=master option will pull the latest master build of Datasette from GitHub.

Owner

simonw commented Nov 29, 2017

@ftrain OK I've shipped the first version of this. Here's the initial documentation:

Create a metadata.json file that looks like this:

{
    "extra_css_urls": [
        "https://simonwillison.net/static/css/all.bf8cd891642c.css"
    ],
    "extra_js_urls": [
        "https://code.jquery.com/jquery-3.2.1.slim.min.js"
    ]
}

Then start datasette like this:

datasette mydb.db --metadata=metadata.json

The CSS and JavaScript files will be linked in the <head> of every page.

You can also specify a SRI (subresource integrity hash) for these assets:

{
    "extra_css_urls": [
        {
            "url": "https://simonwillison.net/static/css/all.bf8cd891642c.css",
            "sri": "sha384-9qIZekWUyjCyDIf2YK1FRoKiPJq4PHt6tp/ulnuuyRBvazd0hG7pWbE99zvwSznI"
        }
    ],
    "extra_js_urls": [
        {
            "url": "https://code.jquery.com/jquery-3.2.1.slim.min.js",
            "sri": "sha256-k2WSCIexGzOj3Euiig+TlR8gA0EmPjuc79OEeY5L45g="
        }
    ]
}

Modern browsers will only execute the stylsheet or JavaScript if the SRI hash
matches the content served. You can generate hashes using www.srihash.org

This isn't shipped in a release yet, but you can still access these features in datasette publish like so:

datasette publish now mydb.db --metadata=metadata.json --branch=master

The --branch=master option will pull the latest master build of Datasette from GitHub.

@simonw

This comment has been minimized.

Show comment
Hide comment
@simonw

simonw Nov 29, 2017

Owner

To style individual columns you'll currently need to use the nth-of-type selector, e.g.:

td:nth-of-type(5):before { white-space: pre }
Owner

simonw commented Nov 29, 2017

To style individual columns you'll currently need to use the nth-of-type selector, e.g.:

td:nth-of-type(5):before { white-space: pre }
@simonw

This comment has been minimized.

Show comment
Hide comment
@simonw

simonw Nov 29, 2017

Owner

(This only addresses point 2 in your issue description - points 1 and point 3 are still to come)

Owner

simonw commented Nov 29, 2017

(This only addresses point 2 in your issue description - points 1 and point 3 are still to come)

simonw added a commit that referenced this issue Nov 29, 2017

@simonw

This comment has been minimized.

Show comment
Hide comment
@simonw

simonw Nov 29, 2017

Owner

OK, that's point 1 covered.

Owner

simonw commented Nov 29, 2017

OK, that's point 1 covered.

simonw added a commit that referenced this issue Nov 30, 2017

CSS styling hooks as classes on the body
Refs #153

Every template now gets CSS classes in the body designed to support custom
styling.

The index template (the top level page at /) gets this:

    <body class="index">

The database template (/dbname/) gets this:

    <body class="db db-dbname">

The table template (/dbname/tablename) gets:

    <body class="table db-dbname table-tablename">

The row template (/dbname/tablename/rowid) gets:

    <body class="row db-dbname table-tablename">

The db-x and table-x classes use the database or table names themselves IF
they are valid CSS identifiers. If they aren't, we strip any invalid
characters out and append a 6 character md5 digest of the original name, in
order to ensure that multiple tables which resolve to the same stripped
character version still have different CSS classes.

Some examples (extracted from the unit tests):

    "simple" => "simple"
    "MixedCase" => "MixedCase"
    "-no-leading-hyphens" => "no-leading-hyphens-65bea6"
    "_no-leading-underscores" => "no-leading-underscores-b921bc"
    "no spaces" => "no-spaces-7088d7"
    "-" => "336d5e"
    "no $ characters" => "no--characters-59e024"
@simonw

This comment has been minimized.

Show comment
Hide comment
@simonw

simonw Nov 30, 2017

Owner

Every template now gets CSS classes in the body designed to support custom
styling.

The index template (the top level page at /) gets this:

<body class="index">

The database template (/dbname/) gets this:

<body class="db db-dbname">

The table template (/dbname/tablename) gets:

<body class="table db-dbname table-tablename">

The row template (/dbname/tablename/rowid) gets:

<body class="row db-dbname table-tablename">

The db-x and table-x classes use the database or table names themselves IF
they are valid CSS identifiers. If they aren't, we strip any invalid
characters out and append a 6 character md5 digest of the original name, in
order to ensure that multiple tables which resolve to the same stripped
character version still have different CSS classes.

Some examples (extracted from the unit tests):

"simple" => "simple"
"MixedCase" => "MixedCase"
"-no-leading-hyphens" => "no-leading-hyphens-65bea6"
"_no-leading-underscores" => "no-leading-underscores-b921bc"
"no spaces" => "no-spaces-7088d7"
"-" => "336d5e"
"no $ characters" => "no--characters-59e024"
Owner

simonw commented Nov 30, 2017

Every template now gets CSS classes in the body designed to support custom
styling.

The index template (the top level page at /) gets this:

<body class="index">

The database template (/dbname/) gets this:

<body class="db db-dbname">

The table template (/dbname/tablename) gets:

<body class="table db-dbname table-tablename">

The row template (/dbname/tablename/rowid) gets:

<body class="row db-dbname table-tablename">

The db-x and table-x classes use the database or table names themselves IF
they are valid CSS identifiers. If they aren't, we strip any invalid
characters out and append a 6 character md5 digest of the original name, in
order to ensure that multiple tables which resolve to the same stripped
character version still have different CSS classes.

Some examples (extracted from the unit tests):

"simple" => "simple"
"MixedCase" => "MixedCase"
"-no-leading-hyphens" => "no-leading-hyphens-65bea6"
"_no-leading-underscores" => "no-leading-underscores-b921bc"
"no spaces" => "no-spaces-7088d7"
"-" => "336d5e"
"no $ characters" => "no--characters-59e024"

@simonw simonw changed the title from Add some conditional formatting per-column to HTML view to Ability to customize presentation of specific columns in HTML view Nov 30, 2017

simonw added a commit that referenced this issue Nov 30, 2017

Removed dependency on sanic-jinja2
I wasn't using any of the functionality it adds on top of raw Jinja2.

Refs #12 and #153

simonw added a commit that referenced this issue Nov 30, 2017

git commit -m "datasette --template-dir=mytemplates/" argument
You can now pass an additional argument specifying a directory to look for
custom templates in.

Datasette will fall back on the default templates if a template is not
found in that directory.

Refs #12, #153

simonw added a commit that referenced this issue Nov 30, 2017

Ability to over-ride templates for individual tables/databases
It is now possible to over-ride templates on a per-database / per-row or per-
table basis.

When you access e.g. /mydatabase/mytable Datasette will look for the following:

    - table-mydatabase-mytable.html
    - table.html

If you provided a --template-dir argument to datasette serve it will look in
that directory first.

The lookup rules are as follows:

    Index page (/):
        index.html

    Database page (/mydatabase):
        database-mydatabase.html
        database.html

    Table page (/mydatabase/mytable):
        table-mydatabase-mytable.html
        table.html

    Row page (/mydatabase/mytable/id):
        row-mydatabase-mytable.html
        row.html

If a table name has spaces or other unexpected characters in it, the template
filename will follow the same rules as our custom <body> CSS classes
introduced in 8ab3a16 - for example, a table called "Food Trucks"
will attempt to load the following templates:

    table-mydatabase-Food-Trucks-399138.html
    table.html

It is possible to extend the default templates using Jinja template
inheritance. If you want to customize EVERY row template with some additional
content you can do so by creating a row.html template like this:

    {% extends "default:row.html" %}

    {% block content %}
    <h1>EXTRA HTML AT THE TOP OF THE CONTENT BLOCK</h1>
    <p>This line renders the original block:</p>
    {{ super() }}
    {% endblock %}

Closes #12, refs #153
@simonw

This comment has been minimized.

Show comment
Hide comment
@simonw

simonw Nov 30, 2017

Owner

It is now possible to over-ride templates on a per-database / per-row or per-
table basis.

When you access e.g. /mydatabase/mytable Datasette will look for the following:

- table-mydatabase-mytable.html
- table.html

If you provided a --template-dir argument to datasette serve it will look in
that directory first.

The lookup rules are as follows:

Index page (/):
    index.html

Database page (/mydatabase):
    database-mydatabase.html
    database.html

Table page (/mydatabase/mytable):
    table-mydatabase-mytable.html
    table.html

Row page (/mydatabase/mytable/id):
    row-mydatabase-mytable.html
    row.html

If a table name has spaces or other unexpected characters in it, the template
filename will follow the same rules as our custom <body> CSS classes
introduced in 8ab3a16 - for example, a table called "Food Trucks"
will attempt to load the following templates:

table-mydatabase-Food-Trucks-399138.html
table.html

It is possible to extend the default templates using Jinja template
inheritance. If you want to customize EVERY row template with some additional
content you can do so by creating a row.html template like this:

{% extends "default:row.html" %}

{% block content %}
<h1>EXTRA HTML AT THE TOP OF THE CONTENT BLOCK</h1>
<p>This line renders the original block:</p>
{{ super() }}
{% endblock %}
Owner

simonw commented Nov 30, 2017

It is now possible to over-ride templates on a per-database / per-row or per-
table basis.

When you access e.g. /mydatabase/mytable Datasette will look for the following:

- table-mydatabase-mytable.html
- table.html

If you provided a --template-dir argument to datasette serve it will look in
that directory first.

The lookup rules are as follows:

Index page (/):
    index.html

Database page (/mydatabase):
    database-mydatabase.html
    database.html

Table page (/mydatabase/mytable):
    table-mydatabase-mytable.html
    table.html

Row page (/mydatabase/mytable/id):
    row-mydatabase-mytable.html
    row.html

If a table name has spaces or other unexpected characters in it, the template
filename will follow the same rules as our custom <body> CSS classes
introduced in 8ab3a16 - for example, a table called "Food Trucks"
will attempt to load the following templates:

table-mydatabase-Food-Trucks-399138.html
table.html

It is possible to extend the default templates using Jinja template
inheritance. If you want to customize EVERY row template with some additional
content you can do so by creating a row.html template like this:

{% extends "default:row.html" %}

{% block content %}
<h1>EXTRA HTML AT THE TOP OF THE CONTENT BLOCK</h1>
<p>This line renders the original block:</p>
{{ super() }}
{% endblock %}
@simonw

This comment has been minimized.

Show comment
Hide comment
@simonw

simonw Nov 30, 2017

Owner

Remaining work on this now lives in a milestone: https://github.com/simonw/datasette/milestone/6

Owner

simonw commented Nov 30, 2017

Remaining work on this now lives in a milestone: https://github.com/simonw/datasette/milestone/6

@ftrain

This comment has been minimized.

Show comment
Hide comment
@ftrain

ftrain Nov 30, 2017

ftrain commented Nov 30, 2017

@simonw

This comment has been minimized.

Show comment
Hide comment
@simonw
Owner

simonw commented Nov 30, 2017

@simonw

This comment has been minimized.

Show comment
Hide comment
@simonw

simonw Dec 7, 2017

Owner

In #159 I added a mechanism for easily customizing per-column displays, and I've added documentation showing an example of using this mechanism to set certain columns to display as unescaped HTML: http://datasette.readthedocs.io/en/latest/custom_templates.html#custom-templates

This fixes item 3, so I'm closing this ticket!

Owner

simonw commented Dec 7, 2017

In #159 I added a mechanism for easily customizing per-column displays, and I've added documentation showing an example of using this mechanism to set certain columns to display as unescaped HTML: http://datasette.readthedocs.io/en/latest/custom_templates.html#custom-templates

This fixes item 3, so I'm closing this ticket!

@simonw simonw closed this Dec 7, 2017

@simonw

This comment has been minimized.

Show comment
Hide comment
@simonw

simonw Dec 10, 2017

Owner

@ftrain Datasette 0.14 is now released with all of the above: https://github.com/simonw/datasette/releases/tag/0.14

Owner

simonw commented Dec 10, 2017

@ftrain Datasette 0.14 is now released with all of the above: https://github.com/simonw/datasette/releases/tag/0.14

@simonw

This comment has been minimized.

Show comment
Hide comment
@simonw

simonw Dec 10, 2017

Owner

Also worth mentioning: as of #160 and #157 the datasette publish now, datasette publish heroku and datasette package commands all know how to bundle up any --static or --template-dir content and include it in the Docker image / Heroku/Now deployment that gets generated.

Owner

simonw commented Dec 10, 2017

Also worth mentioning: as of #160 and #157 the datasette publish now, datasette publish heroku and datasette package commands all know how to bundle up any --static or --template-dir content and include it in the Docker image / Heroku/Now deployment that gets generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment