Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

accept bare URL definitions for sql tables #1098

Merged
merged 7 commits into from Mar 18, 2024
Merged

accept bare URL definitions for sql tables #1098

merged 7 commits into from Mar 18, 2024

Conversation

Fil
Copy link
Contributor

@Fil Fil commented Mar 18, 2024

---
sql:
  adresses: https://static.data.gouv.fr/resources/bureaux-de-vote-et-adresses-de-leurs-electeurs/20230626-135723/table-adresses-reu.parquet
  quakes: https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_day.csv
  gaia: https://uwdata.github.io/mosaic-datasets/data/gaia-5m.parquet
---

```sql echo
SELECT COUNT() FROM quakes;
```

etc.

closes #1071

I believe this is more a bugfix than a new feature (we previously returned /https:/static.data.gouv.fr/resources/burea……….parquet).

Copy link
Member

@mbostock mbostock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure this is the right place to do this — normally we only call resolveFile on things that we know are files (unlike resolveImport, I guess). Look for calls to isAssetPath, which we should also use here. I’ll take a look and see if we can consolidate. Good find on the fix!

@Fil
Copy link
Contributor Author

Fil commented Mar 18, 2024

The caller is renderFiles

function renderFiles(files: Iterable<string>, resolve: (name: string) => string, getLastModified): string {

called where we build the registerTable client code.

}${data?.sql ? `\nimport {registerTable} from ${JSON.stringify(resolveImport("npm:@observablehq/duckdb"))};` : ""}${

@mbostock
Copy link
Member

Yeah, we don’t want to declare a FileAttachment with an external URL, I imagine. Interesting that it works though.

@mbostock mbostock mentioned this pull request Mar 18, 2024
Copy link
Member

@mbostock mbostock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switched to using the FROM-first syntax, e.g.,

CREATE TABLE 'quakes' AS FROM 'https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_day.csv';

docs/sql.md Outdated Show resolved Hide resolved
@Fil Fil enabled auto-merge (squash) March 18, 2024 22:37
@Fil Fil merged commit 777b635 into main Mar 18, 2024
4 checks passed
@Fil Fil deleted the fil/sql-http branch March 18, 2024 22:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support for sql remote URLs
2 participants