Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add bsoup module #853

Merged
merged 1 commit into from Apr 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/modules.md
Expand Up @@ -28,6 +28,7 @@ individual modules, please refer to the Starlib documentation.

| Module | Description |
| --- | --- |
| [`bsoup.star`](https://github.com/qri-io/starlib/blob/master/bsoup) | Beautiful Soup-like functions for HTML |
| [`compress/gzip.star`](https://github.com/qri-io/starlib/blob/master/compress/gzip) | gzip decompressing |
| [`compress/zipfile.star`](https://github.com/qri-io/starlib/blob/master/zipfile) | zip decompressing |
| [`encoding/base64.star`](https://github.com/qri-io/starlib/tree/master/encoding/base64) | Base 64 encoding and decoding |
Expand Down
1 change: 1 addition & 0 deletions go.mod
Expand Up @@ -60,6 +60,7 @@ require (
github.com/cloudflare/circl v1.3.7 // indirect
github.com/cyphar/filepath-securejoin v0.2.4 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/dustmop/soup v1.1.2-0.20190516214245-38228baa104e // indirect
github.com/emirpasic/gods v1.18.1 // indirect
github.com/gabriel-vasile/mimetype v1.4.2 // indirect
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
Expand Down
1 change: 1 addition & 0 deletions go.sum
Expand Up @@ -73,6 +73,7 @@ github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZm
github.com/dgryski/go-sip13 v0.0.0-20181026042036-e10d5fee7954/go.mod h1:vAd38F8PWV+bWy6jNmig1y/TA+kYO4g3RSRF0IAv0no=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/dustmop/soup v1.1.2-0.20190516214245-38228baa104e h1:44fmjqDtdCiUNlSjJVp+w1AOs6na3Y6Ai0aIeseFjkI=
github.com/dustmop/soup v1.1.2-0.20190516214245-38228baa104e/go.mod h1:CgNC6SGbT+Xb8wGGvzilttZL1mc5sQ/5KkcxsZttMIk=
github.com/elazarl/goproxy v0.0.0-20230808193330-2592e75ae04a h1:mATvB/9r/3gvcejNsXKSkQ6lcIaNec2nyfOdlTBR2lU=
github.com/elazarl/goproxy v0.0.0-20230808193330-2592e75ae04a/go.mod h1:Ro8st/ElPeALwNFlcTpWmkr6IoMFfkjXAvTHpevnDsM=
Expand Down
4 changes: 4 additions & 0 deletions runtime/applet.go
Expand Up @@ -13,6 +13,7 @@ import (

starlibgzip "github.com/qri-io/starlib/compress/gzip"
starlibbase64 "github.com/qri-io/starlib/encoding/base64"
starlibbsoup "github.com/qri-io/starlib/bsoup"
starlibcsv "github.com/qri-io/starlib/encoding/csv"
starlibhash "github.com/qri-io/starlib/hash"
starlibhtml "github.com/qri-io/starlib/html"
Expand Down Expand Up @@ -510,6 +511,9 @@ func (a *Applet) loadModule(thread *starlark.Thread, module string) (starlark.St
case "xpath.star":
return xpath.LoadXPathModule()

case "bsoup.star":
return starlibbsoup.LoadModule()

case "compress/gzip.star":
return starlark.StringDict{
starlibgzip.Module.Name: starlibgzip.Module,
Expand Down
3 changes: 3 additions & 0 deletions runtime/applet_test.go
Expand Up @@ -221,6 +221,7 @@ func TestModuleLoading(t *testing.T) {
// Our basic set of modules can be imported
src := `
load("render.star", "render")
load("bsoup.star", "bsoup")
load("encoding/base64.star", "base64")
load("encoding/json.star", "json")
load("http.star", "http")
Expand All @@ -245,6 +246,8 @@ def main():
fail("re broken")
if time.parse_duration("10s").seconds != 10:
fail("time broken")
if bsoup.parseHtml("<h1>foo</h1>").find("h1").get_text() != "foo":
fail("bsoup broken")
return render.Root(child=render.Box())
`
app, err := NewApplet("test.star", []byte(src))
Expand Down