refactor deploy html #362

bcwu · 2023-03-09T00:31:09Z

Description

Refactor deploy html based on the new Manifest and Bundle classes
Add missing unit test coverage for deploy html

- make_html_bundle - make_html_bundle_content

bcwu · 2023-03-09T00:33:53Z

fixes #361

mmarchetti · 2023-03-09T15:44:47Z

I still get a 404 when deploying static content with an index not named index.html.

I deployed with

rsconnect deploy html -n local ./static-test/

and

rsconnect deploy html -n local --entrypoint news.html ./static-test/

bcwu · 2023-03-09T17:47:02Z

also, fixes #359

kgartland-rstudio · 2023-03-09T20:47:23Z

kgartland-rstudio · 2023-03-09T20:48:51Z

When specifying an entrypoint that doesn't exist we don't catch the error and give a reasonable response, instead we throw a NoneType Err:

In this example the --entrypoint blue doesn't exist.

> rsconnect deploy html html-multi-file/index.html --entrypoint blue -n dogfood -t "multiple html" -N
Validating server... 	[OK]
Validating app mode... 	[OK]
Making bundle ... 	[ERROR]: expected str, bytes or os.PathLike object, not NoneType
Traceback (most recent call last):
  File "/Users/kgartland/.pyenv/versions/3.8.2/envs/rsconnect-tag/lib/python3.8/site-packages/rsconnect/main.py", line 95, in wrapper
    result = func(*args, **kwargs)
  File "/Users/kgartland/.pyenv/versions/3.8.2/envs/rsconnect-tag/lib/python3.8/site-packages/rsconnect/main.py", line 1188, in deploy_html
    ce.validate_server()
  File "/Users/kgartland/.pyenv/versions/3.8.2/envs/rsconnect-tag/lib/python3.8/site-packages/rsconnect/log.py", line 186, in wrapper
    result = method(self, *args, **kw)
  File "/Users/kgartland/.pyenv/versions/3.8.2/envs/rsconnect-tag/lib/python3.8/site-packages/rsconnect/api.py", line 636, in make_bundle
    bundle = func(*args, **kwargs)
  File "/Users/kgartland/.pyenv/versions/3.8.2/envs/rsconnect-tag/lib/python3.8/site-packages/rsconnect/bundle.py", line 925, in make_html_bundle
    manifest = create_html_manifest(**locals())
  File "/Users/kgartland/.pyenv/versions/3.8.2/envs/rsconnect-tag/lib/python3.8/site-packages/rsconnect/bundle.py", line 890, in create_html_manifest
    excludes.extend(list_environment_dirs(deploy_dir))
  File "/Users/kgartland/.pyenv/versions/3.8.2/envs/rsconnect-tag/lib/python3.8/site-packages/rsconnect/bundle.py", line 792, in list_environment_dirs
    path = join(directory, name)
  File "/Users/kgartland/.pyenv/versions/3.8.2/lib/python3.8/posixpath.py", line 76, in join
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType
Internal error: expected str, bytes or os.PathLike object, not NoneType

kgartland-rstudio · 2023-03-09T20:53:29Z

When specifying two files in an html deployment, from outside the directory, we state:
No such file or directory, even when the file exists:

With this file structure:

-> tree html-multi-file/
html-multi-file/
├── index.html
├── package.json
├── rsconnect-python
│   └── html-multi-file.json
├── sandbox.config.json
└── second.html

From outside the directory:

> rsconnect deploy html html-multi-file/index.html html-multi-file/second.html -n dogfood -t "multiple html" -N
Validating server... 	[OK]
Validating app mode... 	[OK]
Making bundle ... 	[ERROR]: Unable to include the file second.html in the bundle: No such file or directory
Error: Unable to include the file second.html in the bundle: No such file or directory

When specifying two files in an html deployment, from inside the directory, we state:
Internal error: 'second.html' does not start with [PATH]:

> rsconnect deploy html index.html second.html -n dogfood -t "multiple html" -N
Validating server... 	[OK]
Validating app mode... 	[OK]
Making bundle ... 	[ERROR]: 'second.html' does not start with '/Users/kgartland/work/connect/test/rsconnect-python/bundles/html-multi-file'
Traceback (most recent call last):
  File "/Users/kgartland/.pyenv/versions/3.8.2/envs/rsconnect-tag/lib/python3.8/site-packages/rsconnect/main.py", line 95, in wrapper
    result = func(*args, **kwargs)
  File "/Users/kgartland/.pyenv/versions/3.8.2/envs/rsconnect-tag/lib/python3.8/site-packages/rsconnect/main.py", line 1188, in deploy_html
    ce.validate_server()
  File "/Users/kgartland/.pyenv/versions/3.8.2/envs/rsconnect-tag/lib/python3.8/site-packages/rsconnect/log.py", line 186, in wrapper
    result = method(self, *args, **kw)
  File "/Users/kgartland/.pyenv/versions/3.8.2/envs/rsconnect-tag/lib/python3.8/site-packages/rsconnect/api.py", line 636, in make_bundle
    bundle = func(*args, **kwargs)
  File "/Users/kgartland/.pyenv/versions/3.8.2/envs/rsconnect-tag/lib/python3.8/site-packages/rsconnect/bundle.py", line 941, in make_html_bundle
    return bundle.to_file()
  File "/Users/kgartland/.pyenv/versions/3.8.2/envs/rsconnect-tag/lib/python3.8/site-packages/rsconnect/bundle.py", line 274, in to_file
    rel_path = Path(fp).relative_to(self.deploy_dir) if flatten_to_deploy_dir else None
  File "/Users/kgartland/.pyenv/versions/3.8.2/lib/python3.8/pathlib.py", line 899, in relative_to
    raise ValueError("{!r} does not start with {!r}"
ValueError: 'second.html' does not start with '/Users/kgartland/work/connect/test/rsconnect-python/bundles/html-multi-file'
Internal error: 'second.html' does not start with '/Users/kgartland/work/connect/test/rsconnect-python/bundles/html-multi-file'

The same error is presented, even when providing the full path of the second.html file as the error suggests:

> rsconnect deploy html index.html /Users/kgartland/work/connect/test/rsconnect-python/bundles/html-multi-file/second.html -n dogfood -t "multiple html" -N
Validating server... 	[OK]
Validating app mode... 	[OK]
Making bundle ... 	[ERROR]: 'second.html' does not start with '/Users/kgartland/work/connect/test/rsconnect-python/bundles/html-multi-file'
Traceback (most recent call last):
  File "/Users/kgartland/.pyenv/versions/3.8.2/envs/rsconnect-tag/lib/python3.8/site-packages/rsconnect/main.py", line 95, in wrapper
    result = func(*args, **kwargs)
  File "/Users/kgartland/.pyenv/versions/3.8.2/envs/rsconnect-tag/lib/python3.8/site-packages/rsconnect/main.py", line 1188, in deploy_html
    ce.validate_server()
  File "/Users/kgartland/.pyenv/versions/3.8.2/envs/rsconnect-tag/lib/python3.8/site-packages/rsconnect/log.py", line 186, in wrapper
    result = method(self, *args, **kw)
  File "/Users/kgartland/.pyenv/versions/3.8.2/envs/rsconnect-tag/lib/python3.8/site-packages/rsconnect/api.py", line 636, in make_bundle
    bundle = func(*args, **kwargs)
  File "/Users/kgartland/.pyenv/versions/3.8.2/envs/rsconnect-tag/lib/python3.8/site-packages/rsconnect/bundle.py", line 941, in make_html_bundle
    return bundle.to_file()
  File "/Users/kgartland/.pyenv/versions/3.8.2/envs/rsconnect-tag/lib/python3.8/site-packages/rsconnect/bundle.py", line 274, in to_file
    rel_path = Path(fp).relative_to(self.deploy_dir) if flatten_to_deploy_dir else None
  File "/Users/kgartland/.pyenv/versions/3.8.2/lib/python3.8/pathlib.py", line 899, in relative_to
    raise ValueError("{!r} does not start with {!r}"
ValueError: 'second.html' does not start with '/Users/kgartland/work/connect/test/rsconnect-python/bundles/html-multi-file'
Internal error: 'second.html' does not start with '/Users/kgartland/work/connect/test/rsconnect-python/bundles/html-multi-file'

bcwu · 2023-03-09T23:16:54Z

When specifying an entrypoint that doesn't exist we don't catch the error and give a reasonable response, instead we throw a NoneType Err:

fixed by f5a2146

When specifying two files in an html deployment, from outside the directory, we state: No such file or directory, even when the file exists:

With this file structure:
-> tree html-multi-file/
html-multi-file/
├── index.html
├── package.json
├── rsconnect-python
│   └── html-multi-file.json
├── sandbox.config.json
└── second.html

fixed by 2f8703a

kgartland-rstudio · 2023-03-10T14:51:45Z

When specifying an entrypoint that doesn't exist we don't catch the error and give a reasonable response, instead we throw a NoneType Err:

fixed by f5a2146
When specifying two files in an html deployment, from outside the directory, we state: No such file or directory, even when the file exists:
With this file structure:
-> tree html-multi-file/
html-multi-file/
├── index.html
├── package.json
├── rsconnect-python
│   └── html-multi-file.json
├── sandbox.config.json
└── second.html
fixed by 2f8703a

These fixes look good.

kgartland-rstudio · 2023-03-10T15:48:26Z

Windows testing. When multiple files are in the target directory it fails with this error:

> rsconnect deploy html .\html\html-single-file\ -s https://rsc.radixu.com/ -k APIKEY
Validating server...    [OK]
Validating app mode...  [OK]
Making bundle ...       [ERROR]: 'html\\html-single-file\\nothin.html' is not in the subpath of 'C:\\Users\\User\\html\\html-single-file' OR one path is relative and the other is absolute.
Traceback (most recent call last):
  File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\rsconnect\main.py", line 95, in wrapper
    result = func(*args, **kwargs)
  File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\rsconnect\main.py", line 1190, in deploy_html
    .make_bundle(
  File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\rsconnect\log.py", line 186, in wrapper
    result = method(self, *args, **kw)
  File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\rsconnect\api.py", line 636, in make_bundle
    bundle = func(*args, **kwargs)
  File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\rsconnect\bundle.py", line 940, in make_html_bundle
    return bundle.to_file()
  File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\rsconnect\bundle.py", line 274, in to_file
    rel_path = Path(fp).relative_to(self.deploy_dir) if flatten_to_deploy_dir else None
  File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\pathlib.py", line 818, in relative_to
    raise ValueError("{!r} is not in the subpath of {!r}"
ValueError: 'html\\html-single-file\\nothin.html' is not in the subpath of 'C:\\Users\\User\\html\\html-single-file' OR one path is relative and the other is absolute.
Internal error: 'html\\html-single-file\\nothin.html' is not in the subpath of 'C:\\Users\\User\\html\\html-single-file' OR one path is relative and the other is absolute.

Directory structure:

    Directory: C:\Users\User\html\html-single-file


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----         3/10/2023   6:58 AM                rsconnect-python
-a----         3/10/2023   6:58 AM            144 index.html
------         3/10/2023   6:58 AM              0 nothin.html

kgartland-rstudio · 2023-03-10T16:14:33Z

When specifying an entrypoint where multiple html files exist...

do we expect to require the path to the file? or just the file name itself? Since we're giving the path to the deployment directory, I would expect that the path wouldn't be required in the entrypoint arg.

> rsconnect deploy html bundles/html-multi-file/ --entrypoint second.html -n dogfood
Validating server... 	[OK]
Validating app mode... 	[OK]
Making bundle ... 	[ERROR]: Entrypoint second.html does not exist.
Error: Entrypoint second.html does not exist.

Directory structure:

> tree bundles/html-multi-file/
bundles/html-multi-file/
├── index.html
├── package.json
├── rsconnect-python
│   └── html-multi-file.json
├── sandbox.config.json
└── second.html

When you do specify the full path in the entrypoint arg, it fails:

> rsconnect deploy html bundles/html-multi-file/ --entrypoint bundles/html-multi-file/second.html -n dogfood
Validating server... 	[OK]
Validating app mode... 	[OK]
Making bundle ... 	[ERROR]: 'bundles/html-multi-file/index.html' does not start with '/Users/kgartland/work/connect/test/rsconnect-python/bundles/html-multi-file'
Traceback (most recent call last):
  File "/Users/kgartland/.pyenv/versions/3.8.2/envs/rsconnect-tag/lib/python3.8/site-packages/rsconnect/main.py", line 95, in wrapper
    result = func(*args, **kwargs)
  File "/Users/kgartland/.pyenv/versions/3.8.2/envs/rsconnect-tag/lib/python3.8/site-packages/rsconnect/main.py", line 1188, in deploy_html
    ce.validate_server()
  File "/Users/kgartland/.pyenv/versions/3.8.2/envs/rsconnect-tag/lib/python3.8/site-packages/rsconnect/log.py", line 186, in wrapper
    result = method(self, *args, **kw)
  File "/Users/kgartland/.pyenv/versions/3.8.2/envs/rsconnect-tag/lib/python3.8/site-packages/rsconnect/api.py", line 636, in make_bundle
    bundle = func(*args, **kwargs)
  File "/Users/kgartland/.pyenv/versions/3.8.2/envs/rsconnect-tag/lib/python3.8/site-packages/rsconnect/bundle.py", line 940, in make_html_bundle
    return bundle.to_file()
  File "/Users/kgartland/.pyenv/versions/3.8.2/envs/rsconnect-tag/lib/python3.8/site-packages/rsconnect/bundle.py", line 274, in to_file
    rel_path = Path(fp).relative_to(self.deploy_dir) if flatten_to_deploy_dir else None
  File "/Users/kgartland/.pyenv/versions/3.8.2/lib/python3.8/pathlib.py", line 899, in relative_to
    raise ValueError("{!r} does not start with {!r}"
ValueError: 'bundles/html-multi-file/index.html' does not start with '/Users/kgartland/work/connect/test/rsconnect-python/bundles/html-multi-file'
Internal error: 'bundles/html-multi-file/index.html' does not start with '/Users/kgartland/work/connect/test/rsconnect-python/bundles/html-multi-file'

remove create_abspath_list

bcwu · 2023-03-10T16:35:35Z

Windows testing. When multiple files are in the target directory it fails with this error:
> rsconnect deploy html .\html\html-single-file\ -s https://rsc.radixu.com/ -k APIKEY

fixed by 0415875

When specifying an entrypoint where multiple html files exist...

1. do we expect to require the path to the file? or just the file name itself?  Since we're giving the path to the deployment directory, I would expect that the path wouldn't be required in the entrypoint arg.

This depends on where you are deploying the files. If you are outside the folder, you have to give it the full path.

2. When you _do_ specify the full path in the entrypoint arg, it fails:

fixed by 0415875

kgartland-rstudio · 2023-03-10T16:47:43Z

It looks like this part is still broken:

do we expect to require the path to the file? or just the file name itself? Since we're giving the path to the deployment directory, I would expect that the path wouldn't be required in the entrypoint arg.

> rsconnect deploy html bundles/html-multi-file/ --entrypoint second.html -n dogfood
Validating server... 	[OK]
Validating app mode... 	[OK]
Making bundle ... 	[ERROR]: Entrypoint second.html does not exist.
Error: Entrypoint second.html does not exist.

bcwu · 2023-03-10T16:52:18Z

It looks like this part is still broken:

do we expect to require the path to the file? or just the file name itself? Since we're giving the path to the deployment directory, I would expect that the path wouldn't be required in the entrypoint arg.
> rsconnect deploy html bundles/html-multi-file/ --entrypoint second.html -n dogfood
Validating server... 	[OK]
Validating app mode... 	[OK]
Making bundle ... 	[ERROR]: Entrypoint second.html does not exist.
Error: Entrypoint second.html does not exist.

The full path has to be specified for the entrypoint if you are outside the folder. So in this case:

rsconnect deploy html bundles/html-multi-file/ --entrypoint bundles/html-multi-file/second.html -n dogfood

The other way is to do relative deploy, like so:

cd html bundles/html-multi-file/
rsconnect deploy html ./ --entrypoint second.html -n dogfood

kgartland-rstudio · 2023-03-10T17:06:50Z

It looks like this part is still broken:

do we expect to require the path to the file? or just the file name itself? Since we're giving the path to the deployment directory, I would expect that the path wouldn't be required in the entrypoint arg.
> rsconnect deploy html bundles/html-multi-file/ --entrypoint second.html -n dogfood
Validating server... 	[OK]
Validating app mode... 	[OK]
Making bundle ... 	[ERROR]: Entrypoint second.html does not exist.
Error: Entrypoint second.html does not exist.
The full path has to be specified for the entrypoint if you are outside the folder. So in this case:
rsconnect deploy html bundles/html-multi-file/ --entrypoint bundles/html-multi-file/second.html -n dogfood
The other way is to do relative deploy, like so:
cd html bundles/html-multi-file/
rsconnect deploy html ./ --entrypoint second.html -n dogfood

I realize it works as expected when you're in the directory you're trying to deploy. However that's not the way other content-types work. Are we fine with deviating away from how other deployment types work for html using --entrypoint?

We'll at least have to document around that.

bcwu · 2023-03-10T17:59:49Z

We'll at least have to document around that.

Added documentation on the deploy html behavior in the changelog

Let me know if there's anything that's missed.

CHANGELOG.md

kgartland-rstudio · 2023-03-10T18:51:43Z

Testing on Windows is complete. I think we're good to merge 👍

rsconnect/bundle.py

bcwu added 7 commits March 8, 2023 11:11

add absolute path file list and use it for voila

5566262

return list instead of string when default found

e686448

add deploy html unittest files

afd7d28

deprecate legacy deploy_html functions

d3af4c2

- make_html_bundle - make_html_bundle_content

update infer_entrypoint_candidates to return absolute paths

be57ea1

refactor deploy_html with Manifest and Bundle

07a55c6

add unit tests for deploy_html manifest

32bd359

bcwu force-pushed the bcwu-refactor-deploy-html branch from 010c686 to 32bd359 Compare March 9, 2023 00:32

bcwu requested review from kgartland-rstudio and mmarchetti March 9, 2023 00:34

add primary_html to Manifest

e38cb83

mmarchetti approved these changes Mar 9, 2023

View reviewed changes

bcwu added 2 commits March 9, 2023 17:02

check whether provided path and entrypoint exist

f5a2146

optional absolute paths for validate_extra_files

2f8703a

bcwu force-pushed the bcwu-refactor-deploy-html branch from 8e1f533 to dc011c0 Compare March 10, 2023 16:28

add use_abspath flag to create_file_list

0415875

remove create_abspath_list

bcwu force-pushed the bcwu-refactor-deploy-html branch from dc011c0 to 0415875 Compare March 10, 2023 16:32

add unit tests for make_html_bundle

43b49df

bcwu force-pushed the bcwu-refactor-deploy-html branch from 4b4b312 to 43b49df Compare March 10, 2023 17:06

bcwu force-pushed the bcwu-refactor-deploy-html branch 3 times, most recently from 7b5c7db to 2740862 Compare March 10, 2023 17:55

kgartland-rstudio reviewed Mar 10, 2023

View reviewed changes

CHANGELOG.md Show resolved Hide resolved

bcwu force-pushed the bcwu-refactor-deploy-html branch from 2740862 to 82010ce Compare March 10, 2023 18:47

kgartland-rstudio reviewed Mar 10, 2023

View reviewed changes

rsconnect/bundle.py Outdated Show resolved Hide resolved

bcwu added 2 commits March 10, 2023 13:57

Update changelog

e641d19

correct description

07d182c

bcwu force-pushed the bcwu-refactor-deploy-html branch from 82010ce to 07d182c Compare March 10, 2023 18:57

bcwu merged commit 5b1ea3c into master Mar 10, 2023

bcwu deleted the bcwu-refactor-deploy-html branch March 10, 2023 19:08

refactor deploy html #362

refactor deploy html #362

Uh oh!

Conversation

bcwu commented Mar 9, 2023

Description

Uh oh!

bcwu commented Mar 9, 2023

Uh oh!

mmarchetti commented Mar 9, 2023

Uh oh!

bcwu commented Mar 9, 2023

Uh oh!

kgartland-rstudio commented Mar 9, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kgartland-rstudio commented Mar 9, 2023

Uh oh!

kgartland-rstudio commented Mar 9, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bcwu commented Mar 9, 2023

Uh oh!

kgartland-rstudio commented Mar 10, 2023

Uh oh!

kgartland-rstudio commented Mar 10, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kgartland-rstudio commented Mar 10, 2023

Uh oh!

bcwu commented Mar 10, 2023

Uh oh!

kgartland-rstudio commented Mar 10, 2023

Uh oh!

bcwu commented Mar 10, 2023

Uh oh!

kgartland-rstudio commented Mar 10, 2023

Uh oh!

bcwu commented Mar 10, 2023

Uh oh!

Uh oh!

kgartland-rstudio commented Mar 10, 2023

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

kgartland-rstudio commented Mar 9, 2023 •

edited

Loading

kgartland-rstudio commented Mar 9, 2023 •

edited

Loading

kgartland-rstudio commented Mar 10, 2023 •

edited

Loading