Permission denied after Processing #1

TheClassyTurkey · 2023-12-06T16:57:18Z

I am receiving this permissions error after trying to process the image. I believe the issue is that it is unable to create the folder for the output. I am running this on a mac environment.

/Documents/SD-Main/stable-diffusion-webui/venv/bin/activate: Permission denied
Traceback (most recent call last):
File "/Documents/SD-Main/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/routes.py", line 488, in run_predict
output = await app.get_blocks().process_api(
File "/Documents/SD-Main/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1431, in process_api
result = await self.call_function(
File "/Documents/SD-Main/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1103, in call_function
prediction = await anyio.to_thread.run_sync(
File "/Documents/SD-Main/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/Documents/SD-Main/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "/Documents/SD-Main/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run
result = context.run(func, *args)
File "/Documents/SD-Main/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/utils.py", line 707, in wrapper
response = f(*args, **kwargs)
File "/Documents/SD-Main/stable-diffusion-webui/extensions/sd-webui-old-photo-restoration/scripts/bop.py", line 81, in bop
results = detect_new_results(final_output, cache)
File "/Documents/SD-Main/stable-diffusion-webui/extensions/sd-webui-old-photo-restoration/scripts/bop.py", line 41, in detect_new_results
new_files = {f: os.path.getmtime(os.path.join(output_folder, f)) for f in os.listdir(output_folder)}
FileNotFoundError: [Errno 2] No such file or directory: '/Documents/SD-Main/stable-diffusion-webui/outputs/old-photo-restoration/12.06-10.49.40/final_output'

Haoming02 · 2023-12-07T02:28:18Z

wait wut
GitHub Desktop pls

Haoming02 · 2023-12-07T02:37:48Z

Oh wait. I see the issue now...
It's not related to the output path.

/Documents/SD-Main/stable-diffusion-webui/venv/bin/activate: Permission denied

It wasn't able to start the process, so the output path was not generated to being with...

Haoming02 · 2023-12-07T02:40:11Z

Can you tell me if you're using a Python Virtual Environment?

And if so, how do you activate it? Do you have this file /Documents/SD-Main/stable-diffusion-webui/venv/bin/activate ? Is there some sort of permission issue?

ali0une · 2023-12-09T12:37:57Z

Same issue here : venv/bin/activate: Permission denied.
A1111 on Debian.

Yes A1111 uses a venv to run, it is located at stable-diffusion-webui/venv
To run a python command using this venv you have to :
source /whatever/stable-diffusion-webui/venv/bin/activate
then :
python /whatever/script.py

if i open a terminal

cd stable-diffusion-webui/repositories/BOP-BtL 
source /whatever/stable-diffusion-webui/venv/bin/activate 
python /whatever/stable-diffusion-webui/repositories/BOP-BtL/run.py --GPU 0 --input_folder /whatever/input_folder --output_folder /whatever/stable-diffusion-webui/outputs/old-photo-restoration

it works.

But it doesn't with the current code in the extension.
i don't know how to tell your bop.py script to do what i can do manualy.
Maybe there is a way using A1111 built in functionalities.

Haoming02 · 2023-12-09T14:03:07Z

Please try the new version
Now I also added source in the begining of the commands

ali0une · 2023-12-09T14:56:02Z

Still not working :-/
Updated and run A1111

tried and got :

  File "/whatever/stable-diffusion-webui/extensions/sd-webui-old-photo-restoration/scripts/bop.py", line 85, in bop
    results = [os.path.join(final_output, F) for F in os.listdir(final_output)]
FileNotFoundError: [Errno 2] No such file or directory: '/whatever/stable-diffusion-webui/outputs/old-photo-restoration/12.09-15.31.41/final_output'

Same issue.

Tried to modify a bit bop.py to understand better.

modified like :

    img_path = os.path.abspath(img_path)
    print('img_path: ' + img_path)
    output_path = os.path.abspath(output_path)
    print('output_path :' + output_path)
...
#    process_output = os.path.join(output_path, datetime.datetime.now().strftime("%m.%d-%H.%M.%S"))
#    final_output = os.path.join(process_output, 'final_output')
    process_output = output_path
    print('process_output :' + process_output)
    final_output = output_path
    print('final_output :' + final_output)

Then got this error :
/whatever/stable-diffusion-webui/venv/bin/activate: 1: source: not found

i think source is a bash command to ask the shell to load the venv, but in your script it's the venv python that is trying to launch the source command something like /whatever/stable-diffusion-webui/venv/bin/python source instead of /bin/bash source

Haoming02 · 2023-12-09T16:20:51Z

Hmm, my command is

['source', '~webui/venv/bin/activate', '&&', 'python', 'run.py', ...]`

though

Haoming02 · 2023-12-09T16:22:41Z

Again, the issue is not on the path. The path will exist if it launches the script properly.
Did you still get Permission denied error?

ali0une · 2023-12-09T17:09:05Z

Hmm, my command is

['source', '~webui/venv/bin/activate', '&&', 'python', 'run.py', ...]`

though

The source command is a built-in bash command and you need to get bash to launch it see https://stackoverflow.com/questions/7040592/calling-the-source-command-from-subprocess-popen

Maybe there is another way to execute the /whatever/stable-diffusion-webui/repositories/BOP-BtL/run.py script from an extension.
i don't know, my python knowledge is very limited.

Again, the issue is not on the path. The path will exist if it launches the script properly. Did you still get Permission denied error?

Yes just like i said in #1 (comment)

Haoming02 · 2023-12-11T01:54:23Z

Reddit Discussion

jjlee · 2023-12-11T21:30:28Z

Does this help?

import subprocess
import shlex

def run_in_venv_on_unix(virtualenv_path, command):
    subprocess.check_call(["bash", "-c", f'source {shlex.quote(virtualenv_path)}/bin/activate && exec "$@"', "wrapper_script"] + command)

ali0une · 2023-12-12T17:03:23Z

@jjlee many thanks for your input.

i'd like to integrate this to @Haoming02 code but i have to define some variables like virtualenv_path and wrapper_script right?
i guess command are the parameters given to the run.py script, wrapper_script the run.py script itself, right?
What about virtualenv_path? is it the absolute path to the venv?

jjlee · 2023-12-13T00:21:56Z

You can call run_in_venv_on_unix passing whatever virtualenv path and whatever unix command you like, yes:

run_in_venv_on_unix("/path/to/my/virtualenv_dir", ["python", "whatever.py", "some", "arguments"])

Since virtualenv_path is passed to bash as part of the shell command (the next item in the list after the -c), bash will do whatever it normally does with paths you pass to shell builtin source -- which means if you pass a relative path, it will be treated as relative to the bash process' current working directory (which, the way I've used subprocess, it will inherit from python).

"wrapper_script" is just a listeral string, which is just a placeholder: it could be "eager_frogs" if you prefer. Why? It's not very important and quite confusing (try playing around with this in both python and bash to get it), but have a look at man bash -- quoting from that:

       -c        If the -c option is present, then commands are read from the first non-option argument command_string.  If there are arguments after the command_string, the first argument is assigned to $0 and any  remaining  argu‐
                 ments are assigned to the positional parameters.  The assignment to $0 sets the name of the shell, which is used in warning and error messages.

$0 in bash is the equivalent of python's sys.argv[0], so in that sense it's the "first argument" of the unix command that launched bash. It's worth understanding that shell commands (what bash -c takes as its argument) are strings, but "unix commands" (what the execve system call takes as an argument -- take a look at man -S2 execve) are arrays of strings. That "first item" of a unix command array is special, and in fact execve (2) handles it as a first argument separate from the rest of the array for that reason: what you pass as that argument is the path of the program you're executing (so in bash my-script.sh foo bar, the code in my-script.sh sees $0 as my-script.sh. and $@ is the rest -- in python syntax ["foo", "bar"]). When you call bash -c, bash doesn't have a great choice of string to use for $0 (because it's an inline script, there's no my-script.sh filename to pass to execve's first argument). So bash allows you to specify whatever string you like to pass! (I would guess bash -c doesn't actually call execve, but just interprets the shell command you pass with $0 set to what you asked it to set)

Haoming02 · 2023-12-13T04:02:08Z

Hello, please check out the dev branch~

I now run the functions directly instead of trying to call the scripts. Right now only the first step is implemented. Please test if it works correctly now.

Delete the extension folder first and clone the new dev branch.

Then move the checkpoints folders and the landmarks.dat file from inside the BOP-BtL folder in repositories, to the new extension folder. (The structure is exactly the same) (But only move the models, as I have modified the scripts)

ali0une · 2023-12-17T09:11:09Z

@jjlee once again, many thanks for your input, that's very interesting! i'm copy/pasting this in a .txt file in my tips folder right now!

@Haoming02 the dev branch still fails with exact same errors.
i persist to think you need to think about what @jjlee proposes as i'm pretty sure it's an issue related to how the command is launched via the extension.

i'll use the original repo of BOPBTL with a venv in command line for the moment as this works nice.
Would have been nice to have it integrated and working into A1111 with Linux.

Haoming02 · 2023-12-17T12:17:22Z

@ali0une
The dev branch isn't even using process anymore.
Are you sure you switched correctly?

ali0une · 2023-12-17T12:54:37Z

@ali0une The dev branch isn't even using process anymore. Are you sure you switched correctly?

Hum ... i thought i uninstalled the master branch of the extension properly before installing the dev branch but seems like not. Sorry, my bad.

OK so we have a new error :

Traceback (most recent call last):
  File "/whatever/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "/whatever/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "/whatever/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/whatever/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/whatever/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "/whatever/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "/whatever/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "/whatever/stable-diffusion-webui/extensions/sd-webui-old-photo-restoration/scripts/bop.py", line 55, in bop
    main(args, GLOBAL_CHECKPOINTS_FOLDER)
UnboundLocalError: local variable 'args' referenced before assignment

Haoming02 · 2023-12-17T13:38:49Z

Did you disable Scratch ?

ali0une · 2023-12-17T13:47:26Z

Did you disable Scratch ?

No ... disabled Scratch and it works (without global restoration btw)!

Mapping: You are using the mapping model without global restoration.
Now you are processing some_pic.jpg

Many thanks!

Haoming02 · 2023-12-17T14:45:21Z

Aye finally...
Now I just need to implement the rest of the features~

Haoming02 · 2023-12-19T03:38:35Z

Should be fixed on the main branch now~

TheClassyTurkey · 2023-12-19T22:51:34Z

Thank you for your continued work on this. I upgraded my main branch to the latest release and am now getting this error with no options checked

Running Stage 1: Overall restoration
Traceback (most recent call last):

  File "/whatever/SD-Main/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "/whatever/SD-Main/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "/whatever/SD-Main/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/whatever/SD-Main/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/whatever/SD-Main/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "/whatever/SD-Main/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "/whatever/SD-Main/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "/whatever/SD-Main/stable-diffusion-webui/extensions/sd-webui-old-photo-restoration/scripts/main_function.py", line 142, in bop
    final_output = core_functions(input_path, output_path, gpu_id, scratch, hr, face_res)
  File "/whatever/SD-Main/stable-diffusion-webui/extensions/sd-webui-old-photo-restoration/scripts/main_function.py", line 67, in core_functions
    global_test(args, GLOBAL_CHECKPOINTS_FOLDER)
  File "/whatever/SD-Main/stable-diffusion-webui/extensions/sd-webui-old-photo-restoration/Global/test.py", line 97, in global_test
    opt = TestOptions().parse(custom_args, save=False)
  File "/whatever/SD-Main/stable-diffusion-webui/extensions/sd-webui-old-photo-restoration/Global/options/base_options.py", line 358, in parse
    torch.cuda.set_device(self.opt.gpu_ids[0])
  File "/whatever/SD-Main/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/cuda/__init__.py", line 350, in set_device
    torch._C._cuda_setDevice(device)
AttributeError: module 'torch._C' has no attribute '_cuda_setDevice'

Haoming02 · 2023-12-20T02:02:07Z

The latest commit now handles the cases where no CUDA-compatible GPU is available~

Haoming02 · 2024-01-03T01:54:12Z

Assuming the issue has been fixed.
Do open this Issue again if it's related to running on non-Windows environment~

TheClassyTurkey · 2024-01-03T19:47:11Z

Apologies, I was busy over the holiday and just saw you had tried to fix this issue. I can't reopen the issue as I am not a collaborator.

I pulled the latest commit and am now seeing this error.

Running Stage 1: Overall restoration
initializing the dataloader
model weights loaded
directory of testing image: /somelocation/Documents/photorestore-test
processing photo-restoration-example-001-before.jpg
Traceback (most recent call last):
  File "/somelocation/Documents/SD-Main/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "/somelocation/Documents/SD-Main/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "/somelocationDocuments/SD-Main/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/somelocation/Documents/SD-Main/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/somelocation/Documents/SD-Main/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "/somelocation/Documents/SD-Main/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "/somelocation/Documents/SD-Main/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "/somelocation/Documents/SD-Main/stable-diffusion-webui/extensions/sd-webui-old-photo-restoration/scripts/main_function.py", line 142, in bop
    final_output = core_functions(input_path, output_path, gpu_id, scratch, hr, face_res)
  File "/somelocation/Documents/SD-Main/stable-diffusion-webui/extensions/sd-webui-old-photo-restoration/scripts/main_function.py", line 74, in core_functions
    global_detection(args)
  File "/somelocation/Documents/SD-Main/stable-diffusion-webui/extensions/sd-webui-old-photo-restoration/Global/detection.py", line 179, in global_detection
    main(config)
  File "/somelocation/Documents/SD-Main/stable-diffusion-webui/extensions/sd-webui-old-photo-restoration/Global/detection.py", line 146, in main
    scratch_image_scale = scratch_image_scale.to(config.GPU)
  File "/somelocation/Documents/SD-Main/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/cuda/__init__.py", line 239, in _lazy_init
    raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled

Another thing I noticed was that this message was displayed every time I launch SD. I know there is another issue open related to the "Installing requirements" message appearing.

Python 3.10.13 (main, Aug 24 2023, 22:36:46) [Clang 14.0.3 (clang-1403.0.22.14.1)]
Version: v1.6.0
Commit hash: 5ef669de080814067961f28357256e8fe27544f4
Installing Requirements for Bringing-Old-Photos-Back-to-Life...
Installing Old-Photo-Restoration Requirement: scikit-image
Installing Old-Photo-Restoration Requirement: PyYAML
Installing Old-Photo-Restoration Requirement: opencv-python

Haoming02 closed this as completed in 4d64f29 Dec 7, 2023

Haoming02 reopened this Dec 7, 2023

Haoming02 added a commit that referenced this issue Dec 9, 2023

addressing #1

991243e

Haoming02 closed this as completed Jan 3, 2024

TheClassyTurkey mentioned this issue Jan 3, 2024

Non-Windows Environment - Torch not compiled with CUDA enabled #3

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Permission denied after Processing #1

Permission denied after Processing #1

TheClassyTurkey commented Dec 6, 2023

Haoming02 commented Dec 7, 2023

Haoming02 commented Dec 7, 2023

Haoming02 commented Dec 7, 2023

ali0une commented Dec 9, 2023 •

edited

Loading

Haoming02 commented Dec 9, 2023

ali0une commented Dec 9, 2023 •

edited

Loading

Haoming02 commented Dec 9, 2023 •

edited

Loading

Haoming02 commented Dec 9, 2023

ali0une commented Dec 9, 2023 •

edited

Loading

Haoming02 commented Dec 11, 2023

jjlee commented Dec 11, 2023

ali0une commented Dec 12, 2023

jjlee commented Dec 13, 2023 •

edited

Loading

Haoming02 commented Dec 13, 2023 •

edited

Loading

ali0une commented Dec 17, 2023

Haoming02 commented Dec 17, 2023

ali0une commented Dec 17, 2023

Haoming02 commented Dec 17, 2023

ali0une commented Dec 17, 2023 •

edited

Loading

Haoming02 commented Dec 17, 2023

Haoming02 commented Dec 19, 2023

TheClassyTurkey commented Dec 19, 2023

Haoming02 commented Dec 20, 2023

Haoming02 commented Jan 3, 2024

TheClassyTurkey commented Jan 3, 2024 •

edited

Loading

Permission denied after Processing #1

Permission denied after Processing #1

Comments

TheClassyTurkey commented Dec 6, 2023

Haoming02 commented Dec 7, 2023

Haoming02 commented Dec 7, 2023

Haoming02 commented Dec 7, 2023

ali0une commented Dec 9, 2023 • edited Loading

Haoming02 commented Dec 9, 2023

ali0une commented Dec 9, 2023 • edited Loading

Haoming02 commented Dec 9, 2023 • edited Loading

Haoming02 commented Dec 9, 2023

ali0une commented Dec 9, 2023 • edited Loading

Haoming02 commented Dec 11, 2023

jjlee commented Dec 11, 2023

ali0une commented Dec 12, 2023

jjlee commented Dec 13, 2023 • edited Loading

Haoming02 commented Dec 13, 2023 • edited Loading

ali0une commented Dec 17, 2023

Haoming02 commented Dec 17, 2023

ali0une commented Dec 17, 2023

Haoming02 commented Dec 17, 2023

ali0une commented Dec 17, 2023 • edited Loading

Haoming02 commented Dec 17, 2023

Haoming02 commented Dec 19, 2023

TheClassyTurkey commented Dec 19, 2023

Haoming02 commented Dec 20, 2023

Haoming02 commented Jan 3, 2024

TheClassyTurkey commented Jan 3, 2024 • edited Loading

ali0une commented Dec 9, 2023 •

edited

Loading

ali0une commented Dec 9, 2023 •

edited

Loading

Haoming02 commented Dec 9, 2023 •

edited

Loading

ali0une commented Dec 9, 2023 •

edited

Loading

jjlee commented Dec 13, 2023 •

edited

Loading

Haoming02 commented Dec 13, 2023 •

edited

Loading

ali0une commented Dec 17, 2023 •

edited

Loading

TheClassyTurkey commented Jan 3, 2024 •

edited

Loading