Access is denied colcon.exe using cmd/windows 10

[sloretz]

Create an empty directory and save the following as bootstrap.bat

curl --output colcon.repos https://raw.githubusercontent.com/colcon/colcon.readthedocs.org/master/colcon.repos
python -m venv env
call env/Scripts/activate
pip install -U EmPy setuptools PyYAML pywin32
mkdir src
vcs import src < colcon.repos
type nul > src/colcon-argcomplete/COLCON_IGNORE
python src/colcon-core/bin/colcon build --paths src/*
call "install/prefix.bat"
colcon build

Run cmd, cd to the directory and call bootstrab.bat. The last command has the following output.

Finished <<< colcon-ros [0.70s]
Starting >>> colcon-core
Finished <<< colcon-core [7.42s]
Starting >>> colcon-library-path
Starting >>> colcon-python-setup-py
Starting >>> colcon-bash
Starting >>> colcon-defaults
Finished <<< colcon-library-path [6.19s]
Starting >>> colcon-cmake
Finished <<< colcon-python-setup-py [6.36s]
Finished <<< colcon-bash [6.45s]
Starting >>> colcon-devtools
Starting >>> colcon-metadata
Finished <<< colcon-defaults [8.13s]
Starting >>> colcon-notification
Finished <<< colcon-devtools [10.05s]
Starting >>> colcon-output
Finished <<< colcon-metadata [11.52s]
Starting >>> colcon-package-information
Finished <<< colcon-cmake [12.03s]
Starting >>> colcon-package-selection
Finished <<< colcon-output [12.33s]
Starting >>> colcon-parallel-executor
Finished <<< colcon-package-information [11.86s]
Starting >>> colcon-powershell
Finished <<< colcon-package-selection [14.72s]
Starting >>> colcon-recursive-crawl
--- stderr: colcon-notification
error: [WinError 5] Access is denied: 'D:\\colcon-master\\install\\colcon-notification\\Scripts\\colcon.exe'
---
Failed   <<< colcon-notification         [ Exited with code 1 ]
Aborted  <<< colcon-powershell
Aborted  <<< colcon-parallel-executor
Aborted  <<< colcon-recursive-crawl

Summary: 11 packages finished [47.02s]
  1 package failed: colcon-notification
  3 packages aborted: colcon-parallel-executor colcon-powershell colcon-recursive-crawl
  1 package had stderr output: colcon-notification
  2 packages not processed

I expected the build to succeed. It seems strange that it is looking for colcon.exe inside of colcon-notification.

[dirk-thomas]

Is the problem reproducible / deterministic? As you already mentioned colcon-notification doesn't have colcon.exe executable. It should be (only in) install/colcon-core/Scripts.

[sloretz]

Yes it's reproducible. To make sure I deleted everything except for bootstrap.bat, opened a new cmd shell, and called the batch script (repeating that process 3 times). Each time it finished with the same error in colcon-notification

Edit: Here's pip freeze in a brand new shell

C:\Program Files (x86)\Microsoft Visual Studio 14.0>pip freeze
attrs==17.4.0
catkin-pkg==0.4.1
colorama==0.3.9
coloredlogs==9.0
coverage==4.5.1
cycler==0.10.0
docutils==0.14
empy==3.3.2
enum34==1.1.6
flake8==3.5.0
flake8-blind-except==0.1.1
flake8-builtins==1.0.post0
flake8-class-newline==1.6.0
flake8-comprehensions==1.4.1
flake8-deprecated==1.3
flake8-docstrings==1.3.0
flake8-import-order==0.17
flake8-polyfill==1.0.2
flake8-quotes==0.14.0
humanfriendly==4.10
matplotlib==2.0.2
mccabe==0.6.1
mock==2.0.0
nose==1.3.7
numpy==1.13.1
pbr==3.1.1
pep8==1.7.1
pluggy==0.6.0
py==1.5.2
pycodestyle==2.3.1
pydocstyle==2.1.1
pyflakes==1.6.0
pyparsing==2.2.0
pypiwin32==223
pyreadline==2.1
pytest==3.4.1
pytest-cov==2.5.1
pytest-runner==4.0
python-dateutil==2.6.1
pytz==2017.2
pywin32==223
PyYAML==3.12
six==1.10.0
snowballstemmer==1.2.1
trollius==2.1
vcstool==0.1.32
You are using pip version 9.0.1, however version 10.0.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

[dirk-thomas]

It seems to fail within the setup.py invocation (rather then anywhere in colcon itself). Not sure if there is anything we can do about it in our code.

[sloretz]

It may be a problem with the prefix.bat logic instead of anything in these packages. If I add a COLCON_IGNORE to colcon-notification then on the second invokation of colcon build I see the error in colcon-core instead

Starting >>> colcon-core
--- stderr: colcon-core
error: [Errno 13] Permission denied: 'D:\\colcon-master\\install\\colcon-core\\Scripts\\colcon.exe'
---
Failed   <<< colcon-core         [ Exited with code 1 ]

Summary: 0 packages finished [1.36s]
  1 package failed: colcon-core
  1 package had stderr output: colcon-core
  15 packages not processed

Maybe this issue should be moved to colcon-core instead? Here's the bootstrap.bat that produced the above error.

curl --output colcon.repos https://raw.githubusercontent.com/colcon/colcon.readthedocs.org/master/colcon.repos
python -m venv env
call env/Scripts/activate
pip install -U EmPy setuptools PyYAML pywin32 coloredlogs
mkdir src
vcs import src < colcon.repos
type nul > src/colcon-argcomplete/COLCON_IGNORE
type nul > src/colcon-notification/COLCON_IGNORE
python src/colcon-core/bin/colcon build --paths src/*
call "install/prefix.bat"
colcon build

[dirk-thomas]

In the first build the output of colcon-notification contains the following (which is unique compared to all other packages):

Searching for colcon-core==0.1.0
Best match: colcon-core 0.1.0
Adding colcon-core 0.1.0 to easy-install.pth file
Installing colcon-script.py script to C:\dev\colcon-powershell\install\colcon-notification\Scripts
Installing colcon.exe script to C:\dev\colcon-powershell\install\colcon-notification\Scripts

These installed scripts are the reason why the second build then fails.

[sloretz]

It's not specific to colcon-notification. I get the same error building only colcon-core.

curl --output colcon.repos https://raw.githubusercontent.com/colcon/colcon.readthedocs.org/master/colcon.repos
python -m venv env
call env/Scripts/activate
type nul > env/COLCON_IGNORE
pip install -U EmPy setuptools PyYAML pywin32 coloredlogs
mkdir src
vcs import src < colcon.repos
python src/colcon-core/bin/colcon build --paths src/colcon-core
call "install\setup.bat"
colcon build --paths src/colcon-core

Output of last build step

(env) D:\colcon-master>colcon build --paths src/colcon-core
Starting >>> colcon-core
running egg_info
writing D:\colcon-master\build\colcon-core\colcon_core.egg-info\PKG-INFO
writing dependency_links to D:\colcon-master\build\colcon-core\colcon_core.egg-info\dependency_links.txt
writing entry points to D:\colcon-master\build\colcon-core\colcon_core.egg-info\entry_points.txt
writing requirements to D:\colcon-master\build\colcon-core\colcon_core.egg-info\requires.txt
writing top-level names to D:\colcon-master\build\colcon-core\colcon_core.egg-info\top_level.txt
reading manifest file 'D:\colcon-master\build\colcon-core\colcon_core.egg-info\SOURCES.txt'
writing manifest file 'D:\colcon-master\build\colcon-core\colcon_core.egg-info\SOURCES.txt'
running build
running build_py
running install
running install_lib
running install_egg_info
removing 'D:\colcon-master\install\colcon-core\Lib\site-packages\colcon_core-0.1.0-py3.6.egg-info' (and everything under it)
Copying D:\colcon-master\build\colcon-core\colcon_core.egg-info to D:\colcon-master\install\colcon-core\Lib\site-packages\colcon_core-0.1.0-py3.6.egg-info
running install_scripts
Installing colcon-script.py script to D:\colcon-master\install\colcon-core\Scripts
Installing colcon.exe script to D:\colcon-master\install\colcon-core\Scripts
error: [Errno 13] Permission denied: 'D:\\colcon-master\\install\\colcon-core\\Scripts\\colcon.exe'
Failed   <<< colcon-core        [ Exited with code 1 ]

[dirk-thomas]

It may be a problem with the prefix.bat logic instead of anything in these packages.

The prefix / local_setup scripts are not used to build downstream packages. Each package has a script in its build folder which is being used: e.g. colcon_command_prefix_setup_py.bat

The only thing it does in this case is sourcing the package.bat script of colcon-core.

[dirk-thomas]

The presence of the script in the colcon-notification directory is one problem. It shouldn't be there.

The second problem is the permission problem. I think that may be related to the fact that the build is currently using that specific executable which can't be overwritten at the same time (on Windows).

[dirk-thomas]

The presence of the script in the colcon-notification directory is one problem. It shouldn't be there.

I created a follow up ticket for that: colcon/colcon-core#48

The second problem is the permission problem. I think that may be related to the fact that the build is currently using that specific executable which can't be overwritten at the same time (on Windows).

The initially described problem (access denied) has been addressed in colcon/colcon.readthedocs.org#10 by updating the instructions. Hence I will go ahead and close this ticket.