New project creation fails when pushing to GitHub, but repo created succesfully

[e2jk]

[EDIT] this issue was caused by a space at the beginning of my GitHub PAT. reconfiguring works. Nonetheless, see this post with a question about improving the user experience

Associated Template/Command/Core

After navigating around these 3 issues #748, #749 and 750, I'm facing yet another "interesting" situation.
I have created a personal access token on GitHub giving it these access rights:

(you see that when selecting "workflow", all previous options under "repo" are automatically selected, guaranteeing that these rights are available to my PAT...)

When creating a new project, the empty repository is nicely created on GitHub, but for some reason the git push command gets blocked?
Is there another option I should give to my personal access token? Normally pushing to the repo is handled by the "repo" group...

Describe the bug

To Reproduce

Steps to reproduce the behavior:

1. Run cookietemple with command: create
2. Select: cli, python, project name, description, the rest of the questions answered with their default value (uploading to Github to a non-org, non-private repo)
3. See error in the next collapsible section:

Full `--verbose` output of the command (Click to expand)

C:\Users\<username>\Documents\devel>cookietemple create

     ██████  ██████   ██████  ██   ██ ██ ███████ ████████ ███████ ███    ███ ██████  ██      ███████ 
    ██      ██    ██ ██    ██ ██  ██  ██ ██         ██    ██      ████  ████ ██   ██ ██      ██      
    ██      ██    ██ ██    ██ █████   ██ █████      ██    █████   ██ ████ ██ ██████  ██      █████ 
    ██      ██    ██ ██    ██ ██  ██  ██ ██         ██    ██      ██  ██  ██ ██      ██      ██    
     ██████  ██████   ██████  ██   ██ ██ ███████    ██    ███████ ██      ██ ██      ███████ ███████ 

Run cookietemple --help for an overview of all commands

? Choose the project's domain:  cli
? Choose the project's primary language:  python
? Project name [exploding-springfield]:  timelog_tracker
Looking up timelog_tracker at PyPi!
Looking up timelog_tracker at readthedocs.io!
? Short description of your project [timelog_tracker. A cookietemple based .]:  A utility to track working time in a Microsoft Outlook calendar and generate monthly timesh
eets.
? Initial version of your project [0.1.0]:
? License:  MIT
Automatically creating a Github repository with cookietemple is strongly recommended. Otherwise you will not be able to use all of cookietemple's features!

? Do you want to create a Github repository and push your template to it? [Yes]:  Yes
? Do you want to create an organization repository? [No]:  No
? Do you want your repository to be private? [No]:  No
Fixing too short underlines of *.rst file (usually index.rst)
Running general linting
Running lint checks ----------------------------------------------------------------------------------------------------------------------- 6 of 6 lint_cookietemple_config
Running cli-python linting
Running lint checks ----------------------------------------------------------------------------------------------------------------------------- 2 of 2 python_files_exist
Running autopep8 to fix pep8 issues in place

──────────────────────────────────────────────────────────────────────────────  LINT RESULTS ──────────────────────────────────────────────────────────────────────────────

     [[✔]]    8 tests passed
     [[!]]    1 tests had warnings
     [[✗]]    0 tests failed

─────────────────────────────────────────────────────────────────────────── [[✔]] Tests Passed ────────────────────────────────────────────────────────────────────────────
┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│                                                                                                                                                                         │
│  1 https://cookietemple.readthedocs.io/en/latest/lint.html#general-1 : All required general files were found!                                                           │
│  2 https://cookietemple.readthedocs.io/en/latest/lint.html#general-2 : Dockerfile check passed                                                                          │
│  3 https://cookietemple.readthedocs.io/en/latest/lint.html#general-5 : Versions were consistent over all files                                                          │
│  4 https://cookietemple.readthedocs.io/en/latest/lint.html#general-7 : All required cookietemple.cfg sections were found!                                               │
│  5 https://cookietemple.readthedocs.io/en/latest/lint.html#general-7 : All general config sections passed cookietemple linting!                                         │
│  6 https://cookietemple.readthedocs.io/en/latest/lint.html#cli-python-3 : All required sync blacklisted files are configured!                                           │
│  7 https://cookietemple.readthedocs.io/en/latest/lint.html#cli-python-1 : All required cli-python specific files were found!                                            │
│  8 https://cookietemple.readthedocs.io/en/latest/lint.html#cli-python-1 : File not found check: __pycache__                                                             │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

─────────────────────────────────────────────────────────────────────────── [[!]] Test Warnings ───────────────────────────────────────────────────────────────────────────
┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│                                                                                                                                                                         │
│  1 https://cookietemple.readthedocs.io/en/latest/lint.html#general-3 : TODO string found in publish_package.yml: Configure your PyPI Token to enable automatic          │
│    deployment to PyPi on releases                                                                                                                                       │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
Decrypting personal access token.
Logging into Github
Creating Github repository
Creating labels and default Github settings
Creating repository sync secret
Cloning empty Github repository
Staging template
Pushing template to Github origin main
┌─────────────────────────────────────────────────────────────────── Traceback (most recent call last) ───────────────────────────────────────────────────────────────────┐
│ c:\users\<username>\appdata\local\programs\python\python38-32\lib\runpy.py:193 in _run_module_as_main                                                                       │
│                                                                                                                                                                         │
│   190 │   main_globals = sys.modules["__main__"].__dict__                                                                                                               │
│   191 │   if alter_argv:                                                                                                                                                │
│   192 │   │   sys.argv[0] = mod_spec.origin                                                                                                                             │
│ > 193 │   return _run_code(code, main_globals, None,                                                                                                                    │
│   194 │   │   │   │   │    "__main__", mod_spec)                                                                                                                        │
│   195                                                                                                                                                                   │
│   196 def run_module(mod_name, init_globals=None,                                                                                                                       │
│                                                                                                                                                                         │
│ c:\users\<username>\appdata\local\programs\python\python38-32\lib\runpy.py:86 in _run_code                                                                                  │
│                                                                                                                                                                         │
│    83 │   │   │   │   │      __loader__ = loader,                                                                                                                       │
│    84 │   │   │   │   │      __package__ = pkg_name,                                                                                                                    │
│    85 │   │   │   │   │      __spec__ = mod_spec)                                                                                                                       │
│ >  86 │   exec(code, run_globals)                                                                                                                                       │
│    87 │   return run_globals                                                                                                                                            │
│    88                                                                                                                                                                   │
│    89 def _run_module_code(code, init_globals=None,                                                                                                                     │
│                                                                                                                                                                         │
│ C:\Users\<username>\.local\bin\cookietemple.exe\__main__.py:7 in <module>                                                                                                   │
│                                                                                                                                                                         │
│ [Errno 2] No such file or directory: 'C:\\Users\\<username>\\.local\\bin\\cookietemple.exe\\__main__.py'                                                                    │
│                                                                                                                                                                         │
│ C:\Users\<username>\.local\pipx\venvs\cookietemple\lib\site-packages\cookietemple\__main__.py:53 in main                                                                    │
│                                                                                                                                                                         │
│    50 │   # Is the latest cookietemple version installed? Upgrade if not!                                                                                               │
│    51 │   if not UpgradeCommand.check_cookietemple_latest():                                                                                                            │
│    52 │   │   console.print("[bold blue]Run [green]cookietemple upgrade [blue]to get the                                                                                │
│       latest version.")                                                                                                                                                 │
│ >  53 │   cookietemple_cli()                                                                                                                                            │
│    54                                                                                                                                                                   │
│    55                                                                                                                                                                   │
│    56 @click.group(cls=HelpErrorHandling)                                                                                                                               │
│                                                                                                                                                                         │
│ C:\Users\<username>\.local\pipx\venvs\cookietemple\lib\site-packages\click\core.py:1137 in __call__                                                                         │
│                                                                                                                                                                         │
│   1134 │                                                                                                                                                                │
│   1135 │   def __call__(self, *args: t.Any, **kwargs: t.Any) -> t.Any:                                                                                                  │
│   1136 │   │   """Alias for :meth:`main`."""                                                                                                                            │
│ > 1137 │   │   return self.main(*args, **kwargs)                                                                                                                        │
│   1138                                                                                                                                                                  │
│   1139                                                                                                                                                                  │
│   1140 class Command(BaseCommand):                                                                                                                                      │
│                                                                                                                                                                         │
│ C:\Users\<username>\.local\pipx\venvs\cookietemple\lib\site-packages\click\core.py:1062 in main                                                                             │
│                                                                                                                                                                         │
│   1059 │   │   try:                                                                                                                                                     │
│   1060 │   │   │   try:                                                                                                                                                 │
│   1061 │   │   │   │   with self.make_context(prog_name, args, **extra) as ctx:                                                                                         │
│ > 1062 │   │   │   │   │   rv = self.invoke(ctx)                                                                                                                        │
│   1063 │   │   │   │   │   if not standalone_mode:                                                                                                                      │
│   1064 │   │   │   │   │   │   return rv                                                                                                                                │
│   1065 │   │   │   │   │   # it's not safe to `ctx.exit(rv)` here!                                                                                                      │
│                                                                                                                                                                         │
│ C:\Users\<username>\.local\pipx\venvs\cookietemple\lib\site-packages\click\core.py:1668 in invoke                                                                           │
│                                                                                                                                                                         │
│   1665 │   │   │   │   super().invoke(ctx)                                                                                                                              │
│   1666 │   │   │   │   sub_ctx = cmd.make_context(cmd_name, args, parent=ctx)                                                                                           │
│   1667 │   │   │   │   with sub_ctx:                                                                                                                                    │
│ > 1668 │   │   │   │   │   return _process_result(sub_ctx.command.invoke(sub_ctx))                                                                                      │
│   1669 │   │                                                                                                                                                            │
│   1670 │   │   # In chain mode we create the contexts step by step, but after the                                                                                       │
│   1671 │   │   # base command has been invoked.  Because at that point we do not                                                                                        │
│                                                                                                                                                                         │
│ C:\Users\<username>\.local\pipx\venvs\cookietemple\lib\site-packages\click\core.py:1404 in invoke                                                                           │
│                                                                                                                                                                         │
│   1401 │   │   │   echo(style(message, fg="red"), err=True)                                                                                                             │
│   1402 │   │                                                                                                                                                            │
│   1403 │   │   if self.callback is not None:                                                                                                                            │
│ > 1404 │   │   │   return ctx.invoke(self.callback, **ctx.params)                                                                                                       │
│   1405 │                                                                                                                                                                │
│   1406 │   def shell_complete(self, ctx: Context, incomplete: str) -> t.List["CompletionItem"]:                                                                         │
│   1407 │   │   """Return a list of completions for the incomplete value. Looks                                                                                          │
│                                                                                                                                                                         │
│ C:\Users\<username>\.local\pipx\venvs\cookietemple\lib\site-packages\click\core.py:763 in invoke                                                                            │
│                                                                                                                                                                         │
│    760 │   │                                                                                                                                                            │
│    761 │   │   with augment_usage_errors(__self):                                                                                                                       │
│    762 │   │   │   with ctx:                                                                                                                                            │
│ >  763 │   │   │   │   return __callback(*args, **kwargs)                                                                                                               │
│    764 │                                                                                                                                                                │
│    765 │   def forward(                                                                                                                                                 │
│    766 │   │   __self, __cmd: "Command", *args: t.Any, **kwargs: t.Any  # noqa: B902                                                                                    │
│                                                                                                                                                                         │
│ C:\Users\<username>\.local\pipx\venvs\cookietemple\lib\site-packages\cookietemple\__main__.py:110 in create                                                                 │
│                                                                                                                                                                         │
│   107 │   Next, you will be asked whether you want to use cookietemple's Github support create                                                                          │
│       a repository, push your template and enable a few settings.                                                                                                       │
│   108 │   After the project has been created it will be linted and you will be notified of any                                                                          │
│       TODOs.                                                                                                                                                            │
│   109 │   """                                                                                                                                                           │
│ > 110 │   choose_domain(path, domain, None)                                                                                                                             │
│   111                                                                                                                                                                   │
│   112                                                                                                                                                                   │
│   113 @cookietemple_cli.command(short_help="Lint your existing cookietemple project.",                                                                                  │
│       cls=CustomHelpSubcommand)                                                                                                                                         │
│                                                                                                                                                                         │
│ C:\Users\<username>\.local\pipx\venvs\cookietemple\lib\site-packages\cookietemple\create\create.py:37 in choose_domain                                                      │
│                                                                                                                                                                         │
│   34 │   switcher = {"cli": CliCreator, "web": WebCreator, "gui": GuiCreator, "lib":                                                                                    │
│      LibCreator, "pub": PubCreator}                                                                                                                                     │
│   35 │                                                                                                                                                                  │
│   36 │   creator_obj: Union[CliCreator, WebCreator, GuiCreator, LibCreator, PubCreator] =                                                                               │
│      switcher.get(domain.lower())()  # type: ignore                                                                                                                     │
│ > 37 │   creator_obj.create_template(path, dot_cookietemple)                                                                                                            │
│   38                                                                                                                                                                    │
│                                                                                                                                                                         │
│ C:\Users\<username>\.local\pipx\venvs\cookietemple\lib\site-packages\cookietemple\create\domains\cli_creator.py:84 in create_template                                       │
│                                                                                                                                                                         │
│    81 │   │   )                                                                                                                                                         │
│    82 │   │                                                                                                                                                             │
│    83 │   │   # perform general operations like creating a GitHub repository and general                                                                                │
│       linting                                                                                                                                                           │
│ >  84 │   │   super().process_common_operations(                                                                                                                        │
│    85 │   │   │   path=Path(path).resolve(),                                                                                                                            │
│    86 │   │   │   domain="cli",                                                                                                                                         │
│    87 │   │   │   language=self.cli_struct.language,                                                                                                                    │
│                                                                                                                                                                         │
│ C:\Users\<username>\.local\pipx\venvs\cookietemple\lib\site-packages\cookietemple\create\template_creator.py:83 in process_common_operations                                │
│                                                                                                                                                                         │
│    80 │   │   │   # rename the currently created template to a temporary name, create Github                                                                            │
│       repo, push, remove temporary template                                                                                                                             │
│    81 │   │   │   tmp_project_path = f"{project_path}_cookietemple_tmp"                                                                                                 │
│    82 │   │   │   os.mkdir(tmp_project_path)                                                                                                                            │
│ >  83 │   │   │   create_push_github_repository(project_path, self.creator_ctx,                                                                                         │
│       tmp_project_path)                                                                                                                                                 │
│    84 │   │   │   shutil.rmtree(tmp_project_path, ignore_errors=True)                                                                                                   │
│    85 │   │                                                                                                                                                             │
│    86 │   │   if subdomain:                                                                                                                                             │
│                                                                                                                                                                         │
│ C:\Users\<username>\.local\pipx\venvs\cookietemple\lib\site-packages\cookietemple\create\github_support.py:118 in create_push_github_repository                             │
│                                                                                                                                                                         │
│   115 │   │   console.print(f"[bold blue]Pushing template to Github origin {default_branch}")                                                                           │
│   116 │   │   if default_branch != "master":                                                                                                                            │
│   117 │   │   │   cloned_repo.git.branch("-M", f"{default_branch}")                                                                                                     │
│ > 118 │   │   cloned_repo.remotes.origin.push(refspec=f"{default_branch}:{default_branch}")                                                                             │
│   119 │   │                                                                                                                                                             │
│   120 │   │   # set branch protection (all WF must pass, dismiss stale PR reviews) only when                                                                            │
│       repo is public                                                                                                                                                    │
│   121 │   │   log.debug("Set branch protection rules.")                                                                                                                 │
│                                                                                                                                                                         │
│ C:\Users\<username>\.local\pipx\venvs\cookietemple\lib\site-packages\git\remote.py:884 in push                                                                              │
│                                                                                                                                                                         │
│   881 │   │   kwargs = add_progress(kwargs, self.repo.git, progress)                                                                                                    │
│   882 │   │   proc = self.repo.git.push(self, refspec, porcelain=True, as_process=True,                                                                                 │
│   883 │   │   │   │   │   │   │   │     universal_newlines=True, **kwargs)                                                                                              │
│ > 884 │   │   return self._get_push_info(proc, progress)                                                                                                                │
│   885 │                                                                                                                                                                 │
│   886 │   @property                                                                                                                                                     │
│   887 │   def config_reader(self) -> SectionConstraint:                                                                                                                 │
│                                                                                                                                                                         │
│ C:\Users\<username>\.local\pipx\venvs\cookietemple\lib\site-packages\git\remote.py:764 in _get_push_info                                                                    │
│                                                                                                                                                                         │
│   761 │   │   handle_process_output(proc, stdout_handler, progress_handler, finalizer=None,                                                                             │
│       decode_streams=False)                                                                                                                                             │
│   762 │   │   stderr_text = progress.error_lines and '\n'.join(progress.error_lines) or ''                                                                              │
│   763 │   │   try:                                                                                                                                                      │
│ > 764 │   │   │   proc.wait(stderr=stderr_text)                                                                                                                         │
│   765 │   │   except Exception:                                                                                                                                         │
│   766 │   │   │   if not output:                                                                                                                                        │
│   767 │   │   │   │   raise                                                                                                                                             │
│                                                                                                                                                                         │
│ C:\Users\<username>\.local\pipx\venvs\cookietemple\lib\site-packages\git\cmd.py:447 in wait                                                                                 │
│                                                                                                                                                                         │
│    444 │   │   │   │   if status != 0:                                                                                                                                  │
│    445 │   │   │   │   │   errstr = read_all_from_possibly_closed_stream(self.proc.stderr)                                                                              │
│    446 │   │   │   │   │   log.debug('AutoInterrupt wait stderr: %r' % (errstr,))                                                                                       │
│ >  447 │   │   │   │   │   raise GitCommandError(remove_password_if_present(self.args), status,                                                                         │
│        errstr)                                                                                                                                                          │
│    448 │   │   │   # END status handling                                                                                                                                │
│    449 │   │   │   return status                                                                                                                                        │
│    450                                                                                                                                                                  │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git push --porcelain origin main:main
  stderr: 'fatal: Authentication failed for 'https://github.com/e2jk/timelog_tracker/''

Expected behavior

I would have expected the git push command to succeed, since the repo has been created succesfully.

System:

• OS: Windows 10 Enterprise build 1803, French language
• Python: 3.8.2 (handled by pipx)
• Virtual environment: pipx
• cookietemple: 1.3.9

Additional context

Please ask if any other detail would be needed. In the meantime I'll start with the files created in the folder, and will handle the project creation manually on GitHub.

[e2jk]

Interestingly, when looking at the remote, it looks like there is a space that introduced itself before the PAT:

C:\Users\<username>\Documents\devel\timelog_tracker>git remote -v
origin  https://e2jk: ghp_<rest of the token>@github.com/e2jk/timelog_tracker (fetch)
origin  https://e2jk: ghp_<rest of the token>@github.com/e2jk/timelog_tracker (push)
                     ^
                     see space here

I suppose that when I pasted the PAT while doing cookietemple config, there was a space in front of the PAT.
I would still nonetheless find it weird that it accepted the PAT to do the repo creation, but not for the push. Is there some PAT sanitization happening in the repo creation code that is not used when setting up the remote?

Question: should cookietemple config do a basic sanitation when entering the PAT, to ensure there are no trailing or leading spaces that could come when selecting by hand the PAT from the GitHub page (I remember for some reason the copy icon in GitHub not having had the desired effect, so I selected the code by hand, likely adding that space then - and since the PAT gets replaced by stars in the command line when pasting it, there is no way to identify that the first character is incorrect...)

[e2jk]

After manually changing the remote's URL to remove the space, a manual git push works as expected:
git remote set-url origin https://e2jk:ghp_<rest of the token>@github.com/e2jk/timelog_tracker

I will now delete the repo, update the config to have the space removed from the PAT, and hopefully confirm that the rest of the process works.

[e2jk]

SUCCESS (at last, after the 3 issues listed above and this one ;) )

The question at the bottom of this post is thus to be discussed, why did the PAT with the space get accepted to created the repository, but not to push, and should there be some basic check/sanitation be made on the PAT when doing cookietemple config all?

[Zethson]

SUCCESS (at last, after the 3 issues listed above and this one ;) )

The question at the bottom of this post is thus to be discussed, why did the PAT with the space get accepted to created the repository, but not to push, and should there be some basic check/sanitation be made on the PAT when doing cookietemple config all?

@Imipenem would you be willing to hack some PAT validations? Should be very easy.

@e2jk perfectly agree. This should be validated.