git: Unable to initialize submodules over HTTPS

[recvfrom]

Description

Cloning the repo over HTTPS via the commands provided in the documentation fails to initialize the submodules when SSH isn't configured because the submodule URLs are hardcoded to use SSH.

Steps to Reproduce

Run git clone --recurse-submodules https://github.com/fireeye/capa.git without an SSH key configured

Expected behavior:

submodules are checked out using the same protocol as the main repo

Actual behavior:

submodules are checked out using SSH regardless of how the main repo is checked out

Output:

$ git clone --recurse-submodules https://github.com/fireeye/capa.git
Cloning into 'capa'...
remote: Enumerating objects: 320, done.
remote: Counting objects: 100% (320/320), done.
remote: Compressing objects: 100% (194/194), done.
remote: Total 3824 (delta 183), reused 217 (delta 124), pack-reused 3504
Receiving objects: 100% (3824/3824), 1.98 MiB | 4.03 MiB/s, done.
Resolving deltas: 100% (2689/2689), done.
Submodule 'rules' (git@github.com:fireeye/capa-rules.git) registered for path 'rules'
Submodule 'tests/data' (git@github.com:fireeye/capa-testfiles.git) registered for path 'tests/data'
Cloning into '/home/zelda/workspace/capa/rules'...
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@github.com:fireeye/capa-rules.git' into submodule path '/home/zelda/workspace/capa/rules' failed
Failed to clone 'rules'. Retry scheduled
Cloning into '/home/zelda/workspace/capa/tests/data'...
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@github.com:fireeye/capa-testfiles.git' into submodule path '/home/zelda/workspace/capa/tests/data' failed
Failed to clone 'tests/data'. Retry scheduled
Cloning into '/home/zelda/workspace/capa/rules'...
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@github.com:fireeye/capa-rules.git' into submodule path '/home/zelda/workspace/capa/rules' failed
Failed to clone 'rules' a second time, aborting

Versions

master

Additional Information

This fixes it for me, per https://stackoverflow.com/a/44630028/9457431:

--- a/.gitmodules
+++ b/.gitmodules
@@ -1,6 +1,6 @@
 [submodule "rules"]
        path = rules
-       url = git@github.com:fireeye/capa-rules.git
+       url = ../capa-rules.git
 [submodule "tests/data"]
        path = tests/data
-       url = git@github.com:fireeye/capa-testfiles.git
+       url = ../capa-testfiles.git

The only potential downside is that it would require people who fork capa to also fork the other two repos. Thoughts?

[williballenthin]

hey @recvfrom thanks for the thorough bug report!

The fix looks reasonable to me. I think people that are dev'ing on capa core will have forks of the other two repos. Shouldn't affect people that implement new rules. So, looks like a fix we should merge. Thoughts @mr-tz @mike-hunhoff ?

[mr-tz]

looks good to me!

[williballenthin]

@recvfrom want to propose a PR so that you can get on the GH contributors list? otherwise, i can commit these changes directly. always nice to get the recognition though ;-)

[mike-hunhoff]

agreed, looks good!

[recvfrom]

PR submitted - thank you!