Failure to fetch submodule from GitHub

[gyakushev]

When running docker build <git-url> Docker fails to fetch some submodules, apparently those not pegged to specific tag.

Repro:

$ docker build https://github.com/PX4/Firmware.git
unable to prepare context: unable to 'git clone' to temporary context directory: error initializing submodules: Submodule 'Tools/jMAVSim' (https://github.com/PX4/jMAVSim.git) registered for path 'Tools/jMAVSim'
Submodule 'Tools/sitl_gazebo' (https://github.com/PX4/sitl_gazebo.git) registered for path 'Tools/sitl_gazebo'
Submodule 'boards/atlflight/cmake_hexagon' (https://github.com/PX4/cmake_hexagon.git) registered for path 'boards/atlflight/cmake_hexagon'
Submodule 'cmake/configs/uavcan_board_ident' (https://github.com/PX4/uavcan_board_ident.git) registered for path 'cmake/configs/uavcan_board_ident'
Submodule 'mavlink/include/mavlink/v2.0' (https://github.com/mavlink/c_library_v2.git) registered for path 'mavlink/include/mavlink/v2.0'
Submodule 'msg/tools/gencpp' (https://github.com/PX4/gencpp.git) registered for path 'msg/tools/gencpp'
Submodule 'msg/tools/genmsg' (https://github.com/PX4/genmsg.git) registered for path 'msg/tools/genmsg'
Submodule 'platforms/nuttx/NuttX/apps' (https://github.com/PX4/NuttX-apps.git) registered for path 'platforms/nuttx/NuttX/apps'
Submodule 'platforms/nuttx/NuttX/nuttx' (https://github.com/PX4/NuttX.git) registered for path 'platforms/nuttx/NuttX/nuttx'
Submodule 'src/drivers/gps/devices' (https://github.com/PX4/GpsDrivers.git) registered for path 'src/drivers/gps/devices'
Submodule 'src/drivers/uavcan/libuavcan' (https://github.com/PX4/uavcan.git) registered for path 'src/drivers/uavcan/libuavcan'
Submodule 'src/lib/DriverFramework' (https://github.com/PX4/DriverFramework.git) registered for path 'src/lib/DriverFramework'
Submodule 'src/lib/ecl' (https://github.com/PX4/ecl.git) registered for path 'src/lib/ecl'
Submodule 'src/lib/matrix' (https://github.com/PX4/Matrix.git) registered for path 'src/lib/matrix'
Submodule 'src/modules/micrortps_bridge/micro-CDR' (https://github.com/PX4/micro-CDR.git) registered for path 'src/modules/micrortps_bridge/micro-CDR'
Cloning into '/tmp/docker-build-git605175926/Tools/jMAVSim'...
Cloning into '/tmp/docker-build-git605175926/Tools/sitl_gazebo'...
Cloning into '/tmp/docker-build-git605175926/boards/atlflight/cmake_hexagon'...
Cloning into '/tmp/docker-build-git605175926/cmake/configs/uavcan_board_ident'...
Cloning into '/tmp/docker-build-git605175926/mavlink/include/mavlink/v2.0'...
Cloning into '/tmp/docker-build-git605175926/msg/tools/gencpp'...
Cloning into '/tmp/docker-build-git605175926/msg/tools/genmsg'...
Cloning into '/tmp/docker-build-git605175926/platforms/nuttx/NuttX/apps'...
Cloning into '/tmp/docker-build-git605175926/platforms/nuttx/NuttX/nuttx'...
Cloning into '/tmp/docker-build-git605175926/src/drivers/gps/devices'...
Cloning into '/tmp/docker-build-git605175926/src/drivers/uavcan/libuavcan'...
Cloning into '/tmp/docker-build-git605175926/src/lib/DriverFramework'...
Cloning into '/tmp/docker-build-git605175926/src/lib/ecl'...
Cloning into '/tmp/docker-build-git605175926/src/lib/matrix'...
Cloning into '/tmp/docker-build-git605175926/src/modules/micrortps_bridge/micro-CDR'...
Submodule path 'Tools/jMAVSim': checked out '3bd51e67e022ce59644d33ebdf6570c2ea88ddb6'
Submodule 'jMAVlib' (https://github.com/PX4/jMAVlib.git) registered for path 'Tools/jMAVSim/jMAVlib'
Cloning into '/tmp/docker-build-git605175926/Tools/jMAVSim/jMAVlib'...
Submodule path 'Tools/jMAVSim/jMAVlib': checked out 'b8d4e8e7acfd2e47f4b51f0c2577fba6ef5ba735'
error: Server does not allow request for unadvertised object 22df9475ca0d157e2db066a20f64c35906bf7f25
Fetched in submodule path 'Tools/sitl_gazebo', but it did not contain 22df9475ca0d157e2db066a20f64c35906bf7f25. Direct fetching of that commit failed.
: exit status 1

Fetching the same repo recursively with Git works fine:

$ git clone --recursive https://github.com/PX4/Firmware.git
<no error>

Docker version:

$ docker version
Client: Docker Engine - Community
 Version:           19.03.1
 API version:       1.40
 Go version:        go1.12.5
 Git commit:        74b1e89
 Built:             Thu Jul 25 21:21:05 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.1
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.5
  Git commit:       74b1e89
  Built:            Thu Jul 25 21:19:41 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

From the looks of it, fetch works fine for submodules that are pegged to tagged commit, but fail for other (non-tagged) commits.

[gyakushev]

The reason for this is --depth=1 parameter here: https://github.com/docker/docker-ce/blob/7ddb4221acd9617c5d3de1f3072468b214979b9a/components/engine/builder/remotecontext/git/gitutils.go#L68

Git works fine without that parameter, but fails with it. Going to file an issue with Git now.

[manishtomar]

This is not a distribution issue. If required please consider opening issue based on https://github.com/docker/docker-ce/. Closing.