Instructions for building OpenOCD on windows are incorrect / incomplete (OCD-545)

[littleboot]

Building OpenOcd on windows following the instruction by the letter fails.
Every part of the instruction seems to have a problem, this is really frustrating.
After almost two full days of trying to build openOcd on windows I decided I need some help. I hope someone can assist me to get it working and the instructions can be improved and fixed.
In this issue report below I will describe in detail how I tried building OpenOCD from this github repo, I will also describe the fixes for some of the faulty instructions currently listed on the doc.espressif website.

Download MSYS from:
https://www.msys2.org/

Install:
When the install finished the incorrect shell opens by default, close this shell and open the correct one:
"MSYS2 MinGW X86"

Install Dependencies
Problem 1: "libtool" is missing from the list, I get an error if I don't install this dependency.
Fix: Add "libtool" to the dependencies list.

Problem 2: The download of these tools can timeout
Fix: Added "--disable-download-timeout" to the pacman command.

New cmd with fixes implemented:

pacman -S --disable-download-timeout --noconfirm --needed autoconf automake git make \
mingw-w64-i686-gcc \
mingw-w64-i686-toolchain \
mingw-w64-i686-libtool \
mingw-w64-i686-pkg-config \
mingw-w64-cross-winpthreads-git \
p7zip \
libtool

Download Sources of OpenOCD
Problem 3: "No such file or directory"
Fix: first create directory called "esp" with mkdir command.

New cmd with fixes implemented:

mkdir esp 
cd ~/esp
git clone --recursive https://github.com/espressif/openocd-esp32.git

Downloading libusb
No problems with this command:

wget https://github.com/libusb/libusb/releases/download/v1.0.22/libusb-1.0.22.7z
7z x -olibusb ./libusb-1.0.22.7z

Problem 4: The path provided to the "LDFLAGS" variable is not correct when looking at the downloaded libusb directory. The ".libs" folder does not exsis.
Fix: remove ".libs" from the LDFLAGS path.

New cmd with fix implemented:

export CPPFLAGS="$CPPFLAGS -I${PWD}/libusb/include/libusb-1.0"
export LDFLAGS="$LDFLAGS -L${PWD}/libusb/MinGW32/dll"

Build OpenOCD
Not sure, commands seem correct but the build fails at the "./configure" command because it is unable to find libusb1.0

cd ~/esp/openocd-esp32
export CPPFLAGS="$CPPFLAGS -D__USE_MINGW_ANSI_STDIO=1 -Wno-error"; export CFLAGS="$CFLAGS -Wno-error"
./bootstrap
./configure --disable-doxygen-pdf --enable-ftdi --enable-jlink --enable-ulink --build=i686-w64-mingw32 --host=i686-w64-mingw32
make
cp ../libusb/MinGW32/dll/libusb-1.0.dll ./src
cp /opt/i686-w64-mingw32/bin/libwinpthread-1.dll ./src

The build configuration fail message
It seems the build can't be configured because libusb cannot be found. I'm currently not able to fix this.
A full log to get to this point is provided to the end of this issue report.

checking for LIBUSB1... no
configure: WARNING: libusb-1.x not found, trying legacy libusb-0.1 as a fallback; consider installing libusb-1.x instead
checking for CAPSTONE... no
checking for HIDAPI... no
checking for HIDAPI... no
checking for HIDAPI... no
checking for LIBFTDI... no
checking for LIBFTDI... no
checking for LIBGPIOD... no
checking for LIBJAYLINK... no
configure: error: libusb-1.x is required for the MPSSE mode of FTDI based devices

Other
I'm new to MSYS and had a hard time figuring out were the files were located on my hard disk.
Were can I find the files in the "~" folder on windows?
C:\msys64\home\<username>\

Full log:
Full log when using the fixed commands, (still fails at "./configure" command)
log.txt

[erhankur]

@littleboot thank you for your valuable work and helping us to solve this. I will give a try from scratch on my windows machine and let you know how you can fix the libusb issue.

[igrr]

Perhaps we can update the instructions to reflect the steps we actually use to build Windows versions of OpenOCD? That is, we cross-compile it inside a Debian container, in Docker. That has the benefit that unlike "building on Windows" instructions we have now, we will actually be automatically verifying that the instructions still work.

[littleboot]

The help is much appreciated I'm looking forward to the solution. I'm not familiar with a Debian container in Docker but I will have a look at it tomorrow and report if I'm able to figure it out and get it to work. "Perhaps we can update the instructions to reflect the steps we actually use to build Windows versions of OpenOCD?" yes that would a good solution. Thank you both for the reply

[igrr]

@littleboot Could you describe what you are trying to achieve in the first place? Do you want to build from source instead of using release binaries for some specific reason? Maybe there's an easier way to solve your problem than having you build OpenOCD from source.

[littleboot]

@igrr I have a peculiar problem when I try to flash my device for a second time it fails. And flash needs to be erased using the serial bootloader (I'm not touching the JTAG GPIO's at all in this firmware, Its the unified provision example + very simple RMT code). I will file a new bug report about later today (Need to clean up my notes and gather the logs).
Edit: see #223

Side question:
Is it correct that there is no official way for erasing flash using the JTAG interface? (I don't want to use the serial bootloader it's my opinion Flashing Erasing Debugging should all be possible without the need to connect the serial interface ).

However my reasoning was as follows, I can't figure out the problem. It might be caused by OpenOcd. Lets build OpenOcd from the latest source. Try if it the problem goes away.
When I was reading the instruction for the first time I assumed I would be able to do this within an hour top. But then I tried building OpenOcd from source following the instruction by the letter. I noticed the instruction weren't correct so I went down the rabbit hole trying to get it to work. I gave up after much trying and filled this bug report to report my findings and hopefully get the instruction updated to spare someone else the trouble.
I just can't leave this alone now, knowing it is possible, I to be able to build OpenOcd from source!

[erhankur]

@littleboot you case erase the flash sectors from the JTAG interface.

flash erase_sector bank_id first_sector_num (last_sector_num|'last')

telnet localhost 4444
> halt
> flash erase_sector 0 0 last
PROF: Flash erased in 19239 ms
erased sectors 0 through 1023 on flash bank 0 in 19.238985s

[erhankur]

@littleboot to fix the libusb config issue you need to have pkgconfig directory under the libusb dir. You can have it by building the libusb or creating manually.

To build libusb, you can follow the below instructions that we use in the CI

    - export LIBUSB_VER=libusb-1.0.21
    - wget https://dl.espressif.com/dl/$LIBUSB_VER.tar.gz -O $LIBUSB_VER.tar.gz
    - tar xzf $LIBUSB_VER.tar.gz && rm $LIBUSB_VER.tar.gz
    - pushd $LIBUSB_VER
    - ./bootstrap.sh
    - ./configure --prefix=$PWD/dist --host=${CONF_HOST} --enable-shared=no --enable-static=yes
    - make
    - make install-strip
    - export PKG_CONFIG_PATH=$PWD/dist/lib/pkgconfig
    - popd

Or create manually as mentioned here

In the end you will have pkgconfig/libusb.pc file and you need to export it just before running the configuration script.

export PKG_CONFIG_PATH=$PWD/dist/lib/pkgconfig

[erhankur]

If someone wants to have windows built, can refer here. https://github.com/openocd-org/openocd/blob/master/.github/workflows/snapshot.yml

We are also planning to add action script here for each commit.