esptool: add configurable connection attempts

[jkoelker]

Description of change

Allows specifying the connection attempts. A value of <= 0 will be treated as infinite attempts.

I have a few hundred boards that I need to flash that need to have their pins manually shorted (I forgot to tie IO0 and IO2 for use with the sdcard, or bring them out to a pad, doh). This allows me to start esptool and have it attempt an infinate amount of times until I bridge the pins.

I have tested this change with the following hardware & software combinations:

Linux ESP32 custom board.

I have run the esptool.py automated integration tests with this change and the above hardware. The results were:

I believe the failures were due to my custom programming board not working well with the baud rate.

Running esptool.py tests...                                    
test_flash_header_no_magic_no_rewrite (__main__.TestBootloaderHeaderRewriteCases) ... ok
test_flash_header_rewrite (__main__.TestBootloaderHeaderRewriteCases) ... ok
test_deep_sleep_flash (__main__.TestDeepSleepFlash)                                                                                                                                                                                                            
Regression test for https://github.com/espressif/esptool/issues/351 ... skipped 'ESP8266 only'                                                                                                                                                                 
test_chip_erase (__main__.TestErase) ... ok                                                                                    
test_large_region_erase (__main__.TestErase) ... ok                                                                            
test_region_erase (__main__.TestErase) ... ok                                                                                  
test_correct_offset (__main__.TestFlashDetection)                                                                              
Verify writing at an offset actually writes to that offset. ... ok                               
test_blank_efuse_encrypt_write_abort (__main__.TestFlashEncryption) ... ok
test_blank_efuse_encrypt_write_continue1 (__main__.TestFlashEncryption) ... ok                                                 
test_blank_efuse_encrypt_write_continue2 (__main__.TestFlashEncryption) ... expected failure
test_flash_size_keep (__main__.TestFlashSizes) ... ok
test_high_offset (__main__.TestFlashSizes) ... ok
test_high_offset_no_compression (__main__.TestFlashSizes) ... ok
test_invalid_size_arg (__main__.TestFlashSizes) ... ok
test_large_image (__main__.TestFlashSizes) ... ok     
test_large_no_compression (__main__.TestFlashSizes) ... ok
test_write_no_compression_past_end_fails (__main__.TestFlashSizes) ... ok
test_write_past_end_fails (__main__.TestFlashSizes) ... ok
test_adjacent_flash (__main__.TestFlashing) ... ok
test_adjacent_independent_flash (__main__.TestFlashing) ... ok
test_compressed_nostub_flash (__main__.TestFlashing) ... ERROR

Stdout:                                                        
**************************************************
Running /home/jkoelker/.local/share/virtualenvs/129062aec8de1bb/bin/python /home/jkoelker/projects/esptool/test/../esptool.py --chip esp32 --port /dev/ttyUSB0 --baud 230400 --no-stub write_flash -z 0x0 images/sector.bin 0x1000 images/fifty_kb.bin...
b'esptool.py v2.8-dev\nSerial port /dev/ttyUSB0\nConnecting....\nChip is ESP32D0WDQ5 (revision 1)\nFeatures: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None\nCrystal is 40MHz\nMAC: 30:ae:a4:c6:6e:2c\nChanging baud rate to 230400
\nChanged.\nEnabling default SPI flash mode...\n\nA fatal error occurred: Timed out waiting for packet header\n'               
test_compressible_file (__main__.TestFlashing) ... ok
test_correct_offset (__main__.TestFlashing)
Verify writing at an offset actually writes to that offset. ... ok
test_highspeed_flash (__main__.TestFlashing) ... ERROR

Stdout:                                                        
**************************************************
Running /home/jkoelker/.local/share/virtualenvs/129062aec8de1bb/bin/python /home/jkoelker/projects/esptool/test/../esptool.py --chip esp32 --port /dev/ttyUSB0 --baud 921600 write_flash 0x0 images/fifty_kb.bin...
b'esptool.py v2.8-dev\nSerial port /dev/ttyUSB0\nConnecting......\nChip is ESP32D0WDQ5 (revision 1)\nFeatures: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None\nCrystal is 40MHz\nMAC: 30:ae:a4:c6:6e:2c\nUploading stub...\nRunning
 stub...\nStub running...\nChanging baud rate to 921600\nChanged.\nConfiguring flash size...\n\nA fatal error occurred: Timed out waiting for packet content\n'
test_length_not_aligned_4bytes (__main__.TestFlashing) ... ok
test_length_not_aligned_4bytes_no_compression (__main__.TestFlashing) ... ok
test_no_compression_flash (__main__.TestFlashing) ... ok
test_partition_table_then_bootloader (__main__.TestFlashing) ... ok
test_partition_table_then_bootloader_no_compression (__main__.TestFlashing) ... ok
test_partition_table_then_bootloader_nostub (__main__.TestFlashing) ... ERROR

Stdout:                                                        
**************************************************
Running /home/jkoelker/.local/share/virtualenvs/129062aec8de1bb/bin/python /home/jkoelker/projects/esptool/test/../esptool.py --chip esp32 --port /dev/ttyUSB0 --baud 230400 --no-stub write_flash 0x4000 images/partitions_singleapp.bin...
b'esptool.py v2.8-dev\nSerial port /dev/ttyUSB0\nConnecting......\nChip is ESP32D0WDQ5 (revision 1)\nFeatures: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None\nCrystal is 40MHz\nMAC: 30:ae:a4:c6:6e:2c\nChanging baud rate to 2304
00\nChanged.\nEnabling default SPI flash mode...\n\nA fatal error occurred: Timed out waiting for packet header\n'             
test_short_flash (__main__.TestFlashing) ... ok
test_single_byte (__main__.TestFlashing) ... ok
test_write_no_overlap (__main__.TestFlashing) ... ok
test_write_overlap (__main__.TestFlashing) ... ok
test_write_sector_overlap (__main__.TestFlashing) ... ok
test_zero_length (__main__.TestFlashing) ... ok
test_detect_size_changes_size (__main__.TestKeepImageSettings) ... ok
test_explicit_set_size_freq_mode (__main__.TestKeepImageSettings) ... ok
test_keep_does_not_change_settings (__main__.TestKeepImageSettings) ... ok
test_load_ram (__main__.TestLoadRAM)
Verify load_ram command ... ok                                 
test_read_chip_id (__main__.TestReadIdentityValues) ... skipped 'ESP8266 only'
test_read_mac (__main__.TestReadIdentityValues) ... ok
test_end_sector (__main__.TestSectorBoundaries) ... ok
test_end_sector_uncompressed (__main__.TestSectorBoundaries) ... ok
test_overlap (__main__.TestSectorBoundaries) ... ok
test_verify_failure (__main__.TestVerifyCommand) ... ok
test_verify_success (__main__.TestVerifyCommand) ... ok
test_verify_unaligned_length (__main__.TestVerifyCommand) ... ok

======================================================================
ERROR: test_compressed_nostub_flash (__main__.TestFlashing)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./test_esptool.py", line 256, in test_compressed_nostub_flash
    self.run_esptool("--no-stub write_flash -z 0x0 images/sector.bin 0x1000 images/fifty_kb.bin")
  File "./test_esptool.py", line 82, in run_esptool
    raise e                                                    
  File "./test_esptool.py", line 77, in run_esptool
    output = subprocess.check_output([str(s) for s in cmd], cwd=TEST_DIR, stderr=subprocess.STDOUT)
  File "/usr/lib64/python3.7/subprocess.py", line 395, in check_output
    **kwargs).stdout                                           
  File "/usr/lib64/python3.7/subprocess.py", line 487, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/home/jkoelker/.local/share/virtualenvs/129062aec8de1bb/bin/python', '/home/jkoelker/projects/esptool/test/../esptool.py', '--chip', 'esp32', '--port', '/dev/ttyUSB0', '--baud', '230400', '--no-stub', 'write_flash
', '-z', '0x0', 'images/sector.bin', '0x1000', 'images/fifty_kb.bin']' returned non-zero exit status 2.                        

Stdout:                                                        
**************************************************
Running /home/jkoelker/.local/share/virtualenvs/129062aec8de1bb/bin/python /home/jkoelker/projects/esptool/test/../esptool.py --chip esp32 --port /dev/ttyUSB0 --baud 230400 --no-stub write_flash -z 0x0 images/sector.bin 0x1000 images/fifty_kb.bin...
b'esptool.py v2.8-dev\nSerial port /dev/ttyUSB0\nConnecting....\nChip is ESP32D0WDQ5 (revision 1)\nFeatures: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None\nCrystal is 40MHz\nMAC: 30:ae:a4:c6:6e:2c\nChanging baud rate to 230400
\nChanged.\nEnabling default SPI flash mode...\n\nA fatal error occurred: Timed out waiting for packet header\n'               

======================================================================
ERROR: test_highspeed_flash (__main__.TestFlashing)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./test_esptool.py", line 223, in test_highspeed_flash
    self.run_esptool("write_flash 0x0 images/fifty_kb.bin", baud=921600)
  File "./test_esptool.py", line 82, in run_esptool
    raise e                                                    
  File "./test_esptool.py", line 77, in run_esptool
    output = subprocess.check_output([str(s) for s in cmd], cwd=TEST_DIR, stderr=subprocess.STDOUT)
  File "/usr/lib64/python3.7/subprocess.py", line 395, in check_output
    **kwargs).stdout                                           
  File "/usr/lib64/python3.7/subprocess.py", line 487, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/home/jkoelker/.local/share/virtualenvs/129062aec8de1bb/bin/python', '/home/jkoelker/projects/esptool/test/../esptool.py', '--chip', 'esp32', '--port', '/dev/ttyUSB0', '--baud', '921600', 'write_flash', '0x0', 'im
ages/fifty_kb.bin']' returned non-zero exit status 2.          

Stdout:                                                        
**************************************************
Running /home/jkoelker/.local/share/virtualenvs/129062aec8de1bb/bin/python /home/jkoelker/projects/esptool/test/../esptool.py --chip esp32 --port /dev/ttyUSB0 --baud 921600 write_flash 0x0 images/fifty_kb.bin...
b'esptool.py v2.8-dev\nSerial port /dev/ttyUSB0\nConnecting......\nChip is ESP32D0WDQ5 (revision 1)\nFeatures: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None\nCrystal is 40MHz\nMAC: 30:ae:a4:c6:6e:2c\nUploading stub...\nRunning
 stub...\nStub running...\nChanging baud rate to 921600\nChanged.\nConfiguring flash size...\n\nA fatal error occurred: Timed out waiting for packet content\n'

======================================================================
ERROR: test_partition_table_then_bootloader_nostub (__main__.TestFlashing)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./test_esptool.py", line 274, in test_partition_table_then_bootloader_nostub
    self._test_partition_table_then_bootloader("--no-stub write_flash")
  File "./test_esptool.py", line 261, in _test_partition_table_then_bootloader
    self.run_esptool(args + " 0x4000 images/partitions_singleapp.bin")
  File "./test_esptool.py", line 82, in run_esptool
    raise e                                                    
  File "./test_esptool.py", line 77, in run_esptool
    output = subprocess.check_output([str(s) for s in cmd], cwd=TEST_DIR, stderr=subprocess.STDOUT)
  File "/usr/lib64/python3.7/subprocess.py", line 395, in check_output
    **kwargs).stdout                                           
  File "/usr/lib64/python3.7/subprocess.py", line 487, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/home/jkoelker/.local/share/virtualenvs/129062aec8de1bb/bin/python', '/home/jkoelker/projects/esptool/test/../esptool.py', '--chip', 'esp32', '--port', '/dev/ttyUSB0', '--baud', '230400', '--no-stub', 'write_flash
', '0x4000', 'images/partitions_singleapp.bin']' returned non-zero exit status 2.                                              

Stdout:                                                        
**************************************************
Running /home/jkoelker/.local/share/virtualenvs/129062aec8de1bb/bin/python /home/jkoelker/projects/esptool/test/../esptool.py --chip esp32 --port /dev/ttyUSB0 --baud 230400 --no-stub write_flash 0x4000 images/partitions_singleapp.bin...
b'esptool.py v2.8-dev\nSerial port /dev/ttyUSB0\nConnecting......\nChip is ESP32D0WDQ5 (revision 1)\nFeatures: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None\nCrystal is 40MHz\nMAC: 30:ae:a4:c6:6e:2c\nChanging baud rate to 2304
00\nChanged.\nEnabling default SPI flash mode...\n\nA fatal error occurred: Timed out waiting for packet header\n'             

----------------------------------------------------------------------
Ran 48 tests in 415.313s                                       

FAILED (errors=3, skipped=2, expected failures=1)

[projectgus]

Hi @jkoelker,

Sorry noone got back to you about this back in September.

The changes look good to me, have cherry-picked this to be merged before the v2.9 release. This PR will be updated once it's merged.

Angus

[projectgus]

Cherry-picked as 9e5fd48 with a couple of minor cleanup commits, thanks!