Skip to content

Use requests, bug fix and additional user parameters

Choose a tag to compare

@gdherbert gdherbert released this 07 May 00:52
· 12 commits to main since this release

1. HTTP Library Migration: urllib → requests

  • Replaced urllib.request with requests library for HTTP operations
  • Benefits:
    • Built-in automatic retry mechanism with exponential backoff
    • Better handling of large HTTP responses (fixes IncompleteRead errors)
    • Cleaner, more Pythonic API (e.g., response.json() vs manual decoding)
    • Session-based connection pooling for improved performance

2. SSL Certificate Verification

  • Replaced hardcoded verify=False with intelligent verification logic
  • Features:
    • Verification enabled by default (secure by default)
    • Optional custom CA bundle path support
    • Explicit "ignore verification" flag only when requested
    • Automatic suppression of InsecureRequestWarning when verification is disabled

3. Improved Error Handling

  • Fixed get_all_the_layers() function logic to properly distinguish between success and error responses
  • Added response.raise_for_status() for explicit HTTP error detection
  • Better error messages with included error details from API responses

4. Code Modernization

  • Converted all .format() string operations to f-strings for improved readability

5. Automatic Workspace Creation

  • Script now automatically creates output workspace (folder or fgdb) if it doesn't exist
  • Supports:
    • File geodatabase creation (.gdb)
    • Folder creation
    • Error handling for SDE connections

6. Control overwrite

  • Script now allows user to overwrite the default overwrite behavior. If unchecked it will skip if the output featureclass already exists.

7. Toolbox Parameter Updates

  • Added new optional parameters:
    • ignore_ssl_verification (bool, default: False)
    • ca_bundle_path (string, default: empty)
    • overwrite_output (bool, default: True)

Dependency Changes

  • Added: requests library
  • Added: urllib3.exceptions.InsecureRequestWarning handling
  • Retained: urllib.parse (for URL manipulation, not HTTP)
  • Removed: ssl module (replaced by requests/urllib3 SSL handling)

Breaking Changes

None - fully backward compatible. Default behavior is more secure than previous versions.

Known Issues Fixed

  • http.client.IncompleteRead errors when downloading large datasets
  • SSL verification warnings in ArcGIS Pro
  • Error handling in service layer discovery