refactor: refactor StatefulApiExtractor to fix busy buffer and EOF errors

[caioq]

Summary

This PR updates the StatefulApiExtractor.Execute() method to avoid using a long-lived cursor while writing to the database. Previously, the cursor was kept open during data iteration and insertions, which led to errors like unexpected EOF and busy buffer. This change introduces a two-phase process: first, we collect all raw data IDs using a cursor and then close it. After that, each row is loaded individually by ID and processed safely. This avoids concurrency issues with the database connection and improves stability for high-volume data extractions.

Does this close any open issues?

Closes #7826

Screenshots

Testing with Github connection

Testing with Jira connection

Other Information

This issue was observed during the extractIssues step in the Jira plugin. It occurred consistently on subsequent executions of the task, but not on the first run after restarting the application.
This change ensures a more robust extraction flow without requiring increased max_allowed_packet settings or other MySQL tuning.

[klesh]

Does the error disappear after this patch?
I would recommend to use db.Pluck to fetch all ids at once instead of using db.Cursor

[caioq]

Does the error disappear after this patch? I would recommend to use db.Pluck to fetch all ids at once instead of using db.Cursor

Good idea, done a32009a

Yes, I tested it locally and the error disappeared.
I tested it with 8K Jira Issues and the following database configuration:

mysql:
    image: mysql:8
    volumes:
      - mysql-storage:/var/lib/mysql
    restart: always
    ports:
      - 3306:3306
    environment:
      MYSQL_ROOT_PASSWORD: admin
      MYSQL_DATABASE: lake
      MYSQL_USER: merico
      MYSQL_PASSWORD: merico
      TZ: UTC
    command: --character-set-server=utf8mb4
      --collation-server=utf8mb4_bin
      --skip-log-bin
      --max_allowed_packet=9M
      --net_read_timeout=6
      --net_write_timeout=6

Without this fix, this scenario was already enough to have the error in subsequent executions.

[klesh]

Nice work. Thanks for your contribution.