Skip to content

fix(storage): respect STORAGE_READ_FROM env var regardless of backend mode#9

Merged
nitaibezerra merged 2 commits intomainfrom
fix/storage-adapter-read-from
Dec 26, 2025
Merged

fix(storage): respect STORAGE_READ_FROM env var regardless of backend mode#9
nitaibezerra merged 2 commits intomainfrom
fix/storage-adapter-read-from

Conversation

@nitaibezerra
Copy link
Contributor

Summary

  • Fix StorageAdapter to properly use STORAGE_READ_FROM environment variable
  • Previously, STORAGE_READ_FROM was ignored when STORAGE_BACKEND was not set to dual_write
  • Now STORAGE_READ_FROM takes precedence when explicitly set

This fixes the upload-to-cogfy step which was still reading from HuggingFace despite STORAGE_READ_FROM=postgres.

Test plan

  • Run pipeline with agencies filter to verify upload reads from PostgreSQL

🤖 Generated with Claude Code

nitaibezerra and others added 2 commits December 26, 2025 16:38
… mode

The StorageAdapter was ignoring STORAGE_READ_FROM when STORAGE_BACKEND
was not set to dual_write. Now STORAGE_READ_FROM takes precedence when
explicitly set, allowing read source to be configured independently.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Use proper SQL date casting with ::date for date range filtering
- Remove dependency on Django-style filter operators (__gte, __lte)
- Query now properly includes all records within the date range

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@nitaibezerra nitaibezerra merged commit 7dba184 into main Dec 26, 2025
1 of 2 checks passed
mauriciomendonca pushed a commit that referenced this pull request Mar 20, 2026
Correcoes baseadas no code review:

- #4 (ALTO): Remover BEGIN/COMMIT explicito do rollback SQL 005 que
  quebrava o commit atomico do runner (COMMIT prematuro antes do
  registro em migration_history)

- #1 (MEDIO): Remover server-side cursor desnecessario em
  006_migrate_unique_ids.py — fetchall() carrega tudo em memoria
  de qualquer forma, o name= so adiciona confusao semantica

- #9 (MEDIO): Sanitizar target_version com regex ^[0-9]{3}$ no
  workflow e substituir eval $CMD por execucao direta com bash
  array para prevenir command injection

- #5 (BAIXO): Extrair _execute_with_history() em migrate.py para
  eliminar ~60% de duplicacao entre execute_migration() e
  execute_rollback(), reduzindo risco de divergencia futura

Todos os 44 testes continuam passando.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant