Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 24 additions & 16 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,22 +70,24 @@ jobs:
set -e
mkdir -p build

# Compute short commit hash
# Compute version info
GIT_COMMIT=$(git rev-parse --short HEAD 2>/dev/null || echo "unknown")
VERSION="dev-${GIT_COMMIT}"
BUILD_TIME=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
MODULE="github.com/fbz-tec/pgxport"

# ldflags for internal/version package
LDFLAGS="-X ${MODULE}/internal/version.AppVersion=${VERSION} \
-X ${MODULE}/internal/version.GitCommit=${GIT_COMMIT} \
-X ${MODULE}/internal/version.BuildTime=${BUILD_TIME}"

echo "Building with version: ${VERSION}"

# Build Linux binary
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags="-X main.Version=${VERSION} \
-X main.GitCommit=${GIT_COMMIT} \
-X main.BuildTime=$(date -u +'%Y-%m-%dT%H:%M:%SZ')" \
-o build/pgxport

CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags="${LDFLAGS}" -o build/pgxport

# Build Windows binary
GOOS=windows GOARCH=amd64 go build -ldflags="-X main.Version=${VERSION} \
-X main.GitCommit=${GIT_COMMIT} \
-X main.BuildTime=$(date -u +'%Y-%m-%dT%H:%M:%SZ')" \
-o build/pgxport.exe
GOOS=windows GOARCH=amd64 go build -ldflags="${LDFLAGS}" -o build/pgxport.exe

- name: 📤 Upload Linux artifact
if: github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/heads/release/')
Expand All @@ -104,7 +106,7 @@ jobs:
retention-days: 3

- name: 🧹 Cleanup build directory
if: always() # Ensures cleanup runs even if previous step fails
if: always()
run: |
echo "🧹 Cleaning up build artifacts..."
rm -rf build
Expand Down Expand Up @@ -133,7 +135,12 @@ jobs:
VERSION="${{ github.ref_name }}"
BUILD_TIME=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
GIT_COMMIT=$(git rev-parse --short HEAD)
LDFLAGS="-s -w -X main.Version=${VERSION} -X main.BuildTime=${BUILD_TIME} -X main.GitCommit=${GIT_COMMIT}"
MODULE="github.com/fbz-tec/pgxport"

LDFLAGS="-s -w \
-X ${MODULE}/internal/version.AppVersion=${VERSION} \
-X ${MODULE}/internal/version.BuildTime=${BUILD_TIME} \
-X ${MODULE}/internal/version.GitCommit=${GIT_COMMIT}"

echo "Building binaries for version ${VERSION}..."

Expand Down Expand Up @@ -202,9 +209,10 @@ jobs:
- name: 🧮 Generate checksums
run: |
cd dist
for file in *; do
sha256sum "$file" > "$file.sha256"
done
sha256sum *.tar.gz *.zip > checksums.txt
echo ""
echo "✅ Checksums generated:"
cat checksums.txt

- name: 🚀 Create GitHub Release
uses: softprops/action-gh-release@v2
Expand Down Expand Up @@ -240,4 +248,4 @@ jobs:
dist/*.zip
dist/checksums.txt
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
5 changes: 2 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,16 +21,15 @@ This is the first pre-release of pgxport.
- Custom date/time formats and timezone support
- SQL export with schema-qualified table names
- Batch INSERT statements for SQL exports (`--insert-batch`) for improved import performance
- Comprehensive test coverage
- CI/CD pipeline with automated builds

#### Installation

```bash
go install github.com/fbz-tec/pgxport@v1.0.0
```

Or download pre-built binaries from [GitHub Releases](https://github.com/fbz-tec/pgxport/releases/tag/untagged-3731b225ccbb85fa3000).
Or download pre-built binaries from [GitHub Releases](https://github.com/fbz-tec/pgxport/releases/tag/v1.0.0-rc1).


---

Expand Down
181 changes: 129 additions & 52 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,8 +153,8 @@ pgxport [command] [flags]
| `--delimiter` | `-d` | CSV delimiter character | `,` | No |
| `--no-header` | - | Skip CSV header row in output | `false` | No |
| `--with-copy` | - | Use PostgreSQL native COPY for CSV export (faster for large datasets) | `false` | No |
| `--xml-root-tag` | - | Sets the root XML element name | `results` | No |
| `--xml-row-tag` | - | Sets the row XML element name | `row` | No |
| `--xml-root-tag` | - | Sets the root element name for XML exports | `results` | No |
| `--xml-row-tag` | - | Sets the row element name for XML exports | `row` | No |
| `--fail-on-empty` | - | Exit with error if query returns 0 rows | `false` | No |
| `--table` | `-t` | Table name for SQL INSERT exports (supports schema.table) | - | For SQL format |
| `--insert-batch` | - | Number of rows per INSERT statement for SQL exports | `1` | No |
Expand Down Expand Up @@ -626,69 +626,128 @@ INSERT INTO "users" ("id", "name", "email", "created_at") VALUES

```
pgxport/
├── exporters/ # Modular export package
│ ├── exporter.go # Interface and factory
│ ├── compression.go # Compression writers (gzip,zip)
│ ├── common.go # Shared utilities
│ ├── csv_exporter.go # CSV export implementation
│ ├── json_exporter.go# JSON export implementation
│ ├── xml_exporter.go # XML export implementation
│ └── sql_exporter.go # SQL export implementation
├── logger/ # Logging package
│ └── logger.go # Logger interface and implementation
├── main.go # CLI entry point and orchestration
├── config.go # Configuration management with validation
├── store.go # Database operations (connection, queries)
├── version.go # Version information
├── go.mod # Go module definition
├── go.sum # Go module checksums
├── LICENSE # MIT license file
└── README.md # Documentation
├── cmd/ # CLI entry points
│ ├── root.go # Main command + flags
│ ├── root_test.go
│ └── version.go # Version subcommand
├── core/ # Business logic
│ ├── exporter/ # Export formats (pluggable)
│ │ ├── registry.go # Format registration system
│ │ ├── formatting.go # Shared formatting utilities
│ │ ├── compression.go # Compression support (gzip/zip)
│ │ ├── options.go # Export options struct
│ │ ├── testing_helpers.go
│ │ ├── csv_exporter.go # CSV export implementation
│ │ ├── json_exporter.go # JSON export implementation
│ │ ├── xml_exporter.go # XML export implementation
│ │ └── sql_exporter.go # SQL export implementation
│ │
│ ├── db/ # Database operations
│ │ ├── connection.go # PostgreSQL connection management
│ │ └── connection_test.go
│ │
│ ├── config/ # Configuration management
│ │ ├── config.go # Config loading with validation
│ │ └── config_test.go
│ │
│ └── validation/ # Input validation
│ ├── query_safety.go # Query and parameter validation
│ └── query_safety_test.go
├── internal/ # Private packages
│ ├── logger/ # Logging utilities
│ │ └── logger.go # Structured logging with verbose mode
│ └── version/ # Build information
│ └── version.go # Version, BuildTime, GitCommit
├── main.go # Application entry point
├── go.mod # Go module definition
├── go.sum # Go module checksums
├── Taskfile.yml # Build automation
├── LICENSE # MIT license
└── README.md # This file
```

## 🧩 Architecture

The project follows a clean, modular architecture with separated concerns:
The project follows a clean, layered architecture with clear separation of concerns:

```mermaid
flowchart TD
A[CLI - Cobra] --> B[main.go<br/>Orchestration]
B --> C[config.go<br/>Configuration]
B --> D[store.go<br/>DB Operations]
B --> E[exporters/<br/>Export Logic]
A[CLI - Cobra] --> B[cmd/root.go<br/>Command Handler]
B --> C[core/config<br/>Configuration]
B --> D[core/db<br/>DB Connection]
B --> E[core/exporter<br/>Export Logic]

E --> E1[CSV Exporter]
E --> E2[JSON Exporter]
E --> E3[XML Exporter]
E --> E4[SQL Exporter]
E --> E0[registry.go<br/>Format Registry]
E0 --> E1[CSV Exporter]
E0 --> E2[JSON Exporter]
E0 --> E3[XML Exporter]
E0 --> E4[SQL Exporter]

E --> F[compression.go<br/>gzip/zip]
E --> G[common.go<br/>Shared Utils]
E --> F[formatting.go<br/>Shared Utils]
E --> G[compression.go<br/>gzip/zip]

B --> H[logger/<br/>Logging]
B --> H[internal/logger<br/>Logging]
B --> I[internal/version<br/>Build Info]

D --> J[core/validation<br/>Query Safety]

style B fill:#e1f5ff
style E fill:#ffe1f5
style D fill:#f5ffe1
style C fill:#fff4e1
```

**Architecture Principles:**

- **Layered Structure**: Clear separation between CLI, business logic, and utilities
- **Pluggable Exporters**: Registry pattern allows easy addition of new formats
- **SOLID Principles**: Each package has a single, well-defined responsibility
- **Testability**: Modular design facilitates comprehensive testing

**Component Descriptions:**

- **`exporters/`**: Modular export package with Strategy pattern
- **`exporter.go`**: Defines the `Exporter` interface and factory
- **`compression.go`**: Handles output compression (gzip, zip)
- **`common.go`**: Shared formatting utilities for all exporters
- **`csv_exporter.go`**: CSV export implementation
- **`json_exporter.go`**: JSON export implementation
- **`xml_exporter.go`**: XML export implementation
- **`sql_exporter.go`**: SQL INSERT export implementation
- **`logger/`**: Logging package with structured output
- **`logger.go`**: Logger interface and singleton implementation with debug/verbose support
- **`store.go`**: Handles all database operations (connect, query, return results)
- **`main.go`**: Orchestrates the flow between store and exporters
- **`config.go`**: Manages configuration with validation, defaults, and `.env` file loading

Each exporter is isolated in its own file, making the codebase easy to maintain, test, and extend with new formats.
### CLI Layer (`cmd/`)
- **`root.go`**: Main command orchestration with Cobra framework
- **`version.go`**: Version information subcommand

### Core Business Logic (`core/`)

**`exporter/`** - Export format implementations
- **`registry.go`**: Dynamic format registration using factory pattern
- **`formatting.go`**: Shared formatting utilities (dates, escaping, etc.)
- **`compression.go`**: Output compression (gzip, zip)
- **`options.go`**: Export configuration options
- **`csv_exporter.go`**: CSV format with COPY mode support
- **`json_exporter.go`**: JSON array format
- **`xml_exporter.go`**: XML format with customizable tags
- **`sql_exporter.go`**: SQL INSERT statements with batch support

**`db/`** - PostgreSQL operations
- **`connection.go`**: Database connection management and query execution

**`config/`** - Application configuration
- **`config.go`**: Configuration loading with `.env` support and validation

**`validation/`** - Input validation
- **`query_safety.go`**: Query and parameter validation

### Internal Utilities (`internal/`)

**`logger/`** - Structured logging
- **`logger.go`**: Logger implementation with verbose mode support

**`version/`** - Build metadata
- **`version.go`**: Version information set via ldflags during build

### Key Design Patterns

1. **Registry Pattern**: Exporters self-register at init time, enabling dynamic format support
2. **Factory Pattern**: Each export creates a fresh instance, avoiding state sharing
3. **Strategy Pattern**: Exporters implement a common interface for interchangeable behavior
4. **Dependency Injection**: Components receive dependencies rather than creating them

## 🛠️ Development

Expand All @@ -713,9 +772,13 @@ The project uses the following main dependencies:

```bash
go mod download
go mod tidy
```

The project structure follows clean architecture principles:
- `cmd/` - CLI commands and flags
- `core/` - Business logic (exporter, database, config, validation)
- `internal/` - Private utilities (logger, version)

**3. Configure your database**

Create a `.env` file:
Expand Down Expand Up @@ -744,7 +807,17 @@ go build -o pgxport
go build -o pgxport

# Build with version information
go build -ldflags="-X main.Version=1.0.0" -o pgxport
VERSION=$(git describe --tags --always --dirty 2>/dev/null || echo "dev")
BUILD_TIME=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
GIT_COMMIT=$(git rev-parse --short HEAD 2>/dev/null || echo "unknown")

go build -ldflags="-X github.com/fbz-tec/pgxport/internal/version.AppVersion=${VERSION} \
-X github.com/fbz-tec/pgxport/internal/version.BuildTime=${BUILD_TIME} \
-X github.com/fbz-tec/pgxport/internal/version.GitCommit=${GIT_COMMIT}" \
-o pgxport

# Using Taskfile (recommended)
task build

# Cross-platform builds
GOOS=linux GOARCH=amd64 go build -o pgxport-linux
Expand Down Expand Up @@ -844,9 +917,13 @@ Contributions are welcome! Please feel free to submit a Pull Request.

- Follow Go conventions and use `gofmt`
- Add comments for exported functions
- Keep functions small and focused
- Separate concerns (database vs export logic)
- Write tests for new features
- Keep functions small and focused (single responsibility principle)
- Follow the layered architecture:
- `cmd/` - CLI logic only
- `core/` - Business logic
- `internal/` - Reusable utilities
- New export formats should implement the `Exporter` interface and register via `registry.go`
- Write tests for new features (`*_test.go` files alongside source)

## 📄 License

Expand Down
2 changes: 1 addition & 1 deletion Taskfile.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ vars:
sh: date -u '+%Y-%m-%d_%H:%M:%S'
GIT_COMMIT:
sh: git rev-parse --short HEAD 2>/dev/null || echo "unknown"
LDFLAGS: -ldflags "-X main.Version={{.VERSION}} -X main.BuildTime={{.BUILD_TIME}} -X main.GitCommit={{.GIT_COMMIT}}"
LDFLAGS: -ldflags "-X github.com/fbz-tec/pgxport/internal/version.AppVersion={{.VERSION}} -X github.com/fbz-tec/pgxport/internal/version.BuildTime={{.BUILD_TIME}} -X github.com/fbz-tec/pgxport/internal/version.GitCommit={{.GIT_COMMIT}}"

tasks:
default:
Expand Down
Loading