Feature/golang rewrite#17
Merged
Merged
Conversation
- Create Go project structure under go/ - Port shared constants, utils, thumbnails, file browser, job store - Set up gin web server with embedded static files - Port organizer module (date_sorter, renamer, scanner) - Add organizer API routes - Add Go Makefile targets - Single-pass filepath.WalkDir replaces 82+ Python rglob calls - Generic JobStore with TTL-based cleanup replaces leaky Python dicts - True goroutine parallelism throughout
…ses 4-7) - Inspector: EXIF read/strip via go-exif, integrity check via imaging + ffprobe - Resizer: fit/fill/stretch/pad modes using disintegration/imaging - Converter: image conversion via imaging, video via ffmpeg subprocess - Dupfinder: CPU-only perceptual hashing via goimagehash, video frame extraction - All modules use goroutine workers with semaphore concurrency control - Added shared utilities: IsImage, IsVideo, NormaliseExt - Registered all routes in server.go (inspector, resizer, converter, dupfinder) - Added corona10/goimagehash v1.1.0 dependency
- Shared: add GET /api/browse (in-page dir browser), POST /api/browse/native
- Shared: rename /api/system-info to /api/system_info, add ffmpeg encoders
- Shared: add GET /api/media for full-size media preview
- Organizer: rename /api/organizer/scan to /plan
- Organizer: execute accepts {job_id} in body instead of URL param
- Organizer: status returns unified 'plan' array, plan_count, conflicts, execution
- Organizer: PlanRename accepts start_seq parameter
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| } | ||
|
|
||
| // Avoid overwriting | ||
| if _, err := os.Stat(dest); err == nil { |
| } | ||
|
|
||
| // Avoid overwriting | ||
| if _, err := os.Stat(dest); err == nil { |
|
|
||
| cmd = append(cmd, dest) | ||
|
|
||
| out, err2 := exec.Command(cmd[0], cmd[1:]...).CombinedOutput() |
| return nil | ||
| }) | ||
| } else { | ||
| entries, err := os.ReadDir(folder) |
| } | ||
|
|
||
| origSize := int64(0) | ||
| if info, err := os.Stat(source); err == nil { |
| } | ||
|
|
||
| path = filepath.Clean(path) | ||
| info, err := os.Stat(path) |
| return | ||
| } | ||
|
|
||
| entries, _ := os.ReadDir(path) |
| } | ||
|
|
||
| filePath = filepath.Clean(filePath) | ||
| info, err := os.Stat(filePath) |
| if contentType == "" { | ||
| contentType = "application/octet-stream" | ||
| } | ||
| c.File(filePath) |
|
|
||
| // isDir returns true when path exists and is a directory. | ||
| func isDir(path string) bool { | ||
| info, err := os.Stat(path) |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR rewrites the project from a Python/Flask implementation to a Go implementation (Gin-based), embedding the web UI assets and re-implementing the converter/dupfinder/organizer backends in Go. It also migrates CI/docs tooling from Python/Sphinx to Go/Jekyll and removes the Python source + pytest suite.
Changes:
- Introduces a Go web server (
cmd/morphic,web/) serving embedded templates/static assets and exposing/api/*endpoints for shared, organizer, converter, and dupfinder features. - Adds Go implementations for shared utilities (jobs, scanning, thumbnails, native folder dialog) and module logic (converter, dupfinder, organizer).
- Removes Python source code, tests, and Sphinx docs; updates Makefile/CI/CodeQL/Dependabot/docs to Go + Jekyll.
Reviewed changes
Copilot reviewed 105 out of 128 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
| web/static/style.css | UI styling additions (indeterminate progress, converter/organizer tweaks, stop button, interrupted notice). |
| web/server.go | Gin router setup with embedded templates/static and no-cache middleware. |
| web/routes_shared.go | Shared API endpoints: browse, native browse, thumbnails, system info, media serving. |
| web/routes_organizer.go | Organizer API endpoints: plan/execute/status/cancel. |
| web/routes_dupfinder.go | Dupfinder API endpoints: scan/status/results/cancel/delete. |
| web/routes_converter.go | Converter API endpoints: scan/formats/convert/progress/poll/cancel/delete. |
| internal/shared/utils.go | Go replacements for shared scanning + formatting utilities. |
| internal/shared/thumbnails.go | Thumbnail generation (imaging + ffmpeg fallback) with in-memory cache. |
| internal/shared/jobs.go | Generic job + job store with cancellation context and TTL cleanup. |
| internal/shared/file_browser.go | OS-native folder dialog helpers (zenity/kdialog/osascript/powershell). |
| internal/shared/constants.go | Shared constants (extensions, thresholds, version, excluded folders, aliases). |
| internal/organizer/scanner.go | Organizer job planning/execution orchestration. |
| internal/organizer/renamer.go | Organizer rename templating + rename execution. |
| internal/organizer/date_sorter.go | Organizer EXIF/mtime date extraction + sort planning/execution. |
| internal/dupfinder/images.go | Image perceptual hashing and duplicate grouping. |
| internal/dupfinder/videos.go | Video probing + frame hashing and duplicate grouping. |
| internal/converter/scanner.go | Folder scan inventory for convertible media. |
| internal/converter/constants.go | Video container configs + conversion target tables. |
| internal/converter/converter_test.go | Go unit tests for scanner + image conversion + tables. |
| cmd/morphic/main.go | CLI entry-point: flags, starts server, optional browser open. |
| go.mod | Introduces Go module and dependencies (gin, imaging, go-exif, uuid, etc.). |
| Makefile | Replaces Python/uv targets with Go build/test/vet/run/tidy targets. |
| docs/index.md | New Jekyll/GitHub Pages landing page + API summary. |
| docs/Gemfile | Jekyll/GitHub Pages Ruby dependencies. |
| docs/_config.yml | Jekyll site configuration. |
| docs/index.rst | Removes Sphinx index. |
| docs/getting_started.rst | Removes Sphinx getting started doc. |
| docs/development.rst | Removes Sphinx development doc. |
| docs/configuration.rst | Removes Sphinx configuration doc. |
| docs/conf.py | Removes Sphinx configuration. |
| docs/cli.rst | Removes Sphinx CLI doc. |
| docs/changelog.rst | Removes Sphinx changelog. |
| docs/api/shared.rst | Removes Sphinx API docs. |
| docs/api/frontend.rst | Removes Sphinx API docs. |
| docs/api/dupfinder.rst | Removes Sphinx API docs. |
| docs/api/converter.rst | Removes Sphinx API docs. |
| .github/workflows/CI.yml | Migrates CI to Go (go test, go vet, build) and coverage upload changes. |
| .github/workflows/codeql.yml | Switches CodeQL analysis to Go + workflow scope updates. |
| .github/workflows/documentation.yml | Migrates docs build/deploy to Jekyll (Ruby) and updates paths. |
| .github/workflows/release.yml | Adds release workflow generating release notes and publishing releases. |
| .github/dependabot.yml | Switches Dependabot ecosystem from pip to gomod. |
| .vscode/settings.json | Switches IDE settings from Python pytest to Go test explorer. |
| .idea/misc.xml | Removes Python-specific IDE settings. |
| .gitignore | Replaces Python-focused ignores with Go/GoLand/VSCode ignores. |
| tests/conftest.py | Removes pytest fixtures. |
| tests/shared/test_utils.py | Removes Python shared utils tests. |
| tests/shared/test_thumbnails.py | Removes Python thumbnail tests. |
| tests/shared/test_file_browser.py | Removes Python file browser tests. |
| tests/shared/test_constants.py | Removes Python constants tests. |
| tests/resizer/test_scanner.py | Removes Python resizer tests. |
| tests/resizer/test_operations.py | Removes Python resizer tests. |
| tests/organizer/test_renamer.py | Removes Python organizer tests. |
| tests/organizer/test_date_sorter.py | Removes Python organizer tests. |
| tests/inspector/test_scanner.py | Removes Python inspector tests. |
| tests/inspector/test_integrity.py | Removes Python inspector tests. |
| tests/inspector/test_exif.py | Removes Python inspector tests. |
| src/morphic/init.py | Removes Python package root. |
| src/morphic/shared/constants.py | Removes Python shared constants. |
| src/morphic/shared/utils.py | Removes Python shared utils. |
| src/morphic/shared/thumbnails.py | Removes Python thumbnails. |
| src/morphic/shared/file_browser.py | Removes Python native dialog code. |
| src/morphic/shared/init.py | Removes Python shared exports. |
| src/morphic/converter/constants.py | Removes Python converter constants. |
| src/morphic/converter/scanner.py | Removes Python converter scanner. |
| src/morphic/converter/init.py | Removes Python converter exports. |
| src/morphic/dupfinder/init.py | Removes Python dupfinder exports. |
| src/morphic/organizer/date_sorter.py | Removes Python organizer date sorter. |
| src/morphic/organizer/renamer.py | Removes Python organizer renamer. |
| src/morphic/organizer/scanner.py | Removes Python organizer scanner. |
| src/morphic/organizer/init.py | Removes Python organizer exports. |
| src/morphic/resizer/operations.py | Removes Python resizer ops. |
| src/morphic/resizer/scanner.py | Removes Python resizer scanner. |
| src/morphic/resizer/init.py | Removes Python resizer exports. |
| src/morphic/inspector/exif.py | Removes Python inspector EXIF support. |
| src/morphic/inspector/integrity.py | Removes Python inspector integrity checks. |
| src/morphic/inspector/scanner.py | Removes Python inspector scanner. |
| src/morphic/inspector/init.py | Removes Python inspector exports. |
| src/morphic/frontend/app.py | Removes Flask app factory/CLI. |
| src/morphic/frontend/routes_shared.py | Removes Flask shared routes. |
| src/morphic/frontend/routes_converter.py | Removes Flask converter routes. |
| src/morphic/frontend/routes_dupfinder.py | Removes Flask dupfinder routes. |
| src/morphic/frontend/routes_organizer.py | Removes Flask organizer routes. |
| src/morphic/frontend/routes_inspector.py | Removes Flask inspector routes. |
| src/morphic/frontend/routes_resizer.py | Removes Flask resizer routes. |
| src/morphic/frontend/main.py | Removes Python module entry-point. |
| src/morphic/frontend/init.py | Removes Flask frontend exports. |
Files not reviewed (1)
- .idea/misc.xml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+28
to
+37
| path := c.Query("path") | ||
| if path == "" { | ||
| home, _ := os.UserHomeDir() | ||
| path = home | ||
| } | ||
|
|
||
| path = filepath.Clean(path) | ||
| info, err := os.Stat(path) | ||
| if err != nil || !info.IsDir() { | ||
| c.JSON(http.StatusBadRequest, gin.H{"error": "Not a directory"}) |
Comment on lines
+77
to
+83
| // handleBrowseNative opens the OS-native folder picker dialog. | ||
| func handleBrowseNative(c *gin.Context) { | ||
| folder, available, err := shared.OpenNativeFolderDialog() | ||
| if err != nil { | ||
| c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()}) | ||
| return | ||
| } |
Comment on lines
+189
to
+194
| contentType := mime.TypeByExtension(filepath.Ext(filePath)) | ||
| if contentType == "" { | ||
| contentType = "application/octet-stream" | ||
| } | ||
| c.File(filePath) | ||
| } |
Comment on lines
+23
to
+33
| // Serve static files from embedded FS | ||
| staticFS, _ := fs.Sub(webFS, "static") | ||
| r.StaticFS("/static", http.FS(staticFS)) | ||
|
|
||
| // No-cache middleware (mirrors Python's @app.after_request) | ||
| r.Use(func(c *gin.Context) { | ||
| c.Header("Cache-Control", "no-cache, no-store, must-revalidate") | ||
| c.Header("Pragma", "no-cache") | ||
| c.Header("Expires", "0") | ||
| c.Next() | ||
| }) |
Comment on lines
+116
to
+131
| switch job.Mode { | ||
| case "sort": | ||
| dest := job.Destination | ||
| if dest == "" { | ||
| dest = job.Folder | ||
| } | ||
| plan := PlanSort(paths, job.Template, dest) | ||
| job.mu.Lock() | ||
| job.SortPlan = plan | ||
| job.mu.Unlock() | ||
| case "rename": | ||
| plan := PlanRename(paths, job.Template, job.Operation, job.StartSeq) | ||
| job.mu.Lock() | ||
| job.RenamePlan = plan | ||
| job.mu.Unlock() | ||
| } |
Comment on lines
+15
to
+21
| type conversionJob struct { | ||
| shared.Job | ||
| Total int `json:"total"` | ||
| Completed int `json:"completed"` | ||
| CurrentFile string `json:"current_file"` | ||
| Results []map[string]interface{} `json:"results"` | ||
| } |
Comment on lines
+205
to
+213
| lastStr := c.Query("last") | ||
| last := -1 | ||
| if lastStr != "" { | ||
| for i := 0; i < len(lastStr); i++ { | ||
| if lastStr[i] >= '0' && lastStr[i] <= '9' { | ||
| last = last*10 + int(lastStr[i]-'0') | ||
| } | ||
| } | ||
| } |
Comment on lines
+291
to
+300
| func absPath(p string) (string, error) { | ||
| abs, err := os.Getwd() | ||
| if err != nil { | ||
| return p, err | ||
| } | ||
| if len(p) > 0 && p[0] == '/' { | ||
| return p, nil | ||
| } | ||
| return abs + "/" + p, nil | ||
| } |
Comment on lines
+18
to
+28
| var VideoContainers = []VideoContainerConfig{ | ||
| { | ||
| Name: "MP4", | ||
| Codecs: []string{"h264", "h265", "av1"}, | ||
| Extensions: []string{".mp4", ".m4a", ".m4p", ".m4b", ".m4r", ".m4v"}, | ||
| }, | ||
| { | ||
| Name: "Matroska", | ||
| Codecs: []string{"h264", "h265", "av1", "vp9"}, | ||
| Extensions: []string{".mkv", ".mk3d", ".mka", ".mks"}, | ||
| }, |
Comment on lines
+101
to
+121
| for i := 0; i < numFrames; i++ { | ||
| ts := startTime + float64(i+1)*interval | ||
| frameFile := fmt.Sprintf("/tmp/morphic_frame_%d_%d.jpg", os.Getpid(), i) | ||
|
|
||
| cmd := exec.Command("ffmpeg", "-y", | ||
| "-ss", fmt.Sprintf("%.3f", ts), | ||
| "-i", path, | ||
| "-vframes", "1", | ||
| "-q:v", "2", | ||
| frameFile, | ||
| ) | ||
| cmd.Stdout = nil | ||
| cmd.Stderr = nil | ||
|
|
||
| if err := cmd.Run(); err != nil { | ||
| continue | ||
| } | ||
|
|
||
| img, err := imaging.Open(frameFile) | ||
| os.Remove(frameFile) | ||
| if err != nil { |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.