Skip to content

Conversation

@grantdfoster
Copy link
Collaborator

@grantdfoster grantdfoster commented Jul 29, 2025

1. Job Type System Refactoring

  • 16 commits with substantial changes to the core job type system
  • Moved from string-based job types to a more structured JobType system
  • Added proper type validation and argument unmarshalling capabilities
  • Enhanced separation between job types and their specific arguments

2. New Argument Handling Framework

  • Added args/jobs.go - New centralized job argument definitions
  • Added args/unmarshaller.go (173 new lines) - Sophisticated argument unmarshalling system
  • Enhanced existing argument files:
    • args/twitter.go (+98 lines)
    • args/tiktok.go (+112 lines)
    • args/linkedin.go (+74 lines)
    • args/web.go (+89 lines)

3. API Integration Improvements

  • Default capability handling with Twitter as the primary focus
  • Apify integration for getting followers and other social media operations
  • Single tweet operation support added
  • Better type validation across all job types

4. Platform-Specific Enhancements

  • TikTok validation improvements
  • LinkedIn cleanup and refinements
  • Twitter parameter flexibility (removed query requirements in some cases)
  • Profile type support for various platforms

5. Code Organization & Cleanup

  • Removed LinkedIn from certain job type categories
  • Cleaned up argument marshallers and job type definitions
  • Better validation logic across all supported platforms
  • Enhanced type safety throughout the system

Statistics:

  • 8 files changed
  • +760 lines added, -25 lines removed
  • Mix of new files and substantial enhancements to existing ones

@grantdfoster grantdfoster changed the title chore: adds profile type and apify caps feat: job argument unmarshaller and apify types Jul 30, 2025
@grantdfoster grantdfoster linked an issue Jul 30, 2025 that may be closed by this pull request
1 task
@grantdfoster grantdfoster requested review from mcamou and rapidfix August 5, 2025 19:10
Copy link
Collaborator

@mcamou mcamou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going well! A few comments/suggestions.

grantdfoster and others added 3 commits August 7, 2025 11:24
Co-authored-by: Mario Camou <mcamou@users.noreply.github.com>
Co-authored-by: Mario Camou <mcamou@users.noreply.github.com>
Co-authored-by: Mario Camou <mcamou@users.noreply.github.com>
@mudler mudler requested review from Copilot and mcamou August 8, 2025 06:47
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements a comprehensive refactoring of the job type system, introducing structured argument handling and validation capabilities for different job types including Web, TikTok, Twitter, and LinkedIn operations.

  • Added a centralized argument unmarshalling framework with job-specific validation
  • Enhanced job type system with proper capability validation and type safety
  • Added Apify integration support for Twitter operations with new data structures

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
types/twitter.go Adds ProfileResultApify struct with extensive Twitter profile data fields
types/jobs.go Refactors job type system with capability validation and structured job-capability mappings
args/web.go Implements WebSearchArguments with validation, URL parsing, and depth checking
args/unmarshaller.go Creates centralized job argument unmarshalling framework with type-specific interfaces
args/twitter.go Adds TwitterSearchArguments validation and operation type detection methods
args/tiktok.go Implements TikTokTranscriptionArguments with URL validation and language code checking
args/linkedin.go Adds LinkedInArguments validation with search and profile operation detection

Copy link
Collaborator

@mcamou mcamou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor comments. I'm fine with merging this as it is, but please have a look and see if you agree with the minor stuff here, or if we leave it for later.

@grantdfoster grantdfoster merged commit 603d35b into main Aug 11, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: create json unmarshaller for jobArgs

4 participants