feat: linkedin

[grantdfoster]

Pull Request Summary: LinkedIn Integration for TEE Worker

🎯 Overview

This PR implements comprehensive LinkedIn integration support in the tee-worker repository, enabling LinkedIn profile scraping and search capabilities through the Apify platform. The implementation follows the established patterns for other job types while adding LinkedIn-specific functionality.

📊 Changes Summary

• 13 files changed: 889 additions, 52 deletions
• New LinkedIn job processing with full Apify integration
• Comprehensive test coverage with 359 lines of tests
• Enhanced statistics tracking for LinkedIn operations
• Updated dependencies and build configuration

🚀 Key Features Added

1. LinkedIn Job Processing (internal/jobs/linkedin.go)

• 108 lines of LinkedIn job execution logic
• Apify Integration: Uses harvestapi~linkedin-profile-search actor
• Capability Support: Implements CapSearchByProfile capability
• Error Handling: Comprehensive error handling with proper logging
• Configuration: Requires Apify API key for LinkedIn operations
• Interface Design: Clean interface for testability and mocking

2. LinkedIn Apify Client (internal/jobs/linkedinapify/client.go)

• 83 lines of Apify client implementation
• Profile Search: Executes LinkedIn profile searches via Apify
• Data Processing: Converts Apify responses to structured Profile types
• Statistics Integration: Tracks queries, profiles, and errors
• Cursor Support: Implements pagination with cursor-based navigation
• Validation: API key validation for LinkedIn operations

3. Comprehensive Test Suite (internal/jobs/linkedin_test.go)

• 359 lines of comprehensive test coverage
• Mock Implementation: Complete mock LinkedIn Apify client
• Test Scenarios:
  • Successful profile searches
  • Error handling (missing API key, invalid arguments, client errors)
  • Data marshalling and unmarshalling
  • Capability reporting
• Ginkgo/Gomega: Modern BDD testing framework
• Edge Cases: Covers various failure scenarios and edge cases

4. Enhanced Statistics Tracking (internal/jobs/stats/stats.go)

• 3 new stat types added:
  • LinkedInProfiles: Tracks returned profile count
  • LinkedInQueries: Tracks search query count
  • LinkedInErrors: Tracks error occurrences
• Real-time Metrics: Integrated with existing stats collection system
• Worker Tracking: Per-worker statistics for monitoring

5. Apify Actor Configuration (internal/apify/actors.go)

• LinkedIn Actor: Added harvestapi~linkedin-profile-search actor
• Capability Mapping: Maps LinkedIn job type to search profile capability
• Actor Registry: Integrated with existing actor management system
• Default Configuration: Proper default input setup

6. Job Server Integration (internal/jobserver/jobserver.go)

• LinkedIn Job Support: Added LinkedIn job type to job server
• Worker Registration: Integrated with job worker system
• Capability Detection: Dynamic capability reporting

🔧 Technical Improvements

Build System Updates

• Go Version: Updated to Go 1.24.0 with toolchain 1.24.6
• Ego Framework: Updated to v1.8.0 for TEE support
• Dependencies: Updated all dependencies to latest versions
• Test Target: Added test-linkedin Makefile target

Code Quality

• Interface Design: Clean interfaces for testability
• Error Handling: Comprehensive error handling throughout
• Logging: Structured logging with proper context
• Type Safety: Strong typing with proper validation

Testing Infrastructure

• Mock Framework: Complete mock implementation for testing
• Test Coverage: Comprehensive test scenarios
• BDD Testing: Modern testing approach with Ginkgo/Gomega
• Docker Testing: Integrated with existing Docker test infrastructure

🎯 LinkedIn Capabilities Enabled

1. Profile Search: Search LinkedIn profiles by various criteria
2. Advanced Filtering: Filter by location, company, job title, industry
3. Experience Filtering: Filter by years of experience and seniority
4. Pagination Support: Handle large result sets with cursor-based pagination
5. Data Extraction: Extract comprehensive profile information
6. Error Recovery: Robust error handling and recovery

🔄 Integration Points

Apify Platform

• Actor: harvestapi~linkedin-profile-search
• Authentication: Requires Apify API key
• Data Format: Structured JSON response processing
• Rate Limiting: Handled by Apify platform

TEE Types Integration

• Profile Types: Uses comprehensive LinkedIn profile types from tee-types
• Argument Validation: Leverages tee-types validation system
• Capability System: Integrates with TEE capability framework

Statistics & Monitoring

• Real-time Metrics: Tracks queries, profiles, and errors
• Worker Performance: Per-worker statistics
• Health Monitoring: Error rate tracking

✅ Quality Assurance

Testing

• Unit Tests: Comprehensive test coverage for all components
• Integration Tests: End-to-end testing with mock Apify client
• Error Scenarios: Extensive error condition testing
• Data Validation: Input/output validation testing

Code Standards

• Go Conventions: Follows Go best practices
• Error Handling: Consistent error handling patterns
• Logging: Structured logging throughout
• Documentation: Clear code documentation

Performance

• Efficient Processing: Optimized data processing
• Memory Management: Proper resource cleanup
• Concurrent Safety: Thread-safe operations
• Scalability: Designed for high-volume processing

🚀 Deployment Ready

This PR provides a complete LinkedIn integration solution that:

• Follows established patterns from other job types
• Integrates seamlessly with existing TEE worker infrastructure
• Provides comprehensive testing and monitoring
• Supports production deployment with proper error handling
• Enables LinkedIn profile scraping at scale through Apify

The implementation is production-ready and follows all established patterns and best practices within the TEE worker ecosystem.

[Copilot]

Pull Request Overview

This PR adds LinkedIn profile scraping functionality to the tee-worker using the Apify platform integration. The implementation provides LinkedIn profile search capabilities through a new LinkedInScraper that follows the existing patterns established for other scrapers.

• Adds LinkedIn job support via Apify API with comprehensive testing
• Fixes a type conversion bug in Twitter scraper error handling
• Updates project dependencies and build configurations

Reviewed Changes

Copilot reviewed 12 out of 13 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
internal/jobserver/jobserver.go	Registers LinkedIn job type in the job server
internal/jobs/twitter.go	Fixes string conversion bug in error message
internal/jobs/stats/stats.go	Adds LinkedIn-specific statistics tracking
internal/jobs/linkedinapify/	New package for LinkedIn Apify client implementation
internal/jobs/linkedin*.go	Main LinkedIn scraper implementation and tests
internal/apify/actors.go	Adds LinkedIn actor configuration
go.mod	Updates Go version and dependencies
Makefile	Adds LinkedIn test target
Dockerfile	Updates base image reference

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[Copilot]

Pull Request Overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated no new comments.

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[rapidfix]

Looking good here!