Skip to content

Sycatle/gmodstore-ts

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

1 Commit
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

gmodstore-ts

TypeScript/JavaScript SDK for the GmodStore job marketplace

npm version License: MIT

A TypeScript library to scrape job listings from the GmodStore job marketplace. Uses HTML parsing since GmodStore doesn't provide a public API.

⚠️ Important Notes

  • No official API: This library scrapes HTML from GmodStore's public job pages
  • Fragile: Page structure changes may break the scraper
  • Rate limiting: Be respectful with request frequency
  • Terms of Service: Ensure compliance with GmodStore's ToS before using

Features

  • πŸ” HTML Scraping - Parses job listings from GmodStore pages
  • πŸ“¦ Lightweight - Only one dependency (cheerio)
  • πŸ”’ Type-safe - Full TypeScript support with strict types
  • ⚑ Simple API - Easy to use, minimal setup
  • πŸ›‘οΈ Error handling - Comprehensive error types

Installation

npm install gmodstore-ts
# or
pnpm add gmodstore-ts
# or
yarn add gmodstore-ts

Requirements

  • Node.js 18+ (for native fetch API)
  • TypeScript 5+ (if using TypeScript)

Quick Start

import { GmodStoreClient } from 'gmodstore-ts';

const client = new GmodStoreClient();

// Get all jobs (may be slow, page must load fully)
const jobs = await client.getJobs();
console.log(jobs);

// Get specific number of jobs
const latestJobs = await client.getJobs({ limit: 5 });

// Get a single job by ID
const job = await client.getJobById('gmodstore_12345');

API Reference

GmodStoreClient

Main client class for scraping GmodStore jobs.

Constructor

new GmodStoreClient(config?: ClientConfig)

Options:

  • baseUrl?: string - Base URL (default: https://www.gmodstore.com)
  • userAgent?: string - User agent for requests (default: gmodstore-ts/0.1.0)
  • timeout?: number - Request timeout in milliseconds (default: 10000)

Methods

getJobs(options?: GetJobsOptions): Promise<Job[]>

Scrape job listings from GmodStore browse page.

Options:

  • limit?: number - Maximum number of jobs to return

Returns: Array of Job objects

Throws:

  • APIError - When the request fails
  • NetworkError - When there's a network error
  • ValidationError - When no jobs are found (page structure changed)
getJobById(jobId: string): Promise<Job | null>

Scrape a single job by ID. Returns null if the job is not found.

Parameters:

  • jobId: string - Job ID (with or without 'gmodstore_' prefix)

Returns: Job object or null

Types

Job

interface Job {
  id: string;              // Unique identifier (prefixed with 'gmodstore_')
  title: string;           // Job title
  url: string;             // Direct URL to the job listing
  source: 'gmodstore';     // Source identifier
  description: string | null;  // Job description
  budget: string | null;   // Budget/price information
  category: string | null; // Job category
  applications: number | null;  // Number of applications (always null from scraping)
  createdAt: Date | null;  // Creation timestamp (always null from scraping)
}

Error Types

import { GmodStoreError, APIError, NetworkError, ValidationError } from 'gmodstore-ts';
  • GmodStoreError - Base error class
  • APIError - Request failures (includes status code and response)
  • NetworkError - Network-related issues (includes original error)
  • ValidationError - Parsing failures (page structure changed)

Examples

Basic Usage

import { GmodStoreClient } from 'gmodstore-ts';

const client = new GmodStoreClient();

try {
  const jobs = await client.getJobs({ limit: 10 });
  
  jobs.forEach(job => {
    console.log(`${job.title} - ${job.budget || 'No budget'}`);
    console.log(`  ${job.url}`);
  });
} catch (error) {
  console.error('Failed to fetch jobs:', error);
}

With Error Handling

import { GmodStoreClient, APIError, NetworkError, ValidationError } from 'gmodstore-ts';

const client = new GmodStoreClient({ timeout: 15000 });

try {
  const jobs = await client.getJobs();
  console.log(`Found ${jobs.length} jobs`);
} catch (error) {
  if (error instanceof ValidationError) {
    console.error('Page structure changed, scraper needs update');
  } else if (error instanceof NetworkError) {
    console.error('Network Error:', error.message);
  } else if (error instanceof APIError) {
    console.error(`HTTP Error [${error.statusCode}]:`, error.message);
  } else {
    throw error;
  }
}

React Integration

import { useEffect, useState } from 'react';
import { GmodStoreClient, Job } from 'gmodstore-ts';

function JobsList() {
  const [jobs, setJobs] = useState<Job[]>([]);
  const [loading, setLoading] = useState(true);
  const [error, setError] = useState<string | null>(null);

  useEffect(() => {
    const client = new GmodStoreClient();
    
    client.getJobs({ limit: 20 })
      .then(setJobs)
      .catch(err => setError(err.message))
      .finally(() => setLoading(false));
  }, []);

  if (loading) return <div>Loading jobs...</div>;
  if (error) return <div>Error: {error}</div>;

  return (
    <div>
      {jobs.map(job => (
        <div key={job.id}>
          <h3>{job.title}</h3>
          <p>{job.description}</p>
          <span>{job.budget || 'Budget not specified'}</span>
          <a href={job.url}>View Job</a>
        </div>
      ))}
    </div>
  );
}

Next.js App Router (Server Component)

// app/jobs/page.tsx
import { GmodStoreClient } from 'gmodstore-ts';

export const revalidate = 600; // Revalidate every 10 minutes

export default async function JobsPage() {
  const client = new GmodStoreClient();
  
  try {
    const jobs = await client.getJobs({ limit: 20 });

    return (
      <div>
        <h1>GmodStore Jobs</h1>
        {jobs.map(job => (
          <article key={job.id}>
            <h2>{job.title}</h2>
            {job.description && <p>{job.description}</p>}
            <span>{job.budget || 'Budget not specified'}</span>
            <a href={job.url}>Apply</a>
          </article>
        ))}
      </div>
    );
  } catch (error) {
    return <div>Failed to load jobs. Please try again later.</div>;
  }
}

Express.js API

import express from 'express';
import { GmodStoreClient } from 'gmodstore-ts';

const app = express();
const client = new GmodStoreClient();

// Cache to avoid hammering GmodStore
let cachedJobs: any[] = [];
let lastFetch = 0;
const CACHE_TTL = 10 * 60 * 1000; // 10 minutes

app.get('/api/jobs', async (req, res) => {
  try {
    const now = Date.now();
    
    if (now - lastFetch > CACHE_TTL) {
      const limit = parseInt(req.query.limit as string) || 20;
      cachedJobs = await client.getJobs({ limit });
      lastFetch = now;
    }
    
    res.json({ success: true, data: cachedJobs, cached: now - lastFetch < CACHE_TTL });
  } catch (error) {
    res.status(500).json({ 
      success: false, 
      error: error instanceof Error ? error.message : 'Unknown error' 
    });
  }
});

app.listen(3000, () => console.log('API running on port 3000'));

Limitations

  • No real-time data: Scraping HTML means data may be stale
  • Page structure: Changes to GmodStore's HTML will break the scraper
  • Performance: HTML parsing is slower than API calls
  • Rate limiting: No built-in rate limiting, implement your own caching
  • Incomplete data: Some fields (applications, createdAt) are not available from HTML

Best Practices

  1. Cache results: Don't fetch on every request
  2. Handle errors gracefully: Page structure can change anytime
  3. Respect rate limits: Add delays between requests
  4. Monitor for changes: Watch for ValidationError exceptions
  5. Fallback strategy: Have a backup plan when scraping fails

Development

# Install dependencies
pnpm install

# Build the package
pnpm build

# Run examples
pnpm tsx examples/basic.ts

# Type check
pnpm tsc --noEmit

Migration from API-based version

If you were using an older version that relied on an API:

- const client = new GmodStoreClient({ baseUrl: 'https://api.gmodstore.com' });
+ const client = new GmodStoreClient(); // Now scrapes HTML

  const jobs = await client.getJobs();
+ // Note: Some fields may now be null (applications, createdAt)

License

MIT Β© Sycatle

Related Projects

Contributing

Contributions are welcome, especially:

  • Improved HTML parsing selectors
  • Better error handling
  • Additional job fields extraction
  • Rate limiting helpers

Please feel free to submit a Pull Request.

Support

Disclaimer

This library scrapes publicly available data from GmodStore. It is not affiliated with or endorsed by GmodStore. Use responsibly and in accordance with GmodStore's Terms of Service.

About

TypeScript/JavaScript SDK for scraping the GmodStore job marketplace

Resources

License

Stars

Watchers

Forks