Skip to content

Bulk Operations

Igor Sazonov edited this page Sep 6, 2026 · 1 revision

The SDK exposes selected SocialKit bulk endpoints as experimental methods. They accept a json.RawMessage payload and return json.RawMessage rather than a fixed response type. This is intentional: the upstream OpenAPI response schema only guarantees a success indicator, while the returned data shape is not stable enough for the SDK to promise a strong Go model. 1 2

Use bulk calls only when you can validate the payload and response in your own application. For a stable per-item model, prefer the corresponding non-bulk methods.

Available bulk methods

Service Methods Route pattern
YouTube BulkTranscript, BulkComments, BulkStats, BulkSummarize /youtube/{operation}/bulk
TikTok BulkTranscript, BulkComments, BulkStats, BulkChannelStats, BulkSummarize /tiktok/{operation}/bulk
Instagram BulkTranscript, BulkStats, BulkChannelStats, BulkSummarize /instagram/{operation}/bulk

Each method has the same Go signature:

func (s *YouTubeService) BulkTranscript(
    ctx context.Context,
    body json.RawMessage,
) (json.RawMessage, *socialkit.ResponseMeta, error)

The concrete service name and method vary as listed above, but the input/output contract is the same. 3 4 5

Construct the request body

SocialKit’s OpenAPI contract describes a requests array containing between one and five request objects. Each item must contain a URL; endpoint-specific parameters can be supplied where the upstream API accepts them. Validate URLs and constrain request counts before creating the raw JSON. 1

payload, err := json.Marshal(map[string]interface{}{
    "requests": []map[string]interface{}{
        {"url": "https://www.youtube.com/watch?v=VIDEO_A"},
        {"url": "https://www.youtube.com/watch?v=VIDEO_B"},
    },
})
if err != nil {
    return err
}

raw, meta, err := client.YouTube.BulkTranscript(ctx, json.RawMessage(payload))
if err != nil {
    return err
}

fmt.Printf("status=%d raw=%s\n", meta.HTTPStatus, raw)

Do not hand-build JSON with string interpolation. json.Marshal ensures correct escaping, and json.RawMessage makes the raw-JSON boundary explicit.

Decode the response locally

Because bulk response payloads can evolve, define a small, local view of only the fields your application needs. Keep it permissive, and retain the original json.RawMessage for investigation only if that meets your data-handling policy.

type bulkTranscriptItem struct {
    URL        string `json:"url"`
    Transcript string `json:"transcript"`
    Error      string `json:"error"`
}

type bulkTranscriptResponse struct {
    Results []bulkTranscriptItem `json:"results"`
}

var decoded bulkTranscriptResponse
if err := json.Unmarshal(raw, &decoded); err != nil {
    return fmt.Errorf("decode bulk transcript response: %w", err)
}

for _, item := range decoded.Results {
    if item.Error != "" {
        log.Printf("source failed: url=%s error=%s", item.URL, item.Error)
        continue
    }
    processTranscript(item.URL, item.Transcript)
}

The illustrative structure above is deliberately application-local, not an SDK guarantee. Inspect the current returned payload in a controlled environment and update the local type when the upstream contract changes.

Error and retry strategy

The bulk method itself returns a normal SDK error for a transport or non-2xx API failure. A 2xx response can still contain per-item success or failure information in the raw payload, so distinguish request-level failure from item-level failure in your decoder. 1 6

Layer What to check Suggested handling
SDK request Returned error; sentinel errors; ResponseMeta Handle authentication, validation, rate-limit, server, and transport conditions as usual.
Bulk response envelope Successful HTTP response and parseable JSON Report malformed/unexpected response data as an integration issue.
Individual item Item-specific status, data, or error fields in the raw payload Continue processing successful items; queue only permitted, safe-to-retry failures.

The SDK does not retry by default. If you configure WithRetry, remember that it can retry the POST bulk request. A repeat may incur charges or duplicate work, so enable it only when the payload is safe to submit again under your application’s policy. 6

When not to use a bulk method

Use the standard typed methods if you need strong response models, granular per-item retry, simple observability, or a single URL. Use an asynchronous download workflow instead of a bulk endpoint when the workload is long-form media acquisition. Bulk endpoints are best reserved for carefully bounded batches where reduced request overhead is worth managing an evolving raw response shape. 1 2

References

Clone this wiki locally