Managing your content with the JS SDK

Manage your catalog

The first step of using the SDK is to tell Gally how your data will be structured. The StructureSynchronizer service manages catalog structure synchronization between your e-commerce platform and Gally. This includes catalogs, localized catalogs, metadata, source fields, and field options.

import {
  StructureSynchronizer,
  Catalog,
  LocalizedCatalog,
  Metadata,
  SourceField,
  Label,
} from '@gally/sdk'

const synchronizer = new StructureSynchronizer(config)

Catalogs and Localized Catalogs

Creating Catalogs

// Create a catalog
const catalog = new Catalog('my_shop', 'My Shop')

// Create localized catalogs
const localizedCatalogFr = new LocalizedCatalog(
  catalog,
  'my_shop_fr',
  'My Shop FR',
  'fr_FR',
  'EUR',
)

const localizedCatalogEn = new LocalizedCatalog(
  catalog,
  'my_shop_en',
  'My Shop EN',
  'en_US',
  'USD',
)

Synchronizing Catalogs

// Sync all localized catalogs at once
await synchronizer.syncAllLocalizedCatalogs([
  localizedCatalogFr,
  localizedCatalogEn,
])

// Or sync a single localized catalog
await synchronizer.syncLocalizedCatalog(localizedCatalogFr)

Source Fields

Source fields define the structure of your data (products, categories, etc.).

Creating Source Fields

const metadata = new Metadata('product')

// Text field
const nameField = new SourceField(
  metadata,
  'name',
  'text',
  'Product Name',
  [
    new Label(localizedCatalogFr, 'Nom du produit'),
    new Label(localizedCatalogEn, 'Product Name'),
  ]
)

// Price field
const priceField = new SourceField(
  metadata,
  'price',
  'price',
  'Price',
  [
    new Label(localizedCatalogFr, 'Prix'),
    new Label(localizedCatalogEn, 'Price'),
  ]
)

// Select field (enumeration)
const brandField = new SourceField(
  metadata,
  'brand',
  'select',
  'Brand',
  [
    new Label(localizedCatalogFr, 'Marque'),
    new Label(localizedCatalogEn, 'Brand'),
  ]
)

Field Types

Type	Description	Example
text	Full-text searchable fields	name, description
keyword	Non-analyzed exact-match fields	SKU, reference
select	Enumerated values	brand, color
int	Integer numbers	stock quantity
boolean	True/false values	is_new, in_stock
float	Floating-point numbers	weight, rating
price	Price with customer group support	price
stock	Stock status and quantity	stock
category	Category relationships	categories
reference	Unique identifiers	sku, ean
image	Image URLs	image, thumbnail
object	Complex nested objects	custom data
date	Date/time fields	created_at, updated_at
location	Geographic coordinates	store_location

Synchronizing Source Fields

await synchronizer.syncAllSourceFields([
  nameField,
  priceField,
  brandField,
  // ... more fields
])

Source Field Options

For select type fields, you can define available options:

import { SourceFieldOption, SourceFieldOptionLabel } from '@gally/sdk'

const brandOptions = [
  new SourceFieldOption(
    brandField,
    'nike',
    [
      new SourceFieldOptionLabel(localizedCatalogFr, 'Nike'),
      new SourceFieldOptionLabel(localizedCatalogEn, 'Nike'),
    ],
    1 // position
  ),
  new SourceFieldOption(
    brandField,
    'adidas',
    [
      new SourceFieldOptionLabel(localizedCatalogFr, 'Adidas'),
      new SourceFieldOptionLabel(localizedCatalogEn, 'Adidas'),
    ],
    2
  ),
]

await synchronizer.syncAllSourceFieldOptions(brandOptions)

Complete Synchronization Example

See a complete example with all the steps

import {
  StructureSynchronizer,
  Catalog,
  LocalizedCatalog,
  Metadata,
  SourceField,
  SourceFieldOption,
  Label,
  SourceFieldOptionLabel,
} from '@gally/sdk'

const synchronizer = new StructureSynchronizer(config)

// 1. Create catalog structure
const catalog = new Catalog('my_shop', 'My Shop')
const localizedCatalogFr = new LocalizedCatalog(catalog, 'my_shop_fr', 'My Shop FR', 'fr_FR', 'EUR')
const localizedCatalogEn = new LocalizedCatalog(catalog, 'my_shop_en', 'My Shop EN', 'en_US', 'USD')

// 2. Sync catalogs
await synchronizer.syncAllLocalizedCatalogs([localizedCatalogFr, localizedCatalogEn])

// 3. Define source fields
const productMetadata = new Metadata('product')

const fields = [
  new SourceField(productMetadata, 'sku', 'reference', 'SKU', [
    new Label(localizedCatalogFr, 'Référence'),
    new Label(localizedCatalogEn, 'SKU'),
  ]),
  new SourceField(productMetadata, 'name', 'text', 'Name', [
    new Label(localizedCatalogFr, 'Nom'),
    new Label(localizedCatalogEn, 'Name'),
  ]),
  new SourceField(productMetadata, 'price', 'price', 'Price', [
    new Label(localizedCatalogFr, 'Prix'),
    new Label(localizedCatalogEn, 'Price'),
  ]),
  new SourceField(productMetadata, 'brand', 'select', 'Brand', [
    new Label(localizedCatalogFr, 'Marque'),
    new Label(localizedCatalogEn, 'Brand'),
  ]),
  new SourceField(productMetadata, 'stock', 'stock', 'Stock', [
    new Label(localizedCatalogFr, 'Stock'),
    new Label(localizedCatalogEn, 'Stock'),
  ]),
]

// 4. Sync source fields
await synchronizer.syncAllSourceFields(fields)

// 5. Sync options for select fields
const brandField = fields.find(f => f.code === 'brand')!
const brandOptions = [
  new SourceFieldOption(brandField, 'nike', [
    new SourceFieldOptionLabel(localizedCatalogFr, 'Nike'),
    new SourceFieldOptionLabel(localizedCatalogEn, 'Nike'),
  ], 1),
  new SourceFieldOption(brandField, 'adidas', [
    new SourceFieldOptionLabel(localizedCatalogFr, 'Adidas'),
    new SourceFieldOptionLabel(localizedCatalogEn, 'Adidas'),
  ], 2),
]

await synchronizer.syncAllSourceFieldOptions(brandOptions)

console.log('Catalog structure synchronized successfully!')

Idempotency

All synchronization operations are idempotent - you can safely run them multiple times:

// First run: creates catalogs and fields
await synchronizer.syncAllLocalizedCatalogs([localizedCatalogFr, localizedCatalogEn])
await synchronizer.syncAllSourceFields(fields)

// Second run: updates existing catalogs and fields (no duplicates)
await synchronizer.syncAllLocalizedCatalogs([localizedCatalogFr, localizedCatalogEn])
await synchronizer.syncAllSourceFields(fields)

Error Handling

try {
  await synchronizer.syncAllLocalizedCatalogs([localizedCatalogFr])
} catch (error) {
  if (error.response?.status === 401) {
    console.error('Authentication failed')
  } else if (error.response?.status === 400) {
    console.error('Invalid catalog data:', error.message)
  } else {
    console.error('Sync failed:', error.message)
  }
}

Best Practices

1. Sync structure before indexing - Always sync catalog structure before creating indexes
2. Use consistent codes - Keep catalog and field codes consistent across environments
3. Handle errors gracefully - Implement retry logic for transient failures
4. Batch operations - Use syncAll* methods instead of syncing items individually
5. Version control structure - Store catalog structure in code for reproducibility

Index your catalog

Now that you have defined you catalog structure, you can index your data. The IndexOperation service manages the complete lifecycle of search indexes in Gally: creation, bulk indexing, installation, and management.

import { IndexOperation, Metadata } from '@gally/sdk'

const indexOp = new IndexOperation(config)
const metadata = new Metadata('product')

Index Lifecycle

A typical index workflow:

1. Create - Create a new index
2. Bulk - Index documents into the new index
3. Install - Make the index live (atomic swap)
4. Cleanup - Remove old indexes (optional)

Creating an Index

const index = await indexOp.createIndex(metadata, localizedCatalogFr)

console.log(`Index created: ${index.name}`)
// Example: gally_localized_catalog_my_shop_fr_product_20240319_143052

Index Naming Convention

Indexes are automatically named with a timestamp:

gally_localized_catalog_{catalog_code}_{metadata}_{timestamp}

This allows multiple indexes to coexist during deployment.

Bulk Indexing Documents

Basic Bulk Operation

const documents = [
  {
    id: '1',
    sku: 'PROD-001',
    name: 'Product 1',
    price: [{ price: 29.99, group_id: 0 }],
    stock: { status: true, qty: 100 },
  },
  {
    id: '2',
    sku: 'PROD-002',
    name: 'Product 2',
    price: [{ price: 49.99, group_id: 0 }],
    stock: { status: true, qty: 50 },
  },
]

await indexOp.executeBulk(index, documents)

Large Dataset Indexing

For large datasets, index in batches:

const batchSize = 1000
const totalProducts = 50000

for (let offset = 0; offset < totalProducts; offset += batchSize) {
  const batch = await fetchProductsBatch(offset, batchSize)
  await indexOp.executeBulk(index, batch)
  
  console.log(`Indexed ${offset + batch.length}/${totalProducts} products`)
}

Document Structure

Documents must match your source field definitions:

interface ProductDocument {
  id: string                    // Required: unique identifier
  sku: string                   // reference field
  name: string                  // text field
  description?: string          // optional text field
  price: Array<{                // price field
    price: number
    group_id: number
  }>
  stock: {                      // stock field
    status: boolean
    qty?: number
  }
  brand?: string                // select field
  categories?: string[]         // category field (array of category IDs)
  image?: string                // image field
  // ... other fields
}

Installing an Index

Make an index live (replaces the current live index atomically):

const installedIndex = await indexOp.installIndex(index)

console.log(`Index installed: ${installedIndex.name}`)

Zero-Downtime Deployment

The install operation is atomic:

1. New index is fully prepared
2. Atomic swap to new index
3. Old index remains available for rollback
4. No search downtime

Complete Indexing Example

import { IndexOperation, Metadata } from '@gally/sdk'

async function reindexCatalog(localizedCatalog, products) {
  const indexOp = new IndexOperation(config)
  const metadata = new Metadata('product')
  
  try {
    // 1. Create new index
    console.log('Creating index...')
    const index = await indexOp.createIndex(metadata, localizedCatalog)
    
    // 2. Index documents in batches
    console.log(`Indexing ${products.length} products...`)
    const batchSize = 1000
    
    for (let i = 0; i < products.length; i += batchSize) {
      const batch = products.slice(i, i + batchSize)
      await indexOp.executeBulk(index, batch)
      console.log(`Progress: ${Math.min(i + batchSize, products.length)}/${products.length}`)
    }
    
    // 3. Install index (make it live)
    console.log('Installing index...')
    await indexOp.installIndex(index)
    
    console.log('✓ Reindex completed successfully')
    
    return index
  } catch (error) {
    console.error('Reindex failed:', error.message)
    throw error
  }
}

// Usage
await reindexCatalog(localizedCatalogFr, products)

Index Management

Retrieving Live Index

const liveIndex = await indexOp.getIndexByName(
  'gally_localized_catalog_my_shop_fr_product'
)

console.log(`Current live index: ${liveIndex.name}`)

Listing Indexes

// Get all indexes for a catalog/metadata
const indexes = await indexOp.listIndexes(metadata, localizedCatalogFr)

for (const index of indexes) {
  console.log(`- ${index.name} (${index.isInstalled ? 'LIVE' : 'inactive'})`)
}

Category Indexing

Categories have their own metadata and indexes:

const categoryMetadata = new Metadata('category')

// Create category index
const categoryIndex = await indexOp.createIndex(categoryMetadata, localizedCatalogFr)

// Index categories
const categories = [
  {
    id: '1',
    code: 'cat_shoes',
    name: 'Chaussures',
    position: 1,
  },
  {
    id: '2',
    code: 'cat_clothing',
    name: 'Vêtements',
    position: 2,
  },
]

await indexOp.executeBulk(categoryIndex, categories)
await indexOp.installIndex(categoryIndex)

Price Groups

Prices support customer group segmentation:

const product = {
  id: '1',
  sku: 'PROD-001',
  name: 'Product 1',
  price: [
    { price: 99.99, group_id: 0 },   // Default price
    { price: 89.99, group_id: 1 },   // VIP customers
    { price: 79.99, group_id: 2 },   // Wholesale customers
  ],
}

Stock Management

Stock fields support status and quantity:

// In stock
const inStockProduct = {
  id: '1',
  sku: 'PROD-001',
  stock: {
    status: true,
    qty: 100,
  },
}

// Out of stock
const outOfStockProduct = {
  id: '2',
  sku: 'PROD-002',
  stock: {
    status: false,
    qty: 0,
  },
}

Error Handling

try {
  const index = await indexOp.createIndex(metadata, localizedCatalog)
  await indexOp.executeBulk(index, documents)
  await indexOp.installIndex(index)
} catch (error) {
  if (error.response?.status === 400) {
    console.error('Invalid document structure:', error.message)
  } else if (error.response?.status === 404) {
    console.error('Catalog or metadata not found')
  } else {
    console.error('Indexing failed:', error.message)
  }
}

Best Practices

1. Index in batches - Use batch sizes of 500-1000 documents for optimal performance
2. Create before indexing - Always create a new index before bulk operations
3. Test before installing - Verify index content before making it live
4. Monitor progress - Log progress for long-running operations
5. Handle failures - Implement retry logic for transient failures
6. Keep old indexes - Keep at least one previous index for rollback capability
7. Use consistent IDs - Ensure document IDs are stable across reindexes

Performance Tips

Parallel Indexing

For very large catalogs, index multiple batches in parallel:

async function parallelIndex(index, documents, concurrency = 3) {
  const batchSize = 1000
  const batches = []
  
  for (let i = 0; i < documents.length; i += batchSize) {
    batches.push(documents.slice(i, i + batchSize))
  }
  
  // Process batches with concurrency limit
  for (let i = 0; i < batches.length; i += concurrency) {
    const chunk = batches.slice(i, i + concurrency)
    await Promise.all(chunk.map(batch => indexOp.executeBulk(index, batch)))
    console.log(`Progress: ${Math.min(i + concurrency, batches.length)}/${batches.length} batches`)
  }
}

Incremental Updates

For small updates, create a new index only when necessary:

// For daily full reindex
const shouldReindex = await checkIfReindexNeeded()

if (shouldReindex) {
  await reindexCatalog(localizedCatalog, products)
} else {
  console.log('Skipping reindex - no changes detected')
}

Now that your catalog structure is defined and your data is indexed, you can search your catalog and track user interactions !