Pass an iterator to CreateInBatches

[brvoisin]

Hi all!

Your Question

I tried to pass an iterator to the method CreateInBatches:

	modelsIterator := func(contacts []domain.Contact) iter.Seq[contactModel] {
		return func(yield func(contactModel) bool) {
			for _, contact := range contacts {
				model := domainToModel(contact)
				if !yield(model) {
					return
				}
			}
		}
	}

	tx := db.CreateInBatches(modelsIterator(contacts), 1000)

But I got this error:

unsupported data type: iter.Seq[contactModel]: Table not set, please set it like: db.Model(&user) or db.Table("users")

Are iterators supported by CreateInBatches?

The document you expected this should be explained

It's not mentioned in Create Batch Insert.

Expected answer

Is there a way to create in batches with an iterator to avoid allocating a big array in memory?

[zwell]

The issue arises because you're passing an iter.Seq[contactModel], which is a function type, and stmt.Parse() is unable to recognize function types as valid models. Additionally, since no table is explicitly set via db.Model() or db.Table(), GORM cannot determine which table to interact with.

Solution:

You can resolve the issue by explicitly setting the model type (the target table) using db.Model(&contactModel{}) before calling CreateInBatches().

Example:

db.Model(&contactModel{}).CreateInBatches(yourSliceOrBatch, 1000)

Alternatively, you can first collect the results from iter.Seq into a []contactModel and then pass it to CreateInBatches().

[brvoisin]

Thanks for your answer!

I tried to set the model type, but there is another error.

	modelsIterator := func(contacts []domain.Contact) iter.Seq[contactModel] {
		return func(yield func(contactModel) bool) {
			for _, contact := range contacts {
				model := domainToModel(contact)
				if !yield(model) {
					return
				}
			}
		}
	}

	tx := db.Model(&contactModel{}).CreateInBatches(modelsIterator(contacts), 1000)

Error:

unsupported data

I would like to avoid allocating the whole contact model array in memory before the batch insert.

[zwell]

I apologize for not properly addressing the core of your issue earlier.

You're getting the unsupported data error because CreateInBatches in GORM expects a slice or array, not a generator or function like iter.Seq. Internally, GORM uses reflection to check if the input is a slice:

reflect.ValueOf(value).Kind() == reflect.Slice

Your modelsIterator returns a func(...), which is reflect.Func, so GORM doesn't recognize it and falls back to the default case — leading to the error.

---

✅ Solution

Manually batch the data like this:

batchSize := 1000
batch := make([]contactModel, 0, batchSize)

for _, contact := range contacts {
   model := domainToModel(contact)
   batch = append(batch, model)

   if len(batch) >= batchSize {
      db.Create(&batch)
      batch = batch[:0]
   }
}
if len(batch) > 0 {
   db.Create(&batch)
}

[brvoisin]

So, we can't use an iterator with Gorm. I'll batch the data manually.

Thanks for your answers!