[persistent-template] schema organization and compilation times

[naushadh]

This isn't so much a bug or support request, but something I discovered that could enhance developer experience. See sample project/findings here naushadh/persistent-examples. TLDR: breaking entity definitions into smaller components can yield immense compilation time gains.

Is there any downside to the approach? Should persistManyFileWith be added as an official persistent-template API? Should guides suggest this approach (for compile time gains)?

Possibly related things:

• Long compile time/high memory usage for modules with many deriving clauses
• ghc using 6+ GB of memory for module with Persistent models and Yesod handlers
• Is it possible to construct a persistent schema in pieces?

[MaxGabriel]

I'm OK with all of these approaches. They should probably be caveated as optimizations for advanced users, and not the standard approach. persistManyFileWith sounds good, though I don't know how hard it would be to fix the foreign keys with parseReferences

[naushadh]

fix the foreign keys with parseReferences

Not sure I understand. Somehow use stateful TH within parseReferences to accomplish what persistManyFileWith externally does?

[MaxGabriel]

Oh sorry I misunderstood you, I didn't realize parseReferences was something you were already using.

Ok so I think this should be added as a new API to Persistent for sure