Few questions for more complicated application

[hieven]

Hey, thanks for the great work of showing how to apply clean architecture in Go.

I have a few questions with some more complicated situations and I'd like to hear your thought about them. :)

Q1. Who should handle DB transaction?
According to the answer in #1, you mentioned the transaction must be done in Repository layer. However, if the use case is "When a new article is created, a relation between article and author must be created". Does article repository also create the junction record?

Q2. Who should handle cache?
It's very common to see that service has a "find from cache first or fallback to DB query". Does repository handles this logic or use case layer should does this?

Q3. Imagine there are other services want to subscribe "article created" event, which layer should be responsible for publishing this event?

Thanks

[bxcodec]

Hi @hieven ,
Thank you for the question.
I will answer this based on my experience when developing many projects so far. And I don't know this is the better way or not, it is up to you to decide to follow or not. And also I'm open for advice or suggestion for the better result :)

Q1: Who should handle DB transaction?
In my case(s), Yes, I put the transaction on the repository layer.

Q2: Who should handle cache?
In my opinion, the cache is not specifically handled by any layer. Every layer may have the cache. So to ease this, for all my projects, I always inject a cache interactor (cache usecase) into every layer who need it.

// package cache
type CacheUsecaseInterface interface{
    Get(key string)(interface{},error)
    Set(key string, item interface{})(error)

}

// package article/usecase
type articleUsecase struct {
    // .... other injection 
    cacheUsecase     cache.CacheUsecaseInterface
}

So there will be a modul or package named: cache like the article or author package. It has its own implementation, and not depend to specific cache system. It can be use redis, inmem cache, or anything. But it handled by one package named: cache

Q3: which layer should be responsible for publishing this event?
Neither of them. For my projects, I would make another package, to handle this. Let's say its name eventbus or something. So this package will handle the event. And same as like the cache, I will inject the interactor to any package who need it. So far, I will inject it in usecase layer.

// package eventbus
type EventBusInterface interface{
    Publish(event Event)(error)
}

// package article/usecase
type articleUsecase struct {
    // .... other injection 
    eBus     eventbus.EventBusInterface
}

And for subscribing event from external services, I always made it in delivery layer. Because, in my opinion, it's just another delivery type ( REST, RPC, or Subscribing)

[wherevn]

How to implement the DB transactions if the use case calls 2 or more functions of the repository?

[bxcodec]

Hi @wherevn ,

For my own cases, I just handle it in one repository. I mean, you can't really handle the transaction like with cross-repository because transactions only happen in RDBMS.

So the only way I used usually is to do the transaction in one repository.

For example.

In the article repository, let's imagine, we can post the article even the author still not exists.
So when I insert an article, I will also insert a new author. To handle this I would create a transaction query handler in article repository.

So in article repository, there will be a function like this.

func (r *articleRepository) Insert(ctx context.Context, item *models.Article, author *models.Author) (err error) {
	tx, err := r.DB.BeginTx(ctx, nil)
	if err != nil {
		return
	}
	var authorID = author.ID
	if authorID == "" {
		res, err := tx.Exec("INSERT Author SET .....", author...)
		if err != nil {
			tx.Rollback()
			return err
		}
		// handle the res variable
		// ...
		// get author id from inserted id
		// ...
		authorID = res.LastInsertId()
	}
	res, err := tx.Exec("INSERT Article SET .....", item, authorID)
	if err != nil {
		tx.Rollback()
		return err
	}
	// handle the res variable
	// ....
	err = tx.Commit()
	return
}

A bit tricky, but well, it depends to the business rule. For the example above, the business rule said, we can post an article even we don't have a registered author yet. And if the author is not registered yet, we can create a new one.

But if we look again, it still the article repository jobs to insert a new article even it was using other tables.

[hieven]

Hi @bxcodec, thank you for the detailed answers for all my questions.

Regarding Q1 and the answer you replied to wherevn,
If the case is more complicated, e.g. we need a transaction for entity A+B, A+C and A+C+D
Will it be tedious to keep implementing three different but similar method in Repository layer? (and which repository should own each method)

On the other hand, if we abstract DB transaction and do it in usecase layer, we only need to implement A, B, C, D once.

Besides, whether they should be in a transaction are also business decisions, right? (perhaps we allow some failure since the user's action is idempotent or there are other ways reconciling to fulfill eventual consistent). Does Repository layer should know that business logic or purely focus on how to interact with database?

Regarding Q2 and Q3, please allow me to rephrase them to be more specific.
Right. Both Cache and EventBus are definitely implemented in different packages for easy to reuse.

If we want to publish a article_created event when every time the article is inserted to database and we currently put this event.Publish in usecase layer.

Q3.1: How do we ensure in the future, when another engineer adds a different usecase method also inserting the article record, will not forget to do event publish? (maybe he/she doesn't know there is an existing usecase method or the existing one isn't reusable or he/she just call repository method in new implementation)

[mwei0210]

In the same scenario, anyone managed to find a cleaner solution?

[bxcodec]

Hi @hieven,

I'm sorry for the late reply. I almost forgot this thread 😄

Regarding Q1 and the answer you replied to wherevn,
If the case is more complicated, e.g. we need a transaction for entity A+B, A+C and A+C+D
Will it be tedious to keep implementing three different but similar method in Repository layer? (and which repository should own each method)

I'm not sure yet, it's the ideal way. But what I say is, it can't be helped. The transaction only happens in the repository layer. Let say I'm using NoSQL as my storage, I won't use transaction, because NoSQL doesn't support this. So, the answer is, yeah, it still the repository jobs to handle the transaction and it depends what database I use as the repository. Usecase didn't know anything what happen in the bottom layer, usecase shouldn't know about transaction things.

---

Besides, whether they should be in a transaction are also business decisions, right? (perhaps we allow some failure since the user's action is idempotent or there are other ways reconciling to fulfill eventual consistent). Does Repository layer should know that business logic or purely focus on how to interact with database?

Yeah, this a bit complicated. I can't give the right answer, but if you have a better idea, I would like to hear. And from POV, usecase shouldn't know anything about transactions thing.

But, maybe I want to give you another thing to think, let's say my repository layer was a microservice. I'm not using a database in a repository, but another microservice. Did you still think about the transaction still happened in usecase? How about if that microservice already has a transaction mechanism?

I'm designing this, to be more general usage, and not tightly coupled just to RDBMS only. That's the reason I put the transaction in the repository layer.

---

Q3.1: How do we ensure in the future, when another engineer adds a different usecase method also inserting the article record, will not forget to do event publish? (maybe he/she doesn't know there is an existing usecase method or the existing one isn't reusable or he/she just call repository method in new implementation)

I'm sorry, I can't answer this. The only answer I can give is, do onboarding very well :D, and also, I think every engineer right know usually doing the code review right? That's the right time we should tell that engineer that he forgot to add the event publisher :D

*Oh yeah, if you have better solutions, I'd like to hear it. I also want to learn from other's perspective :)

[dzakaammar]

Hi @bxcodec,

Great work by the way.
I just read this issue and want to share my idea about one of the question.

Regarding Q2, what if we implement the cache repository with a Proxy Pattern. We create a struct that satisfy the repository interface. The struct will accept an object that also satisfy the same interface (usually the real repository implementation itself). And then for each method that has to be implemented, we do some logic based on their functionality.

For example, GetByID method should check for the data existance in cache. If it exists, then simply return it, If it doesn't exists, then the method should call the GetByID from the injected repository object to getting the data. After that, save it to cache.

// package cache
type articleCache {
  // other injection
  repo article.ArticleRepository
}

func NewArticleCache(repo article.ArticleRepository /* other params */) article.ArticleRepository {
  return &articleCache{repo: repo}
}

func (a *articleCache) GetByID(ctx context.Context, id int64) (*models.Article, error) {
  // getting from cache
  // if not found, than getting the data from db

  m, err := a.repo.GetByID(ctx, id)
  if err != nil {
     // do something when error
  }
  
  // save it to cache and return
}

// other method that satisfy article.ArticleRepository

Same implementation with the other command method, like update or delete. We can invalidate/delete the cache after call the same method from the injected repository.

And then, we can initialize the repository in main.go, like this:

func main() {
...
ar := _articleRepo.NewMysqlArticleRepository(dbConn)
ar = _articleCache.NewArticleCache(ar)
...
}

The reason why we implement like this is because we want to cleanly separate the cache implementation from the database repository implementation, but also bounded with the same interface. Because the logic for each implementation in database repository will be followed by how they manage the cache, whether save it or delete it.

Any thoughts about this?

[bxcodec]

Hi @dzakaammar

Thanks for the solutions, yeah, this is quite good. We've done this also in my current projects. This is cleaner and more maintainable.

Thanks again for posting it here.