Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#118: adding support for MultipleResultFunction with status for Doobie #120

Merged
merged 27 commits into from
Jun 25, 2024

Conversation

lsulak
Copy link
Collaborator

@lsulak lsulak commented Mar 25, 2024

What was done:

  • In core, changed / added: DBMultipleResultFunctionWithStatus, DBSingleResultFunctionWithStatus and DBOptionalResultFunctionWithStatus based on DBFunctionWithStatus as it was for non-status classes, i.e. keeping the design consistent across the core module
  • In Doobie added: DoobieOptionalResultFunctionWithStatus and DoobieMultipleResultFunctionWithStatus
  • In Slick, added: SlickOptionalResultFunctionWithStatus and SlickMultipleResultFunctionWithStatus
  • Changed interface of runWithStatus and all dependant methods from F[Either[StatusException, R]] to F[Either[StatusException, FunctionStatusWithData[R]]] (and also their Seq-like versions)
  • Consolidated common package types of the core/status and added type aliases
  • Added StatusAggregation with three simple yet most likely used algorithms for aggregating error statuses, i.e. transformation from F[Seq[Either[StatusException, FunctionStatusWithData[R]]]] to F[Either[StatusException, Seq[FunctionStatusWithData[R]]]]
  • Covered with plenty of unit and integration tests

Depends on #119

Closes #118
Closes #115

Copy link

github-actions bot commented Mar 25, 2024

JaCoCo code coverage report - scala 2.13.12

Total Project Coverage 30.36% 🍏
Module Coverage
fa-db:core Jacoco Report - scala:2.13.12 58.77%
fa-db:slick Jacoco Report - scala:2.13.12 0%
fa-db:doobie Jacoco Report - scala:2.13.12 0%
Files
Module File Coverage [17.47%]
fa-db:core Jacoco Report - scala:2.13.12 StandardStatusHandling.scala 100% 🍏
package.scala 100% 🍏
StatusAggregator.scala 100% 🍏
ByMajorityErrorsStatusAggregator.scala 92.11% 🍏
ByFirstErrorStatusAggregator.scala 85.71% 🍏
ByFirstRowStatusAggregator.scala 82.09% 🍏
DBEngine.scala 11.11%
DBFunction.scala 4.31%
Query.scala 0%
UserDefinedStatusHandling.scala 0%
fa-db:slick Jacoco Report - scala:2.13.12 SlickPgEngine.scala 0%
SlickFunction.scala 0%
SlickQuery.scala 0%
fa-db:doobie Jacoco Report - scala:2.13.12 DoobieFunction.scala 0%
StatusWithData.scala 0%
DoobieQuery.scala 0%
DoobieEngine.scala 0%

Copy link

github-actions bot commented Mar 25, 2024

JaCoCo code coverage report - scala 2.12.17

Total Project Coverage 29.54% 🍏
Module Coverage
fa-db:core Jacoco Report - scala:2.12.17 57.77%
fa-db:slick Jacoco Report - scala:2.12.17 0%
fa-db:doobie Jacoco Report - scala:2.12.17 0%
Files
Module File Coverage [17.42%]
fa-db:core Jacoco Report - scala:2.12.17 StandardStatusHandling.scala 100% 🍏
StatusAggregator.scala 100% 🍏
ByMajorityErrorsStatusAggregator.scala 92.31% 🍏
ByFirstErrorStatusAggregator.scala 85.71% 🍏
ByFirstRowStatusAggregator.scala 81.54% 🍏
package.scala 81.48% 🍏
DBEngine.scala 11.11%
DBFunction.scala 4.2%
Query.scala 0%
UserDefinedStatusHandling.scala 0%
fa-db:slick Jacoco Report - scala:2.12.17 SlickPgEngine.scala 0%
SlickFunction.scala 0%
SlickQuery.scala 0%
fa-db:doobie Jacoco Report - scala:2.12.17 DoobieFunction.scala 0%
StatusWithData.scala 0%
DoobieQuery.scala 0%
DoobieEngine.scala 0%

@lsulak lsulak added the work in progress Work on this item is not yet finished (mainly intended for PRs) label Mar 25, 2024
@lsulak lsulak marked this pull request as draft March 25, 2024 13:07
@lsulak lsulak marked this pull request as ready for review April 3, 2024 15:04
@lsulak lsulak removed the work in progress Work on this item is not yet finished (mainly intended for PRs) label Apr 3, 2024
case Left(statusException) => Left(statusException)
case Right(value) => Right(getResult(value))
): ExceptionOrStatusWithDataRow[R] = {
val o = checkStatus(statusWithData)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using more descriptive names for variables in this method.

Copy link
Collaborator Author

@lsulak lsulak Apr 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, damned! Apologies, this is a result of me debugging this and forgetting to change it back. Rookie mistake :D

* It provides an implementation for aggregating error statuses of a function invocation into a single error
* by choosing the error that occurred the most.
*/
trait ByMajorityErrorsStatusAggregator extends StatusAggregator {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if this is needed. I can hardly imagine someone would want to aggregate errors in this way.

override def aggregate[R](statusesWithData: Seq[ExceptionOrStatusWithDataRow[R]]): ExceptionOrStatusWithDataResultAgg[R] = {
val firstRow = statusesWithData.headOption

val dataFinal = gatherDataWithStatuses(statusesWithData)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to gather data only if first row didn't contain an error.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will change

trait ByFirstErrorStatusAggregator extends StatusAggregator {

override def aggregate[R](statusesWithData: Seq[ExceptionOrStatusWithDataRow[R]]): ExceptionOrStatusWithDataResultAgg[R] = {
val firstError = gatherExceptions(statusesWithData).headOption
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inefficient.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, I'll change this

Base automatically changed from feature/118-db-function-cleanup-pre-118 to master April 26, 2024 09:01
@benedeki benedeki removed the dependent The item depends on some other open item (Issue or PR) label Apr 26, 2024
@benedeki
Copy link
Contributor

* @param data the data of one row (barring the status fields)
* @tparam D the type of the data
*/
case class Row[D](functionStatus: FunctionStatus, data: D)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if Row is a good name. Also why D for the type parameter?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

D simply says that it's the data, or, 'effective/actual data' (sometimes this term can be find in literature). So basically a row without the status-related columns.

Hmm, why not? Row indicates that 1 instance of this should represent 1 row being returned from the DB. I had the name much more complicated, and it's true that Row is pretty simple, naive even, but when I checked the usecases and thought about its internal structure, I think that it's good enough name for this (couldn't come up with anything better and I didn't want the names to be sentences :D )

Also, this naming was not chosen to be 'isolated' to this Row class - we wanted to make it kind of relatable and consistent with the rest of the case classes in this module - see the classes below in this module :)

But as always, I'm open to proposals

*
* Note: D here represents a single row reduced by status-related columns, i.e. a type of data.
*/
type FailedOrRow[D] = Either[StatusException, Row[D]]
Copy link
Contributor

@salamonpavel salamonpavel May 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about StatusExceptionOr[A] - it's more common to have sth like ErrorOr[A] or sth like my suggestion which has the type encoding the failed value in the name.

Copy link
Collaborator Author

@lsulak lsulak May 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm but sometimes it can happen that the type parameter is provided somehow automatically (is the compiler can understand it from the caller, then I won't need to specify it explicitly), which would lead to odd name

On the other hand, we can always explicitly supply it..

I must admit that I haven't really seen such approach in the past, but then you are much more experienced in Scala - I'll think about it; but basically the main idea was to make those 3 classes relatable = so I value 3 classes named well enough and relate-able more, rather than each class perfectly named but not relate-able. WDYT?

*
* Note: D here represents a single row reduced by status-related columns, i.e. a type of data.
*/
type FailedOrRowSet[D] = Either[StatusException, Seq[Row[D]]]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above. I would reconsider the name.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

acknowledged, but see my answer above

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also consider name change, but little different.
FailedOrRows[D] or FailedOrRowSeq[D]? 🤔

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like FailedOrRows, thanks

functionNameOverride: Option[String] = None
)(implicit schema: DBSchema, dBEngine: E)
extends DBFunctionWithStatus[I, R, E, F](functionNameOverride)
with StatusAggregator {
Copy link
Contributor

@salamonpavel salamonpavel May 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like the fact it requires StatusAggregator to be mixed-in. I believe this should be optional and the return value of DBMultipleResultFunctionWithStatus should be a sequence of Either (or its alias). The aggregation should be in my opinion applied optionally later on as one might be interested also in all statuses.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes that's basically the hardest 'design decision' in this PR I think. I appreciate / see the benefits of both approaches, that's why I was kind of undecided yet and 'waiting' for 3rd opinion :D

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay I though about it and:

  • I think status aggregation in general is gonna be used quite common and is therefore very helpful - just need to provide implementation that is convenient for us and users
  • I still really like and prefer mix-in approach - it's consistent and 'expected' from user point of view how things are plugged together, like nice reusable and changeable lego blocks
  • I understand that having a mandatory agg implementation might be too much and sometimes not needed. Therefore I implemented it separately - one with mixin and the other without.
  • With this 'new' approach, I created respective 'utils' ('helper' classes) to both, Doobie and Slick, and sufficiently covered it all with tests.

Does that influence your opinion on this status aggregation related feature?

Copy link

github-actions bot commented May 21, 2024

JaCoCo core code coverage report - scala 2.13.12

Overall Project 57.44% -44.07%
Files changed 40.83%

File Coverage
StandardStatusHandling.scala 100% 🍏
package.scala 100% -909.09%
StatusAggregator.scala 100% 🍏
ByMajorityErrorsStatusAggregator.scala 92.11% -7.89% 🍏
ByFirstErrorStatusAggregator.scala 85.71% -14.29% 🍏
ByFirstRowStatusAggregator.scala 82.09% -17.91% 🍏
DBEngine.scala 11.11% -88.89%
DBFunction.scala 4.01% -52.72%
Query.scala 0% -116.67%
UserDefinedStatusHandling.scala 0% -21.74%

Copy link

github-actions bot commented May 21, 2024

JaCoCo slick code coverage report - scala 2.13.12

Overall Project 87.73% -12.79% 🍏
Files changed 79.32%

File Coverage
SlickPgEngine.scala 92.5% 🍏
SlickFunction.scala 88.97% -24.83%
SlickQuery.scala 84.87% -10.92%

Copy link

github-actions bot commented May 21, 2024

JaCoCo doobie code coverage report - scala 2.13.12

Overall Project 81.6% -14.62% 🍏
Files changed 69.1%

File Coverage
StatusWithData.scala 100% 🍏
DoobieQuery.scala 100% 🍏
DoobieEngine.scala 100% 🍏
DoobieFunction.scala 76.41% -18.75%

@@ -37,21 +37,22 @@ trait QueryWithStatus[A, B, R] {
* @param initialResult - the initial result of the query
* @return the status with data
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* @return the status with data
* @return the data with status

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, changed on all the places

Comment on lines 29 to 30
* @tparam A - the initial result type of the query (a row basically, having status-related columns as well)
* @tparam B - the intermediate result type of the query (a row without status columns, i.e. data only)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about renaming these generics to:
DS - data with status
D - data

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adopted, I like it!

*
* Note: D here represents a single row reduced by status-related columns, i.e. a type of data.
*/
type FailedOrRowSet[D] = Either[StatusException, Seq[Row[D]]]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also consider name change, but little different.
FailedOrRows[D] or FailedOrRowSeq[D]? 🤔

benedeki
benedeki previously approved these changes Jun 21, 2024
Copy link
Contributor

@benedeki benedeki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely an improvement!

…g function util class separately so that users can choose - also available in Slick and Doobie, covered all by tests
Copy link
Contributor

@benedeki benedeki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lsulak lsulak merged commit 2e0631a into master Jun 25, 2024
5 of 6 checks passed
@lsulak lsulak deleted the feature/118-add-DoobieMultipleResultFunctionWithStatus branch June 25, 2024 11:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants