Skip to content

Add further debug logging for ds listing #8247

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Nov 28, 2024
Merged

Add further debug logging for ds listing #8247

Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
f0b1b2f
add debug logging for dataset list route
Nov 28, 2024
82690ac
add extensive logging to listGrouped method of DatasetController
Nov 28, 2024
ed69e48
Merge branch 'master' into add-debug-logging-for-ds-listing
fm3 Nov 28, 2024
4c75c97
dont log full objects
Nov 28, 2024
fe47da7
Merge branch 'add-debug-logging-for-ds-listing' of github.com:scalabl…
Nov 28, 2024
b6e432b
Merge branch 'master' into add-debug-logging-for-ds-listing
fm3 Nov 28, 2024
6295857
restore logging full object except for user objects
Nov 28, 2024
2179355
Merge branch 'add-debug-logging-for-ds-listing' of github.com:scalabl…
Nov 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 21 additions & 6 deletions app/controllers/DatasetController.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -224,28 +224,43 @@ class DatasetController @Inject()(userService: UserService,
implicit ctx: DBAccessContext,
m: MessagesProvider): Fox[List[JsObject]] =
for {
_ <- Fox.successful(())
_ = logger.info(s"datasets: $datasets, requestingUser: ${requestingUser.map(_._id)}")
requestingUserTeamManagerMemberships <- Fox.runOptional(requestingUser)(user =>
userService.teamManagerMembershipsFor(user._id))
userService
.teamManagerMembershipsFor(user._id)) ?~> s"Could not find team manager memberships for user ${requestingUser
.map(_._id)}"
Comment on lines +230 to +232
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Enhance error messages for better debugging

The error messages should provide more context about what failed and why.

        userService
          .teamManagerMembershipsFor(user._id)) ?~> s"Could not find team manager memberships for user ${requestingUser
-        .map(_._id)}"
+        .map(_._id)}. Verify user permissions and team associations."

-          } ?~> s"Could not group by datastore for datasets ${byOrgaTuple._2.map(_.name)}"
+          } ?~> s"Failed to process datasets by datastore. Organization: ${organization._id}, Datastores: ${groupedByDataStore.map(_._1).mkString(", ")}"

-      } ?~> s"Could not group by organization for datasets ${datasets.map(_.name)}"
+      } ?~> s"Failed to process datasets by organization. Dataset count: ${datasets.size}, Organizations: ${groupedByOrga.map(_._1).mkString(", ")}"

Also applies to: 260-260, 262-262

_ = logger.info(
s"requestingUserTeamManagerMemberships: ${requestingUserTeamManagerMemberships.map(_.map(_.toString))}")
groupedByOrga = datasets.groupBy(_._organization).toList
js <- Fox.serialCombined(groupedByOrga) { byOrgaTuple: (String, List[Dataset]) =>
for {
organization <- organizationDAO.findOne(byOrgaTuple._1)
_ <- Fox.successful(())
_ = logger.info(s"byOrgaTuple orga: ${byOrgaTuple._1}, datasets: ${byOrgaTuple._2}")
organization <- organizationDAO.findOne(byOrgaTuple._1) ?~> s"Could not find organization ${byOrgaTuple._1}"
groupedByDataStore = byOrgaTuple._2.groupBy(_._dataStore).toList
_ <- Fox.serialCombined(groupedByDataStore) { byDataStoreTuple: (String, List[Dataset]) =>
{
logger.info(s"datastore: ${byDataStoreTuple._1}, datasets: ${byDataStoreTuple._2}")
Fox.successful(())
}
}
result <- Fox.serialCombined(groupedByDataStore) { byDataStoreTuple: (String, List[Dataset]) =>
for {
dataStore <- dataStoreDAO.findOneByName(byDataStoreTuple._1.trim)(GlobalAccessContext)
dataStore <- dataStoreDAO.findOneByName(byDataStoreTuple._1.trim)(GlobalAccessContext) ?~>
s"Could not find data store ${byDataStoreTuple._1}"
resultByDataStore: Seq[JsObject] <- Fox.serialCombined(byDataStoreTuple._2) { d =>
datasetService.publicWrites(
d,
requestingUser,
Some(organization),
Some(dataStore),
requestingUserTeamManagerMemberships) ?~> Messages("dataset.list.writesFailed", d.name)
}
} ?~> "Could not find public writes for datasets"
} yield resultByDataStore
}
} ?~> s"Could not group by datastore for datasets ${byOrgaTuple._2.map(_._id)}"
} yield result.flatten
}
} ?~> s"Could not group by organization for datasets ${datasets.map(_._id)}"
} yield js.flatten

def accessList(datasetId: String): Action[AnyContent] = sil.SecuredAction.async { implicit request =>
Expand Down