Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Save built-in reports as Tower reports #4760

Merged
merged 2 commits into from
Mar 12, 2024
Merged

Save built-in reports as Tower reports #4760

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e27fd41
Save built-in reports as Tower reports
bentsherman Feb 17, 2024
a8607b5
Publish built-in reports only if specified in tower.yml
bentsherman Feb 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions plugins/nf-tower/src/main/io/seqera/tower/plugin/TowerClient.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -390,6 +390,8 @@ class TowerClient implements TraceObserver {
void onFlowComplete() {
// submit the record
events << new ProcessEvent(completed: true)
// publish runtime reports
reports.publishRuntimeReports()
// wait the submission of pending events
sender.join()
// wait and flush reports content
Expand Down
27 changes: 27 additions & 0 deletions plugins/nf-tower/src/main/io/seqera/tower/plugin/TowerReports.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,10 @@ import groovy.yaml.YamlSlurper
import groovyx.gpars.agent.Agent
import nextflow.Session
import nextflow.file.FileHelper
import nextflow.trace.GraphObserver
import nextflow.trace.ReportObserver
import nextflow.trace.TimelineObserver
import nextflow.trace.TraceFileObserver
/**
* If reports are defined at `nf-<workflow_id>-tower.yml`, collects all published files
* that are reports and writes `nf-<workflow_id>-reports.tsv` file with all the paths.
Expand Down Expand Up @@ -220,4 +224,27 @@ class TowerReports {
final prefix = reportKey.startsWith("**/") ? "" : "**/"
return "glob:${prefix}${reportKey}"
}

/**
* Publish any built-in reports that are enabled to the reports file.
*/
void publishRuntimeReports() {
final config = session.config
final files = []

if( config.navigate('report.enabled') )
files << config.navigate('report.file', ReportObserver.DEF_FILE_NAME)

if( config.navigate('timeline.enabled') )
files << config.navigate('timeline.file', TimelineObserver.DEF_FILE_NAME)

if( config.navigate('trace.enabled') )
files << config.navigate('trace.file', TraceFileObserver.DEF_FILE_NAME)

if( config.navigate('dag.enabled') )
files << config.navigate('dag.file', GraphObserver.DEF_FILE_NAME)

for( def file : files )
filePublish( (file as Path).complete() )
Comment on lines +231 to +248
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems overlapping with the logic in the LogsHandler

https://github.com/nextflow-io/nextflow/blob/3460-publish-nextflow-reports/plugins/nf-tower/src/main/io/seqera/tower/plugin/LogsHandler.groovy#L61-L70

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is the logs handler saving the timeline report but not the others?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because that was the requirement

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What requirement? 😆

In any case, the logs handler is saving the timeline file whereas this PR is adding the built-in reports to the Tower reports list. So there is no overlap

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nearly the timeline is on both

https://github.com/nextflow-io/nextflow/blob/3460-publish-nextflow-reports/plugins/nf-tower/src/main/io/seqera/tower/plugin/LogsHandler.groovy#L65-L70

Therefore if the user provides a custom path it breaks the download in the platform frontend.

But I agree is not a problem of this PR. It should be solved in the platform

}
}