Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tools to clean old incomplete submissions #4540

Open
ajnyga opened this issue Feb 26, 2019 · 42 comments
Open

Tools to clean old incomplete submissions #4540

ajnyga opened this issue Feb 26, 2019 · 42 comments
Assignees
Labels
Community:2:Priority Any issue that has been identified through research or feedback as a major community priority. Enhancement:2:Moderate A new feature or improvement that can be implemented in less than 4 weeks. Hosting Bug reports and feature requests from Publishing Services's hosted clients.

Comments

@ajnyga
Copy link
Collaborator

ajnyga commented Feb 26, 2019

Since a couple of releases ago OJS started to show the incomplete submissions for editors. Some editors have asked whether they can remove these and if there is some automatic way of doing it.

Of course there could be legit incomplete submissions included, but maybe we could have at least a command line tool for removing older incomplete submissions from the database?

@heike-r
Copy link
Contributor

heike-r commented Feb 26, 2019

A command line tool might not be convenient for editors / journal managers. Maybe add an option where the journal manager can enter a time span (in days), after which incomplete submissions are deleted and include the option to never delete them.
In my experience, authors sometimes take several days to complete a submission. We usually contact authors of incomplete submissions (not possible via the incomplete submission but via their OJS user profile) after two weeks, just in case they forgot to press the final button. A few days after this, we delete it by hand from the submission list. Maybe an automated reminder two or three days before deletion would also be possible?

@asmecher
Copy link
Member

@NateWr, do you recall this change? I don't think showing incomplete submissions was an intentional change. Most likely this came with the move to the API/vue.js toolset for submission lists.

@ajnyga
Copy link
Collaborator Author

ajnyga commented Feb 26, 2019

@ajnyga
Copy link
Collaborator Author

ajnyga commented Feb 26, 2019

also https://forum.pkp.sfu.ca/t/delete-incomplete-submission/51229 (did not test if that is true)

@NateWr
Copy link
Contributor

NateWr commented Feb 27, 2019

To clarify, your own incomplete submissions show up in My Queue. Journal Managers and admins see others incomplete submissions in All Active.

This was intentional. If I recall, it was done to help cases where authors get stuck during submission and email the editors for help. Editors wanted to be able to "hop into it" to see for themselves how far the author had gotten.

The All Active list has always been intended as a master list to be used infrequently. However, in conversations at sprints, it seems that many senior editors use it more like a My Queue without requiring them to be assigned.

How and where they should be presented is an open question. I suspect that small journals eager for submissions may want to follow up and encourage the author to complete their submission, but large journals overwhelmed with submissions would probably prefer they are automatically cleaned up and kept out of sight. A separate "incomplete" list might be worth considering down the road.

Anyone who would like to change the default behaviour can do so easily by modifying the parameters passed to the active list. Here's a quick sample plugin to do that (not tested).

@ajnyga
Copy link
Collaborator Author

ajnyga commented Feb 27, 2019

I think it is a good feature to have those visible in the listing, but like one of our editors said, it would be nice to be able to easily remove the older ones easily. As I mentioned above, I would be happy with a simple command line tool for the site admins to manage. This is not something journals ask for all the time. Basically a tool with two parameters: context_path and timelimit.

@NateWr
Copy link
Contributor

NateWr commented Feb 27, 2019

It will probably be a while before we get to submission deletion in our CLI tool, but definitely good to have if you're willing to write the DELETE /submissions/{id} endpoint.

@ajnyga
Copy link
Collaborator Author

ajnyga commented Feb 27, 2019

Not probably something I will have time right now, but can you link an example here of a similar endpoint. Are you simply thinking of extending this https://github.com/pkp/ojs/blob/master/api/v1/submissions/SubmissionHandler.inc.php

@NateWr
Copy link
Contributor

NateWr commented Feb 27, 2019

Oh, looks like we've already got this in https://github.com/pkp/pkp-lib/blob/master/api/v1/_submissions/PKPBackendSubmissionsHandler.inc.php#L43-L52!

Deleting from a CLI tool would just involve getting the submissions you want then passing each one to the delete method in the service class:

$submissionService = Services::get('submission');
$submissions = $submissionService->getMany(['contextId' => X, 'isIncomplete' => true]);
foreach ($submissions as $submission) {
  if (/* submission last modified date or original submission date or something is too old */) {
   $submissionService->delete($submission->getId());
  }
}

@librariam
Copy link
Collaborator

+1 from PKP|PS where we sometimes need to create test/incomplete submissions in client journals and usually delete them right away. In 3.3 it's not possible if created from a JM role (I guess we'll have to be mindful to create them as "author")

@librariam
Copy link
Collaborator

+1 from PKP|PS. A journal on 3.3 is looking to clear a significant (25+) number of incomplete submissions done by authors. Logging in as an author for each is rather time-consuming.

@diegoabadan
Copy link
Contributor

We have cases of journals coming from OJS 2 with hundreds of incomplete submissions.

OJS 2 hid these submissions from the editor so they ended up piling up.

@amandastevens
Copy link
Collaborator

I had a request from a hosted client for both a tool to clear multiple incomplete submissions at once and a separate list of incomplete submissions, separate from the archive.

@amandastevens amandastevens added the Hosting Bug reports and feature requests from Publishing Services's hosted clients. label Nov 26, 2021
@emmauhl
Copy link

emmauhl commented Dec 10, 2021

+1 from PKP|PS - had a client with multiple completely blank submissions with no record of who initiated them in the log (is this what happens when the Journal Manager creates them?), making them arduous to clean up.

@pmangahis
Copy link

+1 from PKP|PS client that would like to remove incomplete submissions on multiple journals as well

@zuphilip
Copy link
Contributor

zuphilip commented Apr 7, 2022

+1 Also the authors themselves should be have the possibility to delete a submission they started. Currently they can only abort it, which will only save it and return.

@NateWr NateWr added the Community:2:Priority Any issue that has been identified through research or feedback as a major community priority. label Apr 21, 2022
ajnyga added a commit to ajnyga/ojs that referenced this issue Jun 16, 2022
ajnyga added a commit to ajnyga/ojs that referenced this issue Jun 16, 2022
ajnyga added a commit to ajnyga/ojs that referenced this issue Jun 16, 2022
ajnyga added a commit to ajnyga/ojs that referenced this issue Jun 16, 2022
ajnyga added a commit to ajnyga/ojs that referenced this issue Jun 16, 2022
@polkeskdi
Copy link

Dear community,

What version of OJS that including this release?

Thanks

ajnyga added a commit to ajnyga/ojs that referenced this issue Jun 20, 2022
@NateWr
Copy link
Contributor

NateWr commented Jun 20, 2022

@polkeskdi this issue is not scheduled against any Milestone, which means it is not planned for any future release.

@polkeskdi
Copy link

Thanks @NateWr

Has the community solved the problem about to clean incomplete submissions? I lack the issue from PKP's forum

Thanks again

@NateWr
Copy link
Contributor

NateWr commented Jun 21, 2022

@polkeskdi no, this has not been solved yet.

@diegoabadan
Copy link
Contributor

To make it easier for people to find, I'm noting references to this issue.

It was covered at the Helsinki 2022 Sprint and the summary can be read at:
https://pkp.sfu.ca/2022/08/05/helsinki-2022-sprint-notes-released-cleaning-incomplete-submissions/

There is a PR at pkp/ojs#3431

Also, we have this comment related to the issue of incomplete submissions no longer showing to editors by default: #7495 (comment)

@NateWr NateWr added the Enhancement:2:Moderate A new feature or improvement that can be implemented in less than 4 weeks. label Sep 5, 2022
@cordoba3030
Copy link

When you solve this problem? When users Journal Manager can delete incomplete submissions or any submissions with simple way!

@01Kuzma
Copy link

01Kuzma commented Aug 18, 2023

Any updates on this?

How could such submissions be deleted or archived? They appeared after upgrade.
image

@asmecher
Copy link
Member

@01Kuzma, I assume you're asking about OJS; what version did you upgrade from, and what version did you upgrade to?

@01Kuzma
Copy link

01Kuzma commented Aug 19, 2023

@asmecher , sorry, I supposed this issue is about OJS.
Current version is 3.3.0.8
There were multiple upgrades, but another person was executing them, so I've now observed such issues.
The latest upgrades were done in a following order:
2.4.6.0 -> 2.4.8.3 -> 3.2.1.3 -> 3.3.0.8

@chunghanlien
Copy link

I would really like to have this feature too!!!
After a period of journal operation, too many incomplete submissions in queue, we cannot confirm this state all the time. Moreover, the author may have uploaded junk files taking up our storage space. It is recommended to add a function to automatically delete these useless submissions while a period of time.

@shantanu198713
Copy link

Hi @asmecher @NateWr

We are using OJS-3.3.0-14 and we have successfully upgraded to OJS-3.4.0-3. Please suggest how to use this Delete Incomplete Submission code, if we can use this code in any of the above-mentioned versions. The All active tab is very heavy due to incomplete submission.

@aleskl
Copy link

aleskl commented Sep 22, 2023

The Delete Incomplete Submission tool is not working for our 3.3.0-15 version, the error thrown is Class 'APP\core\Application' not found. If I compare deleteSubmissions.php code in the Git repository with what I have installed in 3.3.0-15, I can see use APP\facades\Repo; and \APP\core\Application calls that are probably new in OJS 3.4?

@asmecher
Copy link
Member

@aleskl, that code is written to be used with 3.4.0, not 3.3.0; we've added namespaces and made numerous other changes to our conventions to make them more modern since 3.3.0 was released.

@wilsonww
Copy link

Hello team, @asmecher we are running 3.4.0.3. I downloaded deleteIncompleteSubmissions.php and tried running php tools/deleteIncompleteSubmissions.php 600 --dryrun and ran into two issues.

  1. PHP Warning: require(ojs/tools/bootstrap.inc.php): Failed to open stream: No such file or directory in ojs/tools/deleteIncompleteSubmissions.php on line 18 which I mitigated by changing Line 18 to "bootstrap.php" instead of "bootstrap.inc.php" -- this looks like a future or pending change to rename to bootstrap.inc.php?
  2. PHP Fatal error: Uncaught Error: Call to undefined method APP\submission\Repository::getMany() in ojs/tools/deleteIncompleteSubmissions.php:86 which I am unsure on how to debug. Is something else out of sync on my instance? Any intuitions? Thank you!

@asmecher
Copy link
Member

@wilsonww, the tool needs some adjustments before it'll work. @ajnyga, do you have any plans to work on this further?

@diegoabadan
Copy link
Contributor

diegoabadan commented Oct 26, 2023

We intend to soon create a plugin so that editors themselves can delete incomplete submissions, as we receive this request quite frequently. Our priority is OJS 3.3 at the moment.

@diegoabadan
Copy link
Contributor

Hi! We have a preliminary version of the plugin. If you want to try it out (backup first, please), there's a package available for OJS 3.3: https://github.com/lepidus/deleteIncompleteSubmissions

@shantanu198713
Copy link

Hi @diegoabadan

We are using OJS-3.4 and I hope soon we will get a plugin for the same.

@diegoabadan
Copy link
Contributor

We are involved in the release of other plugins, and currently we don't have a deadline, but it is in the plans to have it for OJS 3.4 soon.

@cordoba3030
Copy link

cordoba3030 commented Feb 7, 2024

Who did use this plugin?
How looks is it. Could you paste sccreenshot how to delete... I install plugin, but nothing changed in dashboard interface..
Could you help us..

@cordoba3030
Copy link

We are involved in the release of other plugins, and currently we don't have a deadline, but it is in the plans to have it for OJS 3.4 soon.

Who did use this plugin?
How looks is it. Could you paste sccreenshot how to delete... I install plugin, but nothing changed in dashboard interface..
Could you help us..

@diegoabadan
Copy link
Contributor

Hi @cordoba3030

This first version of the plugin is very simple and not very informative.

You need to check the journal's dashboard for incomplete submissions:

image

Then go to the list of plugins, find it and click on the "Delete submissions..." option.

Once you've deleted the incomplete submissions using the plugin, go back to the list of submissions and the incomplete ones that meet the chosen time criteria should have disappeared.

@cordoba3030
Copy link

cordoba3030 commented Mar 15, 2024

Thanks!! It works!..
I tried from Plugins list.. There is certain button Delete incomplete submissions... I click it .. All incomplete submissions were deleted immediately. Problem solved. Thanks so much!!

@wilsonww
Copy link

wilsonww commented May 29, 2024

Hello @NateWr

I followed your comment from #4540 (comment) and since it was 5 years ago, I tried adopting for 3.4, accounting for the deprecated functions.

Using the context of this sprint https://pkp.sfu.ca/2022/08/05/helsinki-2022-sprint-notes-released-cleaning-incomplete-submissions/, our perspective is from the journal editor's seeing "incomplete" submissions flooding their Active queue, rather than the sprint's focus on the site managers deleting all incomplete submissions.

Your "Hide Incomplete Submissions" plugin seems to be a reasonable workaround, configurable by each journal editor if they want to see incomplete submissions or not.

I've gotten this far but got stuck. I'm able to enable the plugin and ensure it's running on the right page, but I don't understand the filtering & template logic enough to determine next steps.

<?php
namespace APP\plugins\generic\removeIncompleteFromActiveList;

use PKP\plugins\GenericPlugin;
use PKP\plugins\Hook;
use APP\template\TemplateManager;

class RemoveIncompleteFromActiveListPlugin extends GenericPlugin {
  /**
   * Register the plugin
   */
  public function register($category, $path, $mainContextId = NULL) {
  // Register the plugin even when it is not enabled
    $success = parent::register($category, $path);

    if ($success && $this->getEnabled()) {
      // Do something when the plugin is enabled
      Hook::add('TemplateManager::display', [$this, 'modifyTemplateData']);
    }

    return $success;
  }

  /**
  * Provide a name for this plugin
  *
  * The name will appear in the Plugin Gallery where editors can
  * install, enable and disable plugins.
  */
  public function getDisplayName() {
    return 'Hide Incomplete Submissions';
  }

  /**
   * Provide a description for this plugin
   *
   * The description will appear in the Plugin Gallery where editors can
   * install, enable and disable plugins.
   */
  public function getDescription() {
    return 'This plugin will hide incomplete submissions from the Active submissions list.';
  }

  /**
   * Fired when the `TemplateManager::display` hook is called.
   *
   * @param string $hookname
   * @param array $args [$templateMgr, $template, $sendContentType, $charset, $output]
   */
  public function modifyTemplateData($hookName, $args) {

    // Check for the correct template
    $template = $args[1];

    if ($template != 'dashboard/index.tpl') {
      return;
    }

    // Retrieve the TemplateManager
    $templateMgr = $args[0];

    // Attach a custom piece of data to the TemplateManager
    $activeListData = $templateMgr->getTemplateVars('activeListData');
    $activeListData['getParams']['isIncomplete'] = false;

    // Something like this to attach a filter?
    // $filters = $templateMgr->getTemplateVars('filters');

    $templateMgr->assign('activeListData', $activeListData);
  }
}

How I can adopt the core of this code -- to hide submissions with isIncomplete. Can you/anyone provide additional guidance or point me in the right direction on what to try next? Thank you!

@wilsonww
Copy link

wilsonww commented Jun 6, 2024

Quick follow up here. As an alternative, our journal editors mentioned flipping the "Incomplete" filter in the Submissions tab. In other words, can we build a filter for "NOT Incomplete" in order to hide the Incomplete submissions in their queue?

Screenshot 2024-06-06 at 3 27 57 PM

I dug deeper and traced the Submissions API call over to ./lib/pkp/api/v1/_submissions/PKPBackendSubmissionsHandler.php:210: where it appears to only handle isIncomplete -> true rather than listening to the URL parameter and allowing isIncomplete -> false.

Screenshot 2024-06-06 at 3 39 04 PM

I confirmed this is the case, whether putting &isIncomplete=true or &isIncomplete=false has no effect on the Submissions API endpoint.

Screenshot 2024-06-06 at 3 27 08 PM

I traced this to the source ./lib/pkp/classes/submission/Collector.php:373: where I found the core code only accounts for Incomplete submissions.

if ($this->isIncomplete) { $q->where('s.submission_progress', '<>', ''); }

My one idea/hack that worked was adding the else to force Incomplete submissions to be hidden unless explicitly requested by activating the Incomplete filter.

Screenshot 2024-06-06 at 3 45 57 PM

While this works, this applies across my entire OJS instance and I don't think my hacking core pkp/lib is a good solution -- my question for the PKP team (tagging @asmecher here) is can this functionality be added via a plugin (see my plugin attempt in the comment above)? I don't have enough PKP core knowledge to know whether the plugin architecture method is feasible or not to achieve this behavior. Thank you!

@shantanu198713
Copy link

Hi @wilsonww
Following implementation in the suggested code ./lib/pkp/classes/submission/Collector.php:373: also hides the 'Incomplete' submission of the author.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Community:2:Priority Any issue that has been identified through research or feedback as a major community priority. Enhancement:2:Moderate A new feature or improvement that can be implemented in less than 4 weeks. Hosting Bug reports and feature requests from Publishing Services's hosted clients.
Projects
Status: Backlog
Development

No branches or pull requests