TRUNK-6218: Concept Validations should be more lenient with retired concepts

[k4pran]

Description of what I changed

This PR skips validation of concept names and concept maps when the concept being validated is retired. As part of the test there was a requirement to handle these four scenarios:

• We should not validate any concept names for a concept if the concept is retired
• We should validate concept names for a concept when it is being un-retired
• The logic to check for duplicate mappings should ignore retired mappings
• Un-retiring a retired mapping should still check for duplicate mappings

I left a comment on the jira as it seemed to me that checking if the concept was retired also covered the cases where a concept was being unretired. If I am wrong in my understanding I can update this PR accordingly.

Issue I worked on

see https://issues.openmrs.org/browse/TRUNK-6218

Checklist: I completed these to help reviewers :)

• My IDE is configured to follow the code style of this project.

  No? Unsure? -> configure your IDE, format the code and add the changes with git add . && git commit --amend

• I have added tests to cover my changes. (If you refactored
  existing code that was well tested you do not have to add tests)

  No? -> write tests and add them to this commit git add . && git commit --amend

• I ran mvn clean package right before creating this pull request and
  added all formatting changes to my commit.

  No? -> execute above command

• All new and existing tests passed.

  No? -> figure out why and add the fix to your commit. It is your responsibility to make sure your code works.

• My pull request is based on the latest changes of the master branch.

  No? Unsure? -> execute command git pull --rebase upstream master

[ibacher]

Thanks @k4pran! This is very close to perfect. I think the one other thing the ticket asked for was adding some leniency to the ConceptReferenceTermValidator so that we also can implement this part: "The logic to check for duplicate mappings should ignore retired mappings".

This may need some additional work because getConceptReferenceTermByCode() doesn't ignore retired mappings, which it probably should. (I guess the more sophisticated implementation, which may be better, would return an unretired mapping for a code if one exists and a retired mapping for a code if only a retired mapping for the code exists).

[ibacher]

It seems like it would be more efficient to move this if check before the outer loop, since if a concept is retired, we don't need to validate any of it's names.

[ibacher]

Ditto for this.

[k4pran]

This may need some additional work because getConceptReferenceTermByCode() doesn't ignore retired mappings, which it probably should. (I guess the more sophisticated implementation, which may be better, would return an unretired mapping for a code if one exists and a retired mapping for a code if only a retired mapping for the code exists).

Got it thanks. To clarify, if we want to implement a more sophisticated approach is it safe to do this at the DAO level (HibernateConceptDAO.getConceptReferenceTermByCode) ? Then I would update the method to do this following

1. Get an unretired ConceptReferenceTerm if an unretired one exists
2. If no ConceptReferenceTerm exists then get a retired one if it exists
3. If neither exists the code doesn't change

Is that correct?

EDIT: Although upon reviewing the underlying DAO:

		if (terms.isEmpty()) {
			return null;
		} else if (terms.size() > 1) {
			throw new APIException("ConceptReferenceTerm.foundMultipleTermsWithCodeInSource",
				new Object[] { code, conceptSource.getName() });
		}
		return terms.get(0);

It seems to throw an exception if there is more than one match so we would never get a situation where the ConceptReferenceTerm is matched to both a retired and unretired term. I guess we could look through the list and prioritize the unretired term instead of throwing an exception immediately

[ibacher]

@k4pran So, yeah, I think the less sophisticated approach would be something like:

@Override 
public ConceptReferenceTerm getConceptReferenceTermByCode(String code, ConceptSource conceptSource) throws DAOException {
	return getConceptReferenceTermByCode(code, conceptSource, false);
}

@Override
	public ConceptReferenceTerm getConceptReferenceTermByCode(String code, ConceptSource conceptSource, boolean includeRetired) throws DAOException {
		Session session = sessionFactory.getCurrentSession();
		CriteriaBuilder cb = session.getCriteriaBuilder();
		CriteriaQuery<ConceptReferenceTerm> cq = cb.createQuery(ConceptReferenceTerm.class);
		Root<ConceptReferenceTerm> root = cq.from(ConceptReferenceTerm.class);

		Predicate codePredicate = cb.equal(root.get("code"), code);
		Predicate sourcePredicate = cb.equal(root.get("conceptSource"), conceptSource);
        Predicate retiredPredicate = cb.equal(root.get("retired"), retired);

		cq.where(cb.and(codePredicate, sourcePredicate, retiredPredicate));

		List<ConceptReferenceTerm> terms = session.createQuery(cq).getResultList();

		if (terms.isEmpty()) {
			return null;
		} else if (terms.size() > 1) {
			throw new APIException("ConceptReferenceTerm.foundMultipleTermsWithCodeInSource",
				new Object[] { code, conceptSource.getName() });
		}
		return terms.get(0);
	}

What I called the "more sophisticated" approach is something like:

@Override
public ConceptReferenceTerm getConceptReferenceTermByCode(String code, ConceptSource conceptSource) throws DAOException {
	List<ConceptReferenceTerm> conceptReferenceTerms = getConceptReferenceTermByCode(code, conceptSource, true);

	if (conceptReferenceTerms.isEmpty()) {
		return null;
    } else if (conceptReferenceTerms.size() > 1) {
		List<ConceptReferenceTerm> unretiredConceptReferenceTerms = conceptReferenceTerms.stream().filter(term -> !term.getRetired());
        if (unretiredConceptReferenceTerms.size() == 1) {
			return unretiredConceptReferenceTerms.get(0);
		}
		
		// either more than one unretired concept term or more than one retired concept term
		throw new APIException("ConceptReferenceTerm.foundMultipleTermsWithCodeInSource",
				new Object[] { code, conceptSource.getName() });
	}

	return conceptReferenceTerms.get(0);
}

@Override
	public List<ConceptReferenceTerm> getConceptReferenceTermByCode(String code, ConceptSource conceptSource, boolean includeRetired) throws DAOException {
		Session session = sessionFactory.getCurrentSession();
		CriteriaBuilder cb = session.getCriteriaBuilder();
		CriteriaQuery<ConceptReferenceTerm> cq = cb.createQuery(ConceptReferenceTerm.class);
		Root<ConceptReferenceTerm> root = cq.from(ConceptReferenceTerm.class);

		Predicate codePredicate = cb.equal(root.get("code"), code);
		Predicate sourcePredicate = cb.equal(root.get("conceptSource"), conceptSource);
        Predicate retiredPredicate = cb.equal(root.get("retired"), retired);

		cq.where(cb.and(codePredicate, sourcePredicate, retiredPredicate));

		return session.createQuery(cq).getResultList();
	}

And, yeah, this is a change in behaviour because we don't want retired mappings to block unretired mappings.

[ibacher]

LGTM! Thanks @k4pran!