Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maintain correct first name when merging SF contacts #188

Merged
merged 16 commits into from Aug 30, 2018

Conversation

johnduffell
Copy link
Member

@johnduffell johnduffell commented Aug 22, 2018

When we were merging the SF contacts, we decide the winner based on the reliability of the address. This is more likely for print subscriptions.

However, where one of the source contacts was created via identity, that would have a first name, whereas the target SF one may not have.

This PR updates the validation to make sure that all zuora accounts have the same last name before going ahead with the merge. In that case, if the losing account has just a full stop for the name, it will replace it with the one from the identity user. If the first name is present in zuora already, it will write the winning one into the winning contact. If there is no first name from either, it will just replace with a full stop. This will ensure that the zuora sync will work.

I have also added a check that the winning SF contact id really is in the ones connected from zuora. If not then it could indicate an issue.

I have also been doing a bit of tidyup of the function signatures. Rather than explicitly specifying the function type, I have used a trait with a single abstract method. This is experimental and may encourage excessive numbers of parameters, but see what you think.

I think there are a few codacy things to work through and I need to check the important lines have code coverage as I have a feeling I need to add some tests. But have a look at the overall view in the mean time. @pvighi @paulbrown1982 @abhichawla

@@ -88,16 +88,6 @@ lazy val salesforce = all(project in file("lib/salesforce"))
libraryDependencies ++= Seq(okhttp3, logging, scalaz, playJson, scalatest)
)

lazy val `update-sf-identityid` = all(project in file("lib/sf/update-sf-identityid"))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was added as an attempt to share the update SF identity id code. However now that we are updating first name as well, it is diverging a lot so I have given up and had two copies.

_
))
} yield Operation.noHealthcheck {
WireRequestToDomainObject {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

previously we were trying to make a big for comprehension working from the api gateway request level all the way down to the domain level. Then we were trying to artificially split between "validate" and "update" once at the domain level.
I decided it would make more sense to split by the level, so the WireRequestToDomainObject brings us down to the domain level and then steps just worries about dealing with a domain MergeRequest

_ <- validateLastNames(accountAndEmails.map(_.lastName))
maybeIdentityFirstName = firstNameForIdentityAccount(accountAndEmails.map { info => (info.identityId, info.firstName) })
maybeOldFirstName = firstNameForSFContact(mergeRequest.sFPointer.sfContactId, accountAndEmails.map { info => (info.sfContactId, info.firstName) })
firstNameToUse <- firstNameIfNot(maybeOldFirstName, maybeIdentityFirstName)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should probably combine these three previous lines into one function - TODO

@coveralls
Copy link

coveralls commented Aug 22, 2018

Pull Request Test Coverage Report for Build 337

  • 75 of 77 (97.4%) changed or added relevant lines in 9 files are covered.
  • 5 unchanged lines in 5 files lost coverage.
  • Overall coverage increased (+0.4%) to 70.309%

Changes Missing Coverage Covered Lines Changed/Added Lines %
handlers/sf-contact-merge/src/main/scala/com/gu/sf_contact_merge/Handler.scala 24 25 96.0%
handlers/sf-contact-merge/src/main/scala/com/gu/sf_contact_merge/getaccounts/GetZuoraContactDetails.scala 12 13 92.31%
Files with Coverage Reduction New Missed Lines %
lib/handler/src/main/scala/com/gu/util/apigateway/ApiGatewayHandler.scala 1 80.95%
handlers/sf-contact-merge/src/main/scala/com/gu/sf_contact_merge/update/UpdateAccountSFLinks.scala 1 80.0%
handlers/sf-contact-merge/src/main/scala/com/gu/sf_contact_merge/getaccounts/GetIdentityAndZuoraEmailsForAccountsSteps.scala 1 93.33%
src/main/scala/com/gu/util/zuora/ZuoraGetAccountSummary.scala 1 95.24%
lib/restHttp/src/main/scala/com/gu/util/resthttp/RestRequestMaker.scala 1 69.64%
Totals Coverage Status
Change from base Build 298: 0.4%
Covered Lines: 1480
Relevant Lines: 2105

💛 - Coveralls

@guardian guardian deleted a comment Aug 23, 2018
@guardian guardian deleted a comment Aug 23, 2018
@guardian guardian deleted a comment Aug 23, 2018
@guardian guardian deleted a comment Aug 23, 2018

def apply(emailAddresses: List[Option[EmailAddress]]): ApiGatewayOp[Unit] =
def apply[EmailAddress]: AssertSame[EmailAddress] = (emailAddresses: List[EmailAddress]) =>
Copy link
Contributor

@pvighi pvighi Aug 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is now generic right? because all the names and the error message still refer to email addresses


object EnsureNoAccountWithWrongIdentityId { // make sure all accounts are either this identity id or none

val apply: EnsureNoAccountWithWrongIdentityId = //FIXME use the validation from new-product-api
Copy link
Contributor

@pvighi pvighi Aug 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was this FIXME meant to be fixed in this PR or later ?
edit: I see that this FIXME was preexisting, you just moved it so probably it's for later

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if we should really move the validate stuff to a lib, or whether it is simple enough to just copy and paste the one file.

@@ -8,21 +8,21 @@ class AssertSameEmailsTest extends FlatSpec with Matchers {
it should "be all ok with the same email addresses" in {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor: This should probably be called AssertSameTest now and maybe the test descriptions could not refer to emails

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good spot, I'm actually testing assert same specifically in the case of Email where it is an Option. So I think I will leave it the same for now, unless you prefer to have the class as AssertSameTest and just leave the descriptions mentioning it's specifically for the emails case.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have renamed the actual AssertSame type parameter though, as that code is generic

if (emailAddresses.distinct.size == 1) ContinueProcessing(())
else ReturnWithResponse(ApiGatewayResponse.notFound("those zuora accounts had differing emails"))

}

trait AssertSame[EmailAddress] {
def apply(emailAddresses: List[EmailAddress]): ApiGatewayOp[Unit]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we ever use the new-product-api validation for EnsureNoAccountWithWrongIdentityId we can probably use it for this as well

@guardian guardian deleted a comment Aug 23, 2018

class GetFirstNameToUseFirstNameIfNotTest extends FlatSpec with Matchers {

it should "firstNameIfNot both present should use winning one" in {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very minor:
the description of this test would read:
"GetFirstNameToUseFirstNameIfNotTest should firstNameIfNot both present should use winning one"
maybe it should be something like
"firstNameIfNot" should "use winning one if both present"

(maybeOld, maybeIdentity) match {
case (Some(old), _) => Some(old)
case (None, Some(identityFirstName)) => Some(identityFirstName)
case (old, _) => old // will be none
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this different than doing maybeOld orElse MaybeIdentity ?
if it is maybe we don't need this function at all

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh - good spot that's the plain english version too 😆

@guardian guardian deleted a comment Aug 23, 2018

def setOrClearIdentityId(sfContactId: SFContactId, idid: SFContactUpdate): Types.ClientFailableOp[Unit] = {
order = (idid match {
def setOrClearIdentityId(sfContactId: SFContactId, sfUpdateRequest: SFContactUpdate): Types.ClientFailableOp[Unit] = {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pvighi github had a funny and list your comment about idid being a funny name, but anyway I've done a bit of a rejig on this test now - it wasn't great and it still needs a var due to side effects.

@guardian guardian deleted a comment Aug 23, 2018
@@ -12,22 +12,35 @@ object GetEmails {
case class ContactId(value: String) extends AnyVal

case class EmailAddress(value: String) extends AnyVal
case class FirstName(value: String) extends AnyVal
case class LastName(value: String) extends AnyVal
case class Record(emailAddress: Option[EmailAddress], firstName: Option[FirstName], lastName: LastName)
Copy link
Contributor

@pvighi pvighi Aug 30, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The naming is a little confusing now. you still call GetEmails.apply buy it now returns a map of ContactId -> Record.
Also record seems like a really generic name, maybe we could use something more descriptive like personalDetails or ZuoraContactDetails

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good spot, thanks, I'll rename these all

val someFirstName = Some(FirstName("hi"))
val contactIdsFromZuora = List(NameForContactId(SFContactId("winningSfContact"), someFirstName))
val actual = GetFirstNameToUse.firstNameForSFContact(SFContactId("winningSfContact"), contactIdsFromZuora)
actual should be(ContinueProcessing(someFirstName))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

really minor:
don't you think in cases like this we need to test that the correct value is picked by providing some other contacts and asserting that it picks the right one? or would you rather have the minimal test to keep it simple?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't do so because I wanted to test the boolean condition and its outcome rather than the built in find functionality, however I'll add one for completeness


val actual = MoveIdentityId(mock.setOrClearIdentityId)(sfPointer, maybeContactId, Some(FirstName("hello")))

mock.order.reverse should be(List("clear contold setname: DontChangeFirstName", "clear contnew setname: SetFirstName(FirstName(hello))"))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

really really tiny: maybe mock.order could return the reversed list already, otherwise it should be called reverseOrder

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll just get rid of the reversedness in the first place, it won't be slow with only a couple of items in the list...


class MoveIdentityIdTest extends FlatSpec with Matchers {

final class Var() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this could be called mockSetOrClearIdentityId and order could be something like invocationLog or something

"FirstName" -> JsString("firstname")
))
val expected = new PatchRequest(expectedJson, RelativePath("/services/data/v20.0/sobjects/Contact/contactsf"))
actual should be(expected)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 I'm going to try to start using this pattern, it makes it much more testable

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we decide to switch over, there's a lot of tech debt we can plough through to get off the old style request maker stuff.

Copy link
Contributor

@pvighi pvighi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 looks good, left some minor comments mostly about naming that you can ignore if you want

@guardian guardian deleted a comment Aug 30, 2018
@pvighi
Copy link
Contributor

pvighi commented Aug 30, 2018

👍 👍

@guardian guardian deleted a comment Aug 30, 2018
@johnduffell johnduffell merged commit c3ee0e2 into master Aug 30, 2018
@johnduffell johnduffell deleted the copy-firstname-merge branch August 30, 2018 14:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants