Term frequency charts warning message

[DavidFrenchSG]

Hello,

I noticed that when creating a term frequency chart, a "not found in the dataset" warning message appears even when the names provided are in the dataset. Indeed, such a warning message appears in the output on the help page here, despite "Robert" and "Grace" appearing on the chart.

I am very new to Python, so apologies if I have misunderstood something, but I think the problem is in line 240 of term_frequency.py:

vals_not_included = [val for val in vals_to_include if val not in df["value"]]

For reasons I don't really understand, this seems to return every value in vals_to_include, regardless of whether they are in df["value"]

I think this line of code works instead:

vals_not_included = [val for val in vals_to_include if ~df["value"].str.contains(val).any()]

Did that make sense? Have I completely misunderstood something?

Dave

[DavidFrenchSG]

I played around with this a bit more and realised that the proposed solution above would check for partial matches.

I think this works instead:
vals_not_included = [val for val in vals_to_include if val not in df["value"].values]

[RobinL]

@DavidFrenchSG sorry for the delay in responding, I've been on leave - good spot, thanks!

Would you like to do a PR with the suggestion above? I've checked it at my end and it looks good to me. If not, happy to do it myself, but would be nice for you to get credit!

p.s. if you'd ever like to chat, we always like to hear from users in government- feel free to reach out at robinlinacre@hotmail.com and I'll reply from my gov email address

[DavidFrenchSG]

@RobinL

Thank you! I will do a PR with the change early next week, and will drop you an email as well.