You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We are currently unable to run a number of DataFrame filters on .imageLinks() and webgraph() because they have src and/or dest columns instead of url. The DataFrame filters should be able to filter on those columns as well.
@SinghGursimran want this one since we're stuck in a holding pattern on the Python side of things until I sort out the Scala UDF -> Python UDF linkage?
@ruebot Shall I add a new function to incorporate src and dest OR accommodate this within the same function using an extra argument?
Amending the current function would require a change in docs as well...
Based on the chat @lintool and I where having in Slack this morning, it'd be amending the current functions. I think we could just do this with try cases (oh, I don't know what the proper Scala term is for it 😆 ) for url and src. I don't think we need to do dest or image_url, though @ianmilligan1 might have a use case for that. How's that sound?
We are currently unable to run a number of DataFrame filters on
.imageLinks()
andwebgraph()
because they havesrc
and/ordest
columns instead ofurl
. The DataFrame filters should be able to filter on those columns as well.keepUrlsDF
keepDomainsDF
discardUrlsDF
discardDomainsDF
discardUrlPatternsDF
keepUrlPatternsDF
The text was updated successfully, but these errors were encountered: