docs: Join types

[KronosTheLate]

Please describe the issue

If all the join types documented here are available, I find it problematic that the docstring for join only mentiones "inner" and "left". I am actually still not sure what types of join are available, and if I should be using "join" with the "how" argument, or if I should be using the dedicated functions "anti_join", "left_join" etc. I see how it would be massivly breaking to depricate one of them, but having multiple ways to do something thing is more problematic IMO.

Code of Conduct

• I agree to follow this project's Code of Conduct

[jy-andy-hu]

I strong echo to this comment. In addition, I hope there are more specific examples how each type of predicates ("==", "!=" etc.) behaves and how each predicate type corresponds to each different how argument. I appreciate the flexibility and incremental power coming from using both predictates and how, but at the moment, between the predicates and how arguments, new users can find it very confusing just by reading the API reference.

[cpcloud]

I see how it would be massivly breaking to depricate one of them, but having multiple ways to do something thing is more problematic IMO.

At this point it is far too late to deprecate one of these methods.

Better documentation is certainly on the menu though!

[KronosTheLate]

So both will continue to be supported, and it is 100% user preferance? I would appreciate that being expressed in the docstrings, or at least the online documentation.

Whenever I see two similar things, my question is always which one to choose. That of course depends on what is the difference. So perhaps a sentence like "this is equivalent to using join with the how="inner" argument" would suffice for the specific functions. And for join itself, perhaps "this function is equivalent to using one of join_left, ... with the approptiate how argument"?

[jy-andy-hu]

I think you just articulated better than I did. This is what I was hoping to see improvement from the "join" docstring part of the documentation: 1. What are all of the join kinds (in terms of "how" argument) are available 2. how each type of "how" is expressed in predicates (in terms of "==", ">=", "!=" etc..). At the moment, the documentation leaves too much work for users in terms of guessing and trial and error. Listing and mapping all of them out can give users clear ideas and make them more productive.

…

________________________________ From: Dennis Hoelgaard Bal ***@***.***> Sent: Saturday, June 21, 2025 09:24 To: ibis-project/ibis ***@***.***> Cc: Andy Hu ***@***.***>; Comment ***@***.***> Subject: Re: [ibis-project/ibis] docs: Join types (Issue #11364) [https://avatars.githubusercontent.com/u/61620837?s=20&v=4]KronosTheLate left a comment (ibis-project/ibis#11364)<#11364 (comment)> So both will continue to be supported, and it is 100% user preferance? I would appreciate that being expressed in the docstrings, or at least the online documentation. Whenever I see two similar things, my question is always which one to choose. That of course depends on what is the difference. So perhaps a sentence like "this is equivalent to using join with the how="inner" argument would suffice for the specific functions. And for join itself, perhaps "this function is equivalent to using one of join_left, ... with the approptiate how argument"? — Reply to this email directly, view it on GitHub<#11364 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ATYSIVTMZLTR2GPA22SG3H33EVMG3AVCNFSM6AAAAAB7XQDA2KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSOJTGU4DEOBVGQ>. You are receiving this because you commented.

[KronosTheLate]

I think the predicates are clear enough in the sense that from the examples, I see that I can pass:
A) a string that is the column to join on in both tables
B) a predicate of the form t1.col_1 == t2.col_2
C) or a list of the previous forms

For format B, i believe that using non-equality (!= or <) as you suggest is highly uncommon for joining, but that the effect is clear enough: for example if you write t1.col_1 != t2.col_2, then each row in the left table matches all rows that are different in the right table, which would give you all possible combinations. A bit like a cross join I believe. But this is rather uncommon, so personally I feel no need to document this.

What I DO still have questions about when it comes to predicates is this: If my predicate is _.col_1 = _.col_2, which table is swapped in for the _? The left one (i e. the one's join method you are calling) in both cases? Or are we smart enough to swap in different tables in the different places? This is still very unclear to me.

[jy-andy-hu]

When I suggested mapping between predicates and "how" arguments earlier, I am thinking about something like the follow that should be more cleared up: t1.Join(t2, t1[“col1”] >= t2[“col1”], how = "anti") (I making this up, I don't know if "anti" is still valid now) It would be nice to clarify what kind of join behaviour the statement above can be expected .

…

________________________________ From: Dennis Hoelgaard Bal ***@***.***> Sent: Saturday, June 21, 2025 10:32 To: ibis-project/ibis ***@***.***> Cc: Andy Hu ***@***.***>; Comment ***@***.***> Subject: Re: [ibis-project/ibis] docs: Join types (Issue #11364) [https://avatars.githubusercontent.com/u/61620837?s=20&v=4]KronosTheLate left a comment (ibis-project/ibis#11364)<#11364 (comment)> I think the predicates are clear enough in the sense that from the examples, I see that I can pass: A) a string that is the column to join on in both tables B) a predicate of the form t1.col_1 == t2.col_2 C) or a list of one of the previous forms. For format B, i believe that while using non-equality (!= or <) as you suggest is highly uncommon for joining, but that the effect is clear enough: for example if you write t1.col_1 != t2.col_2, then each row in the left table matches all rows that are different in the right table, which would give you all possible combinations. A bit like a cross join I believe. But this is rather uncommon, so personally I feel no need to document this. What I DO still havequestions about is this: If my predicate is _.col_1 = _.col_2, which table is swapped in for the _? The left one (i e. the one's join method you are calling) in both cases? Or are we smart enough to swap in different tables in the different places? This is still very unclear to me. — Reply to this email directly, view it on GitHub<#11364 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ATYSIVWRO46RI7Q2D7OZLFL3EVUJBAVCNFSM6AAAAAB7XQDA2KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSOJTGYYTCNBVHE>. You are receiving this because you commented.

[KronosTheLate]

And I believe that such a combination of predicate and join type is highly uncommon, and that what you use it for is not well defined. It could be that there are a few cases where it is useful, but that they differ wildly. Or, it might never be useful! The fact that it is possible does not mean it requires documentation, if it is nonsensical use of the function.

In fact, because too much information overwhelms, I would be against documenting every combination like this, because I think it would mostly be noise 🤔

[cpcloud]

After reading through the comments on this issue, I think there's some fundamental misunderstanding about the concept of joins.

Specifically the join kind--INNER, LEFT OUTER, etc--is unrelated to predicate(s). You can have an INNER JOIN with any predicate. The predicate need only produce a boolean value. It doesn't even have to include all the tables in the join.

Pandas has an implicit predicate for all of its join APIs: ==. This is probably the most common kind of join predicate.

It would be nice to clarify what kind of join behaviour the statement above can be expected.

For any composable system, the number of specific scenarios that can be constructed will always vastly outnumber the number of scenarios that are reasonable to document.

We can definitely improve the documentation, but documenting all possible joins from a conceptual standpoint is out of scope for the project.

[jy-andy-hu]

I have no problem admitting I could be wrong and I perhaps did ask for too much. However, I am concerned about the documentation and its impact for new ibis users and beginning programmers/analysts who come across this join function. To wrap up my case, I think tidyverse team, especially those work for documentation of join_by() (https://dplyr.tidyverse.org/reference/join_by.html) deserve a round of applause on explaining the intent of their design. I hope ibis join documentation can reach same parity. I really appreciate the ambition behind ibis and I like where ibis is going. I just want to see it work and more friendly to work with. For those of us who have the privilege of understanding how things work should keep those who don't yet in mind.

…

________________________________ From: Phillip Cloud ***@***.***> Sent: Monday, June 23, 2025 08:11 To: ibis-project/ibis ***@***.***> Cc: Andy Hu ***@***.***>; Comment ***@***.***> Subject: Re: [ibis-project/ibis] docs: Join types (Issue #11364) [https://avatars.githubusercontent.com/u/417981?s=20&v=4]cpcloud left a comment (ibis-project/ibis#11364)<#11364 (comment)> After reading through the comments on this issue, I think there's some fundamental misunderstanding about the concept of joins. Specifically the join kind--INNER, LEFT OUTER, etc--is unrelated to predicate(s). You can have an INNER JOIN with any predicate. The predicate need only produce a boolean value. It doesn't even have to include all the tables in the join. Pandas has an implicit predicate for all of its join APIs: ==. This is probably the most common kind of join predicate. It would be nice to clarify what kind of join behaviour the statement above can be expected. For any composable system, the number of specific scenarios that can be constructed will always vastly outnumber the number of scenarios that are reasonable to document. We can definitely improve the documentation, but documenting all possible joins from a conceptual standpoint is out of scope for the project. — Reply to this email directly, view it on GitHub<#11364 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ATYSIVUHLXL5VUIXMTTWEPT3E7VH5AVCNFSM6AAAAAB7XQDA2KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSOJWGI2TEMZUGQ>. You are receiving this because you commented.

[cpcloud]

@jy-andy-hu For sure, tidyverse docs are outstanding!

I don't think you're asking for too much. We are definitely interested in making the documentation as friendly as possible.