Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[YSQL] Support generated columns #5805

Closed
ddorian opened this issue Sep 24, 2020 · 1 comment
Closed

[YSQL] Support generated columns #5805

ddorian opened this issue Sep 24, 2020 · 1 comment
Assignees
@tedyu
Labels
area/ysql Yugabyte SQL (YSQL) kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue

Comments

@ddorian
Copy link
Contributor

ddorian commented Sep 24, 2020 •
edited by yugabyte-ci
Loading

Jira Link: DB-1559
Generated columns are available in PostgreSQL 12: https://www.postgresql.org/docs/12/ddl-generated-columns.html

From slack user: https://yugabyte-db.slack.com/archives/CG0KQF0GG/p1599780766218200
@ddorian ddorian added the area/ysql Yugabyte SQL (YSQL) label Sep 24, 2020
tedyu added a commit that referenced this issue Oct 14, 2020
@tedyu
[#5805] [YSQL] Create an RTE field to record the query's lock mode fo…
a2e4470 
…r each relation

Summary:
Postgres Commit was fdba460a26af919c0b366755d119f384706e670a

This change is needed for subsequent backports (specifically commit b60c39759908bb2a2dbcfc108ec19bdbdcc278e8) for generated columns feature.

Test Plan: Build yugabyte db and run tests via Jenkins

Reviewers: mihnea, jason

Reviewed By: jason

Subscribers: yql

Differential Revision: https://phabricator.dev.yugabyte.com/D9605
tedyu added a commit that referenced this issue Oct 15, 2020
@tedyu
[#5805] [YSQL] Change rewriter/planner/executor/plancache to depend o…
a994c18 
…n RTE rellockmode

Summary: Original Postgres commit was 6e35939febf83069430fedda6f89ab1fbe0f9e10

Test Plan: Build yugabyte db and run tests via Jenkins

Reviewers: mihnea, jason

Reviewed By: jason

Subscribers: yql

Differential Revision: https://phabricator.dev.yugabyte.com/D9634
tedyu added a commit that referenced this issue Oct 15, 2020
@tedyu
Revert "[#5805] [YSQL] Change rewriter/planner/executor/plancache to …
fa91b2a 
…depend on RTE rellockmode"

Summary:
This reverts commit a994c18 which builds on top of commit a2e4470 which
 introduced a backwards-compatibility issue.

Test Plan:
Existing tests.
Manual upgrade from a version before a2e4470.

Reviewers: dmitry, jhe, mihnea

Reviewed By: mihnea

Subscribers: yql

Differential Revision: https://phabricator.dev.yugabyte.com/D9643
tedyu added a commit that referenced this issue Oct 15, 2020
@tedyu
Revert "[#5805] [YSQL] Create an RTE field to record the query's lock…
97f9a09 
… mode for each relation"

Summary: This reverts commit a2e4470 which causes backward compatibility issue.

Test Plan:
Existing tests.
Manual upgrade from a version before a2e4470.

Reviewers: dmitry, mihnea

Reviewed By: mihnea

Subscribers: yql

Differential Revision: https://phabricator.dev.yugabyte.com/D9644
tedyu added a commit that referenced this issue Nov 9, 2020
@tedyu
[#5805] [YSQL] Error position support for defaults and check constraints
92c4f0f 
Summary:
Original Postgres commit was a4a232b1e70229a6ba0e592f6775c019bb171d9a

Generated column backport depends on this.

Commit message was:

    Add support for error position reporting for the expressions contained
    in defaults and check constraint definitions.  This currently works only
    for CREATE TABLE, not ALTER TABLE, because the latter is not set up to
    pass around the original query string.

    Reviewed-by: Fabien COELHO <coelho@cri.ensmp.fr>

Test Plan: Build yugabyte db and run tests via Jenkins

Reviewers: mihnea, dmitry

Reviewed By: dmitry

Subscribers: yql

Differential Revision: https://phabricator.dev.yugabyte.com/D9650
tedyu added a commit that referenced this issue Nov 9, 2020
@tedyu
[#5805] [YSQL] Split ExecStoreTuple into ExecStoreHeapTuple and ExecS…
0558b74 
…toreBufferHeapTuple

Summary:
 Original Postgres commit was 29c94e03c7d05d2b29afa1de32795ce178531246

Commit message was:
    Upcoming changes introduce further types of tuple table slots, in
    preparation of making table storage pluggable. New storage methods
    will have different representation of tuples, therefore the slot
    accessor should refer explicitly to heap tuples.

    Instead of just renaming the functions, split it into one function
    that accepts heap tuples not residing in buffers, and one accepting
    ones in buffers.  Previously one function was used for both, but that
    was a bit awkward already, and splitting will allow us to represent
    slot types for tuples in buffers and normal memory separately.

    This is split out from the patch introducing abstract slots, as this
    largely consists out of mechanical changes.

    Author: Ashutosh Bapat
    Reviewed-By: Andres Freund
    Discussion: https://postgr.es/m/20180220224318.gw4oe5jadhpmcdnm@alap3.anarazel.de

Test Plan: Build yugabyte db and run tests via Jenkins

Reviewers: mihnea, dmitry

Reviewed By: dmitry

Subscribers: yql

Differential Revision: https://phabricator.dev.yugabyte.com/D9655
tedyu added a commit that referenced this issue Nov 9, 2020
@tedyu
[#5805] [YSQL] Fix run-time partition pruning for appends with multip…
570d236 
…le source rels

Summary:
Original Postgres commit was 1c2cb2744bf3d8ad751cd5cf3b347f10f48492b3

The majority of the commit is already in YB DB (from past sync), except for the new field in MergeAppend.

Commit message was:

    The previous coding here supposed that if run-time partitioning applied to
    a particular Append/MergeAppend plan, then all child plans of that node
    must be members of a single partitioning hierarchy.  This is totally wrong,
    since an Append could be formed from a UNION ALL: we could have multiple
    hierarchies sharing the same Append, or child plans that aren't part of any
    hierarchy.

    To fix, restructure the related plan-time and execution-time data
    structures so that we can have a separate list or array for each
    partitioning hierarchy.  Also track subplans that are not part of any
    hierarchy, and make sure they don't get pruned.

    Per reports from Phil Florent and others.  Back-patch to v11, since
    the bug originated there.

    David Rowley, with a lot of cosmetic adjustments by me; thanks also
    to Amit Langote for review.

    Discussion: https://postgr.es/m/HE1PR03MB17068BB27404C90B5B788BCABA7B0@HE1PR03MB1706.eurprd03.prod.outlook.com

Test Plan: Build yugabyte db and run tests via Jenkins

Reviewers: mihnea, dmitry

Reviewed By: dmitry

Subscribers: yql

Differential Revision: https://phabricator.dev.yugabyte.com/D9676
tedyu added a commit that referenced this issue Nov 11, 2020
@tedyu
[#5805] [YSQL] Don't require return slots for nodes without projection
fbf7b6f 
Summary:
Original Postgres commit was 1ef6bd2954c4ec63ff8a2c9c4ebc38251d7ef5c5

Generated column feature depends on this.

Commit message was:

    In a lot of nodes the return slot is not required. That can either be
    because the node doesn't do any projection (say an Append node), or
    because the node does perform projections but the projection is
    optimized away because the projection would yield an identical row.

    Slots aren't that small, especially for wide rows, so it's worthwhile
    to avoid creating them.  It's not possible to just skip creating the
    slot - it's currently used to determine the tuple descriptor returned
    by ExecGetResultType().  So separate the determination of the result
    type from the slot creation.  The work previously done internally
    ExecInitResultTupleSlotTL() can now also be done separately with
    ExecInitResultTypeTL() and ExecInitResultSlot().  That way nodes that
    aren't guaranteed to need a result slot, can use
    ExecInitResultTypeTL() to determine the result type of the node, and
    ExecAssignScanProjectionInfo() (via
    ExecConditionalAssignProjectionInfo()) determines that a result slot
    is needed, it is created with ExecInitResultSlot().

    Besides the advantage of avoiding to create slots that then are
    unused, this is necessary preparation for later patches around tuple
    table slot abstraction. In particular separating the return descriptor
    and slot is a prerequisite to allow JITing of tuple deforming with
    knowledge of the underlying tuple format, and to avoid unnecessarily
    creating JITed tuple deforming for virtual slots.

    This commit removes a redundant argument from
    ExecInitResultTupleSlotTL(). While this commit touches a lot of the
    relevant lines anyway, it'd normally still not worthwhile to cause
    breakage, except that aforementioned later commits will touch *all*
    ExecInitResultTupleSlotTL() callers anyway (but fits worse
    thematically).

    Author: Andres Freund
    Discussion: https://postgr.es/m/20181105210039.hh4vvi4vwoq5ba2q@alap3.anarazel.de

Test Plan: Build yugabyte db and run tests via Jenkins

Reviewers: mihnea, dmitry

Reviewed By: dmitry

Subscribers: yql

Differential Revision: https://phabricator.dev.yugabyte.com/D9652
tedyu added a commit that referenced this issue Dec 2, 2020
@tedyu
[#5805] [YSQL] Use slots more widely in tuple mapping code and make n…
7aff483 
…aming more consistent

Summary:
Original commit was cc2905e963e950d01cd2cb6c860638ce9506c63d

Commit message was:

    It's inefficient to use a single slot for mapping between tuple
    descriptors for multiple tuples, as previously done when using
    ConvertPartitionTupleSlot(), as that means the slot's tuple descriptors
    change for every tuple.

    Previously we also, via ConvertPartitionTupleSlot(), built new tuples
    after the mapping even in cases where we, immediately afterwards,
    access individual columns again.

    Refactor the code so one slot, on demand, is used for each
    partition. That avoids having to change the descriptor (and allows to
    use the more efficient "fixed" tuple slots). Then use slot->slot
    mapping, to avoid unnecessarily forming a tuple.

    As the naming between the tuple and slot mapping functions wasn't
    consistent, rename them to execute_attr_map_{tuple,slot}.  It's likely
    that we'll also rename convert_tuples_by_* to denote that these
    functions "only" build a map, but that's left for later.

    Author: Amit Khandekar and Amit Langote, editorialized by me
    Reviewed-By: Amit Langote, Amit Khandekar, Andres Freund
    Discussion:
        https://postgr.es/m/CAJ3gD9fR0wRNeAE8VqffNTyONS_UfFPRpqxhnD9Q42vZB+Jvpg@mail.gmail.com
        https://postgr.es/m/e4f9d743-cd4b-efb0-7574-da21d86a7f36%40lab.ntt.co.jp

Test Plan: Build yugabyte db and run tests via Jenkins

Reviewers: mihnea, dmitry

Reviewed By: dmitry

Subscribers: yql

Differential Revision: https://phabricator.dev.yugabyte.com/D9883
@yugabyte-ci yugabyte-ci added kind/bug This issue is a bug priority/medium Medium priority issue labels Jun 8, 2022
@yugabyte-ci yugabyte-ci added kind/enhancement This is an enhancement of an existing feature and removed kind/bug This issue is a bug labels Aug 24, 2022
@sushantrmishra
Copy link

This support will come as part of upgrade to Postgres 13.
Issue tracking that task #9797

@rahulb-yb rahulb-yb mentioned this issue Sep 14, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tedyu tedyu

Labels
area/ysql Yugabyte SQL (YSQL) kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue
Projects
YQL-beta
Status: Done
Milestone
No milestone
Development

No branches or pull requests
4 participants
@tedyu @ddorian @sushantrmishra @yugabyte-ci