Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARM平台make error #5

Closed
Hoxxx opened this issue Oct 21, 2021 · 1 comment
Closed

ARM平台make error #5

Hoxxx opened this issue Oct 21, 2021 · 1 comment

Comments

@Hoxxx
Copy link

Hoxxx commented Oct 21, 2021

平台:Kylin Linux Advanced Server release V10 (Tercel)
gcc:gcc (GCC) 7.3.0
报错信息:

[ 56%] Building CXX object storage/xengine/CMakeFiles/xengine_se.dir/core/memtable/art.cc.o
In file included from /data/GalaxyDB/galaxyengine/storage/xengine/core/memtable/art.h:17:0,
                 from /data/GalaxyDB/galaxyengine/storage/xengine/core/memtable/art.cc:15:
/data/GalaxyDB/galaxyengine/storage/xengine/core/memtable/art_node.h:28:10: fatal error: x86intrin.h: No such file or directory
 #include <x86intrin.h>
          ^~~~~~~~~~~~~
compilation terminated.
make[2]: *** [storage/xengine/CMakeFiles/xengine_se.dir/build.make:1194: storage/xengine/CMakeFiles/xengine_se.dir/core/memtable/art.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:3836: storage/xengine/CMakeFiles/xengine_se.dir/all] Error 2

查了下,应该是arm平台上gcc里不包含<x86intrin.h>这个头文件(包括下面的<emmintrin.h>,<immintrin.h>),尝试改为clang,报错信息更多

@vegetabledogg
Copy link
Collaborator

We have fixed this problem and reopen it if any problem again.

xiewajueji pushed a commit that referenced this issue May 5, 2024
…TH VS 2019 [#5] [noclose]

storage\ndb\src\common\transporter\TransporterRegistry.cpp(410,33): warning C4150: deletion of pointer to incomplete type 'SHM_Transporter'; no destructor called

Change-Id: I8d928d6ce403a9f08d2328bc1e8f934c51892c1e
xiewajueji pushed a commit that referenced this issue May 5, 2024
…ARRAY.H:72

Bug #31978439: ASSERTION FAILURE IN BOUNDS_CHECKED_ARRAY::OPERATOR[] AT SQL/SQL_ARRAY.H

The field list could come into an inconsistent state after temporary
tables were created, and it would cause out-of-bounds indexing later.
The chain of events is fairly complicated, and is best illustrated with
a query:

  SELECT * FROM (
    SELECT PI()
    FROM t1 AS table1, t1 AS table2
    ORDER BY PI(), table1.a
  ) AS d1;

1. The outer SELECT list gets an Item_view_ref to PI(), as a visible item.
2. The derived table is merged.
   2.1. ORDER BY is pulled up into the outer query block (this is a MySQL
        extension).
3. ORDER BY is setup for the outer query block.
   3.1. setup_order() looks for PI() in the SELECT list, doesn't find it
        (since it is hidden behind the Item_view_ref), adds PI() as a
        hidden item to make sure it is pulled through temporary tables.
4. A temporary table is created due to join + ORDER BY together.
   4.1. Item_view_ref is peeled away when asking for temporary field.
        The PI() now wants to exist in the field list both as hidden
        and non-hidden, which is not possible since there's only one
        hidden flag per Item.
5. Since we now have a field list where not all hidden items are before all
   visible items, the indexing into the slice arrays breaks when trying to
   fix up ROLLUP items at the very end, and we go out of bounds.

We can't remove the Item_view_ref when merging the derived table, as it
contains metadata information we'd like to keep. So instead, we change
setup_order() to look for such items wrapped into Item_view_ref as a
last resort, so that it doesn't add them twice.

Step #5 (added when we rewrote the streaming aggregation) is what actually
uncovers the bug, but this has been wrong since the field list was
changed from being a linked list, moving the hidden flag into the Item.

Change-Id: I518520d05402464da71a0d062d6fdb0ad6fd4b65
xiewajueji pushed a commit that referenced this issue May 5, 2024
…ARRAY.H:72

Bug #31978439: ASSERTION FAILURE IN BOUNDS_CHECKED_ARRAY::OPERATOR[] AT SQL/SQL_ARRAY.H

The field list could come into an inconsistent state after temporary
tables were created, and it would cause out-of-bounds indexing later.
The chain of events is fairly complicated, and is best illustrated with
a query:

  SELECT * FROM (
    SELECT PI()
    FROM t1 AS table1, t1 AS table2
    ORDER BY PI(), table1.a
  ) AS d1;

1. The outer SELECT list gets an Item_view_ref to PI(), as a visible item.
2. The derived table is merged.
   2.1. ORDER BY is pulled up into the outer query block (this is a MySQL
        extension).
3. ORDER BY is setup for the outer query block.
   3.1. setup_order() looks for PI() in the SELECT list, doesn't find it
        (since it is hidden behind the Item_view_ref), adds PI() as a
        hidden item to make sure it is pulled through temporary tables.
4. A temporary table is created due to join + ORDER BY together.
   4.1. Item_view_ref is peeled away when asking for temporary field.
        The PI() now wants to exist in the field list both as hidden
        and non-hidden, which is not possible since there's only one
        hidden flag per Item.
5. Since we now have a field list where not all hidden items are before all
   visible items, the indexing into the slice arrays breaks when trying to
   fix up ROLLUP items at the very end, and we go out of bounds.

We can't remove the Item_view_ref when merging the derived table, as it
contains metadata information we'd like to keep. So instead, we change
setup_order() to look for such items wrapped into Item_view_ref as a
last resort, so that it doesn't add them twice.

Step #5 (added when we rewrote the streaming aggregation) is what actually
uncovers the bug, but this has been wrong since the field list was
changed from being a linked list, moving the hidden flag into the Item.

Change-Id: I518520d05402464da71a0d062d6fdb0ad6fd4b65
xiewajueji pushed a commit that referenced this issue May 5, 2024
Rename "lock_count" to "external_lock_count", increment the variable
only when transaction has been registered sucessfully and thus
remove two unneccessary decrement calls. Improve description of the
"external_lock_count" variable.

Change-Id: I5bfa91fdca8fa2d377b170696b6086293d8cb743
xiewajueji pushed a commit that referenced this issue May 5, 2024
Use DBUG_TRACE in changed functions

Change-Id: Idfd60e26dc6ec45f61cb43fea6dfe0ec21c615e1
xiewajueji pushed a commit that referenced this issue May 5, 2024
-- Patch #1: Persist secondary load information --

Problem:
We need a way of knowing which tables were loaded to HeatWave after
MySQL restarts due to a crash or a planned shutdown.

Solution:
Add a new "secondary_load" flag to the `options` column of mysql.tables.
This flag is toggled after a successful secondary load or unload. The
information about this flag is also reflected in
INFORMATION_SCHEMA.TABLES.CREATE_OPTIONS.

-- Patch #2 --

The second patch in this worklog triggers the table reload from InnoDB
after MySQL restart.

The recovery framework recognizes that the system restarted by checking
whether tables are present in the Global State. If there are no tables
present, the framework will access the Data Dictionary and find which
tables were loaded before the restart.

This patch introduces the "Data Dictionary Worker" - a MySQL service
recovery worker whose task is to query the INFORMATION_SCHEMA.TABLES
table from a separate thread and find all tables whose secondary_load
flag is set to 1.

All tables that were found in the Data Dictionary will be appended to
the list of tables that have to be reloaded by the framework from
InnoDB.

If an error occurs during restart recovery we will not mark the recovery
as failed. This is done because the types of failures that can occur
when the tables are reloaded after a restart are less critical compared
to previously existing recovery situations. Additionally, this code will
soon have to be adapted for the next worklog in this area so we are
proceeding with the simplest solution that makes sense.

A Global Context variable m_globalStateEmpty is added which indicates
whether the Global State should be recovered from an external source.

-- Patch #3 --

This patch adds the "rapid_reload_on_restart" system variable. This
variable is used to control whether tables should be reloaded after a
restart of mysqld or the HeatWave plugin. This variable is persistable
(i.e., SET PERSIST RAPID_RELOAD_ON_RESTART = TRUE/FALSE).

The default value of this variable is set to false.

The variable can be modified in OFF, IDLE, and SUSPENDED states.

-- Patch #4 --

This patch refactors the recovery code by removing all recovery-related
code from ha_rpd.cc and moving it to separate files:

  - ha_rpd_session_factory.h/cc:
  These files contain the MySQLAdminSessionFactory class, which is used
to create admin sessions in separate threads that can be used to issue
SQL queries.

  - ha_rpd_recovery.h/cc:
  These files contain the MySQLServiceRecoveryWorker,
MySQLServiceRecoveryJob and ObjectStoreRecoveryJob classes which were
previously defined in ha_rpd.cc. This file also contains a function that
creates the RecoveryWorkerFactory object. This object is passed to the
constructor of the Recovery Framework and is used to communicate with
the other section of the code located in rpdrecoveryfwk.h/cc.

This patch also renames rpdrecvryfwk to rpdrecoveryfwk for better
readability.

The include relationship between the files is shown on the following
diagram:

        rpdrecoveryfwk.h◄──────────────rpdrecoveryfwk.cc
            ▲    ▲
            │    │
            │    │
            │    └──────────────────────────┐
            │                               │
        ha_rpd_recovery.h◄─────────────ha_rpd_recovery.cc──┐
            ▲                               │           │
            │                               │           │
            │                               │           │
            │                               ▼           │
        ha_rpd.cc───────────────────────►ha_rpd.h       │
                                            ▲           │
                                            │           │
            ┌───────────────────────────────┘           │
            │                                           ▼
    ha_rpd_session_factory.cc──────►ha_rpd_session_factory.h

Other changes:
  - In agreement with Control Plane, the external Global State is now
  invalidated during recovery framework startup if:
    1) Recovery framework recognizes that it should load the Global
    State from an external source AND,
    2) rapid_reload_on_restart is set to OFF.

  - Addressed review comments for Patch #3, rapid_reload_on_restart is
  now also settable while plugin is ON.

  - Provide a single entry point for processing external Global State
  before starting the recovery framework loop.

  - Change when the Data Dictionary is read. Now we will no longer wait
  for the HeatWave nodes to connect before querying the Data Dictionary.
  We will query it when the recovery framework starts, before accepting
  any actions in the recovery loop.

  - Change the reload flow by inserting fake global state entries for
  tables that need to be reloaded instead of manually adding them to a
  list of tables scheduled for reload. This method will be used for the
  next phase where we will recover from Object Storage so both recovery
  methods will now follow the same flow.

  - Update secondary_load_dd_flag added in Patch #1.

  - Increase timeout in wait_for_server_bootup to 300s to account for
  long MySQL version upgrades.

  - Add reload_on_restart and reload_on_restart_dbg tests to the rapid
  suite.

  - Add PLUGIN_VAR_PERSIST_AS_READ_ONLY flag to "rapid_net_orma_port"
  and "rapid_reload_on_restart" definitions, enabling their
  initialization from persisted values along with "rapid_bootstrap" when
  it is persisted as ON.

  - Fix numerous clang-tidy warnings in recovery code.

  - Prevent suspended_basic and secondary_load_dd_flag tests to run on
  ASAN builds due to an existing issue when reinstalling the RAPID
  plugin.

-- Bug#33752387 --

Problem:
A shutdown of MySQL causes a crash in queries fired by DD worker.

Solution:
Prevent MySQL from killing DD worker's queries by instantiating a
DD_kill_immunizer before the queries are fired.

-- Patch #5 --

Problem:
A table can be loaded before the DD Worker queries the Data Dictionary.
This means that table will be wrongly processed as part of the external
global state.

Solution:
If the table is present in the current in-memory global state we will
not consider it as part of the external global state and we will not
process it by the recovery framework.

-- Bug#34197659 --

Problem:
If a table reload after restart causes OOM the cluster will go into
RECOVERYFAILED state.

Solution:
Recognize when the tables are being reloaded after restart and do not
move the cluster into RECOVERYFAILED. In that case only the current
reload will fail and the reload for other tables will be attempted.

Change-Id: Ic0c2a763bc338ea1ae6a7121ff3d55b456271bf0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants