Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix read column Info from table map event #440

Conversation

sean-k1
Copy link
Collaborator

@sean-k1 sean-k1 commented Aug 24, 2023

fix: #439

Here is how the scenario Goes:

  1. There is a time difference between the event and the actual state of the database.
  2. The actual database is dropped.
  3. The python-replication-mysql application receives a table map event.
  4. Since there is no cached result for the table_id, it tries to retrieve the database's column schema.
  5. There are no column schema results from the database.
  6. Packets containing information about the table map event's columns are still present.
  7. The application exits the if statement, and the packets remain untouched.
  8. Incorrect packets are inserted into the null_bitmask.
        if self.table_id in table_map:
            self.column_schemas = table_map[self.table_id].column_schemas
        else:
            self.column_schemas = self._ctl_connection._get_table_information(self.schema, self.table)

        ordinal_pos_loc = 0

        if len(self.column_schemas) != 0:
            # Read columns meta data
            column_types = bytearray(self.packet.read(self.column_count))
            
        self.null_bitmask = self.packet.read((self.column_count + 7) / 8) #error packet goes here!

timing of event creation and the timing of querying the table structure are different,
ex) drop table but replication did not reach that point 
f self.column_schemas doesn't yield any results, the code will exit the if statement. However, since there is more column information in the remaining packets, if the column.count is not 0, we still need to read the packets.
@sean-k1 sean-k1 changed the title fix bug fix bu read column Info from table map event Aug 24, 2023
@sean-k1 sean-k1 marked this pull request as draft August 24, 2023 07:09
@sean-k1 sean-k1 marked this pull request as ready for review August 24, 2023 07:19
@sean-k1 sean-k1 marked this pull request as draft August 24, 2023 07:20
@sean-k1 sean-k1 marked this pull request as ready for review August 24, 2023 07:22
@sean-k1 sean-k1 changed the title fix bu read column Info from table map event fix read column Info from table map event Aug 24, 2023
@julien-duponchelle julien-duponchelle merged commit c98358e into julien-duponchelle:main Aug 26, 2023
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug Wrong Packet goes null_bitmask from table map event
2 participants