flink-cdc在全量阶段使用了SELECT * FROM TABLE读取数据。 #3058
Unanswered
banyue0618
asked this question in
Q&A
Replies: 1 comment
-
看源码Debezium读取时候使用select *, column.include.list参数只是在生成change event时候使用。 |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
flink-cdc读取数据第一阶段,快照读,采用了select * 读取数据。
![image](https://private-user-images.githubusercontent.com/40993873/302156266-b412bef3-9d1c-481f-90b9-a04e60f4cea1.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTkxNzUwNDQsIm5iZiI6MTcxOTE3NDc0NCwicGF0aCI6Ii80MDk5Mzg3My8zMDIxNTYyNjYtYjQxMmJlZjMtOWQxYy00ODFmLTkwYjktYTA0ZTYwZjRjZWExLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA2MjMlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNjIzVDIwMzIyNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTE5NmMwMjEzNTFkYTA4MjkwODY5NjhmN2QwYzAyMGExZWIzMDNiMjg3YWE0OTJmMmMyZjJhZDViMTVlYThkNmImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.wC9SQ8C7ZCTyMHeckVL4FMIxMxUTBS7N_Mln0bqn3J0)
![image](https://private-user-images.githubusercontent.com/40993873/302152971-5827e8ac-7fb4-47c6-9fa2-bb7613240ff6.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTkxNzUwNDQsIm5iZiI6MTcxOTE3NDc0NCwicGF0aCI6Ii80MDk5Mzg3My8zMDIxNTI5NzEtNTgyN2U4YWMtN2ZiNC00N2M2LTlmYTItYmI3NjEzMjQwZmY2LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA2MjMlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNjIzVDIwMzIyNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWYyNGRlMjc3YzA4ZTBiOWZlNjY0Y2RlYjk4ODE2NjMwY2VkMTQyOWI3OTM0N2Q0ZTJjOTI5M2U1ODZiYTM0ZjUmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.zYMqGXt161nLHgIcH0iujFOl3-qrEjmmGxYhqoSSOkk)
![image](https://private-user-images.githubusercontent.com/40993873/302152574-eae2db88-2d5a-4751-a4ef-ce3ded85907c.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTkxNzUwNDQsIm5iZiI6MTcxOTE3NDc0NCwicGF0aCI6Ii80MDk5Mzg3My8zMDIxNTI1NzQtZWFlMmRiODgtMmQ1YS00NzUxLWE0ZWYtY2UzZGVkODU5MDdjLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA2MjMlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNjIzVDIwMzIyNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTdhNzExNmE4MmFmY2U2ZGRjMmRhNWI5OThlNTY1YTUzMTllMjdjZmZkZTJkNjU0OWZhN2IzMjVkMmZmZWYyMzgmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.aZrhmb9qTlojT6EXn4arFolpOu0oWaCeqiYDBPLXwzg)
请问下应该怎么配置,只读取我需要的字段呢。因为:我的一张表有两百个字段,但是实际上我只需要其中的五个字段,完全没有必要采用星号。
看flink-cdc的源码。直接代码中写死了星号。
目前已经进行的尝试是配置debezium属性,
debezium.column.include.list
,但是根据数据库查询日志,还是使用的select *然后我查看debezium的源码,在构建projection的时候校验了
isColumnsFiltered
,根据这个属性来决定select的字段信息为什么flink-cdc没有这样做呢。
感谢您的帮助。
Beta Was this translation helpful? Give feedback.
All reactions