Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for schema inference for hdfsCluster [2] #35812

Conversation

nikitamikhaylov
Copy link
Member

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Added support for schema inference for hdfsCluster.

Information about CI checks: https://clickhouse.tech/docs/en/development/continuous-integration/

@robot-ch-test-poll1 robot-ch-test-poll1 added the pr-improvement Pull request with some product improvements label Mar 31, 2022
@qoega
Copy link
Member

qoega commented Apr 29, 2022

@Mergifyio update

@mergify
Copy link
Contributor

mergify bot commented Apr 29, 2022

update

❌ Base branch update has failed

merge conflict between base and head
err-code: DBEF1

@ClickHouse ClickHouse deleted a comment from Dstiers11091 May 5, 2022
@nikitamikhaylov nikitamikhaylov force-pushed the revert-35802-revert-35602-schema-inference-hdfs-cluster branch from f4b42de to e611b2a Compare May 17, 2022 13:22
@nikitamikhaylov
Copy link
Member Author

@Mergifyio update

@mergify
Copy link
Contributor

mergify bot commented Jun 1, 2022

update

✅ Branch has been successfully updated

@nikitamikhaylov nikitamikhaylov merged commit fe47735 into master Jun 3, 2022
@nikitamikhaylov nikitamikhaylov deleted the revert-35802-revert-35602-schema-inference-hdfs-cluster branch June 3, 2022 13:54
@tavplubix
Copy link
Member

Hadoop prints trash to tests log:

2022-06-27 05:21:29 00345_index_accurate_comparison:                                        [ OK ] 0.49 sec.
2022-06-27 05:21:30 00338_replicate_array_of_strings:                                       [ OK ] 0.40 sec.
2022-06-27 05:21:30,369 INFO BlockStateChange: BLOCK* processReport 0x84a5eff10894cd08: from storage DS-834c86df-9f25-4de0-8ec5-8423664e7325 node DatanodeRegistration(127.0.0.1:46163, datanodeUuid=016cf04d-2f4d-42ca-bab3-54945fbe4d25, infoPort=32843, infoSecurePort=0, ipcPort=46049, storageInfo=lv=-57;cid=testClusterID;nsid=575950195;c=1656328155273), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0
2022-06-27 05:21:30,369 INFO BlockStateChange: BLOCK* processReport 0x84a5eff10894cd08: from storage DS-13c69541-a384-4aaf-a268-c07b7ad093c4 node DatanodeRegistration(127.0.0.1:46163, datanodeUuid=016cf04d-2f4d-42ca-bab3-54945fbe4d25, infoPort=32843, infoSecurePort=0, ipcPort=46049, storageInfo=lv=-57;cid=testClusterID;nsid=575950195;c=1656328155273), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0
2022-06-27 05:21:30,370 INFO datanode.DataNode: Successfully sent block report 0x84a5eff10894cd08 to namenode: localhost/127.0.0.1:12222,  containing 2 storage report(s), of which we sent 2. The reports had 0 total blocks and used 1 RPC(s). This took 0 msecs to generate and 2 msecs for RPC and NN processing. Got back one command: FinalizeCommand/5.
2022-06-27 05:21:30,370 INFO datanode.DataNode: Got finalize command for block pool BP-1330783287-172.17.0.2-1656328155273
2022-06-27 05:21:30 00331_final_and_prewhere:                                               [ OK ] 0.48 sec.
2022-06-27 05:21:30 00339_parsing_bad_arrays:                                               [ OK ] 1.06 sec.
2022-06-27 05:21:31 00320_between:                                                          [ OK ] 0.32 sec.
2022-06-27 05:21:31 00411_long_accurate_number_comparison_int4:                             [ OK ] 11.45 sec.

https://s3.amazonaws.com/clickhouse-test-reports/38344/da52780cb5a76377d3d6eb52c93752a8c946fe14/stateless_tests__thread__actions__[3/3]/runlog.log

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-improvement Pull request with some product improvements
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants