求助！编译问题：object thrift is not a member of package org.apache.doris

[hectorhe001]

按照官网教程，Doris 源码已经编译过，但是在编译 spark-doris-connector 还是报这个问题，帮忙看看，感谢!

[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/rdd/ScalaValueReader.scala:36: error: object thrift is not a member of package org.apache.doris
[ERROR] import org.apache.doris.thrift.{TScanCloseParams, TScanNextBatchParams, TScanOpenParams, TScanOpenResult}
[ERROR] ^
[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/rdd/ScalaValueReader.scala:79: error: not found: type TScanOpenParams
[ERROR] private val openParams: TScanOpenParams = {
[ERROR] ^
[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/rdd/ScalaValueReader.scala:80: error: not found: type TScanOpenParams
[ERROR] val params = new TScanOpenParams
[ERROR] ^
[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/rdd/ScalaValueReader.scala:130: error: not found: type TScanOpenResult
[ERROR] protected val openResult: TScanOpenResult = lockClient(_.openScanner(openParams))
[ERROR] ^
[ERROR] /build/extension/spark-doris-connector/src/main/java/org/apache/doris/spark/backend/BackendClient.java:33: error: object thrift is not a member of package org.apache.doris
[INFO] import org.apache.doris.thrift.TScanOpenResult;
[INFO] ^
[ERROR] /build/extension/spark-doris-connector/src/main/java/org/apache/doris/spark/backend/BackendClient.java:121: error: not found: type TScanOpenResult
[INFO] public TScanOpenResult openScanner(TScanOpenParams openParams) throws ConnectedFailedException {
[INFO] ^
[ERROR] /build/extension/spark-doris-connector/src/main/java/org/apache/doris/spark/backend/BackendClient.java:32: error: object thrift is not a member of package org.apache.doris
[INFO] import org.apache.doris.thrift.TScanOpenParams;
[INFO] ^
[ERROR] /build/extension/spark-doris-connector/src/main/java/org/apache/doris/spark/backend/BackendClient.java:121: error: not found: type TScanOpenParams
[INFO] public TScanOpenResult openScanner(TScanOpenParams openParams) throws ConnectedFailedException {
[INFO] ^
[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/sql/SchemaUtils.scala:26: error: object thrift is not a member of package org.apache.doris
[ERROR] import org.apache.doris.thrift.TScanColumnDesc
[ERROR] ^
[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/sql/SchemaUtils.scala:104: error: not found: type TScanColumnDesc
[ERROR] def convertToSchema(tscanColumnDescs: Seq[TScanColumnDesc]): Schema = {
[ERROR] ^
[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/rdd/ScalaValueReader.scala:137: error: not found: type TScanNextBatchParams
[ERROR] val nextBatchParams = new TScanNextBatchParams
[ERROR] ^
[ERROR] /build/extension/spark-doris-connector/src/main/java/org/apache/doris/spark/backend/BackendClient.java:28: error: object thrift is not a member of package org.apache.doris
[INFO] import org.apache.doris.thrift.TScanBatchResult;
[INFO] ^
[ERROR] /build/extension/spark-doris-connector/src/main/java/org/apache/doris/spark/backend/BackendClient.java:156: error: not found: type TScanBatchResult
[INFO] public TScanBatchResult getNext(TScanNextBatchParams nextBatchParams) throws DorisException {
[INFO] ^
[ERROR] /build/extension/spark-doris-connector/src/main/java/org/apache/doris/spark/backend/BackendClient.java:31: error: object thrift is not a member of package org.apache.doris
[INFO] import org.apache.doris.thrift.TScanNextBatchParams;
[INFO] ^
[ERROR] /build/extension/spark-doris-connector/src/main/java/org/apache/doris/spark/backend/BackendClient.java:156: error: not found: type TScanNextBatchParams
[INFO] public TScanBatchResult getNext(TScanNextBatchParams nextBatchParams) throws DorisException {
[INFO] ^
[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/rdd/ScalaValueReader.scala:142: error: value isEos is not a member of Nothing
[ERROR] eos.set(nextResult.isEos)
[ERROR] ^
[ERROR] /build/extension/spark-doris-connector/src/main/java/org/apache/doris/spark/serialization/RowBatch.java:45: error: object thrift is not a member of package org.apache.doris
[INFO] import org.apache.doris.thrift.TScanBatchResult;
[INFO] ^
[ERROR] /build/extension/spark-doris-connector/src/main/java/org/apache/doris/spark/serialization/RowBatch.java:84: error: not found: type TScanBatchResult
[INFO] public RowBatch(TScanBatchResult nextResult, Schema schema) throws DorisException {
[INFO] ^
[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/rdd/ScalaValueReader.scala:196: error: not found: type TScanNextBatchParams
[ERROR] val nextBatchParams = new TScanNextBatchParams
[ERROR] ^
[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/rdd/ScalaValueReader.scala:200: error: value isEos is not a member of Nothing
[ERROR] eos.set(nextResult.isEos)
[ERROR] ^
[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/rdd/ScalaValueReader.scala:223: error: not found: type TScanCloseParams
[ERROR] val closeParams = new TScanCloseParams
[ERROR] ^
[ERROR] /build/extension/spark-doris-connector/src/main/java/org/apache/doris/spark/backend/BackendClient.java:29: error: object thrift is not a member of package org.apache.doris
[INFO] import org.apache.doris.thrift.TScanCloseParams;
[INFO] ^
[ERROR] /build/extension/spark-doris-connector/src/main/java/org/apache/doris/spark/backend/BackendClient.java:196: error: not found: type TScanCloseParams
[INFO] public void closeScanner(TScanCloseParams closeParams) {
[INFO] ^
[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/sql/DorisSourceProvider.scala:25: error: object v2 is not a member of package org.apache.spark.sql.sources
[ERROR] import org.apache.spark.sql.sources.v2.writer.streaming.StreamWriter
[ERROR] ^
[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/sql/DorisSourceProvider.scala:26: error: object v2 is not a member of package org.apache.spark.sql.sources
[ERROR] import org.apache.spark.sql.sources.v2.{DataSourceOptions, StreamWriteSupport}
[ERROR] ^
[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/sql/DorisSourceProvider.scala:38: error: not found: type StreamWriteSupport
[ERROR] private[sql] class DorisSourceProvider extends DataSourceRegister with RelationProvider with CreatableRelationProvider with StreamWriteSupport with Logging {
[ERROR] ^
[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/sql/DorisSourceProvider.scala:127: error: not found: type StreamWriter
[ERROR] override def createStreamWriter(queryId: String, structType: StructType, outputMode: OutputMode, dataSourceOptions: DataSourceOptions): StreamWriter = {
[ERROR] ^
[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/sql/DorisSourceProvider.scala:127: error: not found: type DataSourceOptions
[ERROR] override def createStreamWriter(queryId: String, structType: StructType, outputMode: OutputMode, dataSourceOptions: DataSourceOptions): StreamWriter = {
[ERROR] ^
[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/sql/DorisStreamWriter.scala:23: error: object v2 is not a member of package org.apache.spark.sql.sources
[ERROR] import org.apache.spark.sql.sources.v2.writer.streaming.StreamWriter
[ERROR] ^
[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/sql/DorisStreamWriter.scala:35: error: not found: type StreamWriter
[ERROR] class DorisStreamWriter(settings: SparkSettings) extends StreamWriter {
[ERROR] ^
[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/sql/DorisStreamWriter.scala:24: error: object v2 is not a member of package org.apache.spark.sql.sources
[ERROR] import org.apache.spark.sql.sources.v2.writer.{DataWriter, DataWriterFactory, WriterCommitMessage}
[ERROR] ^
[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/sql/DorisStreamWriter.scala:50: error: not found: type DataWriterFactory
[ERROR] case class DorisStreamWriterFactory(settings: SparkSettings) extends DataWriterFactory[Row] {
[ERROR] ^
[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/sql/DorisStreamWriter.scala:38: error: not found: type WriterCommitMessage
[ERROR] override def commit(l: Long, writerCommitMessages: Array[WriterCommitMessage]): Unit = {}
[ERROR] ^
[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/sql/DorisStreamWriter.scala:40: error: not found: type WriterCommitMessage
[ERROR] override def abort(l: Long, writerCommitMessages: Array[WriterCommitMessage]): Unit = {}
[ERROR] ^
[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/sql/DorisStreamWriter.scala:51: error: not found: type DataWriter
[ERROR] override def createDataWriter(partitionId: Int, attemptNumber: Int): DataWriter[Row] = {
[ERROR] ^
[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/sql/DorisStreamWriter.scala:68: error: not found: type DataWriter
[ERROR] class DorisStreamDataWriter(settings: SparkSettings) extends DataWriter[Row] {
[ERROR] ^
[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/sql/DorisStreamWriter.scala:60: error: not found: type WriterCommitMessage
[ERROR] case object DorisWriterCommitMessage extends WriterCommitMessage
[ERROR] ^
[ERROR] /build/extension/spark-doris-connector/src/main/scala/org/apache/doris/spark/sql/DorisStreamWriter.scala:89: error: not found: type WriterCommitMessage
[ERROR] override def commit(): WriterCommitMessage = {
[ERROR] ^
[ERROR] 38 errors found
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 13.054 s
[INFO] Finished at: 2021-10-26T11:49:11Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.1:compile (default-cli) on project doris-spark: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

[hf200012]

version?

[hectorhe001]

感谢！编译环境是 apache/incubator-doris:build-env-1.2，doris 版本: 0.14.0

[hectorhe001]

还有编译 doris 的时候，报错如下，按照官方的教程，是在这个环境下编译的：apache/incubator-doris:build-env-1.2，但是报错了，然后如教程所说，编译 spark-doris-connector 的时候，会报 thrift: command not found ，尝试了很多次，希望官方能提供帮助，感谢！
-====================
[ 99%] Building CXX object src/tools/CMakeFiles/meta_tool.dir/meta_tool.cpp.o
[ 99%] Building CXX object src/service/CMakeFiles/palo_be.dir/doris_main.cpp.o
[ 99%] Linking CXX executable meta_tool
[100%] Linking CXX executable palo_be
collect2: fatal error: ld terminated with signal 9 [Killed]
compilation terminated.
make[2]: *** [src/tools/meta_tool] Error 1
make[2]: *** Deleting file `src/tools/meta_tool'
make[1]: *** [src/tools/CMakeFiles/meta_tool.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

[100%] Built target palo_be
make: *** [all] Error 2

[hf200012]

First, you must compile doris completely, execute sh build.sh in the doris directory, and then execute sh build.sh after entering the Spark Doris Connector. It is recommended that you pull the latest code. Now Spark Connector supports writing and supports Spark 2.x, 3. x version, refer to the official website for the specific compilation method hectorhe001 ***@***.***> 于2021年10月27日周三 下午5:09写道：

…

还有编译 doris 的时候，报错如下，按照官方的教程，是在这个环境下编译的：apache/incubator-doris:build-env-1.2，但是报错了，然后如教程所说，编译 spark-doris-connector 的时候，会报 thrift: command not found ，尝试了很多次，希望官方能提供帮助，感谢！ -==================== [ 99%] Building CXX object src/tools/CMakeFiles/meta_tool.dir/meta_tool.cpp.o [ 99%] Building CXX object src/service/CMakeFiles/palo_be.dir/doris_main.cpp.o [ 99%] Linking CXX executable meta_tool [100%] Linking CXX executable palo_be collect2: fatal error: ld terminated with signal 9 [Killed] compilation terminated. make[2]: *** [src/tools/meta_tool] Error 1 make[2]: *** Deleting file `src/tools/meta_tool' make[1]: *** [src/tools/CMakeFiles/meta_tool.dir/all] Error 2 make[1]: *** Waiting for unfinished jobs.... [100%] Built target palo_be make: *** [all] Error 2 — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#6941 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGIITOTPXC32IVH3LRVLHDUI66TZANCNFSM5GXWJRXQ> .

-- 张家峰

[hectorhe001]

OK，我试一下，感谢！