Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hive table read ORC file error with arm64 and enable hive.cache #14963

Open
ElapsedSoul opened this issue Nov 9, 2022 · 4 comments
Open

Hive table read ORC file error with arm64 and enable hive.cache #14963

ElapsedSoul opened this issue Nov 9, 2022 · 4 comments
Labels
bug Something isn't working

Comments

@ElapsedSoul
Copy link

The cluster run on arm servers,
The cluster enable hive.cache,
The trino version is 389,
The data can access successful on x86 cluster without hive.cache.
Next we wait a time to disable hive.cache to test.

error message:

io.trino.spi.TrinoException: Failed to read ORC file: hdfs://xxxxxxxxx/user/hive/warehouse/xxxxx.db/xxxx/xxx=20220815/000037_0
	at io.trino.plugin.hive.orc.OrcPageSource.handleException(OrcPageSource.java:208)
	at io.trino.plugin.hive.orc.OrcPageSourceFactory.lambda$createOrcPageSource$8(OrcPageSourceFactory.java:393)
	at io.trino.orc.OrcBlockFactory$OrcBlockLoader.load(OrcBlockFactory.java:83)
	at io.trino.spi.block.LazyBlock$LazyData.load(LazyBlock.java:400)
	at io.trino.spi.block.LazyBlock$LazyData.getFullyLoadedBlock(LazyBlock.java:379)
	at io.trino.spi.block.LazyBlock.getLoadedBlock(LazyBlock.java:286)
	at io.trino.plugin.hive.orc.OrcPageSource$SourceColumn$MaskingBlockLoader.load(OrcPageSource.java:379)
	at io.trino.spi.block.LazyBlock$LazyData.load(LazyBlock.java:400)
	at io.trino.spi.block.LazyBlock$LazyData.getFullyLoadedBlock(LazyBlock.java:379)
	at io.trino.spi.block.LazyBlock.getLoadedBlock(LazyBlock.java:286)
	at io.trino.operator.project.DictionaryAwarePageProjection$DictionaryAwarePageProjectionWork.setupDictionaryBlockProjection(DictionaryAwarePageProjection.java:211)
	at io.trino.operator.project.DictionaryAwarePageProjection$DictionaryAwarePageProjectionWork.lambda$getResult$0(DictionaryAwarePageProjection.java:197)
	at io.trino.spi.block.LazyBlock$LazyData.load(LazyBlock.java:400)
	at io.trino.spi.block.LazyBlock$LazyData.getFullyLoadedBlock(LazyBlock.java:379)
	at io.trino.spi.block.LazyBlock.getLoadedBlock(LazyBlock.java:286)
	at io.trino.operator.project.PageProcessor$ProjectSelectedPositions.processBatch(PageProcessor.java:345)
	at io.trino.operator.project.PageProcessor$ProjectSelectedPositions.process(PageProcessor.java:208)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:391)
	at io.trino.operator.WorkProcessorUtils.lambda$flatten$7(WorkProcessorUtils.java:296)
	at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:338)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:391)
	at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:325)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:391)
	at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:325)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:391)
	at io.trino.operator.WorkProcessorUtils.lambda$flatten$7(WorkProcessorUtils.java:296)
	at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:338)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:391)
	at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:325)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:391)
	at io.trino.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:240)
	at io.trino.operator.WorkProcessorUtils.lambda$processStateMonitor$3(WorkProcessorUtils.java:219)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:391)
	at io.trino.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:240)
	at io.trino.operator.WorkProcessorUtils.lambda$finishWhen$4(WorkProcessorUtils.java:234)
	at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:391)
	at io.trino.operator.WorkProcessorSourceOperatorAdapter.getOutput(WorkProcessorSourceOperatorAdapter.java:150)
	at io.trino.operator.Driver.processInternal(Driver.java:410)
	at io.trino.operator.Driver.lambda$process$10(Driver.java:313)
	at io.trino.operator.Driver.tryWithLock(Driver.java:698)
	at io.trino.operator.Driver.process(Driver.java:305)
	at io.trino.operator.Driver.processForDuration(Driver.java:276)
	at io.trino.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:740)
	at io.trino.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:163)
	at io.trino.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:488)
	at io.trino.$gen.Trino_389____20221103_014715_2.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.UnsupportedOperationException: Slice is not backed by a byte array
	at io.airlift.slice.Slice.checkHasByteArray(Slice.java:333)
	at io.airlift.slice.Slice.byteArray(Slice.java:352)
	at io.trino.orc.stream.CompressedOrcChunkLoader.nextChunk(CompressedOrcChunkLoader.java:129)
	at io.trino.orc.stream.OrcInputStream.advance(OrcInputStream.java:204)
	at io.trino.orc.stream.OrcInputStream.read(OrcInputStream.java:83)
	at io.trino.orc.stream.LongInputStreamV2.readValues(LongInputStreamV2.java:64)
	at io.trino.orc.stream.LongInputStreamV2.next(LongInputStreamV2.java:366)
	at io.trino.orc.reader.SliceDictionaryColumnReader.openRowGroup(SliceDictionaryColumnReader.java:242)
	at io.trino.orc.reader.SliceDictionaryColumnReader.readBlock(SliceDictionaryColumnReader.java:119)
	at io.trino.orc.reader.SliceColumnReader.readBlock(SliceColumnReader.java:74)
	at io.trino.orc.OrcBlockFactory$OrcBlockLoader.load(OrcBlockFactory.java:76)
	... 46 more
@findepi findepi added the bug Something isn't working label Nov 17, 2022
@findepi
Copy link
Member

findepi commented Nov 17, 2022

cc @raunaqmorarka

@ElapsedSoul
Copy link
Author

ElapsedSoul commented Nov 21, 2022

Next we wait a time to disable hive.cache to test.

I have done this, and the error is disappeared.
It seems enable hive.cache occur the problem at least on arm64

@hanson2021
Copy link

My Trino-403 evironment also have the same problem on X86_64.Was it solved on new version?

cc @raunaqmorarka

@raunaqmorarka
Copy link
Member

If you are running into issues, I would strongly recommend avoiding usage of the current implementation of hive.cache. It has been unmaintained for a long time and it's unlikely to ever receive any code fixes.
There is an alternative caching mechanism being worked on right now #16375 which should hopefully replace the current one.
cc: @electrum @beinan

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Development

No branches or pull requests

4 participants