Skip to content
This repository has been archived by the owner on Nov 16, 2023. It is now read-only.

Id in schema when reading from Accumulo #29

Closed
anargyri opened this issue Dec 2, 2019 · 0 comments · Fixed by #30
Closed

Id in schema when reading from Accumulo #29

anargyri opened this issue Dec 2, 2019 · 0 comments · Fixed by #30
Assignees
Labels
bug Something isn't working

Comments

@anargyri
Copy link

anargyri commented Dec 2, 2019

If "id" is part of the schema, then spark.read.format("com.microsoft.accumulo").options(props.asScala).schema(schema).load fails with an error.

@anargyri anargyri added the bug Something isn't working label Dec 2, 2019
eisber added a commit that referenced this issue Dec 3, 2019
* use local classloader

* try different class loader

* add println

* add hack for zfs in jimfs

* Update MLeapUtil.scala

Try to add classloader...

* forcefully instantiate zip filesystem provider to overcome scala/spark behavior

* imported OpenJDK ZipFileSystem to overcome JDK dependency

* JDK 8 compat

* more JDK 8 compat

* more JDK 8 compat

* switched to packaged zipfs for iterator

* updated copyright notice as per CELA guidance

* don't re-use the zip file system

* added README for zipfs origin

* added debug print

* switch to log4j logging

* implemented mleap dataframe caching

* added log4j logging
added perf metrics

* changed the groupId to com.microsoft.masc

* package rename

* renamed to com.microsoft.accumulo
introduce guava based cache for MLeap model
try to fix column pruning

* fixed datasource schema passing

* fixed issue #27

* re-added accumulo client to datasource jar

* removed accumulo-core from iterator shaded jar

* added MLeap guid to support fast cache lookup

* fix MLeap model caching

* don't add empty splits (resulting in duplicate records)

* fixing #29 (duplicate rowkey column in schema)
@eisber eisber closed this as completed Dec 3, 2019
@eisber eisber mentioned this issue Dec 3, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants