This repository has been archived by the owner. It is now read-only.

Error on Sample Execution (could not find implicit value for parameter kv) #4

Closed
conikeec opened this Issue May 13, 2012 · 1 comment

Comments

Projects
None yet
2 participants
@conikeec

conikeec commented May 13, 2012

Based on the example provided, I carved out a sample executable and stumbling on the following error on execution

import com.gravity.hbase.schema.{ HRow, DeserializedResult, HbaseTable, Schema }

import com.gravity.hbase.schema._

import com.gravity.hbase.mapreduce._
import java.lang.String
import org.joda.time.{ DateMidnight, DateTime }
import com.gravity.hbase.mapreduce.{ HMapReduceTask, HJob }
import java.net.URL
import com.gravity.hbase.schema._

import WebCrawlingSchema.WebPageRow
import org.apache.hadoop.io.Text

import org.apache.hadoop.conf.Configuration
import org.apache.hadoop.hbase.HBaseConfiguration
import org.joda.time.{ DateMidnight, DateTime }

object HBaseConfig {
lazy val config: Configuration = {
val c = HBaseConfiguration.create
val localConfigPaths = List("resources/hbase-site.xml", "../resources/hbase-site.xml")
localConfigPaths.foreach((p) => {
val path = new org.apache.hadoop.fs.Path(p)
if ((new java.io.File(path.toString)).exists) {
c.addResource(path)

  }
})
c

}
}

object WebCrawlingSchema extends Schema {

implicit val conf = HBaseConfig.config

class WebTable extends HbaseTable[WebTable, String, WebPageRow](tableName = "pages", rowKeyClass = classOf[String]) {
def rowBuilder(result: DeserializedResult) = new WebPageRow(this, result)

val meta = family[String, String, Any]("meta")
val title = column(meta, "title", classOf[String])
val lastCrawled = column(meta, "lastCrawled", classOf[DateTime])

val content = family[String, String, Any]("text", compressed = true)
val article = column(content, "article", classOf[String])
val attributes = column(content, "attrs", classOf[Map[String, String]])

val searchMetrics = family[String, DateMidnight, Long]("searchesByDay")

}

class WebPageRow(table: WebTable, result: DeserializedResult) extends HRow[WebTable, String](result, table) {
def domain = new URL(rowid).getAuthority
}

val WebTable = table(new WebTable)

class SiteMetricsTable extends HbaseTable[SiteMetricsTable, String, SiteMetricsRow](tableName = "site-metrics", rowKeyClass = classOf[String]) {
def rowBuilder(result: DeserializedResult) = new SiteMetricsRow(this, result)

val meta = family[String, String, Any]("meta")
val name = column(meta, "name", classOf[String])

val searchMetrics = family[String, DateMidnight, Long]("searchesByDay")

}

class SiteMetricsRow(table: SiteMetricsTable, result: DeserializedResult) extends HRow[SiteMetricsTable, String](result, table)

val Sites = table(new SiteMetricsTable)

}

object MainClz extends App {

val exTable = WebCrawlingSchema.WebTable.createScript()
println(exTable)
}

Error:

could not find implicit value for parameter kv: com.gravity.hbase.schema.ByteConverter[Map[java.lang.String,java.lang.String]]
[error] val attributes = column(content, "attrs", classOf[Map[String, String]])
[error] ^
[error] one error found

@erraggy

This comment has been minimized.

Contributor

erraggy commented May 13, 2012

Why not use a family instead of a single column?

val attributes = family[String, String, String]("attrs")

If you really did intend to use a single column to store a Map[String, String], then you are the lucky one to find an issue with scala and type inferencing. There are multiple Map types in scala. For instance there is scala.collection.mutable.Map and scala.collection.immutable.Map. If you want to avoid binding to one or the other, then you'll want to add this import statement to the scope of your code import scala.collection._

Here is an example I just tested locally that works:

package com.robnrob

import scala.collection._
import com.gravity.hbase.schema._
import org.apache.hadoop.hbase.HBaseConfiguration

object MySchema extends Schema {
  implicit val conf = HBaseConfiguration.create()

  class MyTable extends HbaseTable[MyTable, String, MyRow](tableName = "mytable", rowKeyClass = classOf[String]) {
    def rowBuilder(result: DeserializedResult) = new MyRow(this, result)

    val meta = family[String, String, Any]("meta")
    val myColumn = column(meta, "mc", classOf[String])
    val myAttributes = column(meta, "ma", classOf[Map[String, String]])
  }

  class MyRow(table: MyTable, result: DeserializedResult) extends HRow[MyTable, String](result, table)

  val MyTable = table(new MyTable)

}

@erraggy erraggy closed this May 13, 2012

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.