Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API for distributed cache #198

Closed
johnynek opened this issue Oct 4, 2012 · 3 comments
Closed

API for distributed cache #198

johnynek opened this issue Oct 4, 2012 · 3 comments

Comments

@johnynek
Copy link
Collaborator

johnynek commented Oct 4, 2012

Make this easier to use from scalding.

@avibryant
Copy link
Contributor

Suggested API:

class Cache[T](path : String) {
  def load : T
}

object Cache {
  def apply[T](value : T) = new Cache[T](serializeAndPlaceInDistCache(value))
}

Usage:

val cachedData = Cache(loadLocalData)
lazy val data = cachedData.load

@ktoso
Copy link
Contributor

ktoso commented Dec 30, 2013

Was thinking of picking this up, but it turns our we have it already:
https://github.com/twitter/scalding/blob/develop/scalding-core/src/main/scala/com/twitter/scalding/filecache/DistributedCacheFile.scala added in 7fd4b15

A nice to have here would be to expose the add__ToClasspath methods, but in general the existing implementation by @slyphon seems to resolve this task.

@johnynek
Copy link
Collaborator Author

yes, closed by #450

@ktoso the classpath stuff would be great to have.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants