Add scala.util.Using, for automatic resource management

[NthPortal]

Resolves scala/bug#11003
Resolves scala/bug#8028 (I think)
(and possibly others)

[NthPortal]

bikeshedding welcome

[Ichoran]

I would think the signature should be [R: Resource, A](=>R)(R => A): Try[A]

[hrhino]

See also discussion at #6347.

[NthPortal]

@Ichoran I guess I have two comments on that

• why => R? Is there an advantage to having its evaluation be deferred?
• there are a few disadvantages to having it return a Try
  • it becomes impossible to not use a Try; on the other hand, as it is, you can just wrap the whole thing in a Try if you want that
  • it makes it uglier to nest/compose using blocks; if you have four resources, you need to create four Trys, which really isn't necessary for the computation

regarding the latter point, this is what the implementation of the four resource method would look like if it returns a Try

using(resource1) { r1 =>
  using(resource2) { r2 =>
    using(resource3) { r3 =>
      using(resource4) { r4 =>
        body(r1, r2, r3, r4)
      }.get
    }.get
  }.get
}

not horrible, but certainly uglier

[Ichoran]

You want Try because throwing exceptions all over the place is messy. Maybe you should have two versions, one that wraps the exceptions and one that doesn't, but the usual case should be to use the Try version since you can't muck that one up.

If you're going to have a Try variant, then you need => R so that resource creation exceptions are also handled cleanly.

And of course you ought to be able to

for {
  r1 <- using(resource1)
  r2 <- using(resource2)
  r3 <- using(resource3)
  r4 <- using(resource4)
} yield body(r1, r2, r3, r4)

which means the whole thing should be an applicative functor, or functor-like (I don't know what the non-identity version is, where map is (F[A], A => B) => G[B] and flatMap is (F[A], A =>G[B]) => G[B]) for a specific permitted G, e.g. Try[A] or Either[ResourceException, A].

[Ichoran]

Okay, this all works. I just have a stub implementation for the actual resource logic, but I envision it like this:

class Using[R](r: => R) {
  import scala.util.{Try, Success, Failure}
  def apply[A](f: R => A)(implicit ev: Using.Resource[R]): Try[A] = map[A](f)(ev)
  def map[A](f: R => A)(implicit ev: Using.Resource[R]): Try[A] = Try{ val myR = r; val ans = f(myR); ev.close(myR); ans }
  def flatMap[A](f: R => Try[A])(implicit ev: Using.Resource[R]): Try[A] = Try{ val myR = r; val ans = f(myR); ev.close(myR); ans }.flatten
}
object Using {
  trait Resource[R]{ def close(r: R): Unit }
  def apply[R](r: => R): Using[R] = new Using(r)
}

Then we have usage like so:

case class Id[A](value: A) {}
object Id { 
  implicit def resourcedId[A]: Using.Resource[Id[A]] = new Using.Resource[Id[A]] { def close(r: Id[A]) = () } 
} 

@ for { u <- Using(Id(2)); v <- Using(Id("salmon")) } yield v.value * u.value 
res2: scala.util.Try[String] = Success("salmonsalmon")

@ 
Using(Id(true)).flatMap{ bool => Using(Id('x')){ c => if (bool.value) c.value.toUpper else c }} 
res5: scala.util.Try[Any] = Success('X')

[Ichoran]

I'm not sure what the name of this not-exactly-a-Monad thing is, if it has one, but the usage is really nice. You just use your stuff, and the right thing happens, and exceptions are caught.

[iravid]

These are relevant:

• https://github.com/iravid/managedt
• https://github.com/typelevel/cats-effect/blob/master/core/shared/src/main/scala/cats/effect/Resource.scala
• https://hackage.haskell.org/package/managed

[NthPortal]

@Ichoran my only concern is how fatal exceptions are handled. I personally think that even in the case of a fatal exception, we ought to attempt to close the resource before re-throwing the exception (especially if it's a NonLocalReturnControl). I will have to have a look at your implementation to see how it handles it.

I think it makes sense to have both, but with the Try one more prominently defined, as you suggest. If we're to have a Using class though, should the throwing methods be renamed? Or is there not too much of a worry of having them be called, since it would require importing the entire contents of object Using?

Also, I would be more inclined to put the implicit Resource in the constructor of a Using, rather than on each of the methods.

[smarter]

See also @densh idea of scoped implicit lifetimes

[NthPortal]

@Ichoran pushed the monad-ish impl you suggested. I'm open to burying the raw methods a bit more to make them less obvious

[Ichoran]

@NthPortal - You can put the typeclasses on the constructor, but then you can't chain the applies. So it would have to be Using(file.open).map(_.slurp) not Using(file.open){ _.slurp } (assuming that there was an open method on file, and that you can slurp an OpenFile).

[Ichoran]

I don't think Using.using is good nomenclature. Maybe Using.direct or Using.unsafe or Using.unwrapped?

[NthPortal]

my only concern is that I'd prefer a method name that, if imported statically, still makes sense.

e.g. Using.direct(file.open) makes sense, but direct(file.open) doesn't really

[Ichoran]

withResource?

[NthPortal]

that more I play with it in my mind, the less happy I am with the name withResource. something about it doesn't flow right.

what do you think of just resource or tryWithResource? The former works well when not imported - Using.resource.

[NthPortal]

or another possibility: tryWith

[NthPortal]

(for comparison)

Using.withResource(new FileInputStream("foo.txt")) { _ => () }
withResource(new FileInputStream("foo.txt")) { _ => () }

Using.resource(new FileInputStream("foo.txt")) { _ => () }
resource(new FileInputStream("foo.txt")) { _ => () }

Using.tryWithResource(new FileInputStream("foo.txt")) { _ => () }
tryWithResource(new FileInputStream("foo.txt")) { _ => () }

Using.tryWith(new FileInputStream("foo.txt")) { _ => () }
tryWith(new FileInputStream("foo.txt")) { _ => () }

[NthPortal]

I think the implementation is pretty much done at this point. The scaladoc could use some improvement, which I would appreciate input on. The implementation is ready for review.

[NthPortal]

should this be in scala.util.control (as it is currently) or in scala.util where Try is?

[NthPortal]

review @kmizu

[NthPortal]

I am open to being convinced that the resource can be generated again using the by-name parameter, and that the Using instance can be used arbitrarily many times.

[kmizu]

@NthPortal Sorry. I forgot to review.

[kmizu]

@NthPortal It looks good to me as far as I can see

[dwijnand]

should this be in scala.util.control (as it is currently) or in scala.util where Try is?

JFYI I noticed it's declared in scala.util.control but defined in scala/util/Using.scala.

[NthPortal]

Good catch!

I expected IntelliJ to automatically change the package statement when I moved the files, but apparently it didn't ¯\_(ツ)_/¯

I'll fix that

[NthPortal]

@joshlemer bonus side effect of simplifying the implicit Resource for AutoClosable!

[joshlemer]

I am just throwing out a suggestion but maybe a factory method

Resource.fromTry[A](f: A => Try[_]): Resource[A]

might come frequently in handy as well

EDIT: Whoops fixed typo in the code

[NthPortal]

@joshlemer It's not immediately apparent to me what that does

[joshlemer]

@NthPortal something like

def fromTry[A](f: A => Try[_]): Resource[A] = (a: A) => f(a).failed.foreach(throw _)

[NthPortal]

@joshlemer what's the value of catching exceptions in a Try just to rethrow them immediately?

[joshlemer]

@NthPortal I was thinking more in the case where a user has a library which returns a Try for some IO operation, rather than itself throwing. In that case, the user would have to implement a Resource[T] which just throws any failed Try's, so it might be handy to have that case covered.

[NthPortal]

@joshlemer aha, I see. I'm not super into it, but I'd love to hear others' thoughts on it

[lrytz]

Very nice, thanks! Asking for a few scaladoc improvements.

[lrytz]

I think the doc should explain what a resource is, how the Resource type class is used, what the user needs to do for non-AutoClosables.

[lrytz]

Since there are two ways of using Using, I think this comment (which is the entry-point) should mention both, say why they exist, what the differences are.

[NthPortal]

When you say two ways, to you mean Using(foo){ _.bar } vs for (f <- Using(foo)) yield f.bar, or Using(foo){ _.bar } vs Using.resource(foo){ _.bar }?

[lrytz]

I meant Using(foo) vs Using.resource(foo), but explaining the possibilities how to use the monadic version is also welcome.

[lrytz]

It's maybe not clear to everyone what the Using construct is

[joshlemer]

@NthPortal is there a reason not to make Using covariant?

[NthPortal]

@joshlemer not that I can think of, but I also can't think of a reason why you'd ever declare it as a parameter to something, so I don't think it makes a difference. If you want to use its result for something, just use a Try.

[SethTisue]

but won't add new things in RC1). @SethTisue, I defer to you as M5 lead, but I've put it back on the M5 radar for now

maybe it could still make M5, but I think it could wait for RC1 if it must. we aren't adding new things post-M5, but I think we should allow exceptions for new things that were already PR'ed well in advance of the M5 deadline. especially something like this that isn't being used yet and has zero effect on code that doesn't use it.

[NthPortal]

@SethTisue I think that's a good idea. I don't know if I can push through the doc changes by tomorrow while also trying to get LazyList done

[NthPortal]

@Ichoran what are your thoughts on removing the apply method, so that there aren't two ways to use it - you have to use a for comprehension (or manual map/flatMap)?

[NthPortal]

This has the added benefit of allowing us to pass the implicit Resource on construction

[Ichoran]

The for-comprehension-only version probably ought to be called Use or Used, not Using, if there's only that option. I generally prefer the apply version, but I don't have a strong preference.

[NthPortal]

I don't feel strongly about it either; I'm just concerned that some others might.

I'm going to leave it as is for now

[SethTisue]

shall we merge as-is, to get feedback from a broader group, and figure there's still time to tweak it for RC1?

[lrytz]

That works for me!

[NthPortal]

(this probably should have been squashed first, oops)

[SethTisue]

doh, my bad!