Null safety feedback: How to use firstWhere?

[xvrh]

EDIT - best solutions so far:

• Use package:collection

import 'package:collection/collection.dart';

void main() {
  var list = ['a', 'b', 'c'];
  var d = list.firstWhereOrNull((e) => e == 'd');
}

• Maintain your own utility extension

extension IterableExtension<T> on Iterable<T> {
  T? firstWhereOrNull(bool Function(T element) test) {
    for (var element in this) {
      if (test(element)) return element;
    }
    return null;
  }
}

void main() {
  var list = ['a', 'b'];
  var d = list.firstWhereOrNull((e) => e == 'd');
}

---

Original question:

I want to search a list and return null when the element is not found

void main() {
  var list = ['a', 'b', 'c'];

  String? d = list.firstWhere((e) => e == 'd', orElse: () => null);
}

https://nullsafety.dartpad.dev/2d0bc36ec1f3a5ade5550c0944702d73

With null safety, I get this error:

main.dart:4:62: Error: A value of type 'Null' can't be assigned to a variable of type 'String'.
  String? d = list.firstWhere((e) => e == 'd', orElse: () => null);

The orElse parameter of firstWhere is declared not nullable:

E firstWhere(bool test(E element), {E orElse()?})

Why not?

E? firstWhere(bool test(E element), {E? orElse()?})

What is the recommended way to search a collection with null safety enabled?

[julemand101]

My guess is that the return type of firstWhere are E and it would be confusing if it was changed to E? since the method are described with the following for the case of no element returning true in the test:

If no element satisfies test, the result of invoking the orElse function is returned. If orElse is omitted, it defaults to throwing a StateError.

A workaround (I hope there are better solutions...) you could do is to make an extension on Iterable like this:

void main() {
  var list = ['a', 'b', 'c'];

  String? d = list.firstWhereOrNull((e) => e == 'd');
  print(d); // null
}

extension FirstWhereOrNullExtension<E> on Iterable<E> {
  E? firstWhereOrNull(bool Function(E) test) {
    for (E element in this) {
      if (test(element)) return element;
    }
    return null;
  }
}

Alternative you could change the list to String? or cast it before calling firstWhere but these does not seem like good solutions from my perspective.

[mraleph]

I am going to transfer this to dart-lang/language.

[mraleph]

Duplicate of dart-lang/language#836

[mraleph]

/cc @leafpetersen @eernstg @lrhn

[pedromassango]

What is the recommended way to search a collection with null safety enabled?

My gues is: return E if it exist or throw (NoSuchElementException) if it doesn't exist and if orElse was not provided.

This make sense. If you want to return null then just use the orElse callback. Dart is becoming non-nullable by default so "by default" we should not expect null values.

[lrhn]

One way would be

List<X> list = ...; 
X? firstOrNull = list.cast<X?>.firstWhere((v) => test(v!), orElse: () => null);

In practice, I'd go for an extension method.

[xvrh]

Maybe the extension can be in the SDK itself then?

extension IterableExtension<E> on Iterable<E> {
  E? findFirst(bool Function(E) test) {}

  E? findLast(bool Function(E) test) {}

  E? findSingle(bool Function(E) test) {}
}

[pedromassango]

Maybe the extension can be in the SDK itself then?

extension IterableExtension<E> on Iterable<E> {
  E? findFirst(bool Function(E) test) {}

  E? findLast(bool Function(E) test) {}

  E? findSingle(bool Function(E) test) {}
}

@lrhn can this be a option? Also Dart can take advantage of extension to create built-in helper functions like Kotlin does, it provide a bunche of extension right inside the language.

[stereotype441]

FWIW, this situation is troublesome for the migraiton tool too. See #42382

[stereotype441]

Note that a similar problem applies to singleWhere (@yjbanov ran into this today)

[munificent]

I've also run into this when migrating tests. I like the idea of the above extension methods and would be happy to see them in the core library if that's feasible.

[lrhn]

It's not impossible. I was targeting them for package:collection (dart-lang/collection#135), and I'm a little disinclined to add extension methods for something that really should be real methods in the library where the real methods would fit in.
(On the other hand, if we ever get interface default methods, I guess you would be able to upgrade extension methods to interface default methods)

[mnordine]

I've used extension methods for the following, following convention with try:

• tryFirst
• tryLast
• trySingle
• tryFirstWhere
• tryLastWhere
• trySingleWhere
• tryCast (instead of throwing TypeError with as, return null)

This makes the intent clear when reading code with these methods.

[stereotype441]

FWIW I'm also running into this when migrating protobufs.

[rodion-m]

Maybe the extension can be in the SDK itself then?

extension IterableExtension<E> on Iterable<E> {
  E? findFirst(bool Function(E) test) {}

  E? findLast(bool Function(E) test) {}

  E? findSingle(bool Function(E) test) {}
}

Why these methods are still not in core? firstWhere in null-safety context at the moment is the pain.

[lukepighetti]

firstWhere in a null safe project is proving to be problematic.

[lrhn]

If you are null safe, then you can use package:collection's new firstWhereOrNull (FAQ).

[sebthom]

@lrhn is there also a workaround for dart-async Stream.firstWhere? https://api.dart.dev/stable/2.12.1/dart-async/Stream/firstWhere.html

[lrhn]

There is no similar method in package:async (yet), so you'll have to make do with:

extension <T> on Stream<T> {
  Future<T?> get firstWhereOrNull async {
    await for (var e in this) {
      return e;
    }
    return null;
  }
}

[Levi-Lesches]

I also have the issue with Streams.

It feels counter-intuitive to me that null-safety is enabled by switching types (T? to T), but with functions like firstWhere, we have to use an entirely different method to get the behavior we want.

While we're on the topic of unintuitive, I'd say that the fact that firstWhere throws is surprising, especially when compared to Map.[] which returns null and List.indexOf which returns -1. The intuition is that null is the N/A sort of value, and it should be used more uniformly, though I understand that there's probably historical context. I also understand that making firstWhere not throw would be a massive breaking change, but that's just my two cents. If there's a way we can change it, I think it should be done.

Using orElse and/or having a try/catch waiting for a StateError both seem like null-checks with extra steps. orElse can be replaced by ?? and a try/catch can be replaced by an if(result == null). In other words, what could be

final int largeNumber = nums.firstWhere((int value) => value > 50) ?? 1000;

is instead:

final int largeNumber =  nums.firstWhere((int value) => value > 50, orElse: () => 1000);

or even worse:

int largeNumber;
try {
  largeNumber = nums.firstWhere((int value) => value > 50);
} on StateError {
  largeNumber = 1000;
}

and that's before null-safety became an issue.

[cedvdb]

Adding a dependency for such basic feature is pushing it

[Levi-Lesches]

@lrhn, I'm trying to figure out why things are the way they are: If breaking changes weren't an issue, would it make sense to change the signature of Iterable.firstWhere to return E?, then allow orElse to return E? as well? Or even better, make it return null if orElse is omitted, instead of throwing an error, but I can see how that might be too breaking (since there's no warning that the try/catch would be useless, but changing to nullable E will give errors and warnings everywhere).

[lrhn]

It would be possible to change firstWhere to return E?. It might even be desirable, and then I'd remove the orElse entirely and let you rely on ?? to introduce the default value. Parameters like orElse are generally a liability in null safety.

However, it would be annoying for people who definitely know that there will be a match. They'd have to write something like (myList.firstWhere(somethingCertain) ?? unreachable) to throw (with a helper like Never get unreachable => throw UnsupportedError("This isn't happening!");).

It's probably better to have both E firstWhere(bool Function(E) test) and E? firstWhereOrNull(bool Function(E) test) (or tryFirstWhere as someone suggested, it just has bad discoverability in auto-complete because it doesn't start with firstWhere).

[Levi-Lesches]

This is the code from api.dart.dev on firstWhere:

E firstWhere(bool test(E element), {E orElse()?}) {
  for (E element in this) {
    if (test(element)) return element;
  }
  if (orElse != null) return orElse();
  throw IterableElementError.noElement();
}

If it weren't for that throw at the end, Dart would complain that this actually returns E?. So it boils down to "is throwing an acceptable return type for firstWhere?" IMO, I think it isn't. firstWhere is used so often, that it's not accurate to say that "not finding the element is an error". Plenty of people use it to check, which is why there are issues like this one. null is the perfect compromise, because those who want to handle failure can use ??, and those who just want to throw an error on failure can use !, or ??. It might be more annoying for those who need the error, but ! isn't so bad, and including those who expect a failure is probably worth it.

[gitaaron]

It would be possible to change firstWhere to return E?. It might even be desirable, and then I'd remove the orElse entirely and let you rely on ?? to introduce the default value. Parameters like orElse are generally a liability in null safety.

However, it would be annoying for people who definitely know that there will be a match. They'd have to write something like (myList.firstWhere(somethingCertain) ?? unreachable) to throw (with a helper like Never get unreachable => throw UnsupportedError("This isn't happening!");).

It's probably better to have both E firstWhere(bool Function(E) test) and E? firstWhereOrNull(bool Function(E) test) (or tryFirstWhere as someone suggested, it just has bad discoverability in auto-complete because it doesn't start with firstWhere).

My train of thought was I would prefer to treat the default as nullable and include a helper method for those that want to throw if an element is not found. ie/ firstWhereGuaranteed . However, that seems like more typing that just firstWhere()! which I think is equally as understandable and goes more with conventions. So I would agree with @Levi-Lesches' previous comment.

[lrhn]

The one issue with just making the return type nullable is that you can no longer distinguish a null from finding no value satisfying where, and where null is a value which satisfies the where.

If you have firstWhere which throws on not finding a match, it differs from list.firstWhere(test)! which returns null on non-match or a match of null.

Not sure how often you'll be doing firstWhere(test) where test accepts null, but our experience with Map.operator[] suggests that people do store null values in collections.

[gitaaron]

A case where 'null' could be returned as an element of the list seems like more of a reason to make the return of 'firstWhere' nullable.

Currently if I run the following -

  List l = [1,2,null];
  int a = l.firstWhere((el) => el==null);
  int b = a + 4;
  print(b);

I believe I get a runtime exception.
But if I were to change it to -

  List l = [1,2,null];
  int a? = l.firstWhere((el) => el==null);
  int b = a + 4;
  print(b);

I believe I get a compile time exception which I would imagine is more preferable and in the spirit of null safety.

[gitaaron]

I am having a hard time understanding why I would want to do something differently if the List contained a null vs. not finding anything. I guess if I wanted to check if null is in the list?

Edit: The two scenarios I can some up with are -

1. I want to throw an exception if null is in the list.
2. I want to see if null is in the list.

In either case I would use a forEach.

Second Edit : In the first case I could also do .firstWhere()! if I wanted so I guess that it is not a use case for differentiating unless I wanted to be explicit about the error I am throwing.

[Levi-Lesches]

A case where 'null' could be returned as an element of the list seems like more of a reason to make the return of 'firstWhere' nullable.

This is another important use-case. Using firstWhere to directly compare to null is not what I had in mind (use contains for that), but comparing the elements to another nullable variable can be useful.

The two scenarios I can some up with are -

1. I want to throw an exception if null is in the list.
2. I want to see if null is in the list.

if (myList.contains(null))  // throw or do something else

Not sure how often you'll be doing firstWhere(test) where test accepts null, but our experience with Map.operator[] suggests that people do store null values in collections.

I think that Map.[] strengthens the argument here, since Map already returns null ambiguously (if the value is null or if the key is not present), and people can manage the difference. That's because they have alternatives -- Map.cotainsKey is usually used if the map holds null values. So we'd need an equivalent for iterable. Maybe add a bool argument to firstWhere: shouldThrow to force an error if no value is found?

E? firstWhere(bool test(E element), {bool shouldThrow = false}) {
 for (E element in this) {
   if (test(element)) return element;
 }
 if (shouldThrow)
   throw IterableElementError.noElement();
}

So now you have this table of return types:

Value of shouldThrow	Element is found	Element is not found
true	E	throws
false	E	E?

So if your elements can be null (E == E?), you set throwError to true to have a clear signal if the element wasn't found. If you don't care, like how people use Map.[] today, you can simply let it return null. I think that's the perfect compromise between being able to handle "not found" gracefully while also allowing a stricter fallback. @lrhn, thoughts?

[sebthom]

In Java/Spring you would declare one method getFirstWhere(...) that throws an Exception in case no element was found and one method findFirstWhere(...) which would return null in case no element was found. Depending on your use case/expectation of the existence of an element you would choose one or the other.

The disadvantage of having a shouldThrow parameter is that you always have E? as return type.

[Levi-Lesches]

The disadvantage of having a shouldThrow parameter is that you always have E? as return type.

I'm assuming you have an iterable of non-nullable elements, because otherwise you'd want to have an E? returned. So now you know that findWhere will never return null when it finds the element, so you can just use ! instead of shouldThrow. shouldThrow is just to resolve ambiguity between "found a nullable result" and "didn't find anything".

Here's what you should do when:

List type	You expect a result	You're just checking
List<int>	int element = firstWhere(condition)!	int? element = firstWhere(condition)
List<int?>	int? element = firstWhere(condition, shouldThrow: true)	int? element = firstWhere(condition)

[sebthom]

I personally don't like this approach as you either have to use an unsafe cast or you have verbosity (shouldThrow:true). Anyways I don't think it matters much as I doubt the return type of firstWhere will ever be changed at this point since this would break existing code for very little benefit. I think adding a second method like firstWhereOrNull or however it is called is probably more sensible if not alone for the fact that it is backwards compatible.

[Levi-Lesches]

True this would be a breaking change, but we shouldn't sacrifice long-term API clarity just for that. Imagine if this had been done during the null-safety implementation: we would have made this change in our migrations and be done with it. Now that we're past that, this would probably have to wait for the next major version, but it should still be done. Besides, by making the return type nullable the compiler can catch cases that need to be fixed (and maybe even fix them with dart fix), instead of having to go find them manually. Updating code isn't the worst thing in the world, we just did it for null safety; this should have been part of that IMO.

As to your first points, both are better than what we currently have today:

• Verbosity: In the case where you can't guarantee there's an element, the nullable return type is much better than a try/catch, and shouldThrow is on par with orElse. When you do have that guarantee, it's only longer by a !.
• Safety: The current implementation throws a runtime-error if an element can't be found. If the point is to preserve that error, then ! is good for you. If you want to be safe, using null allows you to check for it, then promote afterwards.

To demonstrate (assuming we never had firstWhereOrNull):

List<int> nonNullable;
List<int?> nullable;
bool Function(int) condition;

// Throw if the element is not in a non-nullable list
int element = nonNullable.firstWhere(condition)!;  // new
int element = nonNullable.firstWhere(condition);  // current

// Check for an element in a non-nullable list (very common)
int? element = nonNullable.firstWhere(condition);  // new
int? element;  // current
try {
  element = nonNullable.findWhere(condition);
} catch { }

// Throw if an element is not in a nullable list
int? element = nullable.firstWhere(condition, shouldThrow: true);  // new
int? element = nullable.firstWhere(condition);  // current

// Checking for an element in a nullable list
int? element = nullable.firstWhere(condition);  // new
int? element = nullable.firstWhere(condition, orElse: () => null);  // current 

[sebthom]

@Levi-Lesches We can also compare all three approaches. Introducing a shouldThrow parameter would result in mandatory code changes in all four cases. Adding a firstWhereOrNull() is backwards compatible, i.e. needs no changes but allows you to gradually improve your code in three cases.
I also like to point you to https://softwareengineering.stackexchange.com/questions/362900/parameter-to-control-whether-to-throw-an-exception-or-return-null-good-practic

List<int> nonNullable;
List<int?> nullable;
bool Function(int) condition;

// Throw if the element is not in a non-nullable list
int element = nonNullable.firstWhere(condition)!;  // needed change with shouldThrow semantic
int element = nonNullable.firstWhere(condition);   // NO change with firstWhereOrNull()
int element = nonNullable.firstWhere(condition);   // current

// Check for an element in a non-nullable list (very common)
int? element = nonNullable.firstWhere(condition);       // needed change with shouldThrow semantic
int? element = nonNullable.firstWhereOrNull(condition); // optionally improved with firstWhereOrNull()
int? element;  // current
try {
  element = nonNullable.findWhere(condition);
} catch { }

// Throw if an element is not in a nullable list
int? element = nullable.firstWhere(condition, shouldThrow: true);  // needed change shouldThrow semantic
int? element = nullable.firstWhere(condition); // NO change with firstWhereOrNull()
int? element = nullable.firstWhere(condition); // current

// Checking for an element in a nullable list
int? element = nullable.firstWhere(condition);        // needed change with shouldThrow semantic
int? element = nullable.firstWhereOrNull(condition);  // optionally improved with firstWhereOrNull()
int? element = nullable.firstWhere(condition, orElse: () => null);  // current 

Just to be clear, I also see firstWhereOrNull() as a workaround and think that it was a language design error to make the nullability of firstWhere's return type depending on the nullability of the list. firstWhere's return type should be nullable and never throw an exception.

[Levi-Lesches]

I'll note that where you added needed change, the code always improved:

1. Using ! makes it clear there will be a runtime error if the assumption is incorrect. That is not obvious currently
2. The second example is much shorter than the current firstWhere behavior
3. Using shouldThrow makes it clear there will be a runtime error if the assumption is incorrect. That is not obvious currently.
4. The last example is much shorter than the current firstWhere behavior

That's not even mentioning the potential to use ?? and other null operators, or the safer-by-default behavior by not throwing.

I didn't include firstWhereOrNull because I agree that it's a nice temporary solution that doesn't break much. I'm advocating to eventually change firstWhere so that it is clearer and more useful in the long-term.

[lukepighetti]

Just to throw a wrench in the works we could have an operator that converted errors into null values.

[].first ?????????????????????? == null 😅

[Levi-Lesches]

Actually, some have suggested the !! operator for just that. It's like ?? but instead of triggering on a null value, it triggers on an error. I can't find the issue for it, but I'm more in favor of dart-lang/language#361)

[lrhn]

An inline "catch expression" is an interesting idea. I have thought about it a few times, but the grammar needed to express the error type and capture variables gets cumbersome. Just ignoring all of that and reacting to any throw can definitely be shorter. I fear it might not be what people actually want, and it might hide programming errors.
Say int.parse(str.substring(start, end)) !! 0 is intended to catch FormatException, but if you somehow manage to have incorrect start/end indices, it will also happily hide the RangeError you get from substring.
The style guide recommends against catches without on-types for a reason.

(Also, the operator should definitely be ?!? and be pronounced "WAT".)

[Wdestroier]

import 'package:collection/src/iterable_extensions.dart'; shows a warning in VScode, pls find a fix soon.

inline catch is probably not the best solution for this case, but it's a great feature.

Creating an object to pass to singleWhere gets more annoying as the object complexity grows.

final entry = map.entries.singleWhere((entry) => false, orElse: () => MapEntry(ComplexObject1({
  complexObject2: ComplexObject2({ 
    id: ObjectId('');
   }),
  complexObject3: ComplexObject3({ ... }),
}), ComplexObject1({
  complexObject2: ComplexObject2({ 
    id: ObjectId('');
   }),
  complexObject3: ComplexObject3({ ... }),
})));

if (entry.key.complexObject2.id.asString() != '') {
  // Yay, it's not `null`
}

[esDotDev]

It would be nice to get firstWhereOrNull soon, it's really getting annoying having to write code like:

BasicPushNotification? pinned = notifications
        .cast<BasicPushNotification?>()
        .firstWhere((n) => n!.isPinned == true, orElse: () => null);

Yuck.

[lukepighetti]

Actually, some have suggested the !! operator for just that. It's like ?? but instead of triggering on a null value, it triggers on an error. I can't find the issue for it, but I'm more in favor of dart-lang/language#361)

!(isValid()!! ?? false) coming soon to an IDE near you

[cedvdb]

Actually, some have suggested the !! operator for just that.

The !! operator does not force you to do anything which is not null safe since there is no checked exceptions. In this context I'd much prefer the return type to be nullable. It's indeed a bit annoying when you are sure that there is an element in the list, but if you are really sure I don't see why ! would not fit here.

I would be backward to have to add !! when you are sure than the where clause might not be matched all the time instead of adding ! when you are sure it will.

The one issue with just making the return type nullable is that you can no longer distinguish a null from finding no value satisfying where, and where null is a value which satisfies the where.

That's reasonable but that's such an edge case compared to the dozen times you are not in this scenario does it matter ?

[Levi-Lesches]

It's been a while, but I still think it would be beneficial to make firstWhere nullable (note the lack of shouldThrow):

E? firstWhere(bool Function(E) test, {@deprecated E Function()? orElse}) { 
  for (final E element in this) {
    if (test(element)) return element;
  }
  if (orElse != null) return orElse();
  return null;
}

To avoid breaking changes, keep the current semantics of orElse, but mark it @deprecated and encourage ?? instead (although removing it may actually be safe, see below).

I thought a bit more, and I don't think we need to handle a case where firstWhere checks for null. If you want to check that null is in your list, use .contains(null). But if you want the element that equals null... just use null? I'm not sure why we had so many examples in this thread like

List<int?> a;
int? b = myList.firstWhere((c) => c == null);

There is no benefit to checking myList here, just set b = null directly. Thus, using ?? for defaults is never an issue.

Then, the big breaking change that doesn't get caught automatically is people using try/catch (with something useful in the catch) and depending on .firstWhere to throw, like this:

List<int> a;
int? b;

// this won't be affected by making firstWhere nullable, as b will still be null
try { b = a.firstWhere((c) => c % 2 == 0); } catch (_) { /* leave b null */} 

// but this will, as the catch block will never run
try { b = a.firstWhere((c) => c % 2 == 0); } catch (_) { b = 0; } 

But I don't think that's such a big issue either. In the first case, as the comment says, making .firstWhere return null on failures keeps the current behavior. In the case of a useful default, people are hopefully using orElse instead, which is the only case that would really break:

int b = list.firstwhere((c) => c % 2, orElse: () => 0);  // int? cannot be assigned to int

However, this is a relatively quick fix by either changing orElse to ?? or adding a !, which can be automated with a dart fix.

[lrhn]

Would be nice. If I had to redesign firstWhere today, I'd definitely do that. (And, shucks, someone actually told me that orElse was a bad idea when I first introduced it, so ... you were right! We should have designed with null safety in mind all the way 😢.)

It's still not going to happen.
There is too much code out there which will need to be changed from .firstWhere(...) to .firstWhere(...)!. It's just not viable to migrate it all.
We can't just rely on dartfix because a change to the API will affect all code, not just code migrated to the newest language version.
We'll need some kind of API versioning for us to be able to make such a breaking change work. We don't have that.

Also, all the existing third-party implementations of firstWhere which return a non-nullable result will stay valid, and will still throw. We need to migrate those too, but there is nothing which will help us find them (since they're still valid).

[sebthom]

Since Dart 3.0.0 is about streamlining the dart APIs, enabling null safety by default and contains a lot of breaking changes anyways, couldn't this please be fixed/redesigned too now?

[lrhn]

I'd love to, but it's still really not a viable breaking change. Too hard and work-intensive to migrate, too breaking to not migrate.

I don't expect this change can happen until we actually get library versioning, where new code can use the new API, and old code can stay on the old API, on the same object and type. That's not currently on the table (we keep thinking about it, but it's not on the short-list of features we are currently working actively on.)

[lukepighetti]

Until that happens, firstWhereOrNull from collection is OK

[lukepighetti]

I will note that in practice firstWhere is becoming a bit of a footgun. If we could get firstWhereOrNull in the core framework it would be more discoverable