✨ Add fc.scheduler arbitrary

README.md

@@ -102,6 +102,7 @@ fast-check has initially been designed in an attempt to cope with limitations I
 - **Debug:** custom examples in addition of generated ones \[[more](https://github.com/dubzzz/fast-check/blob/master/documentation/1-Guides/Tips.md#add-custom-examples-next-to-generated-ones)\] - *no need to duplicate the code to play the property on custom examples*
 - **Debug:** logger per predicate run \[[more](https://github.com/dubzzz/fast-check/blob/master/documentation/1-Guides/Tips.md#log-within-a-predicate)\] - *simplify your troubleshoot with fc.context and its logging feature*
 - **Unique:** model based approach \[[more](https://github.com/dubzzz/fast-check/blob/master/documentation/1-Guides/Tips.md#model-based-testing-or-ui-test)\]\[[article](https://medium.com/criteo-labs/detecting-the-unexpected-in-web-ui-fuzzing-1f3822c8a3a5)\] - *use the power of property based testing to test UI, APIs or state machines*
+- **Unique:** detect race conditions in your code \[[more](https://github.com/dubzzz/fast-check/blob/master/documentation/1-Guides/Tips.md#detect-race-conditions)\] - *shuffle the way your promises and async calls resolve using the power of property based testing to detect races*
 
 For more details, refer to the documentation in the links above.
 

documentation/1-Guides/Arbitraries.md

@@ -22,6 +22,7 @@ You can refer to the [generated API docs](https://dubzzz.github.io/fast-check/#/
   - [Arbitrary](#arbitrary)
   - [Model runner](#model-runner)
   - [Simplified structure](#simplified-structure)
+- [Race conditions detection](#race-conditions-detection)
 
 ## Boolean (:boolean)
 
@@ -288,3 +289,26 @@ fc.assert(
   )
 );
 ```
+
+## Race conditions detection
+
+In order to ease the detection of race conditions in your code, `fast-check` comes with a built-in asynchronous scheduler.
+The aim of the scheduler - `fc.scheduler()` - is to reorder the order in which your async calls will resolve.
+
+By doing this it can highlight potential race conditions in your code. Please refer to [code snippets](https://codesandbox.io/s/github/dubzzz/fast-check/tree/master/example?hidenavigation=1&module=%2F005-race%2Fautocomplete%2Fmain.spec.tsx&previewwindow=tests) for more details.
+
+`fc.scheduler()` is just an `Arbitrary` providing a `Scheduler` instance. The generated scheduler has the following interface:
+- `schedule: <T>(task: Promise<T>) => Promise<T>` - Wrap an existing promise using the scheduler. The newly created promise will resolve when the scheduler decides to resolve it (see `waitOne` and `waitAll` methods).
+- `scheduleFunction: <TArgs extends any[], T>(asyncFunction: (...args: TArgs) => Promise<T>) => (...args: TArgs) => Promise<T>` - Wrap all the promise produced by an API using the scheduler. `scheduleFunction(callApi)`
+- `scheduleSequence(sequenceBuilders: SchedulerSequenceItem[]): { done: boolean; faulty: boolean }` - Schedule a sequence of operations. Each operation require the previous one to be resolved before being started. Each of the operations will be executed until its end before starting any other scheduled operation.
+- `count(): number` - Number of pending tasks waiting to be scheduled by the scheduler.
+- `waitOne: () => Promise<void>` - Wait one scheduled task to be executed. Throws if there is no more pending tasks.
+- `waitAll: () => Promise<void>` - Wait all scheduled tasks, including the ones that might be created by one of the resolved task. Do not use if `waitAll` call has to be wrapped into an helper function such as `act` that can relaunch new tasks afterwards. In this specific case use a `while` loop running while `count() !== 0` and calling `waitOne` - *see CodeSandbox example on userProfile*.
+
+With:
+```ts
+type SchedulerSequenceItem =
+    { builder: () => Promise<any>; label: string } |
+    (() => Promise<any>)
+;
+```

documentation/1-Guides/Tips.md

@@ -6,6 +6,7 @@ Simple tips to unlock all the power of fast-check with only few changes.
 
 - [Filter invalid combinations using pre-conditions](#filter-invalid-combinations-using-pre-conditions)
 - [Model based testing or UI test](#model-based-testing-or-ui-test)
+- [Detect race conditions](#detect-race-conditions)
 - [Opt for verbose failures](#opt-for-verbose-failures)
 - [Log within a predicate](#log-within-a-predicate)
 - [Preview generated values](#preview-generated-values)
@@ -129,6 +130,77 @@ The code above can easily be applied to other state machines, APIs or UI. In the
 
 **NOTE:** Contrary to other arbitraries, commands built using `fc.commands` requires an extra parameter for replay purposes. In addition of passing `{ seed, path }` to `fc.assert`, `fc.commands` must be called with `{ replayPath: string }`.
 
+## Detect race conditions
+
+Even if JavaScript is mostly a mono-threaded language, it is quite easy to introduce race conditions in your code.
+
+`fast-check` comes with a built-in feature accessible through `fc.scheduler` that will help you to detect such issues earlier during the development. It basically re-orders the execution of your promises or async tasks in order to make it crash under unexpected orderings.
+
+The best way to see it in action is certainly to check the snippets provided in our [CodeSandbox@005-race](https://codesandbox.io/s/github/dubzzz/fast-check/tree/master/example?hidenavigation=1&module=%2F005-race%2Fautocomplete%2Fmain.spec.tsx&previewwindow=tests).
+
+Here is a very simple React-based example that you can play with on [CodeSandbox](https://codesandbox.io/s/github/dubzzz/fast-check/tree/master/example?hidenavigation=1&module=%2F005-race%2FuserProfile%2Fmain.spec.tsx&previewwindow=tests):
+
+```jsx
+/* Component */
+
+import { getUserProfile } from './api.js'
+function UserPageProfile(props) {
+  const { userId } = props;
+  const [userData, setUserData] = React.useState(null);
+
+  React.useEffect(() => {
+    const fetchUser = async () => {
+      const data = await getUserProfile(props.userId);
+      setUserData(data);
+    };
+    fetchUser();
+  }, [userId]);
+
+  if (userData === null) {
+    return <div>Loading...</div>;
+  }
+  return (
+    <div>
+      <div data-testid="user-id">Id: {userData.id}</div>
+      <div data-testid="user-name">Name: {userData.name}</div>
+    </div>
+  );
+}
+
+/* Test with react testing library */
+
+test('should not display data related to another user', () =>
+  fc.assert(
+    fc.asyncProperty(
+      fc.array(fc.uuid(), fc.uuid(), fc.scheduler(),
+      async (uid1, uid2, s) => {
+        // Arrange
+        getUserProfile.mockImplementation(
+          s.scheduleFunction(async (userId) => ({ id: userId, name: userId })));
+
+        // Act
+        const { rerender, queryByTestId } = render(<UserProfilePage userId={uid1} />);
+        s.scheduleSequence([
+          async () => {
+            rerender(<UserProfilePage userId={uid2} />);
+          }
+        ]);
+        while (s.count() !== 0) {
+          await act(async () => {
+            await s.waitOne();
+          });
+        }
+
+        // Assert
+        expect((await queryByTestId('user-id')).textContent).toBe(`Id: ${uid2}`);
+      })
+      .beforeEach(async () => {
+        jest.resetAllMocks();
+        cleanup();
+      })
+  ));
+```
+
 ## Opt for verbose failures
 
 By default, the failures reported by `fast-check` feature most relevant data:

example/005-race/autocomplete/main.spec.tsx

@@ -0,0 +1,156 @@
+import fc from 'fast-check';
+import * as React from 'react';
+
+import AutocompleteField from './src/AutocompleteField';
+//import AutocompleteField from './src/AutocompleteFieldMostRecentQuery';
+//import AutocompleteField from './src/AutocompleteFieldSimple';
+
+import { render, cleanup, fireEvent, act, getNodeText } from '@testing-library/react';
+import '@testing-library/jest-dom/extend-expect';
+
+import { search } from './src/Api';
+
+// If you want to test the behaviour of fast-check in case of a bug
+// You can use the 'enableBug*' attribute passed to <AutocompleteField />
+
+if (!fc.readConfigureGlobal()) {
+  // Global config of Jest has been ignored, we will have a timeout after 5000ms
+  // (CodeSandbox falls in this category)
+  fc.configureGlobal({ interruptAfterTimeLimit: 4000 });
+}
+
+describe('AutocompleteField', () => {
+  it('should suggest results matching the value of the autocomplete field', () =>
+    fc.assert(
+      fc
+        .asyncProperty(AllResultsArbitrary, QueriesArbitrary, fc.scheduler(), async (allResults, queries, s) => {
+          // Arrange
+          const searchImplem: typeof search = s.scheduleFunction(function search(query, maxResults) {
+            return Promise.resolve(allResults.filter(r => r.includes(query)).slice(0, maxResults));
+          });
+
+          // Act
+          const { getByRole, queryAllByRole } = await renderAutoCompleteField(searchImplem);
+          const input = getByRole('input') as HTMLElement;
+          s.scheduleSequence(buildAutocompleteEvents(input, queries));
+
+          // Assert
+          while (s.count() !== 0) {
+            await act(async () => {
+              await s.waitOne();
+            });
+            const autocompletionValue = input.attributes.getNamedItem('value')!.value;
+            const suggestions = (queryAllByRole('listitem') as HTMLElement[]).map(getNodeText);
+            if (!suggestions.every(suggestion => suggestion.includes(autocompletionValue))) {
+              throw new Error(
+                `Invalid suggestions for ${JSON.stringify(autocompletionValue)}, got: ${JSON.stringify(suggestions)}`
+              );
+            }
+          }
+        })
+        .beforeEach(commonBeforeEach)
+    ));
+
+  it('should display more and more sugestions as results come', () =>
+    fc.assert(
+      fc
+        .asyncProperty(AllResultsArbitrary, QueriesArbitrary, fc.scheduler(), async (allResults, queries, s) => {
+          // Arrange
+          const query = queries[queries.length - 1];
+          const searchImplem: typeof search = s.scheduleFunction(function search(query, maxResults) {
+            return Promise.resolve(allResults.filter(r => r.includes(query)).slice(0, maxResults));
+          });
+
+          // Act
+          const { getByRole, queryAllByRole } = await renderAutoCompleteField(searchImplem);
+          const input = getByRole('input') as HTMLElement;
+          for (const event of buildAutocompleteEvents(input, queries)) {
+            await event.builder();
+          } // All the user's inputs have been fired onto the AutocompleField
+
+          // Assert
+          let suggestions: string[] = [];
+          while (s.count() !== 0) {
+            // Resolving one async query in a random order
+            await act(async () => {
+              await s.waitOne();
+            });
+
+            // Read suggestions shown by the component
+            const prevSuggestions = suggestions;
+            suggestions = (queryAllByRole('listitem') as HTMLElement[]).map(getNodeText);
+
+            // We expect the number of suggestions to increase up to the final number
+            // of suggestions for <query> or 10 (max number of suggestions)
+            if (suggestions.length < prevSuggestions.length) {
+              const got = JSON.stringify({
+                prevSuggestions,
+                suggestions
+              });
+              throw new Error(`We expect to have more and more suggestions as we resolve queries, got: ${got}`);
+            }
+          }
+          // At the end we expect to get results matching <query>
+          if (!suggestions.every(s => s.startsWith(query))) {
+            throw new Error(`Must start with ${JSON.stringify(query)}, got: ${JSON.stringify(suggestions)}`);
+          }
+        })
+        .beforeEach(commonBeforeEach)
+    ));
+});
+
+// Helpers
+
+const AllResultsArbitrary = fc.set(fc.string(), 0, 1000);
+const QueriesArbitrary = fc.array(fc.string(), 1, 10);
+
+/** beforeEach helper */
+const commonBeforeEach = async () => {
+  jest.resetAllMocks();
+  cleanup();
+};
+
+/** Render the `<AutocompleteField />` component */
+const renderAutoCompleteField = async (searchImplem: typeof search) => {
+  let getByRole: ReturnType<typeof render>['getByRole'] = null as any;
+  let queryAllByRole: ReturnType<typeof render>['queryAllByRole'] = null as any;
+
+  await act(async () => {
+    const wrapper = render(
+      <AutocompleteField
+        search={searchImplem}
+        // enableBugBetterResults={true}
+        // enableBugUnfilteredResults={true}
+        // enableBugUnrelatedResults={true}
+        // enableBugDoNotDiscardOldQueries={true}
+      />
+    );
+    getByRole = wrapper.getByRole;
+    queryAllByRole = wrapper.queryAllByRole;
+  });
+
+  return { getByRole, queryAllByRole };
+};
+
+/**
+ * Generate a sequence of events that have to be fired onto the component
+ * in order to send it all the queries (characters are fired one by one)
+ */
+const buildAutocompleteEvents = (input: HTMLElement, queries: string[]) => {
+  const autocompleteEvents: Exclude<fc.SchedulerSequenceItem, () => any>[] = [];
+
+  for (const query of queries) {
+    for (let numCharacters = 0; numCharacters <= query.length; ++numCharacters) {
+      const subQuery = query.substring(0, numCharacters);
+      const builder = async () => {
+        await act(async () => {
+          fireEvent.change(input, { target: { value: subQuery } });
+        });
+      };
+      const label = `typing(${JSON.stringify(subQuery)})`;
+      autocompleteEvents.push({ builder, label });
+    }
+  }
+
+  return autocompleteEvents;
+};

example/005-race/autocomplete/src/Api.ts

@@ -0,0 +1,3 @@
+export function search(query: string, maxResults: number): Promise<string[]> {
+  return Promise.resolve([]);
+}

example/005-race/autocomplete/src/AutocompleteField.tsx

@@ -0,0 +1,72 @@
+import React from 'react';
+
+// Injected as a props because CodeSandbox fails to provide jest.mock
+// So it makes such import difficult to test
+//// import { search } from './Api';
+
+type Props = {
+  enableBugUnrelatedResults?: boolean;
+  enableBugBetterResults?: boolean;
+  enableBugUnfilteredResults?: boolean;
+  search: (query: string, maxResults: number) => Promise<string[]>;
+};
+
+export default function AutocompleteField(props: Props) {
+  const lastQueryRef = React.useRef('');
+  const lastSuccessfulQueryRef = React.useRef('');
+  const [query, setQuery] = React.useState(lastQueryRef.current);
+  const [searchResults, setSearchResults] = React.useState([] as string[]);
+
+  React.useEffect(() => {
+    const runQuery = async () => {
+      const results = await props.search(query, 10);
+
+      if (!lastQueryRef.current.startsWith(query) && !props.enableBugUnrelatedResults) {
+        // FIXED BUG:
+        // We show results for queries that are unrelated to the latest started query
+        // eg.: AZ resolves while we look for QS, we show its results even if totally unrelated
+        return;
+      }
+      if (
+        lastQueryRef.current.startsWith(lastSuccessfulQueryRef.current) &&
+        lastSuccessfulQueryRef.current.length > query.length &&
+        !props.enableBugBetterResults
+      ) {
+        // FIXED BUG:
+        // We might update results while we already received results
+        // for a query less strict than the last this one
+        // eg.: We receice AZ while we already have results for AZE
+        return;
+      }
+
+      lastSuccessfulQueryRef.current = query;
+      setSearchResults(results);
+    };
+
+    runQuery();
+  }, [query, props]);
+
+  return (
+    <div>
+      <input
+        role="input"
+        value={query}
+        onChange={evt => {
+          const value = (evt.target as any).value;
+          lastQueryRef.current = value;
+          setQuery(value);
+        }}
+      />
+      <ul>
+        {searchResults
+          // FIXED BUG: We don't filter the results we receive
+          // As we want to display results as soon as possible, even if our searchResults
+          // are related to a past query we want to use them to provide the user with some hints
+          .filter(r => (props.enableBugUnfilteredResults ? true : r.startsWith(query)))
+          .map(r => (
+            <li key={r}>{r}</li>
+          ))}
+      </ul>
+    </div>
+  );
+}