Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bun test #1825

Open
Electroid opened this issue Jan 18, 2023 · 97 comments
Open

bun test #1825

Electroid opened this issue Jan 18, 2023 · 97 comments
Labels
bun:test Something related to the `bun test` runner tracking An umbrella issue for tracking big features

Comments

@Electroid
Copy link
Contributor

Electroid commented Jan 18, 2023

Jest

describe()

test()

Lifecycle hooks

expect()

Mocks

Misc

Vitest

expect()

Mocks

Timers

Misc

jest-extended

expect()

@Electroid Electroid added tracking An umbrella issue for tracking big features bun:test Something related to the `bun test` runner labels Jan 18, 2023
@Electroid Electroid changed the title Promote bun wiptest to bun test bun test Jan 18, 2023
@Electroid Electroid mentioned this issue Jan 18, 2023
83 tasks
@useafterfree
Copy link

ts-jest does some magical hoisting with mock* functions which may or may not be a challenge with bun:
swc-project/swc#5448 (comment)

@nskins
Copy link
Contributor

nskins commented Mar 11, 2023

This issue can be updated to check off these four (they were implemented in PR #1573):

  • .toBeGreaterThan(number | bigint)
  • .toBeGreaterThanOrEqual(number | bigint)
  • .toBeLessThan(number | bigint)
  • .toBeLessThanOrEqual(number | bigint)

@erikshestopal
Copy link

Should jest.mock be added as well? Doesn't seem to be tracked.

@Jarred-Sumner
Copy link
Collaborator

@erikshestopal please give it a try in the canary build, a lot of it is in there

@brapifra
Copy link

brapifra commented Jun 27, 2023

jest.clearAllMocks() is also missing 👀 🙏

image

@robobun robobun changed the title bun test bun test [Converted to "bun test" project] Jul 6, 2023
@robobun robobun changed the title bun test [Converted to "bun test" project] bun test Jul 6, 2023
@itsezc
Copy link

itsezc commented Jul 7, 2023

Would there be any scope to add DOM matchers? If so, I'd love for some parity with the jest-dom library.

@Hanaasagi
Copy link
Collaborator

2023-07-07_10-56

toThrowError is missing. It's a alias for toThrow.

Ref: https://jestjs.io/docs/expect#tothrowerror

@riywo
Copy link

riywo commented Jul 10, 2023

test.each and describe.each are super helpful to build data-driven unit tests.

@simylein
Copy link
Contributor

I am proposing adding https://vitest.dev/api/expect-typeof.html to the vitest list.

@ghiscoding
Copy link

test.each and describe.each are super helpful to build data-driven unit tests.

It's being worked on in PR #4047

@dsanchezjt
Copy link

dsanchezjt commented Aug 10, 2023

Are there plans to add the expect.toBeInTheDocument matcher from @testing-library/jest-dom that many typically use for JSX component testing? Or is that out of scope? Feels like that would be something useful to provide out of the box.

@OleksandrKucherenko
Copy link

OleksandrKucherenko commented Aug 17, 2023

expect.objectContaining

any ideas how to replace this call in tests?

Solution / Workaround

    expect(Object.fromEntries(data.statistics[Kpi.codes])).toHaveProperty(`502`, 1)
    expect(Object.fromEntries(data.statistics[Kpi.codes])).toHaveProperty(`200`, 32)

// instead of:

    expect(Object.fromEntries(data.statistics[Kpi.codes])).toStrictEqual(
      expect.objectContaining({ '200': 32, '502': 1 })
    )

@OleksandrKucherenko
Copy link

OleksandrKucherenko commented Aug 29, 2023

What can you recommend as a replacement for such jest code?

await expect(async () => await askGPT(text, prompt, context)).rejects.toThrowError(`⛔ API error.`)

@simylein
Copy link
Contributor

simylein commented Aug 29, 2023

Shouldn't that be working?

Does this suit your needs?

import { describe, expect, test } from 'bun:test';

const askGPT = async (text: string, prompt: string, context: string): Promise<void> => {
	throw new Error('⛔ API error.');
};

describe('', () => {
	test('', async () => {
		await expect(async () => await askGPT('text', 'prompt', 'context')).toThrow(Error(`⛔ API error.`));
	});
});

@jakeboone02
Copy link
Contributor

@fjpedrosa this should be reported in the jest-dom repo

@sebastianbuechler
Copy link

@sebastianbuechler did you manage to make it work without Typescript complaining? To me is working after exending jest-dom matchers, but seems not to extend the type and Typescript outputs: Property 'toBeInTheDocument' does not exist on type 'Matchers<HTMLElement>'

@fjpedrosa & @jakeboone02 Just half way. I wanted to define it in a setup function so that I don't have to repeat it, but it seems that typescript does not recognize it that way.

I abandoned bun test as it seems just not mature enough if you have already a lot of tests working with vitest and DOM expectations. Maybe better documentation would help here.

@cpt-westphalen
Copy link

cpt-westphalen commented Apr 8, 2024

@sebastianbuechler did you manage to make it work without Typescript complaining? To me is working after exending jest-dom matchers, but seems not to extend the type and Typescript outputs: Property 'toBeInTheDocument' does not exist on type 'Matchers<HTMLElement>'

@fjpedrosa & @jakeboone02 Just half way. I wanted to define it in a setup function so that I don't have to repeat it, but it seems that typescript does not recognize it that way.

I abandoned bun test as it seems just not mature enough if you have already a lot of tests working with vitest and DOM expectations. Maybe better documentation would help here.

I did it. It works.
It took me about five hours.

I'm not experienced at configuring stuff, as you may notice by it, but I kept thinking: if someone's smart enough to code the thing I should be at least smart enough to find out how to use it, lol.

Anyway, for anyone who falls here and needs to know how to configure bun with happy-dom and jest-dom with TypeScript and working code completion, that is how I did it in Vite for working with React:

How to configure Bun + Jest-DOM + Happy DOM in Vite (TypeScript React)

  1. Add dependencies with bun add:

    • bun add -D happy-dom @happy-dom/global-registrator @testing-library/jest-dom @types/web
  2. Add this key to the compilerOptions in tsconfig.json file:

"types": [
      "bun-types", // add Bun global
      "@testing-library/react", // if with react
      "@testing-library/jest-dom",
      "web"
    ],
  1. Create a happydom.ts file inside your src folder with the following (this will be pre-loaded so it works in every test file):
import { GlobalRegistrator } from "@happy-dom/global-registrator";

const oldConsole = console;
GlobalRegistrator.register();
window.console = oldConsole;

import * as matchers from "@testing-library/jest-dom/matchers";
import { expect } from "bun:test";

// Extend the expect object with custom matchers
expect.extend(matchers);

It is ugly, I know. It works, so I'm not touching it again so soon.

  1. Add this to the bunfig.toml file (or create a file named bunfig.toml at the root level of the project, next to package.json):
[test]
preload = "./src/happydom.ts"

If you want to place the happydom file somewhere else, just make sure it is being parsed by TypeScript.

  1. Create a file in your src folder with whatever name and the .d.ts extension (jest-dom-types.d.ts, for example) and this code:
import "@testing-library/jest-dom";
import type { TestingLibraryMatchers } from "@testing-library/jest-dom/matchers";

declare module "bun:test" {
  interface Matchers<R = void>
    extends TestingLibraryMatchers<
      typeof expect.stringContaining,
      R
    > {}
  interface AsymmetricMatchers
    extends TestingLibraryMatchers {}
}
  1. Import expect as you normally would in your test file:

import { describe, expect, it } from "bun:test";

  1. Have blazing fast tests with Testing-Library
    image

@jakeboone02
Copy link
Contributor

@cpt-westphalen Step 6 shouldn't be necessary to do manually. Those types should come out-of-the-box with the library. The same PR that fixed the types for expect.extend generally also added types for bun:test.

But again, this is not the place to hash out these issues. Any bugs or gaps in @testing-library/jest-dom should be filed and discussed in that repo and not this one.

@cpt-westphalen
Copy link

Sorry for flooding here with that. I may replace it with the link to the discussion in jest-dom if you prefer.

@fjpedrosa
Copy link

thanks @cpt-westphalen, I was able to fix the warning with your solution!

@graynk
Copy link

graynk commented Apr 17, 2024

Not sure if I'm missing something, but #5356 is still happening for me on 1.1.3

@paulleonartcalvo
Copy link

Anyone here get the actual matchers working but run into an issue where jest matcher utils seems to not be working? I specifically get the following when failing a test using expect().toBeInTheDocument():

TypeError: this.utils.RECEIVED_COLOR is not a function. (In 'this.utils.RECEIVED_COLOR(this.isNot ? errorFound() : errorNotFound())', 'this.utils.RECEIVED_COLOR' is undefined)

@immayurpanchal
Copy link

Anyone here get the actual matchers working but run into an issue where jest matcher utils seems to not be working? I specifically get the following when failing a test using expect().toBeInTheDocument():

TypeError: this.utils.RECEIVED_COLOR is not a function. (In 'this.utils.RECEIVED_COLOR(this.isNot ? errorFound() : errorNotFound())', 'this.utils.RECEIVED_COLOR' is undefined)

I was facing the same issue where getAllByText() was returning me > 1 items from DOM so used the first item from DOM and .getAllByText()[0] then ran expect().toBeIntheDocument();

@hussain-nz
Copy link

hussain-nz commented May 10, 2024

#1825 (comment)

This is great thank you! I think this is the perfect place to add these instructions. They are extremely necessary to make bun test work with extra matchers that most people are already using and are migrating to bun test now.

@cpt-westphalen I got an error while running a test, any ideas?
Figured out the issue, the error below occurs because I was importing the main library, this is not required when using bun test. i.e. I just had to delete this import statement: import '@testing-library/jest-dom';

bun test v1.1.7 (b0b7db5c)

src\TestFile.test.tsx:
 5 | import 'aria-query';
 6 | import 'chalk';
 7 | import 'lodash/isEqualWith.js';
 8 | import 'css.escape';
 9 |
10 | expect.extend(extensions);
     ^
ReferenceError: Can't find variable: expect
      at C:\Code\frontend\node_modules\@testing-library\jest-dom\dist\index.mjs:10:1

@srosato
Copy link

srosato commented May 13, 2024

Anyone here get the actual matchers working but run into an issue where jest matcher utils seems to not be working? I specifically get the following when failing a test using expect().toBeInTheDocument():

TypeError: this.utils.RECEIVED_COLOR is not a function. (In 'this.utils.RECEIVED_COLOR(this.isNot ? errorFound() : errorNotFound())', 'this.utils.RECEIVED_COLOR' is undefined)

I was facing the same issue where getAllByText() was returning me > 1 items from DOM so used the first item from DOM and .getAllByText()[0] then ran expect().toBeIntheDocument();

On my end I was also getting the error 'Multiple elements found' and I added this to my happydom.ts preload file

beforeEach(() => {
  document.body.innerHTML = '';
})

I still run into the same error as @paulleonartcalvo with TypeError: this.utils.RECEIVED_COLOR is not a function.. Have yet to find how to resolve this one. Have you guys found a way?

@alioshr
Copy link

alioshr commented May 20, 2024

But toMatchInlineSnapshot is the only missing API that blocks me from using bun test.

Is it much harder to implement than toMatchSnapshot ?

For me it is very important to have jest.requireActual() as well as toMatchInlineSnapshot.

With these two I guess that I could plan a PoC at work.

Do you have a roadmap / expectations to have these two done?

@rbwestphalen
Copy link

@paulleonartcalvo @immayurpanchal @hussain-s6 @srosato everyone that has the TypeError: this.utils.RECEIVED_COLOR is not a function... I think this is a 'bad error', it only happened to me when there were actually a problem with the test, like finding more than one element with the criteria on a getBy* or finding an element on a .not.toBeInTheDocument assertion. I've always managed to make the test work after messing around debugging it... do you have a case where you are absolutely sure it should be working but is throwing that error?

@srosato
Copy link

srosato commented May 24, 2024

@rbwestphalen My tests that fail with this error have no problem passing with jest and jsdom. I will try to dig more into it when I get a chance

@Kleywalker
Copy link

Kleywalker commented May 27, 2024

Temporary implementation for toHaveBeenCalledBefore and toHaveBeenCalledAfter.
(inspired by jest-extended)
It would be great if this could be native! 😊

== testSetup.ts ==

import { expect as bunExpect, mock as originalMock } from 'bun:test';

// Define the type for the original mock function
type OriginalMockFunction = (...args: unknown[]) => unknown;

// Define the type for our extended mock function
type ExtendedMockFunction = OriginalMockFunction & {
  callOrder: number[];
  originalMock: OriginalMockFunction;
};

// Create an extended mock function
function createMock(): ExtendedMockFunction {
  const original = originalMock() as OriginalMockFunction;
  const mockFunction = ((...args: unknown[]) => {
    mockFunction.callOrder.push(++createMock.callOrderCounter);
    return original(...args);
  }) as ExtendedMockFunction;

  mockFunction.callOrder = [] as number[];
  mockFunction.originalMock = original;

  return mockFunction;
}

createMock.callOrderCounter = 0;

// Custom matchers
function toHaveBeenCalledBefore(
  this: { utils: any },
  received: ExtendedMockFunction,
  secondMock: ExtendedMockFunction,
) {
  const firstCallOrder = received.callOrder[0];
  const secondCallOrder = secondMock.callOrder[0];

  if (firstCallOrder < secondCallOrder) {
    return {
      pass: true,
      message: () =>
        `Expected ${received.originalMock.name} to have been called before ${secondMock.originalMock.name}`,
    };
  } else {
    return {
      pass: false,
      message: () =>
        `Expected ${received.originalMock.name} to have been called before ${secondMock.originalMock.name}, but it was called after`,
    };
  }
}

function toHaveBeenCalledAfter(
  this: { utils: any },
  received: ExtendedMockFunction,
  firstMock: ExtendedMockFunction,
) {
  const firstCallOrder = firstMock.callOrder[0];
  const secondCallOrder = received.callOrder[0];

  if (secondCallOrder > firstCallOrder) {
    return {
      pass: true,
      message: () =>
        `Expected ${received.originalMock.name} to have been called after ${firstMock.originalMock.name}`,
    };
  } else {
    return {
      pass: false,
      message: () =>
        `Expected ${received.originalMock.name} to have been called after ${firstMock.originalMock.name}, but it was called before`,
    };
  }
}

// Custom matchers interface
interface CustomMatchers<T = unknown, R = void> {
  toHaveBeenCalledAfter(firstMock: T): R;
  toHaveBeenCalledBefore(secondMock: T): R;
}

// Ensure the custom matchers interface extends the Bun matchers interface with consistent type parameters
declare module 'bun:test' {
  interface Matchers<T = unknown, R = void> extends CustomMatchers<T, R> {}
}

// Add custom matchers to expect
const expectWithMatchers = bunExpect as typeof bunExpect & {
  extend: (
    matchers: Record<
      string,
      (
        this: { utils: any },
        ...args: any[]
      ) => { pass: boolean; message: () => string }
    >,
  ) => void;
};

// Add custom matchers to expect
expectWithMatchers.extend({ toHaveBeenCalledBefore, toHaveBeenCalledAfter });

// Override the mock function in bun:test
const bunTest = require('bun:test');
bunTest.mock = createMock;

== order.test.ts ==

import './testSetup';

import { describe, expect, it, mock } from 'bun:test';

describe('Function Call Order Tests', () => {
  it('should call initialize before process', () => {
    const initialize = mock();
    const process = mock();

    // Simulate the function calls
    initialize();
    process();

    // Verify the call order
    expect(initialize).toHaveBeenCalledBefore(process);
  });

  it('should call finalize after process', () => {
    const process = mock();
    const finalize = mock();

    // Simulate the function calls
    process();
    finalize();

    // Verify the call order
    expect(finalize).toHaveBeenCalledAfter(process);
  });

  it('should call fetchData before processData', () => {
    const fetchData = mock();
    const processData = mock();

    // Simulate the function calls
    fetchData();
    processData();

    // Verify the call order
    expect(fetchData).toHaveBeenCalledBefore(processData);
  });

  it('should call saveData after processData', () => {
    const processData = mock();
    const saveData = mock();

    // Simulate the function calls
    processData();
    saveData();

    // Verify the call order
    expect(saveData).toHaveBeenCalledAfter(processData);
  });

  it('should call setup before execute and then cleanup', () => {
    const setup = mock();
    const execute = mock();
    const cleanup = mock();

    // Simulate the function calls
    setup();
    execute();
    cleanup();

    // Verify the call order
    expect(setup).toHaveBeenCalledBefore(execute);
    expect(execute).toHaveBeenCalledBefore(cleanup);
  });
});

== Preload ==
bun test --preload ./setupTests.ts
Preloading should work as well but I only tested the code above with direct import.

@ejini6969
Copy link

ejini6969 commented Jul 1, 2024

  • Seems like toContainAnyKeys has been implemented, but yet to be ticked in the checkbox above.
  • .toBeBetween(startDate, endDate) has not been implemented too, but it is ticked above.

@donaldpipowitch
Copy link

Don't know where/how to report this, but as Bun is focused so much on performance and the docs have statements like "Running 266 React SSR tests faster than Jest can print its version number." I can only imagine I run into some problem/bug.

I was finally able to run tests of an existing project in Bun instead of Jest and Bun takes ~36.1s while Jest needs ~12.8s. It's a bit hard to create a minimal example, but I use happydom and Testing Library. Is this result expected? What are realistic numbers?

@Jarred-Sumner
Copy link
Collaborator

@donaldpipowitch That is definitely not expected. Would love to be able to profile your test suite and see what's going on.

If you're unable to send code for us to look at, if you're on a mac you could try:

  1. Download the -profile build of Bun, e.g. https://github.com/oven-sh/bun/releases/download/bun-v1.1.18/bun-darwin-aarch64-profile.zip
  2. Run xcrun xctrace record --template 'Time Profiler' --launch -- ./bun-profile test
  3. DM me the .trace file on Discord or email jarred@bun.sh

Instruments comes with Xcode. It'll look something like this:
image

@donaldpipowitch
Copy link

Thanks for the quick response. As this is likely a bug then (or bigger misconfiguration from my side) I'll try to create a minimal test case.

@donaldpipowitch
Copy link

@Jarred-Sumner Stupid question, but could it simply be the case that bun is not parallelizing the tests? I tried to break it down, but it becomes less obvious if you run single tests (for single tests bun is a bit faster).

  • whole test suite
    • bun: 36.03s user 1.79s system 104% cpu 36.080 total
    • jest: 67.84s user 6.81s system 583% cpu 12.785 total
  • individual test
    • bun: 1465.93ms
    • jest: 1799 ms

@Jarred-Sumner
Copy link
Collaborator

bun does run all tests single-threaded, but often the cost of parallelizing tests is so high that it doesn't end up being a performance win to use threads

is it using timers? we haven't implemented fake timers yet, and IIRC, the testing library code does something like wait 500 milliseconds of IRL wall clock time in certain cases when there's no jest fake timers

@donaldpipowitch
Copy link

We don't run jest.useFakeTimers() in our code base (not sure if there is another way to use timers or if this is a default behavior by now). (I also commented out two usages of setTimeout, but same result.) Will create the .trace now as it looks like the minimal repo will not really show the problem.

@donaldpipowitch
Copy link

@Jarred-Sumner , sorry to bother you, I was just curious if you got the .trace file? (Please don't feel pressured. I just want to check, if it reached you.)

@kelvinwop
Copy link

dang... no toBeInTheDocument... what the heck bun...

@coolsoftwaretyler
Copy link

Hey folks, just a follow up on my comment from a few months ago. One of our contributors figured out the differences between bun test and jest, specifically:

AFAICT, bun test maintains globals across modules (unlike jest), so you have to be careful to reset it.

Which was breaking assumptions we had from running Jest and trying to use bun:test as a fully drop in replacement.

I think this was mostly consumer error on our end, I don't think Bun needs to change, but I thought it might be helpful to leave some breadcrumbs here in case you're also running into issues swapping bun:test in for a Jest set up that made assumptions similar to ours.

But with this change, we're finally realizing 20x improvements on test suite speed. Phenomenal. Huge. So worth it.

@donaldpipowitch
Copy link

I was finally able to run tests of an existing project in Bun instead of Jest and Bun takes ~36.1s while Jest needs ~12.8s. It's a bit hard to create a minimal example, but I use happydom and Testing Library. Is this result expected? What are realistic numbers?

Re-tested it with 1.1.22 and get the same result.

@dlindenkreuz
Copy link

https://vitest.dev/api/expect-typeof.html

To clarify @simylein's request — this is about type checker tests for TypeScript, not typeof tests: https://vitest.dev/guide/testing-types.html

Haven't seen it added to the master list yet, so here I am raising my hand as well 👋

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bun:test Something related to the `bun test` runner tracking An umbrella issue for tracking big features
Projects
None yet
Development

No branches or pull requests