`@mediapipe/selfie_segmentation` npm package can be bundled by Rollup, but raises an error in runtime. #2883

sile · 2021-12-13T08:34:54Z

First of all, thank you for publishing this great library.
I'm new to this library and TypeScript, so not sure that this is actually a bug.
Please let me know if you have any misunderstandings 🙏

System information (Please provide as much relevant information as possible)

Have I written custom code: Yes
OS Platform and Distribution: mac OS Monterey 12.0.1
Browser and version: Google Chrome 96.0.4664.93
Programming Language and version: TypeScript v4.5.3
MediaPipe version: @mediapipe/selfie_segmentatoin-v0.1.1632777926
Solution: Selfie Segmentation

Describe the current behavior:

Failed to instantiate the SelfieSegmentation class from a JavaScript file bundled by Rollup.

The screenshot below shows the error message (please refer to the following Standalone code to reproduce the issue section about what is the context of index.html):

Describe the expected behavior:

Be able to instantiate SelfieSegmentatoin class without errors (the Hello World! message below is emitted after the instantiation).

Standalone code to reproduce the issue:

I created a small example package, that provides a class only instantiating SelfieSegmentation, to reproduce this issue.

// Files.
$ ls
index.html index.ts package.json rollup.config.js

// Build this package.
$ npm install
$ npm run build

// Open `index.html` on a browser (Chrome in my case).
// The result has been shown at the first screenshot.
$ open index.html

index.html:

<html>
  <head>
    <meta charset="utf-8">
    <title>Example</title>
  </head>
  <body>
    <script src="bundle.js"></script>
    <script>
      const selfie = new window.Example.SelfieSegmentationWrapper();
      console.log("Hello World!");
    </script>
  </body>
</html>

index.ts:

import {SelfieSegmentation} from "@mediapipe/selfie_segmentation";

class SelfieSegmentationWrapper {
    private inner: SelfieSegmentation;

    constructor() {
        this.inner = new SelfieSegmentation({
            locateFile: (file: string) => {
                return `"https://cdn.jsdelivr.net/npm/@mediapipe/selfie_segmentation/${file}`;
            },
        });
        this.inner.setOptions({
            modelSelection: 1,
        });
    }
}

export { SelfieSegmentationWrapper };

package.json:

{
  "name": "example",
  "version": "0.0.0",
  "scripts": {
    "build": "rollup -c ./rollup.config.js"
  },
  "devDependencies": {
    "@rollup/plugin-commonjs": "^21.0.1",
    "@rollup/plugin-node-resolve": "^13.0.6",
    "@rollup/plugin-typescript": "^8.3.0",
    "rollup": "^2.61.1",
    "typescript": "^4.5.3"
  },
  "dependencies": {
    "@mediapipe/selfie_segmentation": "^0.1.1632777926"
  }
}

rollup.config.js:

import typescript from '@rollup/plugin-typescript';
import commonjs from '@rollup/plugin-commonjs';
import resolve from '@rollup/plugin-node-resolve';

export default {
    input: 'index.ts',
    plugins: [
        typescript(),
        commonjs(),
        resolve()
    ],
    output: {
        sourcemap: false,
        file: 'bundle.js',
        format: 'umd',
        name: 'Example',
    }
};

Other info / Complete Logs :

I don't exactly know why it works but if the line exports.SelfieSegmentation = SelfieSegmentation; is appended to selfie_segmentation.js, the problem is disappeared:

$ echo 'exports.SelfieSegmentation = SelfieSegmentation;' >> node_modules/@mediapipe/selfie_segmentation/selfie_segmentation.js
$ npm run build

// Now, the "Hello World!" message is printed as shown in the second screenshot.
$ open index.html

The text was updated successfully, but these errors were encountered:

englishtom · 2021-12-29T01:32:00Z

Just adding that I am experiencing the same issue with the @mediapipe/hands package. Using the package installed via npm works fine but if I compile it with Rollup then I am given TypeError: JT.Hands is not a constructor.

You can see the same issue with the CDN version of the package using this standalone script:

<html>
<head>
    <script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/hand-pose-detection"></script>
    <script src="https://cdn.jsdelivr.net/npm/@mediapipe/hands"></script>
</head>

<body>
    <img id="myImg" src="https://victordibia.com/handtrack.js/images/samples/0.jpg"/>

    <script>
        const img = document.getElementById('myImg');
        const model = handPoseDetection.SupportedModels.MediaPipeHands;
        const detectorConfig = {
            runtime: 'mediapipe',
            modelType: 'full',
            solutionPath: 'https://cdn.jsdelivr.net/npm/@mediapipe/hands@0.4.1635986972',
        };

        const detector = handPoseDetection.createDetector(model, detectorConfig).then(() => {
            console.log('Ready to detect', detector);

            detector.estimateHands(img).then((hands) => {
                console.log('Got results!', hands);  
            });
        });

    </script>
</body>
</html>

tyrmullen · 2022-01-20T02:34:45Z

@englishtom Does @sile's workaround of adding exports.Hands = Hands work for you?

I'm not familiar with Rollup myself, so I don't really have any insights to offer here.

englishtom · 2022-01-20T10:24:04Z

@tyrmullen It does not, however, using the Hands library directly without TensorFlow works well for me so that is the approach I have moved forwards with.

utileetdulce · 2022-02-07T11:28:12Z

This might help to understand the issue:

Rollup CommonJS plugin can't work with obfuscated packages because it relies on static analysis to determine its exports.

Using obfuscated packages: Uncaught (in promise) TypeError: {......} is not a constructor vitejs/vite#4680 (comment)

holtwick · 2022-03-07T10:03:55Z

By @bluwy :

Note: The core issue of @mediapipe/selfie_segmentation is that exports a non-ESM file in pkg.module, which should be ESM. I'd suggest contacting the package author to fix it, but for now I've documented the difference of dependencies handled in dev and build that might help in debugging the issue. vitejs/vite#4680 (comment)

Does someone have a workaround for this issue?

sile · 2022-03-07T10:27:21Z

Just FYI, I implemented a custom rollup plugin that appends the line exports.SelfieSegmentation = SelfieSegmentation; to the selfie_segmentation.js during the bundle phase as our workaround: https://github.com/shiguredo/media-processors/blob/develop/packages/virtual-background/rollup.config.js#L73

holtwick · 2022-03-07T10:48:22Z

Just FYI, I implemented a custom rollup plugin that appends the line exports.SelfieSegmentation = SelfieSegmentation; to the selfie_segmentation.js during the bundle phase as our workaround: https://github.com/shiguredo/media-processors/blob/develop/packages/virtual-background/rollup.config.js#L73

Awesome, thanks so much. This works for me!

I use this plugin now in my vite.config.ts under build.rollupOptions.plugins.

mattrossman · 2022-04-02T21:53:24Z

Just FYI, I implemented a custom rollup plugin that appends the line exports.SelfieSegmentation = SelfieSegmentation; to the selfie_segmentation.js during the bundle phase as our workaround: https://github.com/shiguredo/media-processors/blob/develop/packages/virtual-background/rollup.config.js#L73

Thanks for sharing. In my case the error was related to MediaPipe Facemesh, so I adjusted the plugin as follows:

function mediapipe_workaround() {
  return {
    name: "mediapipe_workaround",
    load(id) {
      if (path.basename(id) === "face_mesh.js") {
        let code = fs.readFileSync(id, "utf-8")
        code += "exports.FaceMesh = FaceMesh;"
        return { code }
      } else {
        return null
      }
    },
  }
}

google-ai-edge/mediapipe#2883

kuaashish · 2023-04-19T08:57:59Z

Hello @sile,
We are upgrading the MediaPipe Legacy Solutions to new MediaPipe solutions However, the libraries, documentation, and source code for all the MediaPipe Legacy Solutions will continue to be available in our GitHub repository and through library distribution services, such as Maven and NPM.

You can continue to use those legacy solutions in your applications if you choose. Though, we would request you to check new MediaPipe solutions which can help you more easily build and customize ML solutions for your applications. These new solutions will provide a superset of capabilities available in the legacy solutions.

sile · 2023-04-20T02:46:10Z

@kuaashish Thank you for letting me know about the new solutions. I tried using the Image Segmenter (with label=person) for selfie segmentation purposes. It was able to build without the error shared by this issue.
However, I decided to continue using the legacy selfie segmentation package as the segmentation mask generated by the Image Segmenter seemed too rough (and it took longer wall clock time to process each frame).
I will consider switching to the new solutions once an alternative for the legacy selfie segmentation becomes available.

kuaashish · 2023-04-21T12:45:45Z

@sile,
Thank you checking out our new solutions. Others legacy solutions source code will be available in the Github if you choose.
Further, We need your confirmation to move issue to close if this is no longer issue from your end.

sile · 2023-04-21T13:08:36Z

@kuaashish I see. It's okay to close this issue 👍

spencer17x · 2023-06-20T15:41:31Z

Maybe you can use vite-plugin-mediapipe：https://github.com/Spencer17x/arca/tree/main/packages/vite-plugin/vite-plugin-mediapipe

Bersaelor · 2023-08-10T20:50:28Z

Just FYI, I implemented a custom rollup plugin that appends the line exports.SelfieSegmentation = SelfieSegmentation; to the selfie_segmentation.js during the bundle phase as our workaround: https://github.com/shiguredo/media-processors/blob/develop/packages/virtual-background/rollup.config.js#L73

Thanks for sharing. In my case the error was related to MediaPipe Facemesh, so I adjusted the plugin as follows:
function mediapipe_workaround() {
  return {
    name: "mediapipe_workaround",
    load(id) {
      if (path.basename(id) === "face_mesh.js") {
        let code = fs.readFileSync(id, "utf-8")
        code += "exports.FaceMesh = FaceMesh;"
        return { code }
      } else {
        return null
      }
    },
  }
}

Hey @mattrossman , I'm encountering the exact same issue, trying to bundle mediapipe/facemesh into a vite built application ( also Ik.FaceMesh is not a constructor )

The link you quoted doesn't go anywhere anymore, do you happen to still have a full vite plugin that the method you wrote goes into?

mattrossman · 2023-08-25T13:33:57Z

@Bersaelor That mediapipe_workaround method in the quoted message is the full plugin, you call it within vite's build.rollupOptions.plugins.

For posterity, here's a permalink to the original by @sile that I based it on (it looks like they just renamed the file extension which broke that other link). And here's where it's used in the config.

sile added the type:bug Bug in the Source Code of MediaPipe Solution label Dec 13, 2021

google-ml-butler bot assigned sgowroji Dec 13, 2021

sgowroji assigned tyrmullen and unassigned sgowroji Dec 13, 2021

tyrmullen assigned mhays-google and unassigned tyrmullen Jan 20, 2022

eyemono-moe added a commit to eyemono-moe/svg-face that referenced this issue Jul 20, 2022

viteでのビルド後に動かなくなる問題を修正

9db14e3

google-ai-edge/mediapipe#2883

dionjwa added a commit to metapages/metaframe-hands that referenced this issue Oct 18, 2022

Fix build issue google-ai-edge/mediapipe#2883

d20db34

This was referenced Feb 13, 2023

Objectron javascript library doesn't work after building the app #4086

Closed

TypeError: hands.Hands is not a constructor #3796

Closed

sapphi-red mentioned this issue Feb 24, 2023

Vite build is prepending undefined variables to some deps in production /dist/index.js file vitejs/vite#12188

Closed

7 tasks

rmajasol mentioned this issue Mar 2, 2023

Vite build is prepending undefined variables to @mediapipe deps in production /dist/index.js file #4120

Closed

kuaashish assigned kuaashish and unassigned mhays-google Apr 19, 2023

kuaashish removed the stat:awaiting googler Waiting for Google Engineer's Response label Apr 19, 2023

kuaashish added the stat:awaiting response Waiting for user response label Apr 19, 2023

google-ml-butler bot removed the stat:awaiting response Waiting for user response label Apr 20, 2023

kuaashish added the stat:awaiting response Waiting for user response label Apr 21, 2023

google-ml-butler bot removed the stat:awaiting response Waiting for user response label Apr 21, 2023

kuaashish closed this as completed Apr 21, 2023

gaikwadrahul8 mentioned this issue Jul 28, 2023

SelfieSegmentation is not a constructor tensorflow/tfjs#7165

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`@mediapipe/selfie_segmentation` npm package can be bundled by Rollup, but raises an error in runtime. #2883

`@mediapipe/selfie_segmentation` npm package can be bundled by Rollup, but raises an error in runtime. #2883

sile commented Dec 13, 2021 •

edited

Loading

englishtom commented Dec 29, 2021 •

edited

Loading

tyrmullen commented Jan 20, 2022

englishtom commented Jan 20, 2022

utileetdulce commented Feb 7, 2022

holtwick commented Mar 7, 2022

sile commented Mar 7, 2022

holtwick commented Mar 7, 2022

mattrossman commented Apr 2, 2022

kuaashish commented Apr 19, 2023

sile commented Apr 20, 2023 •

edited

Loading

kuaashish commented Apr 21, 2023

sile commented Apr 21, 2023

spencer17x commented Jun 20, 2023

Bersaelor commented Aug 10, 2023

mattrossman commented Aug 25, 2023 •

edited

Loading

@mediapipe/selfie_segmentation npm package can be bundled by Rollup, but raises an error in runtime. #2883

@mediapipe/selfie_segmentation npm package can be bundled by Rollup, but raises an error in runtime. #2883

Comments

sile commented Dec 13, 2021 • edited Loading

englishtom commented Dec 29, 2021 • edited Loading

tyrmullen commented Jan 20, 2022

englishtom commented Jan 20, 2022

utileetdulce commented Feb 7, 2022

holtwick commented Mar 7, 2022

sile commented Mar 7, 2022

holtwick commented Mar 7, 2022

mattrossman commented Apr 2, 2022

kuaashish commented Apr 19, 2023

sile commented Apr 20, 2023 • edited Loading

kuaashish commented Apr 21, 2023

sile commented Apr 21, 2023

spencer17x commented Jun 20, 2023

Bersaelor commented Aug 10, 2023

mattrossman commented Aug 25, 2023 • edited Loading

`@mediapipe/selfie_segmentation` npm package can be bundled by Rollup, but raises an error in runtime. #2883

`@mediapipe/selfie_segmentation` npm package can be bundled by Rollup, but raises an error in runtime. #2883

sile commented Dec 13, 2021 •

edited

Loading

englishtom commented Dec 29, 2021 •

edited

Loading

sile commented Apr 20, 2023 •

edited

Loading

mattrossman commented Aug 25, 2023 •

edited

Loading