Skip to content
This repository has been archived by the owner on Mar 14, 2019. It is now read-only.

GridFS: 5593bfe857f397093508f1c5 does not exist #731

Open
neopostmodern opened this issue Jul 1, 2015 · 24 comments
Open

GridFS: 5593bfe857f397093508f1c5 does not exist #731

neopostmodern opened this issue Jul 1, 2015 · 24 comments

Comments

@neopostmodern
Copy link

I've hacked together some code that uploads GIFs, decomposes them into frames and eventually inserts all of them into GridFS. It works.

But there is one specific GIF for which it fails:

(STDERR) 
(STDERR) /home/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1dgxmmd++os.linux.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/connection/base.js:246
(STDERR)         throw message;      
(STDERR)               ^
(STDERR) Error: 5593bfe857f397093508f1c5 does not exist
(STDERR)     at /home/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1dgxmmd++os.linux.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/gridfs/gridstore.js:200:22
(STDERR)     at /home/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1dgxmmd++os.linux.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/collection/query.js:164:5
(STDERR)     at /home/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1dgxmmd++os.linux.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/cursor.js:778:35
(STDERR)     at Cursor.close (/home/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1dgxmmd++os.linux.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/cursor.js:1009:5)
(STDERR)     at Cursor.nextObject (/home/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1dgxmmd++os.linux.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/cursor.js:778:17)
(STDERR)     at commandHandler (/home/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1dgxmmd++os.linux.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/cursor.js:741:14)
(STDERR)     at /home/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1dgxmmd++os.linux.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/db.js:1903:9
(STDERR)     at Server.Base._callHandler (/home/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1dgxmmd++os.linux.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/connection/base.js:453:41)
(STDERR)     at /home/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1dgxmmd++os.linux.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/connection/server.js:488:18
(STDERR)     at [object Object].MongoReply.parseBody (/home/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1dgxmmd++os.linux.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/responses/mongo_reply.js:68:5)
=> Exited with code: 8

Please notice that the 5593bfe857f397093508f1c5 is the exact same on my development machine and the meteor test server, referenced above.

Upload code:

Gifs.insert file, (error, fileObj) ->
# Inserted new doc with ID fileObj._id, and kicked off the data upload using HTTP

    if error?
        Materialize.toast error, 4000, "red"
        state.set 'status', 'default'
        return

    uploadTracker = Tracker.autorun ->
        reactiveFileObj = Gifs.findOne fileObj._id

        if reactiveFileObj.isUploaded()
            uploadTracker.stop()

            Meteor.call 'multichannel.initialize-project', { fileId: fileObj._id }, (error, result) ->
                if error?
                    Materialize.toast error, 4000, "red"
                    state.set 'status', 'default'
                    return

                Materialize.toast "Successfully created project", 2000
                Router.go 'home'
'multichannel.initialize-project': (options) ->
    if not @userId?
        throw new Meteor.Error 401, "Please log in to create a multichannel"

    gif = Gifs.findOne options.fileId

    @unblock()

    Projects.insert(
        name: gif.name()
        originalGif: gif._id
        frames: []
        userId: @userId
        likes: 0
    , (error, projectId) =>
        if error?
            console.dir error
            return

        Meteor.users.update _id: @userId,
            $push:
                projects: projectId

        frameIndex = 0

        GifExplode.Do gif.createReadStream('gifs'), (frame) ->
            _frameIndex = frameIndex
            frameIndex += 1
            fsFrame = new FS.File()
            fsFrame.attachData(frame, { type: 'image/gif' })
            fsFrame.name(Meteor.uuid() + ".gif")
            Gifs.insert(
                    fsFrame
            , (error, frameObject) ->
                if error?
                    console.dir error
                    return

                Projects.update(_id: projectId,
                    $push:
                        frames:
                            $each: [
                                gif: frameObject._id
                                index: _frameIndex
                            ]
                            $sort: index: 1
                            $slice: -50 
                , (error) ->
                    if error?
                        console.dir error
                        return
                )
            )
    )

The frame in the callback is a stream.

Or see the entire source code.
@jimmiebtlr
Copy link

I believe I'm experiencing the same issue, but with a different string.

Error: 55945c2b6389174f15989a5b does not exist

Which occurs fairly rarely on upload of an image, and crashes the meteor server.

@neopostmodern
Copy link
Author

This seems to be a duplicate of #408

@neopostmodern neopostmodern mentioned this issue Jul 14, 2015
Closed
@fishdude
Copy link

fishdude commented Sep 9, 2015

Getting the same thing here.

 Error: 55efccefe3653fa204dd0996 does not exist

@adamgins
Copy link

adamgins commented Oct 7, 2015

Hi, looks I have something similar:

I just started running my app on Galaxy and I am seeing the error below when uploading an image.

I am using Collections FS with S3

2015-10-08 06:22:16+11:00about to mark all notifications read for:PpEu7cHpEShno2zEC
ab79
2015-10-08 06:22:46+11:00 at [object Object].MongoReply.parseBody (/app/bundle/programs/server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/responses/mongo_reply.js:68:5)
ab79
2015-10-08 06:22:46+11:00 at /app/bundle/programs/server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/connection/server.js:488:18
ab79
2015-10-08 06:22:46+11:00 at Server.Base._callHandler (/app/bundle/programs/server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/connection/base.js:453:41)
ab79
2015-10-08 06:22:46+11:00 at /app/bundle/programs/server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/db.js:1903:9
ab79
2015-10-08 06:22:46+11:00 at commandHandler (/app/bundle/programs/server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/cursor.js:741:14)
ab79
2015-10-08 06:22:46+11:00 at Cursor.nextObject (/app/bundle/programs/server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/cursor.js:778:17)
ab79
2015-10-08 06:22:46+11:00 at Cursor.close (/app/bundle/programs/server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/cursor.js:1009:5)
ab79
2015-10-08 06:22:46+11:00 at /app/bundle/programs/server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/cursor.js:778:35
ab79
2015-10-08 06:22:46+11:00 at /app/bundle/programs/server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/collection/query.js:164:5
ab79
2015-10-08 06:22:46+11:00 at /app/bundle/programs/server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/gridfs/gridstore.js:200:22
ab79
2015-10-08 06:22:46+11:00Error: 56157104bd99051c008fc328 does not exist
ab79
2015-10-08 06:22:46+11:00Application exited with code: 7 signal: null
ab79
2015-10-08 06:22:46+11:00Application process closed with code: 7 signal: null

My app currently runs OK on on AWS Opsworks/Docker... so this is just happening on Galaxy.

@vioan
Copy link

vioan commented Oct 26, 2015

I've got a similar error for my app:

W20151026-18:11:16.942(1)? (STDERR)
W20151026-18:11:16.946(1)? (STDERR) /Users/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1juhiel++os.osx.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/connection/base.js:246
W20151026-18:11:16.946(1)? (STDERR)         throw message;
W20151026-18:11:16.946(1)? (STDERR)               ^
W20151026-18:11:16.950(1)? (STDERR) Error: 562e5eb4411fe142a6c59e47 does not exist
W20151026-18:11:16.951(1)? (STDERR)     at /Users/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1juhiel++os.osx.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/gridfs/gridstore.js:200:22
W20151026-18:11:16.951(1)? (STDERR)     at /Users/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1juhiel++os.osx.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/collection/query.js:164:5
W20151026-18:11:16.951(1)? (STDERR)     at /Users/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1juhiel++os.osx.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/cursor.js:778:35
W20151026-18:11:16.952(1)? (STDERR)     at Cursor.close (/Users/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1juhiel++os.osx.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/cursor.js:1009:5)
W20151026-18:11:16.954(1)? (STDERR)     at Cursor.nextObject (/Users/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1juhiel++os.osx.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/cursor.js:778:17)
W20151026-18:11:16.955(1)? (STDERR)     at commandHandler (/Users/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1juhiel++os.osx.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/cursor.js:741:14)
W20151026-18:11:16.955(1)? (STDERR)     at /Users/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1juhiel++os.osx.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/db.js:1903:9
W20151026-18:11:16.957(1)? (STDERR)     at Server.Base._callHandler (/Users/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1juhiel++os.osx.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/connection/base.js:453:41)
W20151026-18:11:16.958(1)? (STDERR)     at /Users/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1juhiel++os.osx.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/connection/server.js:488:18
W20151026-18:11:16.958(1)? (STDERR)     at [object Object].MongoReply.parseBody (/Users/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1juhiel++os.osx.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/responses/mongo_reply.js:68:5)

@dnish
Copy link

dnish commented Oct 29, 2015

Having the same error, whole server crashes:

[server-crash] 56324147e575535820228f38 does not exist

@dnish dnish mentioned this issue Oct 29, 2015
Open
@griv
Copy link

griv commented Nov 20, 2015

Having the same error here (local Mac dev):
Error: 564e67463289527a61dc4262 does not exist

@tagrudev
Copy link
Contributor

Same thing in here

App 29314 stderr: Error: 564ecf6ab3f9228272e2f31a does not exist
App 29314 stderr:     at //[...]//server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/gridfs/gridstore.js:200:22
App 29314 stderr:     at //[...]//server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/collection/query.js:164:5
App 29314 stderr:     at //[...]//server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/cursor.js:778:35
App 29314 stderr:     at Cursor.close (//[...]//npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/cursor.js:1009:5)
App 29314 stderr:     at Cursor.nextObject (//[...]//server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/cursor.js:778:17)
App 29314 stderr:     at commandHandler (//[...]//server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/cursor.js:741:14)
App 29314 stderr:     at //[...]//server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/db.js:1903:9
App 29314 stderr:     at Server.Base._callHandler (//[...]//server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/connection/base.js:453:41)
App 29314 stderr:     at //[...]//server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/connection/server.js:488:18
App 29314 stderr:     at [object Object].MongoReply.parseBody (/[...]/server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/responses/mongo_reply.js:68:5)

It restarts the server, which is a big pain, I've searched all of the collections and couldn't find this id.

I wonder where is it coming from ?

@tagrudev
Copy link
Contributor

That's kind of a big problem since it restarts the server anyone ?

@dnish
Copy link

dnish commented Nov 26, 2015

@tagrudev Clear all temp collections of CollectionFS

@tagrudev
Copy link
Contributor

@dnish you mean all temp mongo collections ? If so I've already tried that, it's not working

@dnish
Copy link

dnish commented Nov 26, 2015

Yeah all ones that are related to CollectionFS like chunks or the temp ones.

@tagrudev
Copy link
Contributor

I've already tried cleaning the collectons, but that doesn't solve the issue (as I said I've tried finding the id that the collectionFS is trying to access and it's nowhere to be found), which is kind of strange - I think the problem comes from CollectionFS trying to access unfinished / failed upload file that's been chunked

@vigile
Copy link

vigile commented Dec 5, 2015

Have the same problem!

@louis49
Copy link

louis49 commented Jan 12, 2016

This fix works : Eliga@86939dc

louis49 pushed a commit to louis49/Meteor-CollectionFS that referenced this issue Jan 12, 2016
Fixing issue Meteor-Community-Packages#731 and Meteor-Community-Packa…
a12d2b1 
…ges#408
@louis49 louis49 mentioned this issue Jan 12, 2016
Closed
@kjami kjami mentioned this issue Jan 14, 2016
Open
@jackkav
Copy link

jackkav commented Jan 21, 2016

@louis49 Your suggested fix and your own fix do not appear to be related, care to explain?

@louis49
Copy link

louis49 commented Jan 21, 2016

Neither, after using some time I realize that this does not work 100% of the time in very specific conditions:
Multinode
"Sticky-session" not activated

After applying other unpublished corrections there are some bug (but not crash)

We base our tests on a bulk of 1000 files < 6 MB

@jackkav
Copy link

jackkav commented Jan 21, 2016

Thanks for the speedy reply, it would appear the temp store doesn't handle multiple application servers accessing it particularly well. In our implementation we are using CollectionFS to store things on the local server filesystem and manage the access. We have been hitting this heavily with tens of thousands of files 10-100MB. The problem doesn't seem to occur when the app and db are on the same server and we only run one app server. I haven't tested it yet but I'm hoping the welelay fix could prevent additional application server from causing the tempstore to crash collectionFS.
I also noticed using the FS store still copied each file into the temp store on the db, 13GB in our case. This is problematic when the db is not on the same physical machine, but if the tempstore could be switched off, with welelay's fix, perhaps this won't be an issue.

@louis49
Copy link

louis49 commented Jan 21, 2016

OK, i found it!
I found a working solution, the last bug were finally on a other package of our project.
I check all and put the final correction tomorrow.

@ChrisCinelli
Copy link
Contributor

@jackkav If "The problem doesn't seem to occur when the app and db are on the same server and we only run one app server." sounds like a race condition.

@hwillson
Copy link

Hi all - Just a quick FYI; I just had this problem as well and clearing the cfs._tempstore.chunks collection fixed it for me:

console> db['cfs._tempstore.chunks'].remove({})

@ScyDev
Copy link

ScyDev commented May 12, 2016

In our case it was actually the upload and storing of huge images that caused this problem. An image of 2.5M and roughly 4000x3000px was too much.

@dnish
Copy link

dnish commented May 12, 2016

@hwillson Yeah, we did this too, but the error occured just a few days later again.

@tobitech
Copy link

This was working fine in my Meteor app until I adopted the new project structure in the Meteor guide and I started getting this error - 59c7a6c4ed1108551a335157 does not exist.
Please does any one have a solution to this? could it be because my collection definition is inside imports/api folder?

@nztqa nztqa mentioned this issue Nov 28, 2017
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests