Not Ok for storage

[cramhead]

I saw that @BraveKenny was successful in getting images stored and retrieved in "Best/Correct way to store images in public dirrectory and get their url in the callback?"
That's awesome!

Unfortunately, when I attempt to add the to the user's profile.profileImage I get

"Meteor.makeErrorType.errorClass {error: 409, reason: "MongoError: not okForStorage", details: undefined, message: "MongoError: not okForStorage [409]", errorType: "Meteor.Error"…}details: undefinederror: 409errorType: "Meteor.Error"message: "MongoError: not okForStorage [409]"

My fileObj look like the following. It has references to dataman and such in there. A bit more complex then I expected. I did make sure to add cfs-ejson-file

I made sure the user has a profile and profile.profileImage property.

'change .fileUploader': function(event, template) {
      FS.Utility.eachFile(event, function(file) {
        var uId = Meteor.userId();
        if (uId) {
          var fileObj = Images.insert(file);
          Meteor.users.update({
            _id: uId
          }, {
            $set: {
              'profile.profileImage': fileObj
            }
          }, function(err, result) {
            if (err) {
              console.log("Error updating user. " + err.detail);
            } else {
              console.log('set profileImage');
            }
          });
        }
      });
    }

I'm I missing something here?

Thanks
Marc

[aldeed]

I don't know for sure if it is the problem, but given that that is client code, you will want to update Meteor.users in an insert callback. insert requires a callback on the client. I don't remember whether it actually returns an FS.File instance on the client. fileObj might be undefined.

[aldeed]

Also, in some circumstances you can get this error with pre-2.6 MongoDB. See meteor/meteor#1890

[cramhead]

Interesting.
So I changed it to using a callback, but it has the same error.
I'm using the default mongo instance that comes with meteor.

I'm wondering it it could be related to bson_ext. Genghisapp complains about it, it is installed, but I'm not really sure how to add it to my load path.

Other then that I can't think of anything that could be problematic.

[aldeed]

Just a thought: did you meteor add cfs-ejson-file? That is, if you stop the app and do meteor list --using, does it show cfs-ejson-file? Sometimes I add it to smart.json directly and forget to do meteor add, which might result in something like this error.

[cramhead]

@aldeed Good idea. I had made sure to add it, but it was not in my smart.json file.
I remove it, added to smart.json and mrt add cfs-ejson-file. Unfortunately, the same error.

My meteor list --using is:
standard-app-packages
autopublish
insecure
stylus
collectionFS
accounts-base
accounts-password
accounts-ui
cfs-filesystem
cfs-ejson-file
cfs-s3

Also, I tried referencing the git master branch of cfs-ejson-file, but no joy.

[cramhead]

Well, I tried switching from my local mongo instance to mongoHQ, but unfortunately I get

W20140708-15:37:02.557(-7)? (STDERR)
W20140708-15:37:02.558(-7)? (STDERR) stream.js:94
W20140708-15:37:02.559(-7)? (STDERR)       throw er; // Unhandled stream error in pipe.
W20140708-15:37:02.559(-7)? (STDERR)             ^
W20140708-15:37:02.559(-7)? (STDERR) Error: ENOENT, open '/Users/cram/Projects/METEOR/nestedImages/.meteor/local/cfs/files/_tempstore/images-2ByQpxqvYoQ6vLQCb-0.chunk'
=> Exited with code: 8

I did a meteor reset as wiped the cfs folder in .meteor/local, but still get the error.

[copleykj]

Personally I would store a reference to the record or the URL. If you store
the fileObj in the users collection you're duplicating a large amount of
data and losing all the benefits of the transform on the CFS collection
when you preform a find. Just my 2 satoshi though.
On Jul 8, 2014 6:50 PM, "Marc" notifications@github.com wrote:

Well, I tried switching from my local mongo instance to mongoHQ, but
unfortunately I get

W20140708-15:37:02.557(-7)? (STDERR)
W20140708-15:37:02.558(-7)? (STDERR) stream.js:94
W20140708-15:37:02.559(-7)? (STDERR) throw er; // Unhandled stream error in pipe.
W20140708-15:37:02.559(-7)? (STDERR) ^
W20140708-15:37:02.559(-7)? (STDERR) Error: ENOENT, open '/Users/cram/Projects/METEOR/nestedImages/.meteor/local/cfs/files/_tempstore/images-2ByQpxqvYoQ6vLQCb-0.chunk'
=> Exited with code: 8

I did a meteor reset as wiped the cfs folder in .meteor/local, but still
get the error.

—
Reply to this email directly or view it on GitHub
#356 (comment)
.

[aldeed]

@copleykj, it actually does not store a lot of data. The cfs-ejson-file pkg adds a custom EJSON type for FS.File so when you store one, it actually stores only a simple pointer and when you retrieve it later Meteor converts that pointer back into the full FS.File.

[copleykj]

oh, wow I did not know that.. That's pretty awesome!

On Tue, Jul 8, 2014 at 8:35 PM, Eric Dobbertin notifications@github.com
wrote:

@copleykj https://github.com/copleykj, it actually does not store a lot
of data. The cfs-ejson-file pkg adds a custom EJSON type for FS.File so
when you store one, it actually stores only a simple pointer and when you
retrieve it later Meteor converts that pointer back into the full FS.File.

—
Reply to this email directly or view it on GitHub
#356 (comment)
.

[cramhead]

So I think got to a better error message now.

I20140708-20:13:41.381(-7)? Exception while invoking method '/cfs.images.filerecord/insert' Error: Only documents (plain objects) may be inserted into MongoDB
I20140708-20:13:41.382(-7)?     at Object.Future.wait (/Users/cram/.meteor/tools/858c88b520/lib/node_modules/fibers/future.js:323:16)
I20140708-20:13:41.382(-7)?     at null.<anonymous> (packages/meteor/helpers.js:111)
I20140708-20:13:41.382(-7)?     at MongoConnection.(anonymous function) [as insert] (packages/mongo-livedata/mongo_driver.js:607)
I20140708-20:13:41.382(-7)?     at Meteor.Collection._validatedInsert (packages/mongo-livedata/collection.js:794)
I20140708-20:13:41.382(-7)?     at m.(anonymous function) (packages/mongo-livedata/collection.js:704)
I20140708-20:13:41.383(-7)?     at maybeAuditArgumentChecks (packages/livedata/livedata_server.js:1487)
I20140708-20:13:41.383(-7)?     at packages/livedata/livedata_server.js:643
I20140708-20:13:41.383(-7)?     at _.extend.withValue (packages/meteor/dynamics_nodejs.js:56)
I20140708-20:13:41.383(-7)?     at packages/livedata/livedata_server.js:642
I20140708-20:13:41.384(-7)?     at _.extend.withValue (packages/meteor/dynamics_nodejs.js:56)
I20140708-20:13:41.384(-7)?     - - - - -
I20140708-20:13:41.384(-7)?     at MongoConnection._insert (packages/mongo-livedata/mongo_driver.js:322)
I20140708-20:13:41.384(-7)?     at null.<anonymous> (packages/meteor/helpers.js:109)
I20140708-20:13:41.384(-7)?     at MongoConnection.(anonymous function) [as insert] (packages/mongo-livedata/mongo_driver.js:607)
I20140708-20:13:41.385(-7)?     at Meteor.Collection._validatedInsert (packages/mongo-livedata/collection.js:794)
I20140708-20:13:41.385(-7)?     at m.(anonymous function) (packages/mongo-livedata/collection.js:704)
I20140708-20:13:41.385(-7)?     at maybeAuditArgumentChecks (packages/livedata/livedata_server.js:1487)
I20140708-20:13:41.385(-7)?     at packages/livedata/livedata_server.js:643
I20140708-20:13:41.385(-7)?     at _.extend.withValue (packages/meteor/dynamics_nodejs.js:56)
I20140708-20:13:41.386(-7)?     at packages/livedata/livedata_server.js:642
I20140708-20:13:41.386(-7)?     at _.extend.withValue (packages/meteor/dynamics_nodejs.js:56)

I'm wondering if this is something new in Meteor 0.8.2.

Also, I took a closer look at the fileObj that I'm trying to save. I don't know how it is not an ordinary JavaScript object, but I did notice there are closures in the controlledByDeps and attachData properties.

[raix]

So it's complaning about storing into the filerecord, should investigate further - maybe try rigging a filerecordClone function on that insert making sure it's pure data + investigate the change in meteor. The error message is new, "Error: Only documents (plain objects)..."

[cramhead]

@raix Thanks for the tips.
I tried with fileObj.clone(), fileObj.clone({full:true}) file.clone({full: false}) and even EJSON.clone(fileObj).
They all fail. Using EJSON.stringify(fileObj) does save the escaped fileObj in correctly, but it's not to useful for anything but making sure I'm not doing something super bone headed.
Also, I tried calling delete on $type, type, $value, value, but I expect that they are made after the call to update.
I have made a git repo. (https://github.com/cramhead/nestedImage2). If anyone feels they have a moment to try it out, please do.

[raix]

@aldeed do you remember the change made by @glasser - Cant remember it other than the error description and a reference to cfs

[cramhead]

So I tried @aldeed minimal sample. It outputs the same result in pre and post meteor update.
I expect that the fix has not made it into Meteor yet.

[[[[[ ~/Projects/METEOR/ejson-test ]]]]]

=> Started proxy.
=> Meteor 0.8.2 is available. Update this project with 'meteor update'.
=> Started MongoDB.
=> Started your app.

=> App running at: http://localhost:3000/
I20140709-11:49:12.229(-7)? { [MongoError: not okForStorage]
I20140709-11:49:12.760(-7)?   name: 'MongoError',
I20140709-11:49:12.760(-7)?   err: 'not okForStorage',
I20140709-11:49:12.760(-7)?   code: 12527,
I20140709-11:49:12.760(-7)?   n: 0,
I20140709-11:49:12.761(-7)?   lastOp: { _bsontype: 'Timestamp', low_: 0, high_: 0 },
I20140709-11:49:12.761(-7)?   connectionId: 9,
I20140709-11:49:12.761(-7)?   ok: 1 }
^CLava:ejson-test cram$ meteor update
ejson-test: updated to Meteor 0.8.2.

-- Notice --

0.8.0: Meteor has a new live templating engine: Blaze!
       See https://github.com/meteor/meteor/wiki/Using-Blaze for what's new.

0.8.1: We closed a security hole in our OAuth client. If you are using
       OAuth-based accounts (such as the `accounts-google` or
       `accounts-twitter` packages), we recommend that you log out
       all your users by running this command from a MongoDB shell:

         $ db.users.update({}, { $set: { 'services.resume.loginTokens':
           [] } }, { multi: true });

Lava:ejson-test cram$ mrt

Stand back while Meteorite does its thing

Done installing smart packages

Ok, everything's ready. Here comes Meteor!

[[[[[ ~/Projects/METEOR/ejson-test ]]]]]

=> Started proxy.
=> Started MongoDB.
=> Started your app.

=> App running at: http://localhost:3000/
I20140709-11:50:13.078(-7)? { [MongoError: not okForStorage]
I20140709-11:50:13.418(-7)?   name: 'MongoError',
I20140709-11:50:13.419(-7)?   err: 'not okForStorage',
I20140709-11:50:13.419(-7)?   code: 12527,
I20140709-11:50:13.419(-7)?   n: 0,
I20140709-11:50:13.419(-7)?   lastOp: { _bsontype: 'Timestamp', low_: 0, high_: 0 },
I20140709-11:50:13.419(-7)?   connectionId: 7,
I20140709-11:50:13.419(-7)?   ok: 1 }

[cramhead]

So I'm I correct that no one is successfully storing the ejson reference to a cfs file in their projects? Is everyone use the fileObj._id?

[aldeed]

I have successfully done it, but I think it might depend on what mongo operator you try to use or something. Like $push might work and not $set, or not within a sub object (ie, not when there's a dot). I don't remember the exact details and I'm not able to test it out right now. I also know that you can't insert an FS.File itself, as the full doc, but it does not seem like you are doing that, although that's what the error message sounds like.

[cramhead]

@aldeed Indeed, you are correct. I changed the code to using $push. I did add the ejson reference as a sub-document. Yippee!

As for the insertion of full doc, I'm not to sure what you mean. I've tried the inserting the result of the callback form, e.g. Images.insert(fsFile, function(err, fileObj) {... and var fileObj = Images.insert(fsFile). The current version I have posted on github uses var cloney = fileObj.clone(); and attempts to update the Meteor.users collection by setting profile.profileImage to cloney.

[aldeed]

By inserting a full doc, I mean NormalCollection.insert(fsFile). I don't know why you would do that, but I remember it coming up for some reason before.

I guess we will have to figure out guidelines for storing ejson refs until the bugs are fixed and all operators are supported.

[raniafekry]

I have Meteor 0.8.2 , I did add collectionFS , mrt add collectionFS , it is listed under mrt list
but I keep getting this error and can't start my application with the example code
ReferenceError: FS is not defined
W20140729-11:28:50.960(-7)? (STDERR) at app/server/orientation.js:36:18
W20140729-11:28:50.961(-7)? (STDERR) at app/server/orientation.js:46:3

which I have this code server/orientatin.js
var Images = new FS.Collection("images", {
stores: [new FS.Store.FileSystem("images", {path: "~/uploads"})]
});

and the following code in client/orientation.js
var Images = new FS.Collection("images", {
stores: [new FS.Store.FileSystem("images", {path: "~/uploads"})]
});
Template.myForm.events({
'change .myFileInput': function(event, template) {
var files = event.target.files;
for (var i = 0, ln = files.length; i < ln; i++) {
Images.insert(files[i], function (err, fileObj) {
//Inserted new doc with ID fileObj._id, and kicked off the data upload using HTTP
});
}
}
});

I did mrt update , same result ,

I appreciate your help

Thanks

Rania

[aldeed]

@raniafekry , see if this fixes it:

$ meteor add collectionFS

Use meteor, not mrt.

As long as it shows up when you do meteor list --using and it's listed in smart.json and you do mrt update, FS should be defined.

[raniafekry]

Thanks Eric , it works great with meteor not mrt

I was able to have the delete and download button working , can I set
button for view ..which open uploaded text file in another window browser
or in the same page

Thanks

Rania

On Tue, Jul 29, 2014 at 12:02 PM, Eric Dobbertin notifications@github.com
wrote:

@raniafekry https://github.com/raniafekry , see if this fixes it:

$ meteor add collectionFS

Use meteor, not mrt.

As long as it shows up when you do meteor list --using and it's listed in
smart.json and you do mrt update, FS should be defined.

—
Reply to this email directly or view it on GitHub
#356 (comment)
.

[hkirk]

Anyone knows why $push works and $set doesn't?

[aldeed]

I don't really understand why, but I think it's fixed by this commit, which isn't until mongo version 2.5.2. Meteor locally uses 2.4.9. Someone could try running against a DB running on 2.5.2+ to see if $set works.

[gauravtiwari]

Hi @aldeed I am getting the same error when using Collection-FS. It works for new inserts (or create) but not for update. Any clue on how to fix this?

Meteor.users.update({_id: current_user}, { $set: { profile: {coverphoto: fileObj}}});

P.S. - I am trying to update the fileOBJ reference in the users collections after FileObj is created (insert(file)).

[nooitaf]

@gauravtiwari just a wild guess.... maybe it has to be treated like queries?

 Meteor.users.update({_id: current_user}, { $set: { 'profile.coverphoto': fileObj } } );

[gauravtiwari]

@nooitaf Ralf, tried it but same result.

[aldeed]

@gauravtiwari, is that with Meteor's built-in mongo? I don't think there is any solution other than to wait until Meteor uses MongoDB 2.5.2+ by default or to connect to your own mongoDB instance that is on 2.5.2+.

[gauravtiwari]

@aldeed Hi Eric, Yes it's the one built-in. Will it work with the local mongo instance - 2.6.3? I just tried connecting to my local instance using mongo_url but seems like a lot of functions from CFS-upload-http isn't working. $setoninsert (file upload functions).

Thanks for looking into this.

[aldeed]

I seem to recall that there are a number of known issues running Meteor against 2.6+. Are you able to test against 2.5.2 instead? I will not have time to test this out for another week or so at least.

[danii1]

My guess - it's a problem with ejson serializing, field names should not cointain $ sign:
http://docs.mongodb.org/manual/reference/limits/#naming-restrictions
When we serializing fileObj reference, we getting fields like EJSON$type, EJSON$value. Can someone rename this fields and check if $set works after that?

[danii1]

Ok, after digging around I come to this meteor/meteor#1890 it's definitely mongodb+ejson issue with $ in the middle of field names and not collectionfs problem, 2.5.2 should work. Argh, so annoying, I'm using Meteor Up for deployment and it installs stable 2.4 branch on server setup.

[aldeed]

This should be working now that Meteor is using 2.6 by default.