DownloadUserData not working

[jcatarino]

Description:

Downloading user data (created because of GDPR) is not working. No errors show up on the log. No email is received.

Steps to reproduce:

1. Go to "A Minha Conta"
2. Click "Transferir os meus dados"

Expected behavior:

I would expect to get an email in the next few days of clicking the button.

Actual behavior:

nothing happens, no email, no data, no errors.

Server Setup Information:

• Version of Rocket.Chat Server: 0.74.3
• Operating System: Ubuntu
• Deployment Method: snap
• Number of Running Instances: 1
• DB Replicaset Oplog: Enabled
• NodeJS Version: 8.11.3 - x64
• MongoDB Version:

Additional context

Relevant logs:

�[34mI20190416-17:30:00.723(0) rocketchat:logger server.js:199 SyncedCron ➔ info Starting "Generate download files for user data".
�[34mI20190416-17:30:00.893(0) rocketchat:logger server.js:199 SyncedCron ➔ info Not running "Generate download files for user data" again.
�[34mI20190416-17:30:00.934(0) rocketchat:logger server.js:199 SyncedCron ➔ info Finished "Generate download files for user data".
�[34mI20190416-17:30:00.950(0) rocketchat:logger server.js:199 SyncedCron ➔ info Not running "Generate download files for user data" again.

[jcatarino]

Could someone check if it is just me? Thanks

[jcatarino]

Similar to #11082

[ghost]

The same problem exists on my server.

[jcatarino]

Data Download doesn't work on open.rocket.chat either. This needs urgent fix.

[Hudell]

We detected that the open.rocket.chat server was not processing the files fast enough to keep up with the demand. It's working, but it has a huge list of other files to process first. We are changing the server settings to process a higher number of files in order to export everything pending and keep up with the demand from now on.

[reetp]

On v1.3.2

I asked for my data, chose Export My Data (JSON) and a day later got this:

https://chat.myserver.info/ufs/FileSystem:UserDataFiles/4Cv78QJQsSfZQkYd4/2019-09-12-Joe%2520Blogs.zip

Click the link and get File not found

Same as #11082

Now testing on HTML but guess the answer will be the same.

FWIW I can see the admin/User Data Download section shows it is enabled, but the System Path is empty. So I presume the system is trying to save it to whatever the default path is, but there is no clue what it is, or what the alternative might be.

Why doesn't it use the File Upload path or something sensible?

And there is nothing in the documentation to explain any of it.

Add it to the list on:
RocketChat/feature-requests#135

This really is poor, and unacceptable.

[reetp]

Possibly related. Shouldn't it be mkdir -p ??

SyncedCron ➔ info Starting "Generate download files for user data".
{ Error: ENOENT: no such file or directory, mkdir '/tmp/userData/hYHbjXWQMSRMu2szG/full'
    at Object.fs.mkdirSync (fs.js:885:18)
    at createDir (app/user-data-download/server/cronProcessDownloads.js:36:6)
    at continueExportingRoom (app/user-data-download/server/cronProcessDownloads.js:189:2)
    at exportOperation.roomList.forEach.exportOpRoomData (app/user-data-download/server/cronProcessDownloads.js:419:5)
    at Array.forEach (<anonymous>)
    at continueExportOperation (app/user-data-download/server/cronProcessDownloads.js:418:29)
    at cursor.forEach.exportOperation (app/user-data-download/server/cronProcessDownloads.js:468:3)
    at SynchronousCursor.forEach (packages/mongo/mongo_driver.js:1107:16)
    at Cursor.(anonymous function) [as forEach] (packages/mongo/mongo_driver.js:887:44)
    at Object.processDataDownloads [as job] (app/user-data-download/server/cronProcessDownloads.js:463:9)
    at packages/littledata_synced-cron.js:251:26
    at scheduleTimeout (packages/littledata_synced-cron.js:304:9)
    at packages/littledata_synced-cron.js:356:42
    at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1304:12)
    at packages/meteor.js:620:25
    at runWithEnvironment (packages/meteor.js:1356:24)
  errno: -2,
  code: 'ENOENT',
  syscall: 'mkdir',
  path: '/tmp/userData/hYHbjXWQMSRMu2szG/full' }
SyncedCron ➔ info Finished "Generate download files for user data".

root@test:/app/bundle# mkdir /tmp/userData/hYHbjXWQMSRMu2szG/full
mkdir: cannot create directory '/tmp/userData/hYHbjXWQMSRMu2szG/full': No such file or directory

root@test:/app/bundle# mkdir -p /tmp/userData/hYHbjXWQMSRMu2szG/full

[jwermers]

Possibly related. Shouldn't it be mkdir -p ??

SyncedCron ➔ info Starting "Generate download files for user data".
{ Error: ENOENT: no such file or directory, mkdir '/tmp/userData/hYHbjXWQMSRMu2szG/full'
    at Object.fs.mkdirSync (fs.js:885:18)
    at createDir (app/user-data-download/server/cronProcessDownloads.js:36:6)
    at continueExportingRoom (app/user-data-download/server/cronProcessDownloads.js:189:2)
    at exportOperation.roomList.forEach.exportOpRoomData (app/user-data-download/server/cronProcessDownloads.js:419:5)
    at Array.forEach (<anonymous>)
    at continueExportOperation (app/user-data-download/server/cronProcessDownloads.js:418:29)
    at cursor.forEach.exportOperation (app/user-data-download/server/cronProcessDownloads.js:468:3)
    at SynchronousCursor.forEach (packages/mongo/mongo_driver.js:1107:16)
    at Cursor.(anonymous function) [as forEach] (packages/mongo/mongo_driver.js:887:44)
    at Object.processDataDownloads [as job] (app/user-data-download/server/cronProcessDownloads.js:463:9)
    at packages/littledata_synced-cron.js:251:26
    at scheduleTimeout (packages/littledata_synced-cron.js:304:9)
    at packages/littledata_synced-cron.js:356:42
    at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1304:12)
    at packages/meteor.js:620:25
    at runWithEnvironment (packages/meteor.js:1356:24)
  errno: -2,
  code: 'ENOENT',
  syscall: 'mkdir',
  path: '/tmp/userData/hYHbjXWQMSRMu2szG/full' }
SyncedCron ➔ info Finished "Generate download files for user data".

root@test:/app/bundle# mkdir /tmp/userData/hYHbjXWQMSRMu2szG/full
mkdir: cannot create directory '/tmp/userData/hYHbjXWQMSRMu2szG/full': No such file or directory

root@test:/app/bundle# mkdir -p /tmp/userData/hYHbjXWQMSRMu2szG/full

Possibly fs.mkdirSync needs to be recursive "fs.mkdirSync(targetDir, { recursive: true })"

[engelgabriel]

@Hudell can you take a look at this and the issue?

[engelgabriel]

Looks like adding the Possibly fs.mkdirSync needs to be recursive "fs.mkdirSync(targetDir, { recursive: true })" would really fix this.