Migrating-from-slack mmctl onboarding doc fails

[glennschler]

Summary

Following the instructions Migrating from Slack Using the Mattermost mmetl Tool and Bulk Import fail if attachments are included in slack export.

Steps to reproduce

Version: MM Version 6.4.1 with mysql self hosted
Fresh install, with one new team including an admin user

• Data files attached to use in steps to reproduce, and steps to workaround

  • mattermost-bulk-import.zip
  • mattermost-bulk-import.workaround.zip

• The steps below reference an export zip file which was prepared from a Slack export of a single day, of only a single message including a single image attachment. The Slack export file is not attached to this issue. Instead, attached to this issue is the mattermost-bulk-import.zip file created during step 4 of the document

  • If not using the prepared mattermost-bulk-import.zip, then use your own slack export from a single day which includes an image attachment, and follow the official mm onboarding document. Then ignore all the steps below. The import will fail.

• The steps to reproduce jump ahead to use the prepared mattermost-bulk-import.zip. I have already completed steps 1-4 of processing the Slack export file

• As a reminder, this is the second command of step 4. This is a pertinent reminder, though not a step to reproduce this issue
  zip -r mattermost-bulk-import.zip bulk-export-attachments mattermost_import.jsonl

• Here is a view to the bulk zip:

$ zipinfo mattermost-bulk-import.zip
Archive:  mattermost-bulk-import.zip
Zip file size: 12979 bytes, number of entries: 3
drwxr-xr-x  3.0 unx        0 bx stor 22-Mar-08 18:39 bulk-export-attachments/
-rw-rw-r--  3.0 unx    11818 bx stor 22-Mar-08 18:39 bulk-export-attachments/F035BT84PTQ_test-image.png
-rw-rw-r--  3.0 unx     2323 tx defN 22-Mar-08 18:37 mattermost_import.jsonl
3 files, 14141 bytes uncompressed, 12379 bytes compressed:  12.5%

Reproduce issue steps:

1. First create a team named "testteam" though UI or the mmctl cli
   mmctl --json team create --name $mm_team --email testuser@example.com --display-name "test team"

2. Now continue at step 5. Import into Mattermost. These are those steps:

• Now you can start the import process. Once you have mmctl installed and authenticated use this command to upload mattermost-bulk-export.zip:

  mmctl import upload ./mattermost-bulk-import.zip

• Run this command to list the available imports:

  mmctl import list available

• Run this command to process the import. Replace with the name you got from the mmctl import list available command:

  mmctl import process <IMPORT FILE NAME>
  

• Finally, run this command to view the status of the import process job. If the job status shows as pending, then wait before running the command again. The --json flag is required to view the possible error message. Replace with the id you got from the mmctl import list process command:

  mmctl import job show <JOB ID> --json

• The output of this job show command shows the error $ ./dev/mmctl/mmctl import job show ax617ptmo3bi7c7e5mcj4cmc8a --json

  [
    {
      "id": "ax617ptmo3bi7c7e5mcj4cmc8a",
      "type": "import_process",
      "priority": 0,
      "create_at": 1646768985598,
      "start_at": 1646768994783,
      "last_activity_at": 1646768997030,
      "status": "error",
      "progress": -1,
      "data": {
        "error": "Error while processing bulk import attachments. — attachment \"data/bulk-export-attachments/F035BT84PTQ_test-image.png\" not found in map",
        "import_file": "7coajdui93ndmpqrzf3dsyzdnc_mattermost-bulk-import.zip",
        "line_number": "9"
      }
    }
  ]

Expected behavior

The import should complete without error. In the MM UI, the team name "testteam", in channel named "General" should show a single message with a single attachment

Observed behavior (that appears unintentional)

Analysis of the error message indicate that the job process has incorrectly joined the path string of the attachment named bulk-export-attachments/F035BT84PTQ_test-image.png with a path named data. There is no path named data in the mattermost-bulk-import.zip

Possible fixes

A workaround, if not able to resolve at this time, is to modify the documentation at step 4 with instructions to move the bulk-export-attachments folder into a new folder named data.

• First delete the previously created bulk zip file
  rm mattermost-bulk-import.zip

Now the example document text for the second half of step 4, following the ./mmetl transform slack command

• Create a new empty folder named "data". On Linux and Mac you can use this command:
  mkdir data

• Move the bulk-export-attachments. On Linux and Mac you can use this command:
  mv bulk-export-attachments data

• Next you have to create a zip file with the mattermost_import.jsonl file and the directory named "data" that contains the bulk-export-attachments which contains the attachments. On Linux and Mac you can use this command:
  zip -r mattermost-bulk-import data mattermost_import.jsonl

• Zipinfo shows the contents of the zip
  $ zipinfo ../../mattermost/mattermost-bulk-import.workaround.zip

  Archive:  ../../mattermost/mattermost-bulk-import.workaround.zip
  Zip file size: 13137 bytes, number of entries: 4
  drwxrwxr-x  3.0 unx        0 bx stor 22-Mar-08 17:25 data/
  drwxr-xr-x  3.0 unx        0 bx stor 22-Mar-08 15:39 data/bulk-export-attachments/
  -rw-rw-r--  3.0 unx    11818 bx stor 22-Mar-08 15:39 data/bulk-export-attachments/F035BT84PTQ_test-image.png
  -rw-rw-r--  3.0 unx     2323 tx defN 22-Mar-08 15:37 mattermost_import.jsonl
  4 files, 14141 bytes uncompressed, 12379 bytes compressed:  12.5%

• Now continue step 5 which will complete successfully

t$ ./dev/mmctl/mmctl import job show xw5b47irsfgj5jg78um5erj1mc --json
[
  {
    "id": "xw5b47irsfgj5jg78um5erj1mc",
    "type": "import_process",
    "priority": 0,
    "create_at": 1646771455872,
    "start_at": 1646771470111,
    "last_activity_at": 1646771475673,
    "status": "success",
    "progress": 0,
    "data": {
      "import_file": "edi1nnt81789fmnkcnmsax3qjc_mattermost-bulk-import.zip"
    }
  }
]

[amyblais]

Opened a ticket: https://mattermost.atlassian.net/browse/MM-42423.