Import from slack error

[accountforcity]

Hi. There is some error during import from slack:

2022-05-17 13:27:10.519 INFO [] XXXXXXXXXXXXXX -> created
2022-05-17 13:27:10.520 INFO [] YYYYYYYYYYYYYY -> created
2022-05-17 13:27:10.559 INFO [] ######### IMPORTING STREAMS FINISHED #########

2022-05-17 13:27:12.827 INFO [] ######### IMPORTING MESSAGES STARTED #########

Traceback (most recent call last):
  File "./manage.py", line 157, in <module>
    execute_from_command_line(sys.argv)
  File "./manage.py", line 122, in execute_from_command_line
    utility.execute()
  File "/srv/zulip-venv-cache/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/zulip-py3-venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/srv/zulip-venv-cache/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/zulip-py3-venv/lib/python3.8/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/srv/zulip-venv-cache/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/zulip-py3-venv/lib/python3.8/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/home/zulip/deployments/2022-05-17-XX-XX-XX/zerver/management/commands/convert_slack_data.py", line 59, in handle
    do_convert_data(path, output_dir, token, threads=num_threads)
  File "/home/zulip/deployments/2022-05-17-XX-XX-XX/zerver/data_import/slack.py", line 1323, in do_convert_data
    reactions, uploads_list, zerver_attachment = convert_slack_workspace_messages(
  File "/home/zulip/deployments/2022-05-17-XX-XX-XX/zerver/data_import/slack.py", line 753, in convert_slack_workspace_messages
    ) = channel_message_to_zerver_message(
  File "/home/zulip/deployments/2022-05-17-XX-XX-XX/zerver/data_import/slack.py", line 916, in channel_message_to_zerver_message
    build_reactions(
  File "/home/zulip/deployments/2022-05-17-XX-XX-XX/zerver/data_import/slack.py", line 1143, in build_reactions
    reaction_dict["user_profile"] = slack_user_id_to_zulip_user_id[slack_user_id]
KeyError: 'U013ACBP1SP'

U013ACBP1SP - some slack user

Help me please.

[timabbott]

@accountforcity can you stop by chat.zulip.org to debug interactively? We'll need to do a bit of print-debugging to determine why that user is not being correctly discovered and added to the slack_user_id_to_zulip_user_id.

Slack doesn't document their export format, so we'll need to work with you to determine what Slack feature this user ID corresponds to and where its details live in the Slack export.

[zulipbot]

Hello @zulip/server-misc members, this issue was labeled with the "area: export/import" label, so you may want to check it out!

[timabbott]

From a PM from the user, it looks like the user in question is Slack's Google calendar integration bot account. So I think what we should do to try to reproduce this is setup that integration in a test organization, do a data export, and see if we can reproduce this.

@yuroitaki if you have a bit of time, it'd be awesome to try reproducing this.

[accountforcity]

Additional Information:

Slack import files with user information

integration_logs.json

Line 2957: {"user_id":"U013ACBP1SP","user_name":"X.YYYYYYY","date":"1636620635","change_type":"expanded","app_type":"Google Calendar","app_id":"ADZ494LHY"},
Line 2958: {"user_id":"U013ACBP1SP","user_name":"X.YYYYYYY","date":"1636620635","change_type":"added","app_type":"Google Calendar","app_id":"ADZ494LHY","scope":"identify,users.profile:write"},
Line 4905: {"user_id":"U013ACBP1SP","user_name":"X.YYYYYYY","date":"1628769040","change_type":"expanded","app_type":"Google Calendar","app_id":"ADZ494LHY"},
Line 4906: {"user_id":"U013ACBP1SP","user_name":"X.YYYYYYY","date":"1628769039","change_type":"added","app_type":"Google Calendar","app_id":"ADZ494LHY","scope":"identify,users.profile:write"},

2022-04-12.json

Line 159: "U013ACBP1SP",
Line 274: "U013ACBP1SP",

users.json

Line 1127: {"id":"U013ACBP1SP","team_id":"TAWTD6X0R","name":"X.YYYYYYY","deleted":true,"profile":{"title":"\u041c \u043f \u043e","phone":"+00000000000","skype":"","real_name":"\u0410 \u041a","real_name_normalized":"XXXXXXXXXXX YYYYYYY","display_name":"\u0410","display_name_normalized":"XXXXXXXXXXX YYYYYYY","fields":{},"status_text":"","status_emoji":"","status_emoji_display_info":[],"status_expiration":0,"avatar_hash":"aaa0000000000000a0a0","image_original":"https:\/\/avatars.slack-edge.com\/2021-10-06\/000000000000_aaa0000000000000a0a0_original.png","is_custom_image":true,"email":"X.YYYYYYY@ZZZZZZZZZZ.com","huddle_state":"default_unset","huddle_state_expiration_ts":0,"first_name":"\u0410","last_name":"\u041a","image_24":"https:\/\/avatars.slack-edge.com\/2021-10-06\/000000000000_aaa0000000000000a0a0_00.png","image_32":"https:\/\/avatars.slack-edge.com\/2021-10-06\/000000000000_aaa0000000000000a0a0_00.png","image_48":"https:\/\/avatars.slack-edge.com\/2021-10-06\/000000000000_aaa0000000000000a0a0_00.png","image_72":"https:\/\/avatars.slack-edge.com\/2021-10-06\/000000000000_aaa0000000000000a0a0_00.png","image_192":"https:\/\/avatars.slack-edge.com\/2021-10-06\/000000000000_aaa0000000000000a0a0_00.png","image_512":"https:\/\/avatars.slack-edge.com\/2021-10-06\/000000000000_aaa0000000000000a0a0_00.png","image_1024":"https:\/\/avatars.slack-edge.com\/2021-10-06\/000000000000_aaa0000000000000a0a0_00.png","status_text_canonical":"","team":"TAWTD6X0R"},"is_bot":false,"is_app_user":false,"updated":1650035061},

Some data obfuscated