Skip to content

[WIP] feat: Migration for PGP-contacts #6818

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 27 commits into from
May 6, 2025

Conversation

Hocuri
Copy link
Collaborator

@Hocuri Hocuri commented Apr 24, 2025

Based on #6796

TODO:

  • Decide what to do about the TODOs in the code
  • Make sure that errors during migration are properly shown in the UI

To decide:

  • What should we do when some members in a group can encrypt, and some can't. Right now, the group will be a PGP-group, and every non-PGP member will be silently removed. Should we employ some heuristics, like, if at least 33% of members can't encrypt, then it's an email-group? Or, if any member can't encrypt, then it's an ad-hoc (unencrypted) group?
  • If a group has some member for whom we have a key, but the peerstate is "reset". Should they count as can't encrypt?
    • In the current implementation, members without a key are silently (i.e. without an info message) removed from DC groups, and peerstate is ignored.
  • Before PGP-contacts, a contact could be verified, but have empty verifier, because at some point in the past, we didn't record who introduced whom. Do we want to persist the information that it's unclear who introduced this contact? In this case, we can put the contact's own id into the verifier column, and define this to mean "unknown introducer".
    • We're perstisting this information, which I implemented already

@Hocuri Hocuri mentioned this pull request Apr 24, 2025
11 tasks
@Hocuri Hocuri force-pushed the hoc/pgp-contacts-migration branch from c5b26a0 to 6a95b3e Compare April 24, 2025 16:01
@link2xt link2xt force-pushed the link2xt/pgp-contacts branch from 3c4255a to d912d59 Compare April 25, 2025 02:57
@Hocuri Hocuri force-pushed the hoc/pgp-contacts-migration branch 6 times, most recently from 781a76b to f3fb5a2 Compare April 30, 2025 12:16
@link2xt link2xt force-pushed the link2xt/pgp-contacts branch from f5a2a18 to 3cdd704 Compare April 30, 2025 13:19
@link2xt link2xt force-pushed the hoc/pgp-contacts-migration branch 3 times, most recently from c1456d6 to c47c105 Compare April 30, 2025 13:24
@Hocuri Hocuri force-pushed the hoc/pgp-contacts-migration branch from c47c105 to cb02753 Compare April 30, 2025 14:19
@link2xt link2xt force-pushed the hoc/pgp-contacts-migration branch 3 times, most recently from 5bc12ea to 9b16d25 Compare May 2, 2025 17:53
@link2xt
Copy link
Collaborator

link2xt commented May 2, 2025

During migration DC community chat shrinked from 114 members (not past members) to 110 members.

The log has these lines for the community chat:

05-02 21:09:36.672 23347 23379 🔵 DeltaChat: [accId=1] src/sql/migrations.rs:1682: Migrating chat 127 to PGP contacts: 13->907 20->913 35->928 137->1016 138->1018 140->1020 141->1021 144->1024 148->1028 150->1030 152->1032 159->1039 163->1043 164->1044 180->1060 185->1065 187->1067 193->1071 204->1082 211->1089 213->1091 216->1094 218->928 220->1097 221->1098 226->1103 227->1104 229->1106 231->1108 233->1110 242->1119 243->1120 244->1121 245->1122 247->1124 250->1127 252->1129 255->1132 257->1134 262->1138 267->1142 269->1144 273->1148 274->1149 280->1155 286->1160 287->1161 288->1162 289->1163 290->928 291->1164 292->1165 296->1169 298->928 301->1173 303->1175 307->1179 309->1181 310->1182 313->1185 314->1186 337->1209 338->1210 342->1213 346->0 347->1218 348->1219 352->1222 355->1225 369->1239 372->1242 373->1243 375->1245 378->1248 382->1252 383->1253 384->1254 385->1255 386->1256 387->1257 389->1259 394->1262 395->1263 396->1264 397->1265 407->1274 409->1276 410->1277 413->1280 414->1281 417->1283 422->1288 428->1294 429->1295 431->1297 438->1304 439->1305 448->1314 449->1315 451->1317 453->1319 455->1321 458->1324 471->1337 482->1348 483->1349 485->1351 494->1360 495->1361 500->1366 506->1372 508->1374 509->1375 511->1377 515->1381 518->1384 522->1388 523->1389 524->1390 527->1393 529->1395 533->1399 534->1400 537->1403 541->1407 542->1408 543->1409 544->1410 546->1412 547->1413 548->1414 549->1415 552->1418 553->1419 554->1420 555->1421 556->1422 558->1424 559->1425 560->1426 562->1428 567->1433 568->1434 569->1435 571->1437 572->1438 573->1439 574->1440 575->1441 576->1442 577->1443 578->1444 579->1445 580->1446 581->1447 582->1448 583->0 584->1449 585->1450 586->1210 587->0 588->1451 589->1452 590->1453 591->1454 592->1455 593->1456 594->1457 595->1458 596->1459 597->1460 598->928 599->1461 600->1462 642->1504 652->1513 659->1519 660->1520 661->1521 663->1523 664->1524 673->1533 676->1536 677->1537 678->1538 679->1539 680->1540 683->1543 685->1545 686->1546 687->1547 689->1549 690->1550 691->1551 692->1552 693->1553 694->1554 695->1555 698->1558 699->1559 700->1560 703->1562 704->1563 705->1564 706->1565 708->1567 709->1568 710->1569 711->1570 713->1572 714->1573 715->1574 716->1575 721->1580 722->0 723->1582 727->1585 728->1586 729->1587 730->1588 731->1217 732->1589 733->1590 734->1591 735->1592 736->1593 737->1594 738->1595 739->1596 740->1597 741->1598 742->0 743->1600 744->1601 745->1602 746->1603 747->1604 748->1605 749->1606 750->1607 751->1600 752->1608 753->1599 754->1609 756->1611 757->1581 758->0 759->1612 760->1613 761->1614 762->1615 763->1616 764->1617 765->1618 766->1619 767->1620 768->1621 769->1622 771->1624 772->1625 773->1626 775->1217 776->1627 780->1631 781->1632 782->1633 783->1634 784->1635 785->1636 786->1637 787->1638 788->1639 789->1640 790->1641 791->1642 792->1407 797->1646 798->1647 799->1648 800->1649 801->1650 802->1651 803->1652 804->1653 805->1654 806->1655 808->1657 809->1658 810->1659 812->1407 813->1661 816->1664 818->1666 819->1667 820->1668 821->1669 831->1674 832->1592 833->1675 835->1676 836->1677 839->1680 840->1681 843->1684 853->1144 855->1695 857->1697 858->1698 860->1700 861->1701 863->1703 864->1704 865->1705 869->1709 870->1710 871->1711 872->1712 873->1713 874->1714 876->1716 877->1717 878->1718 879->1719 888->1728 889->1729 891->1731 892->1732 893->1733 894->1734 895->1735 898->1738 899->1739 900->1740 901->1741
05-02 21:09:36.673 23347 23379 🔵 DeltaChat: [accId=1] src/sql/migrations.rs:1708: Chat partner is in the chat 127 multiple times. Deleting 928, then trying to update 218->928 again
05-02 21:09:36.673 23347 23379 🔵 DeltaChat: [accId=1] src/sql/migrations.rs:1708: Chat partner is in the chat 127 multiple times. Deleting 290, then trying to update 290->928 again
05-02 21:09:36.673 23347 23379 🔵 DeltaChat: [accId=1] src/sql/migrations.rs:1708: Chat partner is in the chat 127 multiple times. Deleting 298, then trying to update 298->928 again
05-02 21:09:36.673 23347 23379 🔵 DeltaChat: [accId=1] src/sql/migrations.rs:1708: Chat partner is in the chat 127 multiple times. Deleting 1210, then trying to update 586->1210 again
05-02 21:09:36.673 23347 23379 🔵 DeltaChat: [accId=1] src/sql/migrations.rs:1708: Chat partner is in the chat 127 multiple times. Deleting 598, then trying to update 598->928 again
05-02 21:09:36.673 23347 23379 🔵 DeltaChat: [accId=1] src/sql/migrations.rs:1708: Chat partner is in the chat 127 multiple times. Deleting 751, then trying to update 751->1600 again
05-02 21:09:36.673 23347 23379 🔵 DeltaChat: [accId=1] src/sql/migrations.rs:1708: Chat partner is in the chat 127 multiple times. Deleting 1217, then trying to update 775->1217 again
05-02 21:09:36.674 23347 23379 🔵 DeltaChat: [accId=1] src/sql/migrations.rs:1708: Chat partner is in the chat 127 multiple times. Deleting 792, then trying to update 792->1407 again
05-02 21:09:36.674 23347 23379 🔵 DeltaChat: [accId=1] src/sql/migrations.rs:1708: Chat partner is in the chat 127 multiple times. Deleting 1407, then trying to update 812->1407 again
05-02 21:09:36.674 23347 23379 🔵 DeltaChat: [accId=1] src/sql/migrations.rs:1708: Chat partner is in the chat 127 multiple times. Deleting 832, then trying to update 832->1592 again
05-02 21:09:36.674 23347 23379 🔵 DeltaChat: [accId=1] src/sql/migrations.rs:1708: Chat partner is in the chat 127 multiple times. Deleting 1144, then trying to update 853->1144 again

This deleted some real members, likely because there are past members without the key or with the same key as they did AEAP in the past.

Contact 35 is migrated into 928. Then according to the log we try to migrate 298 (this is the current email address), fail, remove 928 and try again. Probably at this point we have the correct contact with correct address in the chat.

Then we try to migrate 298, which is a past member. We fail, so we remove correct 928 and migrate 298 into 928.
Then the same for 598 which is also a past member.

In the end 928 has the wrong email address.

We should have migrated 218, but somehow got contact 905 in the chat.

@link2xt link2xt force-pushed the hoc/pgp-contacts-migration branch from 074f5f2 to 21ea8af Compare May 6, 2025 15:27
@link2xt link2xt force-pushed the link2xt/pgp-contacts branch from c156eca to ec7c421 Compare May 6, 2025 15:28
@link2xt link2xt force-pushed the hoc/pgp-contacts-migration branch from 21ea8af to 63fa13a Compare May 6, 2025 15:35
@link2xt link2xt merged commit 4ee7e8a into link2xt/pgp-contacts May 6, 2025
27 of 28 checks passed
@link2xt link2xt deleted the hoc/pgp-contacts-migration branch May 6, 2025 19:22
@link2xt
Copy link
Collaborator

link2xt commented May 6, 2025

Merged migration into PGP-contacts PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants