Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge master branch #4874

Merged
merged 142 commits into from
Oct 25, 2023
Merged

Merge master branch #4874

merged 142 commits into from
Oct 25, 2023

Conversation

link2xt
Copy link
Collaborator

@link2xt link2xt commented Oct 25, 2023

Changelog

I have generated a changelog for overview of what changed.

Features / Changes

  • [breaking] Verified 1:1 chats (#4315).
    • Remove unused DC_STR_PROTECTION_(EN)ABLED* strings
    • Remove unused dc_set_chat_protection()
  • [breaking] Add lockfile to account manager (#4310).
  • Don't show a contact as verified if their key changed since the verification (#4574).
  • deltachat-rpc-server: Add --openrpc option.
  • Add bot field to contact (#4821).
  • Replace Config::SendSyncMsgs with SyncMsgs (#4817).

API-Changes

  • jsonrpc: Add resend_messages.
  • Api!(Rust): Remove unused function is_verified_ex() (#4551)
    No one used it anymore, and all occurences I could find (on GitHub)[https://github.com/search?q=%22is_verified_ex%22&type=code&p=1] are either forks of deltachat-core-rust or of deltachat-core (which is the old C core)..
  • [breaking] Make MsgId.delete_from_db() private.
  • Api!(deltachat-jsonrpc): use kind as a tag for all union types
  • json-rpc: Force stickers to be sent as stickers (#4819).

Build system

  • Remove examples/simple.rs.
  • Use Rust 1.71.0 and increase MSRV to 1.66.0.
  • python: Pin sphinx to 7.1.2.
  • coredeps: Install perl-IPC-Cmd.
  • coredeps: Only run yum if it is available.

CI

  • Remove comment about python from rust tests.
  • Run Rust tests with RUST_BACKTRACE set.
  • Increase MSRV to 1.67.0.
  • Remove misplaced comment.
  • Rename async tests into JSON-RPC tests.

Documentation

  • Document how logs and error messages should be formatted.
  • Improve JSON-RPC API documentation.

Fixes

  • Don't create 1:1 chat as protected for contact who doesn't prefer to encrypt (#4538).
  • Allow to save a draft if the verification is broken (#4542).
  • Fix info-message orderings of verified 1:1 chats (#4545).
  • Fix example; this was changed some time ago, see https://docs.webxdc.org/spec.html#sendupdate
    .
  • Receive_imf: Update peerstate from db after handling Securejoin handshake (#4600).
  • Sort old incoming messages below all outgoing ones (#4621).
  • Delete messages from SMTP queue only on user demand (#4579).
  • Do not mark non-verified group chats as verified when using securejoin.
  • Receive_imf: Set protection only for Chattype::Single (#4597).
  • Return from dc_get_chatlist(DC_GCL_FOR_FORWARDING) only chats where we can send (#4616).
  • Clear VerifiedOneOnOneChats config on backup (#4681).
  • sql: Order migrations the same as on stable branch.

Miscellaneous Tasks

  • Update trust-dns-resolver to hickory-resolver.
  • Update dependencies

Other

  • Clarify transitive behaviour of dc_contact_is_verfified()
  • Document configured_addr

Refactor

  • Use slices and vectors instead of Keyring wrapper.
  • e2ee: Do not return anything from ensure_secret_key_exists().
  • Flatten and simplify imports.
  • Replace DcKey.load_self trait method with functions.
  • Make last_added_location_id an Option.
  • Move dc_preconfigure_keypair() implementation into deltachat crate.
  • Hide DcSecretKey trait from the API.
  • Use SQL transaction in MsgId.delete_from_db().
  • Remove Chattype::Undefined.

Tests

  • Remove unnecessary inner_set_protection() call (#4539).
  • Test that get_system_info() works over RPC backup import.
  • Directly unwrap in TestContext::get_chat() (#4614).
  • Don't accidentally accept that a chat protection is broken (#4550).
  • W/a message reordering in test_reaction_to_partially_fetched_msg().
  • Test_openrpc_command_line: Check that deltachat-rpc-server exists with 0.
  • Extend test_qr_join_chat to check that the group is not verified.
  • Adjust expected info message in test_verified_group_vs_delete_server_after.

link2xt and others added 30 commits July 7, 2023 21:56
Implement #4188

BREAKING CHANGE: Remove unused DC_STR_PROTECTION_(EN)ABLED* strings
BREAKING CHANGE: Remove unused dc_set_chat_protection()
When `cargo test` is executed,
all examples are built by default
to ensure that they can be compiled.

This is a documented and expected behaviour,
even though it was previously reported as a bug:
<rust-lang/cargo#6675>

In particular, `examples/simple.rs` is built into
a 67M binary `target/debug/examples/simple`.
This is unnecessary to do so every time
you change a line in the `deltachat` crate
and want to rerun the tests.

Workaround is to run `cargo test --tests`,
but it is easy to forget and is not discoverable
unless you read the "Target Selection" section of `cargo help test`.

We have a maintained example at https://github.com/deltachat-bot/echo,
so there is no need for an example in the core repository.
1:1 chats are automatically created as protected if the contact is
verified, there is no need to explicitly do this.

Plus, by removing this call, the test also tests that automatically
creating 1:1 chats as protected works.
If the verification is broken, `can_send()` is false.

But if the user was typing a message right when a verification-breaking message came in, the UI still needs to be able to save it as a draft.

Steps to reproduce the bug:
  - Set a draft
  - Your chat partner breaks verification
  - Go back to the chats list
  - Go to the chat again
  - Accept the breakage
  - Expected: The draft is still there
  - Bug behavior: The draft is gone
Rust 1.66 is required by constant_time_eq 0.3.0.
looks like this was fogotten when changing the chat protection stock strings
Correctly handle messages with old timestamps for verified chats:

 * They must not be sorted over a protection-changed info message

 * If they change the protection, then they must not be sorted over existing other messages, because then the protection-changed info message would also be above these existing messages.


This PR fixes this:

 1. Even seen messages can't be sorted into already-noticed messages anymore. **This also changes DC's behavior in the absence of verified 1:1 chats**. Before this PR, messages that are marked as seen when they are downloaded will always be sorted by their timestamp, even if it's very old.

 2. protection-changed info messages are always sorted to the bottom.

    **Edit:**

 3. There is an exception to rule 1: Outgoing messages are still allowed to be sorted purely by their timestamp, and don't influence old messages. This is to the problem described at [*].


Together, these rules also make sure that the protection-changed info message is always right above the message causing the change.

[*] If we receive messages from two different folders, e.g. `Sent` and `Inbox`, then this will lead to wrong message ordering in many cases. I need to think about this more, or maybe someone else has an idea. One new idea that came to my mind is:

 * Always sort noticed messages under the newest info message (this PR sorts them under the newest noticed message, master sorts them purely by their sent timestamp)

 * Always sort unnoticed messages under the newest noticed message (that's the same behavior as in this PR and on master)

 * Always sort protection-changed info messages to the bottom (as in this PR)


However, after a talk with @link2xt we instead decided to add rule 3. (see above) because it seemed a little bit easier.
No one used it anymore, and all occurences I could find (on
GitHub)[https://github.com/search?q=%22is_verified_ex%22&type=code&p=1]
are either forks of deltachat-core-rust or of deltachat-core (which is
the old C core).
This change removes all traces of dc_keyring_t,
which was a C implementation of dynamically sized array.
Opening the same account (context) from multiple processes is dangerous, can result in duplicate
downloads of the same message etc. Same for account manager, attempts to modify the same
accounts.toml even if done atomically with may result in corrupted files as atomic replacement
procedure does not expect that multiple processes may write to the same temporary file.

accounts.toml cannot be used as a lockfile because it is replaced during atomic update. Therefore, a
new file next to accounts.toml is needed to prevent starting second account manager in the same
directory.

But iOS needs to be able to open accounts from multiple processes at the same time. This is required
as the "share-to-DC extension" is a separate process by iOS design -- this process may or may not be
started while the main app is running. Accounts are not altered however by this extension, so let's
add to the `Accounts::new()` constructor an `rdwr` parameter which allows to read the accounts
config w/o locking the lockfile.
dependabot bot and others added 26 commits October 1, 2023 21:42
Bumps [tokio-util](https://github.com/tokio-rs/tokio) from 0.7.8 to 0.7.9.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](tokio-rs/tokio@tokio-util-0.7.8...tokio-util-0.7.9)

---
updated-dependencies:
- dependency-name: tokio-util
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.47 to 1.0.49.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](dtolnay/thiserror@1.0.47...1.0.49)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [regex](https://github.com/rust-lang/regex) from 1.9.5 to 1.9.6.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](rust-lang/regex@1.9.5...1.9.6)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [smallvec](https://github.com/servo/rust-smallvec) from 1.11.0 to 1.11.1.
- [Release notes](https://github.com/servo/rust-smallvec/releases)
- [Commits](servo/rust-smallvec@v1.11.0...v1.11.1)

---
updated-dependencies:
- dependency-name: smallvec
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [sha2](https://github.com/RustCrypto/hashes) from 0.10.7 to 0.10.8.
- [Commits](RustCrypto/hashes@sha2-v0.10.7...sha2-v0.10.8)

---
updated-dependencies:
- dependency-name: sha2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [webpki](https://github.com/briansmith/webpki) from 0.22.0 to 0.22.2.
- [Commits](https://github.com/briansmith/webpki/commits)

---
updated-dependencies:
- dependency-name: webpki
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [proptest](https://github.com/proptest-rs/proptest) from 1.2.0 to 1.3.1.
- [Release notes](https://github.com/proptest-rs/proptest/releases)
- [Changelog](https://github.com/proptest-rs/proptest/blob/master/CHANGELOG.md)
- [Commits](proptest-rs/proptest@v1.2.0...v1.3.1)

---
updated-dependencies:
- dependency-name: proptest
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Resolved conflicts due to asyncio removal.
Release 1.125.0
This approach uses a param field to enable forcing the sticker
`viewtype`. The first commit has the memory-only flag implemented, but
this flag is not persistent through the database conversion needed for
draft/undraft. That's why `param` has to be used.

follow up to #4814 
fixes #4739

---------

Co-authored-by: Septias <scoreplayer2000@gmail.comclear>
…erver_after

Test was written for stable branch
and has to be adjusted for verified 1:1 chats branch
Release 1.126.1
Copy link
Member

@r10s r10s left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably okay :)

we'll do another 1.41 release anyways (tbh, i though much more things were in the last core releases :)

@link2xt link2xt merged commit 113356a into stable Oct 25, 2023
27 checks passed
@link2xt link2xt deleted the merge-master branch October 25, 2023 19:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants