Add save file generator, compatibility test, and generate a savefile #1051

jhert0 · 2018-08-04T20:59:14Z

The save file will be used to ensure that #1046 is compatible with older save files. The output of the save generator was:

Connected to the tox network.
Wrote tox save to save.tox
INFORMATION
----------------------------------
Tox ID: B70E97D41F69B7F4C42A5BC7BD7A76B95B8030BE1B7C0E9E6FC19FC4ABEB195B4C762C7D800B.
Nospam: 4C762C7D.
Name: name.
Status message: Hello World.
Number of friends: 1.
----------------------------------

The added friend was toxirc: A922A51E1C91205B9F7992E2273107D47C72E8AE909C61C28A77A4A2A115431B14592AB38A3B.

This change is

iphydf

Please don't submit the savefile in this PR. It's not used, so it serves no purpose. Once you add a test for loading it, you can submit the data for it as well.

iphydf · 2018-08-09T22:57:20Z

other/save-generator.c

+int main(int argc, char *argv[])
+{
+    if (argc < 3) {
+        printf("Usage: ./save-generator <name> <friend id>\n");


I wonder if we could make this tool a little more versatile, maybe have some way of generating a savefile with multiple friends, groups, etc. I'm happy to have this version submitted as an initial version, but I think the interesting bits of save parsing happen when you have more than 1 of a thing.

iphydf · 2018-08-11T13:17:27Z

other/save-generator.c

+#define SAVE_FILE "save.tox"
+#define STATUS_MESSAGE "Hello World"
+
+static const char *bootstrap_ip = "185.14.30.213";


const char *const

iphydf · 2018-08-11T13:17:53Z

other/save-generator.c

+#include "../testing/misc_tools.h"
+#include "../toxcore/ccompat.h"
+
+#define SAVE_FILE "save.tox"


Why are these macros and the constants below are variables?

iphydf · 2018-08-11T13:22:42Z

other/save-generator.c

+    while (true) {
+        tox_iterate(tox, nullptr);
+
+        if (tox_self_get_connection_status(tox) == TOX_CONNECTION_UDP) {


Prefer callbacks. This main function is quite long. Given that we'll want to extend it in the future, it's probably good to split it into a few functions.

iphydf · 2018-08-11T13:22:54Z

other/save-generator.c

+    }
+
+    const uint8_t *name = (uint8_t *)argv[1];
+    bool ret = tox_self_set_name(tox, name, strlen((char *)name), nullptr);


Prefer error pointers.

iphydf · 2018-08-11T13:23:03Z

other/save-generator.c

+        return -1;
+    }
+
+    tox_bootstrap(tox, bootstrap_ip, udp_port, key, nullptr);


No error checking done.

iphydf · 2018-08-11T13:27:19Z

other/save-generator.c

+    uint32_t friend_num = tox_friend_add(tox, address, (const uint8_t *)msg, strlen(msg), nullptr);
+    free(address);
+
+    if (friend_num == UINT32_MAX) {


Friend number in case of failure is unspecified. This is relying on knowledge of internals. Prefer error enums.

iphydf · 2018-08-11T13:27:52Z

other/save-generator.c

+    tox_self_get_address(tox, tox_id);
+    to_hex(tox_id_str, tox_id, TOX_ADDRESS_SIZE);
+
+    char nospam_str[TOX_NOSPAM_SIZE * 2];


Not enough space for the null terminator.

jhert0

Reviewable status: 0 of 1 LGTMs obtained

other/save-generator.c, line 10 at r1 (raw file):

Previously, iphydf wrote…

Why are these macros and the constants below are variables?

Done.

other/save-generator.c, line 13 at r1 (raw file):

Previously, iphydf wrote…

const char *const

Done.

other/save-generator.c, line 37 at r1 (raw file):

Previously, iphydf wrote…

I wonder if we could make this tool a little more versatile, maybe have some way of generating a savefile with multiple friends, groups, etc. I'm happy to have this version submitted as an initial version, but I think the interesting bits of save parsing happen when you have more than 1 of a thing.

I added in adding multiple friends. I'll add in groupchats when persistent groupchats is merged.

other/save-generator.c, line 55 at r1 (raw file):

Previously, iphydf wrote…

No error checking done.

Done.

other/save-generator.c, line 61 at r1 (raw file):

Previously, iphydf wrote…

Prefer callbacks. This main function is quite long. Given that we'll want to extend it in the future, it's probably good to split it into a few functions.

Done.

other/save-generator.c, line 70 at r1 (raw file):

Previously, iphydf wrote…

Prefer error pointers.

Done.

other/save-generator.c, line 87 at r1 (raw file):

Previously, iphydf wrote…

Friend number in case of failure is unspecified. This is relying on knowledge of internals. Prefer error enums.

Done.

other/save-generator.c, line 117 at r1 (raw file):

Previously, iphydf wrote…

Not enough space for the null terminator.

I just use the length to write the nospam out same with the tox id.

nurupo · 2018-08-11T23:55:04Z

I think such things should go into other/fun directory, not just other.

@iphydf Btw, I have written a bit different save file generator several weeks ago, do you think it makes sense to add it to the repo too? It doesn't use toxcore at all though, it crafts the save file using the current save file binary format, much like other/fun/make-funny-savefile.py does.

jhert0

Reviewable status: 1 change requests, 0 of 1 approvals obtained (waiting on @iphydf)

other/save-generator.c, line 117 at r1 (raw file):

Previously, endoffile78 (Endoffile) wrote…

I just use the length to write the nospam out same with the tox id.

Done.

iphydf · 2018-08-12T23:36:04Z

@nurupo yes, and it seems like a good idea to put these things into other/fun.

iphydf

Reviewable status: 1 change requests, 0 of 1 approvals obtained (waiting on @iphydf and @endoffile78)

other/save-generator.c, line 50 at r2 (raw file):

    if (err != TOX_ERR_BOOTSTRAP_OK) {
        printf("Failed to bootstrap. Error number: %u", err);

Error numbers are signed ints (enum values), so %d.

other/save-generator.c, line 89 at r2 (raw file):

    printf("Name: %s.\n", name);
    printf("Status message: %s.\n", STATUS_MESSAGE);
    printf("Number of friends: %lu.\n", tox_self_get_friend_list_size(tox));

%zu

other/save-generator.c, line 133 at r2 (raw file):

    }

    int num_friends = atoi(argv[2]);

This isn't used.

other/save-generator.c, line 138 at r2 (raw file):

        uint8_t *address = hex_string_to_bin(argv[i]);
        Tox_Err_Friend_Add friend_err;
        uint32_t friend_num = tox_friend_add(tox, address, (const uint8_t *)REQUEST_MESSAGE, strlen(REQUEST_MESSAGE),

Remove dead assignment.

jhert0

Reviewable status: 1 change requests, 0 of 1 approvals obtained (waiting on @iphydf)

other/save-generator.c, line 50 at r2 (raw file):

Previously, iphydf wrote…

Error numbers are signed ints (enum values), so %d.

Done.

other/save-generator.c, line 89 at r2 (raw file):

Previously, iphydf wrote…

%zu

Done.

other/save-generator.c, line 133 at r2 (raw file):

Previously, iphydf wrote…

This isn't used.

Done.

other/save-generator.c, line 138 at r2 (raw file):

Previously, iphydf wrote…

Remove dead assignment.

Done.

iphydf

Reviewed 1 of 5 files at r1, 1 of 4 files at r3.
Reviewable status: 1 change requests, 0 of 1 approvals obtained (waiting on @endoffile78)

auto_tests/data/save.tox, line 0 at r3 (raw file):
This file should be moved to a different PR where we actually use it.

other/save-generator.c, line 50 at r2 (raw file):

Previously, endoffile78 (Endoffile) wrote…

Done.

Not really.

iphydf

Reviewable status: 1 change requests, 0 of 1 approvals obtained (waiting on @endoffile78)

other/fun/save-generator.c, line 137 at r3 (raw file):

        if (friend_err != TOX_ERR_FRIEND_ADD_OK) {
            printf("Failed to add friend number %u. Error number: %d\n", i - 1, friend_err);

i is int, so %d, or make i unsigned and keep %u.

iphydf

Reviewable status: 1 change requests, 0 of 1 approvals obtained (waiting on @endoffile78)

other/fun/save-generator.c, line 3 at r3 (raw file):

#include <stdio.h>
#include <string.h>
#include <stdint.h>

Also sort these includes.

other/fun/save-generator.c, line 7 at r3 (raw file):

#include "../../toxcore/tox.h"
#include "../../testing/misc_tools.h"

Sort includes.

jhert0

Reviewable status: 1 change requests, 0 of 1 approvals obtained (waiting on @endoffile78 and @iphydf)

auto_tests/data/save.tox, line at r3 (raw file):

Previously, iphydf wrote…

This file should be moved to a different PR where we actually use it.

Added a test that uses it

other/fun/save-generator.c, line 3 at r3 (raw file):

Previously, iphydf wrote…

Also sort these includes.

Done.

other/fun/save-generator.c, line 7 at r3 (raw file):

Previously, iphydf wrote…

Sort includes.

Done.

other/fun/save-generator.c, line 137 at r3 (raw file):

Previously, iphydf wrote…

i is int, so %d, or make i unsigned and keep %u.

Done.

iphydf

Reviewable status: 1 change requests, 0 of 1 approvals obtained (waiting on @endoffile78)

auto_tests/save_compatibility_test.c, line 74 at r4 (raw file):

    struct Tox_Options options;

    memset(&options, 0, sizeof(struct Tox_Options));

Use regular struct zero-initialisation instead of memset.

auto_tests/save_compatibility_test.c, line 94 at r4 (raw file):

    uint8_t name[name_size];
    tox_self_get_name(tox, name);
    ck_assert_msg(strncmp((char *)name, NAME, name_size) == 0, "Names do not match, expected %s got %s.", NAME, name);

const char *

auto_tests/save_compatibility_test.c, line 103 at r4 (raw file):

    tox_self_get_status_message(tox, status_message);
    ck_assert_msg(strncmp((char *)status_message, STATUS_MESSAGE, status_message_size) == 0,
                  "Status messages do not match, expeceted %s got %s.",

expected

auto_tests/save_compatibility_test.c, line 114 at r4 (raw file):

    size_t length = snprintf(nospam_str, sizeof(nospam_str), "%08X", nospam);
    nospam_str[length] = '\0';
    ck_assert_msg(strcmp(nospam_str, NOSPAM) == 0, "Nospam does not match, expected %s got %s.", NOSPAM, nospam_str);

Error messages here should start with lowercase and not have a "." at the end.

other/fun/save-generator.c, line 3 at r3 (raw file):

Previously, endoffile78 (Endoffile) wrote…

Done.

Not done.

other/fun/save-generator.c, line 112 at r4 (raw file):

    while (!connected) {
        tox_iterate(tox, nullptr);

Pass &connected to tox_iterate, so it ends up in userdata for the connection callback.

jhert0

Reviewable status: 1 change requests, 0 of 1 approvals obtained (waiting on @iphydf)

auto_tests/save_compatibility_test.c, line 74 at r4 (raw file):

Previously, iphydf wrote…

Use regular struct zero-initialisation instead of memset.

Done.

auto_tests/save_compatibility_test.c, line 94 at r4 (raw file):

Previously, iphydf wrote…

const char *

Done.

auto_tests/save_compatibility_test.c, line 103 at r4 (raw file):

Previously, iphydf wrote…

expected

Done.

auto_tests/save_compatibility_test.c, line 114 at r4 (raw file):

Previously, iphydf wrote…

Error messages here should start with lowercase and not have a "." at the end.

Done.

other/fun/save-generator.c, line 112 at r4 (raw file):

Previously, iphydf wrote…

Pass &connected to tox_iterate, so it ends up in userdata for the connection callback.

Done.

iphydf

Reviewed 2 of 4 files at r3, 1 of 3 files at r4, 2 of 2 files at r5.
Reviewable status: 1 change requests, 0 of 1 approvals obtained

iphydf

Reviewable status: complete! 1 of 1 approvals obtained (waiting on @endoffile78)

auto_tests/save_compatibility_test.c, line 67 at r5 (raw file):

    *length = size;

    return data;

Add fclose here.

iphydf

Can you add the generator tool to other/fun/BUILD.bazel?

Reviewable status: 1 change requests, 0 of 1 approvals obtained (waiting on @endoffile78)

auto_tests/save_compatibility_test.c, line 98 at r5 (raw file):

                  STATUS_MESSAGE_SIZE, status_message_size);

    uint8_t status_message[status_message_size];

You'll need to use the VLA macro here, or preferably avoid VLAs altogether. Rather malloc and free, or allocate the maximum size (from tox.h).

iphydf

still more things to do.

Reviewable status: 1 change requests, 0 of 1 approvals obtained (waiting on @endoffile78)

jhert0

Reviewable status: 1 change requests, 0 of 1 approvals obtained (waiting on @iphydf)

auto_tests/save_compatibility_test.c, line 67 at r5 (raw file):

Previously, iphydf wrote…

Add fclose here.

Done.

auto_tests/save_compatibility_test.c, line 98 at r5 (raw file):

Previously, iphydf wrote…

You'll need to use the VLA macro here, or preferably avoid VLAs altogether. Rather malloc and free, or allocate the maximum size (from tox.h).

Done.

iphydf

Reviewed 4 of 4 files at r6.
Reviewable status: 1 change requests, 0 of 1 approvals obtained (waiting on @endoffile78)

CMakeLists.txt, line 442 at r6 (raw file):

auto_test(typing)
auto_test(version)
auto_test(save_compatibility)

Sort these.

jhert0

Reviewable status: complete! 1 of 1 approvals obtained (waiting on @iphydf)

CMakeLists.txt, line 442 at r6 (raw file):

Previously, iphydf wrote…

Sort these.

Done.

iphydf

Reviewable status: complete! 1 of 1 approvals obtained (waiting on @iphydf and @endoffile78)

other/fun/BUILD.bazel, line 34 at r7 (raw file):

    deps = [
        "//c-toxcore/testing:misc_tools",
        "//c-toxcore/toxcore"

Add "," at end of line.

other/fun/BUILD.bazel, line 35 at r7 (raw file):

        "//c-toxcore/testing:misc_tools",
        "//c-toxcore/toxcore"
    ]

Here too.

Try running buildifier on this file, it'll fix these kinds of style issues.

iphydf · 2018-08-18T23:37:13Z

After fixing the last style issues, rebase on master and I'll merge it.

… file

jhert0 force-pushed the save-generator branch 3 times, most recently from dc14553 to fcec1b3 Compare August 4, 2018 21:32

jhert0 mentioned this pull request Aug 4, 2018

Finish the messenger state plugin system #1046

Merged

jhert0 force-pushed the save-generator branch from a267520 to ed6c940 Compare August 7, 2018 19:25

iphydf added this to the v0.2.x milestone Aug 8, 2018

iphydf requested changes Aug 9, 2018

View reviewed changes

iphydf requested changes Aug 11, 2018

View reviewed changes

jhert0 force-pushed the save-generator branch from ed6c940 to 0ca9db8 Compare August 11, 2018 18:05

jhert0 commented Aug 11, 2018

View reviewed changes

jhert0 force-pushed the save-generator branch from 8405d08 to 74f6068 Compare August 11, 2018 21:13

jhert0 commented Aug 12, 2018

View reviewed changes

iphydf requested changes Aug 12, 2018

View reviewed changes

jhert0 force-pushed the save-generator branch from 74f6068 to 3170886 Compare August 13, 2018 05:45

jhert0 commented Aug 13, 2018

View reviewed changes

iphydf requested changes Aug 13, 2018

View reviewed changes

jhert0 force-pushed the save-generator branch from 3170886 to c79ec0b Compare August 15, 2018 04:30

jhert0 commented Aug 15, 2018

View reviewed changes

jhert0 changed the title ~~Add save file generator and generate a save file~~ Add save file generator, compability test, and generate a savefile Aug 15, 2018

iphydf requested changes Aug 15, 2018

View reviewed changes

jhert0 force-pushed the save-generator branch from 5eeac14 to d206282 Compare August 15, 2018 22:37

jhert0 commented Aug 15, 2018

View reviewed changes

iphydf approved these changes Aug 15, 2018

View reviewed changes

iphydf requested changes Aug 15, 2018

View reviewed changes

iphydf reviewed Aug 15, 2018

View reviewed changes

jhert0 force-pushed the save-generator branch from d206282 to 8fc582f Compare August 16, 2018 03:07

jhert0 commented Aug 16, 2018

View reviewed changes

jhert0 force-pushed the save-generator branch 2 times, most recently from 1c64bbe to 2b17ed4 Compare August 16, 2018 21:59

iphydf approved these changes Aug 18, 2018

View reviewed changes

iphydf changed the title ~~Add save file generator, compability test, and generate a savefile~~ Add save file generator, compatibility test, and generate a savefile Aug 18, 2018

jhert0 force-pushed the save-generator branch from 2b17ed4 to b9a8458 Compare August 18, 2018 23:00

jhert0 commented Aug 18, 2018

View reviewed changes

iphydf requested changes Aug 18, 2018

View reviewed changes

Add save file generator, save compatibility test, and generate a save…

30960dc

… file

iphydf force-pushed the save-generator branch from b9a8458 to 30960dc Compare August 19, 2018 10:38

iphydf approved these changes Aug 19, 2018

View reviewed changes

iphydf merged commit 30960dc into TokTok:master Aug 19, 2018

robinlinden modified the milestones: v0.2.x, v0.2.7 Aug 30, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add save file generator, compatibility test, and generate a savefile #1051

Add save file generator, compatibility test, and generate a savefile #1051

jhert0 commented Aug 4, 2018 •

edited by iphydf

Loading

iphydf left a comment

iphydf Aug 9, 2018

iphydf Aug 11, 2018

iphydf Aug 11, 2018

iphydf Aug 11, 2018

iphydf Aug 11, 2018

iphydf Aug 11, 2018

iphydf Aug 11, 2018

iphydf Aug 11, 2018

jhert0 left a comment

nurupo commented Aug 11, 2018

jhert0 left a comment

iphydf commented Aug 12, 2018

iphydf left a comment

jhert0 left a comment

iphydf left a comment

iphydf left a comment

iphydf left a comment

jhert0 left a comment

iphydf left a comment

jhert0 left a comment

iphydf left a comment

iphydf left a comment

iphydf left a comment

iphydf left a comment

jhert0 left a comment

iphydf left a comment

jhert0 left a comment

iphydf left a comment

iphydf commented Aug 18, 2018

Add save file generator, compatibility test, and generate a savefile #1051

Add save file generator, compatibility test, and generate a savefile #1051

Conversation

jhert0 commented Aug 4, 2018 • edited by iphydf Loading

iphydf left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jhert0 left a comment

Choose a reason for hiding this comment

nurupo commented Aug 11, 2018

jhert0 left a comment

Choose a reason for hiding this comment

iphydf commented Aug 12, 2018

iphydf left a comment

Choose a reason for hiding this comment

jhert0 left a comment

Choose a reason for hiding this comment

iphydf left a comment

Choose a reason for hiding this comment

iphydf left a comment

Choose a reason for hiding this comment

iphydf left a comment

Choose a reason for hiding this comment

jhert0 left a comment

Choose a reason for hiding this comment

iphydf left a comment

Choose a reason for hiding this comment

jhert0 left a comment

Choose a reason for hiding this comment

iphydf left a comment

Choose a reason for hiding this comment

iphydf left a comment

Choose a reason for hiding this comment

iphydf left a comment

Choose a reason for hiding this comment

iphydf left a comment

Choose a reason for hiding this comment

jhert0 left a comment

Choose a reason for hiding this comment

iphydf left a comment

Choose a reason for hiding this comment

jhert0 left a comment

Choose a reason for hiding this comment

iphydf left a comment

Choose a reason for hiding this comment

iphydf commented Aug 18, 2018

jhert0 commented Aug 4, 2018 •

edited by iphydf

Loading