Unable to build on macOS (10.15.5)

[michaelsmoody]

When attempting to build on macOS (Darwin/Mach), despite my best efforts, the following occurs:

backupframe/../common_be.h:120:10: error: no matching function for call to 'bytesToHexString'

I haven't been able to figure this one out. It happens on every file that uses that function, which seems extensive. Any guidance is appreciated.

Thanks,
Michael

[bepaald]

Hm, that's strange. I have very little experience with MacOS, I just make sure I write standards compliant C++ code, test on Linux with various compilers (gcc and clang) and cross-compile a windows binary just because I can do it fairly easily from Linux. It would be nice to get it to build (and run) on MacOS though, so let's see if we can figure out what's happening.

What compiler are you using (and which version)? Are you using the supplied script to compile?

[michaelsmoody]

I'm attempted using the BUILDSCRIPT, as well as manually, with g++ and clang, as well as with openssl and crypt++, all of them fail in the same way.

[michaelsmoody]

Where does the function come from? bytesToHexString?

[bepaald]

I'm attempted using the BUILDSCRIPT, as well as manually, with g++ and clang, as well as with openssl and crypt++, all of them fail in the same way.

Which versions of g++ and clang are you using?

Where does the function come from? bytesToHexString?

If you check the file 'common_be.h', they are all there:

Declarations:

signalbackup-tools/common_be.h

Lines 58 to 60 in f15d495

	std::string bytesToHexString(std::pair<std::shared_ptr<unsigned char []>, unsigned int> const &data, bool unformatted = false);
	std::string bytesToHexString(std::pair<unsigned char *, unsigned int> const &data, bool unformatted = false);
	std::string bytesToHexString(unsigned char const *data, unsigned int length, bool unformatted = false);

Definitions:

signalbackup-tools/common_be.h

Lines 118 to 138 in f15d495

	inline std::string bepaald::bytesToHexString(std::pair<std::shared_ptr<unsigned char []>, unsigned int> const &data, bool unformatted)
	{
	return bytesToHexString(data.first.get(), data.second, unformatted);
	}
	inline std::string bepaald::bytesToHexString(std::pair<unsigned char *, unsigned int> const &data, bool unformatted/* = false*/)
	{
	return bytesToHexString(data.first, data.second, unformatted);
	}
	inline std::string bepaald::bytesToHexString(unsigned char const *data, unsigned int length, bool unformatted/* = false*/)
	{
	std::ostringstream oss;
	if (!unformatted)
	oss << "(hex:) ";
	for (uint i = 0; i < length; ++i)
	oss << std::hex << std::setfill('0') << std::setw(2)
	<< (static_cast<int32_t>(data[i]) & 0xFF)
	<< ((i == length - 1 || unformatted) ? "" : " ");
	return oss.str();
	}

As you can see, your error (on line 120) is actually inside one of the three bytesToHexString functions, where it can't seem to find the third.

As far as I know, this is valid C++ code, but here's two things to try:
- maybe try either reordering the functions so that the third function (at line 128) appears before the others.
- Or: specify the namespace in the function call (that is, change line 120 to bepaald::bytesToHexString(data.first.get(), data.second, unformatted);

Let me know if that makes any difference.

EDIT
I installed MacOS Catalina on a virtual machine and reproduced your problem. The suggestions I made above do not help. I don't know why it's not working (I'm about 100% certain it should just work), but I will look into it. I don't know how long it will take though.

[bepaald]

Apparently, clang is the problem (on MacOS, it works fine on Linux). g++ does not give the same error, it's just that the '/usr/bin/g++' binary on Mac is actually secretly just a copy of clang++. You can see this when you call g++ --version.

If you install gcc (for example via homebrew: brew install gcc), and make sure to call the real g++ binary, the program compiles fine:

Note I'm calling the g++ binary by its full path. Also, the -I and -L options are only needed because I did not install openssl in a default search path. (The three warnings generated are not in my code, but in openssl.) I did not test the program any further, it is possible problems appear when running on an actual backup file.

I will look into why clang is producing an error on Mac, but maybe this already helps you.

[michaelsmoody]

Thanks for that. I did actually specifically use g++ via gcc installed by brew, and did have the same problem (with the full path specified). I'll perhaps try to install a different version of gcc. (I was already aware of the secret alias of g++ to clang on macOS).

Can you textually post your full compile command line with all arguments? Given that it's an image, hard to copy/paste.

Thanks for looking into this. What specific version of gcc did you install? 7? 8? 9?

[bepaald]

Thanks for that. I did actually specifically use g++ via gcc installed by brew, and did have the same problem (with the full path specified). I'll perhaps try to install a different version of gcc. (I was already aware of the secret alias of g++ to clang on macOS). [...] What specific version of gcc did you install? 7? 8? 9?

I think I never compiled this with gcc 7, but it always used to compile with 8, 9 worked fine (and still does) and currently gcc 10 is working fine as well. Anyway, I installed gcc by just doing brew install gcc, which gave me version 9.3.0. I installed openssl by doing brew install openssl.

Can you textually post your full compile command line with all arguments? Given that it's an image, hard to copy/paste.

Haha, yeah I had the same problem trying to copy from the VM, that's why I just did the screenshot :)

The command to compile should be, from the signalbackup-tools directory (forgiving any typos):

/usr/local/bin/g++-9 -I/usr/local/opt/openssl/include/ -std=c++2a -Wall -Wextra -Wshadow -Woverloaded-virtual -pedantic -fomit-frame-pointer -O3 -march=native -flto -o signalbackup-tools */*.cc *.cc -L/usr/local/opt/openssl/lib/ -lcrypto -lsqlite3
<--------A---------> <----------------B--------------> <---------------------------------------------------------C-------------------------------------------------> <---------D---------> <----E----> <-------------F-------------> <--------G------->

Where:
A: Explicit call to the actual g++ binary.
B: An include path to search (otherwise g++ will not find the openssl includes), there may be some other way to get this dir in your default path, but I don't know how and didn't bother to find out.
C: Options for the compiler. Apart from -std=c++2a they are all optional, change them at will if you feel strongly about them.
D: Output file
E: Input files
F: Basically the same as B, but for the library path
G: Tell the compiler to link against openssl and sqlite

I still haven't thoroughly tested the resulting binary because of having some trouble getting the files into the VM (it's refusing most of my thumb drives) and a lack of time to make work of it. But I will. In the meantime, if you get it compiled and working, I'd love to hear it!

EDIT
I have now tested the binary on one large backup file. it reads and writes (reencrypts with a new password) properly. I haven't tested with a broken backup, but I have no reason to think that will not work fine as well.

[Arzte]

I can confirm that the command in the last comment works, and produces a functional binary, I ran into this same issue compiling on macOS, and using that command allowed it to compile into a functional binary.

I'm on macOS Catalina 10.15.4 gcc 9.3.0

[bepaald]

Thanks! I will close this issue, assuming it is solved now, I might even update the README to include MacOS instructions.

Anyone can feel free to continue posting here or in a new issue as you see fit.

[HiperFall]

Hello! I need help. I get some issue. What am I doing wrong? I've installed everything you wrote.

Running /usr/local/bin/g++-11 -I/usr/local/opt/openssl/include/ -std=c++2a -Wall -Wextra -Wshadow -Woverloaded-virtual -pedantic -fomit-frame-pointer -O3 -march=native -flto -o signalbackup-tools */*.cc *.cc -L/usr/local/opt/openssl/lib/ -lcrypto -lsqlite3 inside the "signalbackup-tools" folder.

In file included from /usr/local/Cellar/gcc/11.2.0/include/c++/11.2.0/bits/max_size_type.h:37,
from /usr/local/Cellar/gcc/11.2.0/include/c++/11.2.0/bits/ranges_base.h:38,
from /usr/local/Cellar/gcc/11.2.0/include/c++/11.2.0/string_view:48,
from /usr/local/Cellar/gcc/11.2.0/include/c++/11.2.0/bits/basic_string.h:48,
from /usr/local/Cellar/gcc/11.2.0/include/c++/11.2.0/string:55,
from /usr/local/Cellar/gcc/11.2.0/include/c++/11.2.0/bits/locale_classes.h:40,
from /usr/local/Cellar/gcc/11.2.0/include/c++/11.2.0/bits/ios_base.h:41,
from /usr/local/Cellar/gcc/11.2.0/include/c++/11.2.0/ios:42,
from /usr/local/Cellar/gcc/11.2.0/include/c++/11.2.0/ostream:38,
from /usr/local/Cellar/gcc/11.2.0/include/c++/11.2.0/iostream:39,
from main.cc:20:
/usr/local/Cellar/gcc/11.2.0/include/c++/11.2.0/numbers:60:7: internal compiler error: Illegal instruction: 4
60 | = _Enable_if_floating<_Tp>(2.718281828459045235360287471352662498L);
| ^
Please submit a full bug report,
with preprocessed source if appropriate.
See https://github.com/Homebrew/homebrew-core/issues for instructions.

[bepaald]

Hi! What you are seeing is not a problem in my program but an error in the compiler. Not sure if this is an actual bug in gcc 11.2 or with the homebrew package specifically, but you should probably do what it says at the end:

Please submit a full bug report,
with preprocessed source if appropriate.
See https://github.com/Homebrew/homebrew-core/issues for instructions.

Since getting this problem fixed will probably take some time, maybe you should try installing an older version of gcc, reading the above comments gcc 9 was apparently working. I still see that version (and others) in the homebrew github, so it should be available. Not sure how to install a specific version though, maybe try brew install gcc-9 or brew install gcc@9? And maybe remove version 11 first, just to be safe?

[HiperFall]

It didn't work for me, so I just installed Fedora as a second system and it worked... 😅

But I have some questions. In May I accidentally lost my backup and restored a backup from January. So I have a gap between January and May. I dumped the whole database in a separate folder. I also obtained a backup from my friend, I had only chatting with him. So, what are the steps now to properly edit the database, which tables I have to edit, and which I don't. And if I finish, how can I re-encrypt it?

[bepaald]

Ok, if I understand correctly, you want to import a single conversation from someone else's backup? This should be possible but is certainly not trivial. I suggest you open a new issue for this, and I'll try to walk you through it. Though I must admit I'm not 100% sure how to do this yet, you'd be the first.

Cropping your friends backup to the right conversation and to the right dates is easy (you could do that using the --croptodates and --croptothreads functions, see the README), but actually reversing the messages, so that the outgoing messages turn into incoming ones and vice versa is going to be exciting.

Please forgive me if I ever don't reply for a few days, sometimes I have busy periods (right now as it happens).

[HiperFall]

I am busy too, so it's not a problem. Thanks a lot for your answer. Maybe there is a way to completely reverse all the messages in the friend's backup and then cropping the backup? I will open an issue here as soon as I have free time. Maybe, if reversing messages is possible in friend's backup, then there is no need to crop it, I'll just have to reverse messages and restore the whole backup. But the question is how I can do that.

[HiperFall]

I just want to make sure that you're alive. Have you read my comment? @bepaald

[bepaald]

Hi @HiperFall! Yes I'm still alive, thank you for your concern. I must admit I more or less forgot about your problem, since I asked you to create a new issue, and you said you were going to. If you had, github would have mailed me a reminder :)

Anyway, I created one, so let's continue over there!

[fast-reflexes]

Thanks a lot for this thread! KUDOS ❤️ to you for creating this wonderful tool but also for being so helpful when people reach out. Worked like a charm for me after just changing some paths and then I could extract all the things I wanted :) Thanks again!