F.serializer #422

joka921 · 2021-07-16T09:00:06Z

TODO comment.

hannahbast

Code review in 1-1 with Johannes, thanks for the great mini library!

Johannes said: add some tests also for serializing hash maps and strings.

hannahbast · 2021-07-21T14:14:44Z

src/util/Serializer/Serializer.h

+  Storage&& data() && { return std::move(_data); }
+
+ private:
+  std::vector<char> _data;


Storage _data;

hannahbast · 2021-07-21T14:58:13Z

src/util/Serializer/FileSerializer.h

+
+  SerializationPosition getCurrentPosition() const { return _file.tell(); }
+
+  File&& moveFileOut() && { return std::move(_file); }


moveFileOut -> file

hannahbast · 2021-07-21T14:58:58Z

src/util/Serializer/FileSerializer.h

+    _file.seek(position, SEEK_SET);
+  }
+
+  File&& moveFileOut() && { return std::move(_file); }


hannahbast · 2021-07-21T15:26:30Z

src/index/Index.Text.cpp

@@ -131,7 +131,7 @@ void Index::passContextFileIntoVector(const string& contextFile,
  // this has to be repeated completely here because we have the possibility to
  // only add a text index. In that case the Vocabulary has never been
  // initialized before
-  _vocab = Vocabulary<CompressedString, TripleComponentComparator>();
+  _vocab = std::move(Vocabulary<CompressedString, TripleComponentComparator>());
  readConfiguration();


Why? Was there a problem before?

hannahbast · 2021-07-21T15:30:18Z

test/IndexTest.cpp

-    ASSERT_EQ(Id(0), *reinterpret_cast<Id*>(buf + bytesDone));
+    EXPECT_EQ(Id(0), *reinterpret_cast<Id*>(buf + bytesDone));
    bytesDone += sizeof(Id);
-    ASSERT_EQ(4u, *reinterpret_cast<Id*>(buf + bytesDone));
+    EXPECT_EQ(4u, *reinterpret_cast<Id*>(buf + bytesDone));
    bytesDone += sizeof(Id);
-    ASSERT_EQ(0u, *reinterpret_cast<Id*>(buf + bytesDone));
+    EXPECT_EQ(0u, *reinterpret_cast<Id*>(buf + bytesDone));
    bytesDone += sizeof(Id);
-    ASSERT_EQ(5u, *reinterpret_cast<Id*>(buf + bytesDone));
+    EXPECT_EQ(5u, *reinterpret_cast<Id*>(buf + bytesDone));
    bytesDone += sizeof(Id);

    // Relation b2
-    ASSERT_EQ(Id(0), *reinterpret_cast<Id*>(buf + bytesDone));
+    EXPECT_EQ(Id(0), *reinterpret_cast<Id*>(buf + bytesDone));
    bytesDone += sizeof(Id);
-    ASSERT_EQ(4u, *reinterpret_cast<Id*>(buf + bytesDone));
+    EXPECT_EQ(4u, *reinterpret_cast<Id*>(buf + bytesDone));
    bytesDone += sizeof(Id);
-    ASSERT_EQ(1u, *reinterpret_cast<Id*>(buf + bytesDone));
+    EXPECT_EQ(1u, *reinterpret_cast<Id*>(buf + bytesDone));
    bytesDone += sizeof(Id);
-    ASSERT_EQ(5u, *reinterpret_cast<Id*>(buf + bytesDone));
+    EXPECT_EQ(5u, *reinterpret_cast<Id*>(buf + bytesDone));
    bytesDone += sizeof(Id);
    // No LHS & RHS
-    ASSERT_EQ(index._PSO.metaData().getOffsetAfter(), bytesDone);
+    EXPECT_EQ(index._PSO.metaData().getOffsetAfter(), bytesDone);


Back to ASSERT_EQ

hannahbast · 2021-07-21T15:31:32Z

test/SerializerTest.cpp

+};
+
+auto testWithFileSerialization = [](auto testFunction) {
+  const std::string filename = "serializationTestTmpt.txtx";


serializationTest.tmp

hannahbast · 2021-07-21T15:33:45Z

test/SerializerTest.cpp

+};
+
+TEST(Serializer, Simple) {
+  auto simpleIntTest = [](auto& writer, auto makeReader) {


makeReaderFromWriter ?

hannahbast · 2021-07-21T15:35:51Z

test/SerializerTest.cpp

+  };
+
+  testWithAllSerializers(testmanyPrimitives);
+}


Missing newline

- It has a symmetric interface, similar to boost::serialization etc. That way, typically only one function has to be written for a type to handle reading AND writing. - It avoids the error-prone handling of manual calls to writing and reading of single bytes - We currently use it for the writing of the IndexMetaData, but it can be used in several more places (e.g. the Vocabulary) in the future.

joka921 force-pushed the f.serializer branch from ea90aea to 8931b22 Compare July 16, 2021 09:04

joka921 marked this pull request as ready for review July 16, 2021 09:16

hannahbast approved these changes Jul 21, 2021

View reviewed changes

joka921 force-pushed the f.serializer branch from e928da8 to f1f933f Compare July 22, 2021 08:06

Some changes from Hannah's review.

86ae112

joka921 force-pushed the f.serializer branch from f1f933f to 86ae112 Compare July 22, 2021 08:07

joka921 and others added 3 commits July 22, 2021 12:07

Readded the missing MetaData.

99b2ede

Addid a missing newline.

7654264

Merge branch 'master' into f.serializer

698cd50

joka921 merged commit 0630290 into ad-freiburg:master Jul 22, 2021

joka921 deleted the f.serializer branch July 22, 2021 13:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

F.serializer #422

F.serializer #422

joka921 commented Jul 16, 2021

hannahbast left a comment

hannahbast Jul 21, 2021

hannahbast Jul 21, 2021

hannahbast Jul 21, 2021

hannahbast Jul 21, 2021

hannahbast Jul 21, 2021

hannahbast Jul 21, 2021

hannahbast Jul 21, 2021

hannahbast Jul 21, 2021


		SerializationPosition getCurrentPosition() const { return _file.tell(); }

		File&& moveFileOut() && { return std::move(_file); }

F.serializer #422

F.serializer #422

Conversation

joka921 commented Jul 16, 2021

hannahbast left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment