AURORA: GFF3Writer #300

Nostritius · 2018-06-19T20:31:20Z

I thought long about my first attempt on a gff3 writer, but ultimately decided that it would be better to write an external class for exporting gff3, since the internal structure of the GFF3File class makes it very difficult to modify values. Also it is difficult to get the necessary information for exporting in an easy way. This class is only for generating a gff3 tree and export it. I have made many unit tests and also tested it against the original kotor executable by replacing a simple file by one created by the writer, and it behaved as i expected.

DrMcCoy · 2018-06-25T09:47:31Z

https://gist.github.com/DrMcCoy/43b2c2fbed18779be7ac2e631c2eacb4

Nostritius · 2018-06-25T10:10:53Z

The memory leaks should now be fixed.

DrMcCoy · 2018-06-25T13:19:40Z

src/aurora/gff3writer.cpp

+		} else {
+			// If the values are complex (greater then 4 bytes) write the index to the field data.
+			stream.writeUint32LE(fieldDataIndex);
+			switch (field.type) {


Isn't that a repeat of lines 61ff?

Move that into a method getFieldSize() or something like that.

DrMcCoy · 2018-06-25T13:20:41Z

src/aurora/gff3writer.cpp

+
+	// Write labels.
+	for (size_t i = 0; i < _labels.size(); ++i) {
+		Common::UString label = _labels[i];


Is that copy there necessary?

DrMcCoy · 2018-06-25T13:26:23Z

src/aurora/gff3writer.cpp

+namespace Aurora {
+
+GFF3Writer::GFF3Writer(uint32 id, uint32 version) : _id(id), _version(version) {
+	_structs.push_back(GFF3WriterStructPtr(new GFF3WriterStruct(this)));


Can you use boost::make_shared instead of new'ing the struct into the shared_ptr (here and everywhere else as well)?

I.e. boost::make_shared<GFF3WriterStruct>(this) should be equivalent to boost::shared_ptr<GFF3WriterStruct>(new GFF3WriterStruct(this)), and gets rid of the explicit new.

DrMcCoy · 2018-06-25T13:29:06Z

src/aurora/gff3writer.cpp

+		 * Determine if this field has simple values (less equal 32 bit) which are written in the field
+		 * or complex values, bigger than 32bit like strings written in the field data section.
+		 */
+		bool simple =


That's a const.

DrMcCoy · 2018-06-25T13:30:56Z

src/aurora/gff3writer.cpp

+				break;
+			case GFF3Struct::kFieldTypeResRef:
+				stream.writeByte(MIN(static_cast<byte>(16), static_cast<byte>(field.stringValue.size())));
+				stream.write(field.stringValue.c_str(), field.stringValue.size());


You still might write a longer string than 16 bytes here

DrMcCoy · 2018-06-25T13:37:59Z

src/aurora/gff3writer.cpp

+	field.type = GFF3Struct::kFieldTypeByte;
+	field.labelIndex = _parent->addLabel(label);
+	field.uint32Value = value;
+	_parent->_fields.push_back(field);


Lots of code duplication here in these methods. Creating the field index and adding the field, i.e. everything except setting the actual value, can be abstracted into a method.

Nostritius · 2018-06-25T15:55:29Z

The issues should now be fixed

DrMcCoy · 2018-06-26T11:26:01Z

src/aurora/locstring.h

 	/** Write the LocString to a write stream. */
-	void writeLocString(Common::WriteStream &stream, bool withNullTerminate = false);
+	void writeLocString(Common::WriteStream &stream, bool withNullTerminate = false) const ;


There's now an extra space between the const and the ;

DrMcCoy · 2018-06-26T11:26:53Z

src/aurora/gff3writer.cpp

+
+	// Insert the struct to the field vector.
+	_parent->_fields.push_back(field);
+


Trailing whitespace

DrMcCoy · 2018-06-26T11:29:17Z

src/aurora/gff3writer.cpp

+void GFF3WriterStruct::addByte(const Common::UString &label, byte value) {
+	size_t index = _parent->createField(GFF3Struct::kFieldTypeByte, label);
+	_parent->_fields[index].uint32Value = value;
+	_fieldIndices.push_back(index);


I'd go even further. GFF3Writer::Field &GFF3WriterStruct::createField(GFF3Struct::FieldType type, const Common::UString &label). Calls _parent->createField(), put the index into _fieldIndices and returns a direct reference to the field.

Then you only need to do

createField(GFF3Struct::kFieldTypeByte, label).uint32Value = value;

etc. in each method.

DrMcCoy · 2018-06-26T11:29:44Z

src/aurora/gff3writer.h

+
+class GFF3Writer {
+public:
+	//TODO: Add a constructor consuming a GFF3File object.


DrMcCoy · 2018-06-26T11:30:06Z

src/aurora/gff3writer.h

+	/** Get the toplevel struct. */
+	GFF3WriterStructPtr getTopLevelStruct();
+
+	/** Write the gff3 to stream. */


Please, capitalize GFF and GFF3.

DrMcCoy · 2018-06-26T11:31:44Z

src/aurora/gff3writer.h

+			stringValue = field.stringValue;
+			locStringValue = field.locStringValue;
+
+			if (field.voidData.get() != 0) {


You should still do an unconditional field.voidData.reset() before that. Otherwise, "a = b", with a.voidData filled and b.voidData empty will not empty a.voidData.

DrMcCoy · 2018-06-26T11:32:35Z

src/aurora/gff3writer.cpp

+	}
+
+	// Write fields.
+	unsigned int fieldDataIndex = 0;


Why the move to unsigned int instead of size_t here?

DrMcCoy · 2018-06-26T11:34:20Z

src/aurora/gff3writer.cpp

+	}
+}
+
+uint32 GFF3Writer::getFieldDataSize(const Aurora::GFF3Writer::Field &field) {


That can even be a static function in this file's scope. It doesn't need to be a method of GFF3Writer.

Nostritius · 2018-06-27T14:46:07Z

The issues should now be fixed

DrMcCoy · 2018-07-26T19:35:39Z

Merged as 4971a26...da35c58, thanks! :)

Nostritius mentioned this pull request Jun 19, 2018

AURORA: Add support for writing gff3 (WiP) #274

Closed

Nostritius force-pushed the aurora_gff3writer branch 3 times, most recently from c7adebe to 29f5a1e Compare June 24, 2018 16:01

Nostritius force-pushed the aurora_gff3writer branch from 29f5a1e to 3fb0982 Compare June 25, 2018 10:10

DrMcCoy requested changes Jun 25, 2018

View reviewed changes

Nostritius force-pushed the aurora_gff3writer branch from 3fb0982 to 9d15b0f Compare June 25, 2018 15:54

DrMcCoy requested changes Jun 26, 2018

View reviewed changes

Nostritius force-pushed the aurora_gff3writer branch from 9d15b0f to 061090d Compare June 27, 2018 14:40

Nostritius added 3 commits June 27, 2018 16:43

AURORA: Add a writer for gff3 files

3e0421e

TESTS: Add unit tests for the gff3writer class

0291f5e

AURORA: Make LocString write and getWrittenSize const

c947e90

Nostritius force-pushed the aurora_gff3writer branch from 061090d to c947e90 Compare June 27, 2018 14:45

DrMcCoy added this to File format writers in TODO: Standalone Jul 9, 2018

DrMcCoy closed this Jul 26, 2018

Nostritius deleted the aurora_gff3writer branch December 8, 2018 11:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AURORA: GFF3Writer #300

AURORA: GFF3Writer #300

Nostritius commented Jun 19, 2018

DrMcCoy commented Jun 25, 2018

Nostritius commented Jun 25, 2018

DrMcCoy Jun 25, 2018

DrMcCoy Jun 25, 2018

DrMcCoy Jun 25, 2018

DrMcCoy Jun 25, 2018

DrMcCoy Jun 25, 2018

DrMcCoy Jun 25, 2018

Nostritius commented Jun 25, 2018

DrMcCoy Jun 26, 2018

DrMcCoy Jun 26, 2018

DrMcCoy Jun 26, 2018

DrMcCoy Jun 26, 2018

DrMcCoy Jun 26, 2018

DrMcCoy Jun 26, 2018

DrMcCoy Jun 26, 2018

DrMcCoy Jun 26, 2018

Nostritius commented Jun 27, 2018

DrMcCoy commented Jul 26, 2018


		// Insert the struct to the field vector.
		_parent->_fields.push_back(field);

AURORA: GFF3Writer #300

AURORA: GFF3Writer #300

Conversation

Nostritius commented Jun 19, 2018

DrMcCoy commented Jun 25, 2018

Nostritius commented Jun 25, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Nostritius commented Jun 25, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Nostritius commented Jun 27, 2018

DrMcCoy commented Jul 26, 2018