Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1620 Compare Module and Data Source keyword #1646

Merged
merged 74 commits into from
Sep 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
340fcb3
WIP- adding compare module and data source keyword
rhinoella Aug 18, 2023
ccd0c02
WIP- adding compare module and data source keyword
rhinoella Aug 18, 2023
3e624c3
Deserialise function for datasource keyword
rhinoella Aug 18, 2023
06c248b
Initialising default tag_ in data1d class
rhinoella Aug 18, 2023
2031e30
Created data base classes
rhinoella Aug 22, 2023
7b36fb7
Created data base classes
rhinoella Aug 22, 2023
ac3e151
Use of unique_ptr
rhinoella Aug 23, 2023
713c352
Defined serialisation functions
rhinoella Aug 23, 2023
ea78dcf
WIP Process func
rhinoella Aug 23, 2023
19e20fa
Merge develop into branch to get new rangeVector keyword
rhinoella Aug 24, 2023
037e26e
Merging develop into branch to get rangeVector keyword
rhinoella Aug 24, 2023
c9acf4f
registered module and added process function
rhinoella Aug 24, 2023
cb2a533
WIP compare module widget
rhinoella Aug 29, 2023
21b6407
WIP compare module widget
rhinoella Aug 29, 2023
d3a2aea
WIP adding ui
rhinoella Aug 30, 2023
7025184
WIP adding ui
rhinoella Aug 30, 2023
a02c417
Fixed memory leak and modified function reference
rhinoella Aug 31, 2023
49dbb11
Adding dataSource class and modifying data storage method
rhinoella Sep 1, 2023
489c882
DataSource class
rhinoella Sep 4, 2023
e53e659
DataSourceKeywordBase
rhinoella Sep 5, 2023
48fe93f
DataSource object
rhinoella Sep 5, 2023
a24f22b
Merge branch 'develop' into 1620-compare-module
rhinoella Sep 5, 2023
0dcde7d
Change comments
rhinoella Sep 5, 2023
ac41dbd
Added to cmake list
rhinoella Sep 6, 2023
8573483
DataSource class updates
rhinoella Sep 6, 2023
e5f848f
Compare module process function developed
rhinoella Sep 6, 2023
cf6d49c
Merge branch '1620-compare-module' of github.com:disorderedmaterials/…
rhinoella Sep 6, 2023
9518a78
Updated dataExists() function so we do not access null data
rhinoella Sep 6, 2023
316f1f0
Improved comments for dataSource.h class
rhinoella Sep 6, 2023
143506c
Removed DataFileAndFormat base class (unneeded)
rhinoella Sep 6, 2023
48007a2
Updated comments
rhinoella Sep 6, 2023
d8dbf2e
Updated comments
rhinoella Sep 6, 2023
6684905
Updated comments on compare.h
rhinoella Sep 6, 2023
ee0d70f
Updated comments
rhinoella Sep 6, 2023
dbc6f73
Removed GUI implementation from this branch
rhinoella Sep 6, 2023
883b1fe
Added function to return data ref in compare module
rhinoella Sep 6, 2023
3e00d30
Adding template keyword to addData() call to fix mac build error
rhinoella Sep 6, 2023
00bd84c
Minor fixes: formatting, comments
rhinoella Sep 7, 2023
79e2f79
Add serialise function to dataSource class, uninherrited Serialisable
rhinoella Sep 7, 2023
a729a75
Added TOML support for dataSourceKeyword
rhinoella Sep 7, 2023
0fe886f
Comments
rhinoella Sep 7, 2023
71b364d
Changed capitisation
rhinoella Sep 7, 2023
ff5c0b0
Stored data in processing module data
rhinoella Sep 7, 2023
0e10419
DataSource now stores dataName
rhinoella Sep 7, 2023
b29b2fd
Removed error and changed data storage naming
rhinoella Sep 7, 2023
3685fba
Changing error reporting
rhinoella Sep 11, 2023
1d0d359
Adjusted data naming
rhinoella Sep 11, 2023
be2a3dd
Removed sourcing data success reporting
rhinoella Sep 11, 2023
f15e64b
Remove unused variables
rhinoella Sep 11, 2023
4fc4527
Apply suggestions from code review
rhinoella Sep 11, 2023
58e75d1
Implementing std::variant
rhinoella Sep 13, 2023
fb4746d
Moving functions out of header
rhinoella Sep 14, 2023
01541ef
Removed format type from template
rhinoella Sep 14, 2023
637a9c7
Moving functions out of header
rhinoella Sep 14, 2023
84675e0
Updated comments
rhinoella Sep 14, 2023
1d0d0f1
Update comments dataSource.h
rhinoella Sep 14, 2023
3517172
Removed unneeded copy constructors
rhinoella Sep 14, 2023
ccd1500
Formatting
rhinoella Sep 14, 2023
2a1b250
Include type traits
rhinoella Sep 14, 2023
6a62e35
Removed illegal template lambda
rhinoella Sep 14, 2023
c5bbc7c
fixed search function
rhinoella Sep 14, 2023
f66ac74
Added foward declarations to fix macos/windows builds
rhinoella Sep 15, 2023
a69c2c5
Apply suggestions from code review
rhinoella Sep 15, 2023
a844f31
Changed 'class' specifier to 'typename'
rhinoella Sep 15, 2023
f64fdce
Changed 'class' specifier to 'typename'
rhinoella Sep 15, 2023
42c1171
Changed 'class' specifier to 'typename'
rhinoella Sep 15, 2023
d3f7788
Adding forward declarations to fileformat classes in datasourcekeyword
rhinoella Sep 15, 2023
7f8493f
Rearranging cmake builds so that IO is compilled first
rhinoella Sep 15, 2023
5ca39e7
Forward declarations
rhinoella Sep 16, 2023
4d909d5
Removed forward declaration in Data1DBase
rhinoella Sep 18, 2023
40a26aa
Resolve Windows issue.
trisyoungs Sep 18, 2023
9240d5c
Removing typename keyword
rhinoella Sep 18, 2023
f5d6736
Undoing changes to CMakeLists.txt
rhinoella Sep 18, 2023
0b8d76b
Undoing changes to src/CMakeLists.txt
rhinoella Sep 18, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/classes/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ add_library(
configuration_sites.cpp
configuration_upkeep.cpp
coreData.cpp
dataSource.cpp
distributor.cpp
isotopeData.cpp
isotopologue.cpp
Expand Down Expand Up @@ -81,6 +82,7 @@ add_library(
changeStore.h
configuration.h
coreData.h
dataSource.h
distributor.h
empiricalFormula.h
interactionPotential.h
Expand Down
168 changes: 168 additions & 0 deletions src/classes/dataSource.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
// SPDX-License-Identifier: GPL-3.0-or-later
// Copyright (c) 2023 Team Dissolve and contributors

#include "classes/dataSource.h"
#include "templates/optionalRef.h"
#include <type_traits>

// Return enum options for data source types
EnumOptions<DataSource::DataSourceType> DataSource::dataSourceTypes()
{
return EnumOptions<DataSource::DataSourceType>("DataSourceType", {{Internal, "Internal"}, {External, "External"}});
}

// Return data name (tag or filename)
std::string_view DataSource::dataName() const { return dataName_; }

// Return whether or not the data has been sourced and exists
bool DataSource::dataExists() const
{
return (!internalDataSource_.empty() && dataSourceType_ == Internal) ||
(!externalDataSource_.valueless_by_exception() && !data_.valueless_by_exception() && dataSourceType_ == External);
}

// Return data source type enum
DataSource::DataSourceType DataSource::dataSourceType() const { return dataSourceType_; }

// Return internal data source
std::optional<std::string> DataSource::internalDataSource() const
{
if (!internalDataSource_.empty())
{
return internalDataSource_;
}
return std::nullopt;
}

// Return external data source
DataSource::Format &DataSource::externalDataSource() { return externalDataSource_; }

// Function to source data (only required for internal data sources)
bool DataSource::sourceData(GenericList &processingModuleData)
{
if (!dataExists())
{
return false;
}

// For Internal data sources, try to locate the data now
if (dataSourceType_ == Internal)
rhinoella marked this conversation as resolved.
Show resolved Hide resolved
{
if (std::holds_alternative<Data1D>(data_) || std::holds_alternative<SampledData1D>(data_))
{
// Locate target data from tag and cast to base
auto optData = processingModuleData.searchBase<Data1DBase, Data1D, SampledData1D>(internalDataSource_);
if (!optData)
{
return Messenger::error("No data with tag '{}' exists.\n", internalDataSource_);
}
// Set data
data_ = optData->get();
}
else
{
// For other data types just visit the variant
return std::visit(
[=](auto data)
{
// Locate target data from tag
auto optData = processingModuleData.search<const decltype(data)>(internalDataSource_);
if (!optData)
{
return Messenger::error("No data with tag '{}' exists.\n", internalDataSource_);
}
// Set data
data_ = optData->get();
return true;
},
data_);
}
}

// Return whether or not variant contains value
rhinoella marked this conversation as resolved.
Show resolved Hide resolved
return !data_.valueless_by_exception();
}

// Set internal data source
void DataSource::addData(std::string_view internalDataSource)
{
dataSourceType_ = Internal;
internalDataSource_ = internalDataSource;
// Set data name to be data tag
dataName_ = internalDataSource;
}

// Write through specified LineParser
bool DataSource::serialise(LineParser &parser, std::string_view keywordName, std::string_view prefix) const
{
if (!dataExists())
{
return false;
}

// Write source: internal/external
if (!parser.writeLineF("{}{} ", prefix, dataSourceTypes().keyword(dataSourceType_)))
{
return false;
}

// For internal data sources just need to write the tag. Externals write the file and format
if (dataSourceType_ == Internal)
{
if (!parser.writeLineF("'{}'\n", internalDataSource_))
return false;
}
else
{
return std::visit(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually a really nice use of variant/visit!

[&](auto &data)
{
// Write filename and format
if (!data.writeFilenameAndFormat(parser, prefix))
{
return false;
}

// Write extra keywords
if (!data.writeBlock(parser, prefix)) // Express as a serialisable value
{
return false;
}

// Write extra keywords
if (!data.writeBlock(parser, prefix)) // Express as a serialisable value
{
return false;
}

// End the block
if (!parser.writeLineF("End{}", dataSourceTypes().keyword(External)))
{
return false;
}

return true;
},
externalDataSource_);
}

return true;
}

// Express as a serialisable value
SerialisedValue DataSource::serialise() const
{
SerialisedValue result;

result["dataSourceType"] = dataSourceTypes().keyword(dataSourceType_);
if (dataSourceType_ == Internal)
{
result["data"] = internalDataSource_;
}
else
{
result["data"] = std::visit([](auto &data) { return data.serialise(); }, externalDataSource_);
}
rhinoella marked this conversation as resolved.
Show resolved Hide resolved

return result;
}
94 changes: 94 additions & 0 deletions src/classes/dataSource.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
// SPDX-License-Identifier: GPL-3.0-or-later
// Copyright (c) 2023 Team Dissolve and contributors

#pragma once

#include "base/serialiser.h"
#include "io/fileAndFormat.h"
#include "io/import/data1D.h"
#include "io/import/data2D.h"
#include "io/import/data3D.h"
#include "items/list.h"
#include "keywords/dataSourceBase.h"
#include "math/data1D.h"
#include "math/data2D.h"
#include "math/data3D.h"
#include "math/sampledData1D.h"
#include <variant>

class DataSource
{
public:
using DataType = std::variant<Data1D, SampledData1D, Data2D, Data3D>;
using Format = std::variant<Data1DImportFileFormat, Data2DImportFileFormat, Data3DImportFileFormat>;
rhinoella marked this conversation as resolved.
Show resolved Hide resolved

public:
DataSource() = default;
~DataSource() = default;

public:
// Data Source Types
enum DataSourceType
{
Internal,
External
};
// Return enum options for DataSourceType
static EnumOptions<DataSourceType> dataSourceTypes();
// Return data source type enum
DataSourceType dataSourceType() const;

/*
* Data
*/
private:
// Name of data (tag or filename)
std::string dataName_;
// Type of data being stored
DataSourceType dataSourceType_;
// String to hold internal data tag (if internal)
std::string internalDataSource_;
// Variant to hold file and format of external data (if external)
Format externalDataSource_;
// Variant to hold data
DataType data_;

public:
// Return data name
std::string_view dataName() const;
// Return if data exists and has been initialised
bool dataExists() const;
// Return internal data source
std::optional<std::string> internalDataSource() const;
// Return external data source
Format &externalDataSource();
// Source specified data (only required for internal data sources)
bool sourceData(GenericList &processingModuleData);
// Set internal data source
void addData(std::string_view internalDataSource);
// Overloaded function to add external data
template <class D> void addData(D data, typename D::Formatter &fileAndFormat)
{
dataSourceType_ = External;
// Create format object in place in variant
externalDataSource_.emplace<typename D::Formatter>(fileAndFormat);
data_ = data;
// Set data name to be base filename
dataName_ = fileAndFormat.filename().substr(fileAndFormat.filename().find_last_of("/\\") + 1);
}
// Returns data in the requested type
template <class D> D data() const
{
assert(dataExists());
return std::get<D>(data_);
}

rhinoella marked this conversation as resolved.
Show resolved Hide resolved
/*
* I/O
*/
public:
// Write through specified LineParser
bool serialise(LineParser &parser, std::string_view keywordName, std::string_view prefix) const;
trisyoungs marked this conversation as resolved.
Show resolved Hide resolved
// Express as a serialisable value
SerialisedValue serialise() const;
};
Loading
Loading