Skip to content

Implement nullable reflected fields #37

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jkalias wants to merge 5 commits into from
Closed

Implement nullable reflected fields #37

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
5806a12
Implement nullable reflected fields
jkalias Jul 18, 2026
6296389
Add test include path for SQLite internals
jkalias Jul 18, 2026
aefc4f9
Use portable temp path in schema test
jkalias Jul 18, 2026
232bdf0
Address nullable review feedback
jkalias Jul 18, 2026
d1bb7f6
Fix legacy null hydration tests with NOT NULL schema
jkalias Jul 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,12 @@ FetchContent_Declare(
googletest
URL https://github.com/google/googletest/archive/refs/tags/release-1.12.1.zip
)
FetchContent_MakeAvailable(googletest)
FetchContent_Declare(
functional_cpp
URL https://github.com/jkalias/functional_cpp/archive/refs/tags/1.1.1.zip
SOURCE_SUBDIR src
)
FetchContent_MakeAvailable(googletest functional_cpp)

# Set compiler settings based on the current platform
if(CMAKE_HOST_UNIX)
Expand Down
9 changes: 9 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,8 +133,17 @@ Supported field macros:
| `std::wstring` | `MEMBER_TEXT(name)` |
| `bool` | `MEMBER_BOOL(name)` |
| `sqlite_reflection::TimePoint` | `MEMBER_DATETIME(name)` |
| `fcpp::optional_t<int64_t>` | `MEMBER_INT_NULLABLE(name)` |
| `fcpp::optional_t<double>` | `MEMBER_REAL_NULLABLE(name)` |
| `fcpp::optional_t<std::wstring>` | `MEMBER_TEXT_NULLABLE(name)` |
| `fcpp::optional_t<bool>` | `MEMBER_BOOL_NULLABLE(name)` |
| `fcpp::optional_t<sqlite_reflection::TimePoint>` | `MEMBER_DATETIME_NULLABLE(name)` |
| member function declaration | `FUNC(signature)` |

Nullable macros store SQL `NULL` as an empty optional and bind an empty optional back as SQL `NULL`; present optional values round-trip using the same storage class as their non-nullable counterparts, so `NULL`, an empty string, and numeric/boolean zero remain distinguishable after fetch. The optional type comes from [`functional_cpp`](https://github.com/jkalias/functional_cpp): under C++17 and later `fcpp::optional_t<T>` aliases `std::optional<T>`, while C++11 builds use the library fallback. Use `IsNull(&T::field)` and `IsNotNull(&T::field)` predicates for null checks. Value predicates such as `Equal(&T::nullable_field, value)` compare against a present contained value; pass the contained `T` value, not an optional. If an unset optional is passed as the comparison value, the predicate constructor throws `std::invalid_argument`; use `IsNull` for that query instead.

**Schema nullability caveat.** Newly created tables now declare non-nullable reflected fields as `NOT NULL` and omit `NOT NULL` for nullable fields. Because initialization uses `CREATE TABLE IF NOT EXISTS`, existing database files keep their previous column definitions until you recreate or migrate those tables, just like the `AUTOINCREMENT` caveat below.

**Layout constraint.** Reflectable records must be simple, standard-layout structs: no base
classes, no virtual functions, no virtual/multiple inheritance. Member access is computed from
`offsetof`/pointer-to-member byte offsets, which are only well-defined for such types; a struct
Expand Down
103 changes: 103 additions & 0 deletions include/query_predicates.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ struct REFLECTION_EXPORT SqlValue {
bool bool_value;
double real_value;
std::string text_value;
bool is_null;
};

/// The base class of all WHERE predicates used in SQLite queries
Expand Down Expand Up @@ -103,13 +104,26 @@ class REFLECTION_EXPORT QueryPredicate : public QueryPredicateBase {
: QueryPredicate(fn, value, symbol,
[&](void* v, SqliteStorageClass storage_class) { return GetSqlValue(v, storage_class); }) {}

template <typename T, typename R>
QueryPredicate(fcpp::optional_t<R> T::* fn, R value, const std::string& symbol)
: QueryPredicate(fn, fcpp::optional_t<R>(value), symbol, [&](void* v, SqliteStorageClass storage_class) {
return GetOptionalSqlValue(v, storage_class);
}) {}

template <typename T, typename R>
QueryPredicate(fcpp::optional_t<R> T::* fn, fcpp::optional_t<R> value, const std::string& symbol)
: QueryPredicate(fn, value, symbol, [&](void* v, SqliteStorageClass storage_class) {
return GetOptionalSqlValue(v, storage_class);
}) {}

QueryPredicate(const std::string& symbol, const std::string& member_name, const SqlValue& value)
: symbol_(symbol), member_name_(member_name), value_(value) {}

/// Returns the value used for the current query, against which the struct member
/// (defined from the pointer-to-member function) will be compared. The value needs
/// to be type-erased, so that the header file is not bloated with unnecessary implementation details
virtual SqlValue GetSqlValue(void* v, SqliteStorageClass storage_class) const;
virtual SqlValue GetOptionalSqlValue(void* v, SqliteStorageClass storage_class) const;

/// The symbol used for the comparison, for example "=" for equality
std::string symbol_;
Expand All @@ -122,6 +136,47 @@ class REFLECTION_EXPORT QueryPredicate : public QueryPredicateBase {
SqlValue value_;
};

/// A wrapper for an empty predicate, used to fetch all elements of an SQLite table
class REFLECTION_EXPORT NullPredicate : public QueryPredicateBase {
public:
template <typename T, typename R>
NullPredicate(R T::* fn, const std::string& symbol) : symbol_(symbol) {
auto record = GetRecordFromTypeId(typeid(T).name());
auto offset = OffsetFromStart(fn);
for (auto i = 0; i < record.member_metadata.size(); ++i) {
if (record.member_metadata[i].offset == offset) {
member_name_ = record.member_metadata[i].name;
return;
}
}
throw std::runtime_error("No registered member of '" + record.name +
"' matches the given pointer-to-member (type id: " + typeid(T).name() + ")");
}

std::string Evaluate() const override;
std::vector<SqlValue> Bindings() const override;
QueryPredicateBase* Clone() const override;

protected:
NullPredicate(const std::string& symbol, const std::string& member_name)
: symbol_(symbol), member_name_(member_name) {}

std::string symbol_;
std::string member_name_;
};

class REFLECTION_EXPORT IsNull final : public NullPredicate {
public:
template <typename T, typename R>
explicit IsNull(R T::* fn) : NullPredicate(fn, "IS NULL") {}
};

class REFLECTION_EXPORT IsNotNull final : public NullPredicate {
public:
template <typename T, typename R>
explicit IsNotNull(R T::* fn) : NullPredicate(fn, "IS NOT NULL") {}
};

/// A wrapper for an empty predicate, used to fetch all elements of an SQLite table
class REFLECTION_EXPORT EmptyPredicate final : public QueryPredicateBase {
public:
Expand All @@ -137,6 +192,12 @@ class REFLECTION_EXPORT Equal final : public QueryPredicate {
template <typename T, typename R>
explicit Equal(R T::* fn, R value) : QueryPredicate(fn, value, "=") {}

template <typename T, typename R>
explicit Equal(fcpp::optional_t<R> T::* fn, R value) : QueryPredicate(fn, value, "=") {}

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 18, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject optional control values for nullable predicates

When callers pass an fcpp::optional_t<T> control value to a nullable field, e.g. Equal(&NullableRecord::optional_text, fcpp::optional_t<std::wstring>{}), overload resolution bypasses this contained-value overload and selects the generic Equal(R T::*, R) overload instead. That path treats the optional object's storage as the contained T, so empty optionals can bind garbage or throw rather than performing a null-aware comparison; either delete optional-valued controls or route them to IS NULL/contained-value handling.

Useful? React with 👍 / 👎.


template <typename T, typename R>
explicit Equal(fcpp::optional_t<R> T::* fn, fcpp::optional_t<R> value) : QueryPredicate(fn, value, "=") {}

template <typename T>
explicit Equal(int64_t T::* fn, int value) : Equal(fn, (int64_t)value) {}

Expand All @@ -151,6 +212,12 @@ class REFLECTION_EXPORT Unequal final : public QueryPredicate {
template <typename T, typename R>
explicit Unequal(R T::* fn, R value) : QueryPredicate(fn, value, "!=") {}

template <typename T, typename R>
explicit Unequal(fcpp::optional_t<R> T::* fn, R value) : QueryPredicate(fn, value, "!=") {}

template <typename T, typename R>
explicit Unequal(fcpp::optional_t<R> T::* fn, fcpp::optional_t<R> value) : QueryPredicate(fn, value, "!=") {}

template <typename T>
explicit Unequal(int64_t T::* fn, int value) : Unequal(fn, (int64_t)value) {}

Expand All @@ -167,6 +234,18 @@ class REFLECTION_EXPORT Like final : public QueryPredicate {
: QueryPredicate(fn, value, "LIKE",
[&](void* v, SqliteStorageClass storage_class) { return GetSqlValue(v, storage_class); }) {}

template <typename T, typename R>
explicit Like(fcpp::optional_t<R> T::* fn, R value)
: QueryPredicate(fn, fcpp::optional_t<R>(value), "LIKE", [&](void* v, SqliteStorageClass storage_class) {
return GetOptionalSqlValue(v, storage_class);

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 18, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Apply LIKE binding semantics to nullable fields

When a Like predicate is created for a nullable field, this path unwraps the optional through the base helper and never applies Like::GetSqlValue, which is the code that converts the binding to text, wraps it in %...%, and escapes user-supplied %/_. In C++17/GCC builds where this compiles, Like(&T::optional_text, std::wstring(L"%")) binds raw % and matches every non-NULL row instead of a literal percent, while ordinary substring searches become exact matches.

Useful? React with 👍 / 👎.

}) {}

template <typename T, typename R>
explicit Like(fcpp::optional_t<R> T::* fn, fcpp::optional_t<R> value)
: QueryPredicate(fn, value, "LIKE", [&](void* v, SqliteStorageClass storage_class) {
return GetOptionalSqlValue(v, storage_class);
}) {}

template <typename T>
explicit Like(int64_t T::* fn, int value) : Like(fn, (int64_t)value) {}

Expand All @@ -189,6 +268,12 @@ class REFLECTION_EXPORT GreaterThan final : public QueryPredicate {

template <typename T>
explicit GreaterThan(double T::* fn, double value) : QueryPredicate(fn, value, ">") {}

template <typename T>
explicit GreaterThan(fcpp::optional_t<int64_t> T::* fn, int64_t value) : QueryPredicate(fn, value, ">") {}

template <typename T>
explicit GreaterThan(fcpp::optional_t<double> T::* fn, double value) : QueryPredicate(fn, value, ">") {}
};

/// A wrapper for a comparison predicate, for which the value of the
Expand All @@ -203,6 +288,12 @@ class REFLECTION_EXPORT GreaterThanOrEqual final : public QueryPredicate {

template <typename T>
explicit GreaterThanOrEqual(double T::* fn, double value) : QueryPredicate(fn, value, ">=") {}

template <typename T>
explicit GreaterThanOrEqual(fcpp::optional_t<int64_t> T::* fn, int64_t value) : QueryPredicate(fn, value, ">=") {}

template <typename T>
explicit GreaterThanOrEqual(fcpp::optional_t<double> T::* fn, double value) : QueryPredicate(fn, value, ">=") {}
};

/// A wrapper for a comparison predicate, for which the value of the
Expand All @@ -217,6 +308,12 @@ class REFLECTION_EXPORT SmallerThan final : public QueryPredicate {

template <typename T>
explicit SmallerThan(double T::* fn, double value) : QueryPredicate(fn, value, "<") {}

template <typename T>
explicit SmallerThan(fcpp::optional_t<int64_t> T::* fn, int64_t value) : QueryPredicate(fn, value, "<") {}

template <typename T>
explicit SmallerThan(fcpp::optional_t<double> T::* fn, double value) : QueryPredicate(fn, value, "<") {}
};

/// A wrapper for a comparison predicate, for which the value of the
Expand All @@ -231,6 +328,12 @@ class REFLECTION_EXPORT SmallerThanOrEqual final : public QueryPredicate {

template <typename T>
explicit SmallerThanOrEqual(double T::* fn, double value) : QueryPredicate(fn, value, "<=") {}

template <typename T>
explicit SmallerThanOrEqual(fcpp::optional_t<int64_t> T::* fn, int64_t value) : QueryPredicate(fn, value, "<=") {}

template <typename T>
explicit SmallerThanOrEqual(fcpp::optional_t<double> T::* fn, double value) : QueryPredicate(fn, value, "<=") {}
};

/// A wrapper of a compound predicate, which combines two other predicates,
Expand Down
43 changes: 40 additions & 3 deletions include/reflection.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
#include <typeinfo>
#include <vector>

#include "optional.h"
#include "reflection_export.h"

/// The storage class in an SQLite column for a given member of a struct, for which reflection is enabled
Expand All @@ -47,11 +48,12 @@ struct REFLECTION_EXPORT Reflection {
/// This holds the metadata of a given struct member
class MemberMetadata {
public:
MemberMetadata(const std::string& _name, SqliteStorageClass _storage_class, size_t _offset)
MemberMetadata(const std::string& _name, SqliteStorageClass _storage_class, size_t _offset, bool _nullable = false)
: name(_name),
storage_class(_storage_class),
sqlite_column_name(ToSqliteColumnName(_storage_class)),
offset(_offset) {}
offset(_offset),
nullable(_nullable) {}

/// The struct variable member name, as defined in the source code
std::string name;
Expand All @@ -66,6 +68,9 @@ struct REFLECTION_EXPORT Reflection {
/// The memory offset in bytes of this member from the struct's start, including any padding bits
size_t offset;

/// Whether this member is represented by optional_t<T> and may carry SQL NULL
bool nullable;

private:
/// Helper for conversion between member storage class and SQLite column name
static const char* ToSqliteColumnName(const SqliteStorageClass storage_class) {
Expand Down Expand Up @@ -113,7 +118,9 @@ size_t OffsetFromStart(R T::* fn) {
#define CAT(A, B) CAT_NOEXPAND(A, B)

#define DEFINE_MEMBER(R, T) \
reflectable.member_metadata.push_back(Reflection::MemberMetadata(STR(R), T, offsetof(struct REFLECTABLE, R)));
reflectable.member_metadata.push_back(Reflection::MemberMetadata(STR(R), T, offsetof(struct REFLECTABLE, R), false));
#define DEFINE_MEMBER_NULLABLE(R, T) \
reflectable.member_metadata.push_back(Reflection::MemberMetadata(STR(R), T, offsetof(struct REFLECTABLE, R), true));

/// A singleton object which holds all reflectable structs, and is guaranteed to be
/// instantiated before main.cpp starts
Expand Down Expand Up @@ -167,6 +174,11 @@ struct REFLECTABLE_DLL_EXPORT REFLECTABLE {
#define MEMBER_TEXT(R) MEMBER_DECLARE(std::wstring, R)
#define MEMBER_DATETIME(R) MEMBER_DECLARE(sqlite_reflection::TimePoint, R)
#define MEMBER_BOOL(R) MEMBER_DECLARE(bool, R)
#define MEMBER_INT_NULLABLE(R) MEMBER_DECLARE(fcpp::optional_t<int64_t>, R)
#define MEMBER_REAL_NULLABLE(R) MEMBER_DECLARE(fcpp::optional_t<double>, R)
#define MEMBER_TEXT_NULLABLE(R) MEMBER_DECLARE(fcpp::optional_t<std::wstring>, R)
#define MEMBER_DATETIME_NULLABLE(R) MEMBER_DECLARE(fcpp::optional_t<sqlite_reflection::TimePoint>, R)
#define MEMBER_BOOL_NULLABLE(R) MEMBER_DECLARE(fcpp::optional_t<bool>, R)
#define FUNC(SIGNATURE)
FIELDS
#undef MEMBER_DECLARE
Expand All @@ -175,6 +187,11 @@ struct REFLECTABLE_DLL_EXPORT REFLECTABLE {
#undef MEMBER_TEXT
#undef MEMBER_DATETIME
#undef MEMBER_BOOL
#undef MEMBER_INT_NULLABLE
#undef MEMBER_REAL_NULLABLE
#undef MEMBER_TEXT_NULLABLE
#undef MEMBER_DATETIME_NULLABLE
#undef MEMBER_BOOL_NULLABLE
#undef FUNC
int64_t id;

Expand All @@ -184,13 +201,23 @@ struct REFLECTABLE_DLL_EXPORT REFLECTABLE {
#define MEMBER_TEXT(R)
#define MEMBER_DATETIME(R)
#define MEMBER_BOOL(R)
#define MEMBER_INT_NULLABLE(R)
#define MEMBER_REAL_NULLABLE(R)
#define MEMBER_TEXT_NULLABLE(R)
#define MEMBER_DATETIME_NULLABLE(R)
#define MEMBER_BOOL_NULLABLE(R)
#define FUNC(SIGNATURE) SIGNATURE;
FIELDS
#undef MEMBER_INT
#undef MEMBER_REAL
#undef MEMBER_TEXT
#undef MEMBER_DATETIME
#undef MEMBER_BOOL
#undef MEMBER_INT_NULLABLE
#undef MEMBER_REAL_NULLABLE
#undef MEMBER_TEXT_NULLABLE
#undef MEMBER_DATETIME_NULLABLE
#undef MEMBER_BOOL_NULLABLE
#undef FUNC
};

Expand Down Expand Up @@ -232,13 +259,23 @@ static std::string CAT(Register, REFLECTABLE)() {
#define MEMBER_TEXT(R) DEFINE_MEMBER(R, SqliteStorageClass::kText)
#define MEMBER_DATETIME(R) DEFINE_MEMBER(R, SqliteStorageClass::kDateTime)
#define MEMBER_BOOL(R) DEFINE_MEMBER(R, SqliteStorageClass::kBool)
#define MEMBER_INT_NULLABLE(R) DEFINE_MEMBER_NULLABLE(R, SqliteStorageClass::kInt)
#define MEMBER_REAL_NULLABLE(R) DEFINE_MEMBER_NULLABLE(R, SqliteStorageClass::kReal)
#define MEMBER_TEXT_NULLABLE(R) DEFINE_MEMBER_NULLABLE(R, SqliteStorageClass::kText)
#define MEMBER_DATETIME_NULLABLE(R) DEFINE_MEMBER_NULLABLE(R, SqliteStorageClass::kDateTime)
#define MEMBER_BOOL_NULLABLE(R) DEFINE_MEMBER_NULLABLE(R, SqliteStorageClass::kBool)
#define FUNC(SIGNATURE)
FIELDS
#undef MEMBER_INT
#undef MEMBER_REAL
#undef MEMBER_TEXT
#undef MEMBER_DATETIME
#undef MEMBER_BOOL
#undef MEMBER_INT_NULLABLE
#undef MEMBER_REAL_NULLABLE
#undef MEMBER_TEXT_NULLABLE
#undef MEMBER_DATETIME_NULLABLE
#undef MEMBER_BOOL_NULLABLE
#undef FUNC
}
return name;
Expand Down
2 changes: 2 additions & 0 deletions src/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ source_group("internal" FILES ${HEADERS_INTERNAL})

# Set Properties->General->Configuration Type to Dynamic Library (.dll/.so/.dylib)
add_library(${LIBNAME} SHARED ${HEADERS} ${HEADERS_INTERNAL} ${SOURCES})
target_link_libraries(${LIBNAME} PUBLIC fcpp)
target_include_directories(${LIBNAME} PUBLIC ${functional_cpp_SOURCE_DIR}/include)

if(CMAKE_HOST_UNIX AND NOT CMAKE_HOST_APPLE)
target_link_libraries(${LIBNAME} PUBLIC tbb dl)
Expand Down
Loading
Loading