Skip to content

Commit

Permalink
Allowed the use of the Aerospike Key instead of re-writing a key field.
Browse files Browse the repository at this point in the history
Added a new annotation field "storeInPkOnly" to use the PK field as
storing the key, rather than explicitly setting a bin in the database.
- Added unit tests for the same, both reactive and normal
- Tested with annotations, YAML config, code config.
  • Loading branch information
tim-aero committed Jun 14, 2024
1 parent b21833f commit dbd6665
Show file tree
Hide file tree
Showing 11 changed files with 308 additions and 23 deletions.
55 changes: 53 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -484,7 +484,57 @@ public String getKey() {
Note that it is not required to have a key on an object annotated with @AerospikeRecord. This is because an object can be embedded in another object (as a map or list) and hence not require a key to identify it to the database.
Also, the existence of @AerospikeKey on a field does not imply that the field will get stored in the database explicitly. Use @AerospikeBin or mapAll attribute to ensure that the key gets mapped to the database too.
Also, the existence of `@AerospikeKey` on a field does not imply that the field will get stored in the database explicitly. Use `@AerospikeBin` or `mapAll` attribute to ensure that the key gets mapped to the database too.
By default, the key will always be stored in a separate column in the database. So for a class defined as
```java
@AerospikeRecord(namespace = "test", set = "testSet")
public static class A {
@AerospikeKey
private long key;
private String value;
}
```
there will be a bin in the database called `key`, whose value will be the same as the value used in the primary key. This is because Aerospike does not implicitly store the value of the key in the database, but rather uses a hash of the primary key as a unique representation. So the value in the database might look like:
```
aql> select * from test.testSet
+-----+--------+
| key | value |
+-----+--------+
| 1 | "test" |
+-----+--------+
```
If it is desired to force the primary key to be stored in the database and NOT have key added explicitly as a column then two things must be set:
1. The `@AerospikeRecord` annotation must have `sendKey = true`
2. The `@AerospikeKey` annotation must have `storeInPkOnly = true`
So the object would look like:
```java
@AerospikeRecord(namespace = "test", set = "testSet", sendKey = true)
public static class A {
@AerospikeKey(storeInPkOnly = true)
private long key;
private String value;
}
```
When data is inserted, the field `key` is not saved, but rather the key is saved as the primary key. When the value is read from the database, the stored primary key is put back into the `key` field. So the data in the database might be:
```
aql> select * from test.testSet
+----+--------+
| PK | value |
+----+--------+
| 1 | "test" |
+----+--------+
```
----
Expand Down Expand Up @@ -679,7 +729,7 @@ Here are how standard Java types are mapped to Aerospike types:
| Map<?,?> | Map |
| Object Reference (@AerospikeRecord) | List or Map |
These types are built into the converter. However, if you wish to change them, you can use a [Custom Object Converter](#Custom-Object-Converters). For example, if you want Dates stored in the database as a string, you could do:
These types are built into the converter. However, if you wish to change them, you can use a Custom Object Converter](#custom-object-converters). For example, if you want Dates stored in the database as a string, you could do:
```java
public static class DateConverter {
Expand Down Expand Up @@ -1975,6 +2025,7 @@ The key structure is used to specify the key to a record. Keys are optional in s

The key structure contains:
- **field**: The name of the field which to which this key is mapped. If this is provided, the getter and setter cannot be provided.
- **storeInPkOnly**: Do not store the primary key column in the database, but rather use the `sendKey` facility related to Aerospike to save the key in the database. When the record is read, the value will be pulled back and place in the key field.
- **getter**: The getter method used to populate the key. This must be used in conjunction with a setter method, and excludes the use of the field attribute.
- **setter**: The setter method used to map data back to the Java key. This is used in conjunction with the getter method and precludes the use of the field attribute. Note that the return type of the getter must match the type of the first parameter of the setter, and the setter can have either 1 or 2 parameters, with the second (optional) parameter being either of type [com.aerospike.client.Key](https://www.aerospike.com/apidocs/java/com/aerospike/client/Key.html) or Object.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,6 @@
* The setter attribute is used only on Methods where the method is used to set the key on lazy object instantiation
*/
boolean setter() default false;

boolean storeInPkOnly() default false;
}
8 changes: 4 additions & 4 deletions src/main/java/com/aerospike/mapper/tools/AeroMapper.java
Original file line number Diff line number Diff line change
Expand Up @@ -221,7 +221,7 @@ private <T> T read(Policy readPolicy, @NotNull Class<T> clazz, @NotNull Key key,
try {
ThreadLocalKeySaver.save(key);
LoadedObjectResolver.begin();
return mappingConverter.convertToObject(clazz, record, entry, resolveDependencies);
return mappingConverter.convertToObject(clazz, key, record, entry, resolveDependencies);
} catch (ReflectiveOperationException e) {
throw new AerospikeException(e);
} finally {
Expand Down Expand Up @@ -252,7 +252,7 @@ private <T> T[] readBatch(BatchPolicy batchPolicy, @NotNull Class<T> clazz, @Not
} else {
try {
ThreadLocalKeySaver.save(keys[i]);
T result = mappingConverter.convertToObject(clazz, records[i], entry, false);
T result = mappingConverter.convertToObject(clazz, keys[i], records[i], entry, false);
results[i] = result;
} catch (ReflectiveOperationException e) {
throw new AerospikeException(e);
Expand Down Expand Up @@ -372,7 +372,7 @@ public <T> void scan(ScanPolicy policy, @NotNull Class<T> clazz, @NotNull Proces
AtomicBoolean userTerminated = new AtomicBoolean(false);
try {
mClient.scanAll(policy, namespace, setName, (key, record) -> {
T object = this.getMappingConverter().convertToObject(clazz, record);
T object = this.getMappingConverter().convertToObject(clazz, key, record);
if (!processor.process(object)) {
userTerminated.set(true);
throw new AerospikeException.ScanTerminated();
Expand Down Expand Up @@ -420,7 +420,7 @@ public <T> void query(QueryPolicy policy, @NotNull Class<T> clazz, @NotNull Proc
RecordSet recordSet = mClient.query(policy, statement);
try {
while (recordSet.next()) {
T object = this.getMappingConverter().convertToObject(clazz, recordSet.getRecord());
T object = this.getMappingConverter().convertToObject(clazz, recordSet.getKey(), recordSet.getRecord());
if (!processor.process(object)) {
break;
}
Expand Down
38 changes: 33 additions & 5 deletions src/main/java/com/aerospike/mapper/tools/ClassCacheEntry.java
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ public class ClassCacheEntry<T> {
private final Class<T> clazz;
private ValueType key;
private String keyName = null;
private boolean keyOnlyInPK = false;
private final TreeMap<String, ValueType> values = new TreeMap<>();
private ClassCacheEntry<?> superClazz;
private int binCount;
Expand Down Expand Up @@ -665,9 +666,18 @@ private void loadFieldsFromClass() {
if (key != null) {
throw new AerospikeException("Class " + clazz.getName() + " cannot have a more than one key");
}
AerospikeKey keyAnnotation = thisField.getAnnotation(AerospikeKey.class);
boolean storeInPkOnly = (keyAnnotation != null && keyAnnotation.storeInPkOnly());
if (keyConfig != null && keyConfig.getStoreInPkOnly() != null) {
storeInPkOnly = keyConfig.getStoreInPkOnly();
}
if (storeInPkOnly && (this.sendKey == null || !this.sendKey)) {
throw new AerospikeException("Class " + clazz.getName() + " attempts to store primary key information inside the aerospike key, but sendKey is not true at the record level");
}
AnnotatedType annotatedType = new AnnotatedType(config, thisField);
TypeMapper typeMapper = TypeUtils.getMapper(thisField.getType(), annotatedType, this.mapper);
this.key = new ValueType.FieldValue(thisField, typeMapper, annotatedType);
this.keyOnlyInPK = storeInPkOnly;
isKey = true;
}

Expand Down Expand Up @@ -850,6 +860,10 @@ public Bin[] getBins(Object instance, boolean allowNullBins, String[] binNames)
while (thisClass != null) {
Set<String> keys = thisClass.values.keySet();
for (String name : keys) {
if (name.equals(thisClass.keyName) && thisClass.keyOnlyInPK) {
// Do not explicitly write the key to the bin
continue;
}
if (contains(binNames, name)) {
ValueType value = (ValueType) thisClass.values.get(name);
Object javaValue = value.get(instance);
Expand Down Expand Up @@ -948,15 +962,15 @@ public List<Object> getList(Object instance, boolean skipKey, boolean needsType)
}

public T constructAndHydrate(Map<String, Object> map) {
return constructAndHydrate(null, map);
return constructAndHydrate(null, null, map);
}

public T constructAndHydrate(Record record) {
return constructAndHydrate(record, null);
public T constructAndHydrate(Key key, Record record) {
return constructAndHydrate(key, record, null);
}

@SuppressWarnings("unchecked")
private T constructAndHydrate(Record record, Map<String, Object> map) {
private T constructAndHydrate(Key key, Record record, Map<String, Object> map) {
Map<String, Object> valueMap = new HashMap<>();
try {
ClassCacheEntry<?> thisClass = this;
Expand All @@ -976,7 +990,21 @@ private T constructAndHydrate(Record record, Map<String, Object> map) {
while (thisClass != null) {
for (String name : thisClass.values.keySet()) {
ValueType value = thisClass.values.get(name);
Object aerospikeValue = record == null ? map.get(name) : record.getValue(name);
Object aerospikeValue;
if (record == null) {
aerospikeValue = map.get(name);
}
else if (name.equals(thisClass.keyName) && thisClass.keyOnlyInPK) {
if (key.userKey != null) {
aerospikeValue = key.userKey.getObject();
}
else {
throw new AerospikeException("Key field on class " + className + " was <null> for key " + key + ". Was the record saved passing 'sendKey = true'? ");
}
}
else {
aerospikeValue = record.getValue(name);
}
valueMap.put(name, value.getTypeMapper().fromAerospikeFormat(aerospikeValue));
}
if (result == null) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -201,7 +201,7 @@ private <T> Mono<T> read(Policy readPolicy, @NotNull Class<T> clazz, @NotNull Ke
.map(keyRecord -> {
try {
ThreadLocalKeySaver.save(key);
return mappingConverter.convertToObject(clazz, keyRecord.record, entry, resolveDependencies);
return mappingConverter.convertToObject(clazz, key, keyRecord.record, entry, resolveDependencies);
} catch (ReflectiveOperationException e) {
throw new AerospikeException(e);
} finally {
Expand Down Expand Up @@ -230,7 +230,7 @@ private <T> Flux<T> readBatch(BatchPolicy batchPolicy, @NotNull Class<T> clazz,
.map(keyRecord -> {
try {
ThreadLocalKeySaver.save(keyRecord.key);
return mappingConverter.convertToObject(clazz, keyRecord.record, entry, true);
return mappingConverter.convertToObject(clazz, keyRecord.key, keyRecord.record, entry, true);
} catch (ReflectiveOperationException e) {
throw new AerospikeException(e);
} finally {
Expand Down Expand Up @@ -324,7 +324,7 @@ public <T> Flux<T> scan(ScanPolicy policy, @NotNull Class<T> clazz, int recordsP
String setName = entry.getSetName();

return reactorClient.scanAll(policy, namespace, setName)
.map(keyRecord -> getMappingConverter().convertToObject(clazz, keyRecord.record));
.map(keyRecord -> getMappingConverter().convertToObject(clazz, keyRecord.key, keyRecord.record));
}

@Override
Expand All @@ -344,7 +344,7 @@ public <T> Flux<T> query(QueryPolicy policy, @NotNull Class<T> clazz, Filter fil
statement.setSetName(entry.getSetName());

return reactorClient.query(policy, statement)
.map(keyRecord -> getMappingConverter().convertToObject(clazz, keyRecord.record));
.map(keyRecord -> getMappingConverter().convertToObject(clazz, keyRecord.key, keyRecord.record));
}

@Override
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -231,6 +231,16 @@ public Builder withKeyField(String fieldName) {
return this;
}

public Builder withKeyFieldAndStorePkOnly(String fieldName, boolean storePkOnly) {
if (this.classConfig.getKey() == null) {
this.classConfig.setKey(new KeyConfig());
}
this.validateFieldExists(fieldName);
this.classConfig.getKey().setField(fieldName);
this.classConfig.getKey().setStoreInPkOnly(storePkOnly);
return this;
}

public Builder withKeyGetterAndSetterOf(String getterName, String setterName) {
if (this.classConfig.getKey() == null) {
this.classConfig.setKey(new KeyConfig());
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ public class KeyConfig {
private String field;
private String getter;
private String setter;
private Boolean storeInPkOnly;

public String getField() {
return field;
Expand All @@ -18,7 +19,14 @@ public String getGetter() {
public String getSetter() {
return setter;
}

public Boolean getStoreInPkOnly() {
return storeInPkOnly;
}

public void setStoreInPkOnly(boolean value) {
this.storeInPkOnly = value;
}

public void setField(String field) {
this.field = field;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -78,9 +78,9 @@ public <T> T translateFromAerospike(@NotNull Object obj, @NotNull Class<T> expec
* @return A virtual list.
* @throws AerospikeException an AerospikeException will be thrown in case of an encountering a ReflectiveOperationException.
*/
public <T> T convertToObject(Class<T> clazz, Record record) {
public <T> T convertToObject(Class<T> clazz, Key key, Record record) {
try {
return convertToObject(clazz, record, null);
return convertToObject(clazz, key, record, null);
} catch (ReflectiveOperationException e) {
throw new AerospikeException(e);
}
Expand All @@ -96,18 +96,18 @@ public <T> T convertToObject(Class<T> clazz, Record record) {
* @return A virtual list.
* @throws AerospikeException an AerospikeException will be thrown in case of an encountering a ReflectiveOperationException.
*/
public <T> T convertToObject(Class<T> clazz, Record record, ClassCacheEntry<T> entry) throws ReflectiveOperationException {
return this.convertToObject(clazz, record, entry, true);
public <T> T convertToObject(Class<T> clazz, Key key, Record record, ClassCacheEntry<T> entry) throws ReflectiveOperationException {
return this.convertToObject(clazz, key, record, entry, true);
}

/**
* This method should not be used, it is public only to allow mappers to see it.
*/
public <T> T convertToObject(Class<T> clazz, Record record, ClassCacheEntry<T> entry, boolean resolveDependencies) throws ReflectiveOperationException {
public <T> T convertToObject(Class<T> clazz, Key key, Record record, ClassCacheEntry<T> entry, boolean resolveDependencies) throws ReflectiveOperationException {
if (entry == null) {
entry = ClassCache.getInstance().loadClass(clazz, mapper);
}
T result = entry.constructAndHydrate(record);
T result = entry.constructAndHydrate(key, record);
if (resolveDependencies) {
resolveDependencies(entry);
}
Expand Down Expand Up @@ -252,7 +252,7 @@ public void resolveDependencies(ClassCacheEntry<?> parentEntity) {
DeferredObjectLoader.DeferredObjectSetter thisObjectSetter = deferredObjects.get(i);
try {
ThreadLocalKeySaver.save(keys[i]);
Object result = records[i] == null ? null : convertToObject((Class) thisObjectSetter.getObject().getType(), records[i], classCacheEntryList.get(i), false);
Object result = records[i] == null ? null : convertToObject((Class) thisObjectSetter.getObject().getType(), keys[i], records[i], classCacheEntryList.get(i), false);
thisObjectSetter.getSetter().setValue(result);
} catch (ReflectiveOperationException e) {
throw new AerospikeException(e);
Expand Down
2 changes: 1 addition & 1 deletion src/test/java/com/aerospike/mapper/AeroMapperBaseTest.java
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ public static void setupClass() {
ClientPolicy policy = new ClientPolicy();
// Set event loops to use in asynchronous commands.
policy.eventLoops = new NioEventLoops(1);
client = new AerospikeClient(policy, "localhost", 3000);
client = new AerospikeClient(policy, "localhost", 3100);
}

@AfterAll
Expand Down
Loading

0 comments on commit dbd6665

Please sign in to comment.