-
Notifications
You must be signed in to change notification settings - Fork 0
Wiki of NoJPA
NoJPA is an open-source framework built by LESS IS MORE, designed for developers by developers. NoJPA makes all the difficult technical stuff - nice and easy. This means that the developer can focus on implementing business logic and not get stuck in the technical things. NoJPA - is inspired by functionality in the JPA framework from SUN/Oracle. LESS IS MORE have removed all the “nonsense”, made it intelligent and enabled it for NoSQL databases. NoJPA is transparent no matter if it is running against traditional SQL databases or NoSQL databases. NoJPA - is a “type strong” framework, which means - if it compiles, it runs.
NoJPA is open source under Apache License 2.0
- How to create a ModelObject?
- How to create a database?
- How to alter the database, when running in production?
- How to save a ModelObject to a SQL and NoSQL database?
- How to search in a SQL database?
- How to search in a NoSQL - database?
- How to search in an Array?
- How to search by ID in NQL?
- How to use a visitor in NoJPA?
- NoJPA Annotations
- Project Samples
- TOP 5 Coolest features
- NoJPAMapper
- The future and history of NoJPA
The project is always split into two parts: backend and frontend
Inside the backend you can find such packages as:
-
controllerAll the web controllers go here. -
modelAll the ModelObjects go here. -
springAlmost always we use spring for our projects, all the spring related configuration goes here. -
servicesAll the services that interact with our database go here. -
utilsstatic and stateless methods that help do business work go here. -
helperclasses that can be stateful or require an instance to be created go here.
Inside the frontend we have:
-
webappwhere you will findviews,css,js,images,lib. -
resourceshere we usually place .properties files and it's also the place we keep our solr core's config.
In order to use NoJPA simply add these dependencies to your pom.xml file:
<!-- nojpa -->
<dependency>
<groupId>nojpa</groupId>
<artifactId>nojpa_orm</artifactId>
<version>0.1-SNAPSHOT</version>
<exclusions>
<exclusion>
<groupId>org.apache.solr</groupId>
<artifactId>solr-solrj</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.solr</groupId>
<artifactId>solr-core</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>nojpa</groupId>
<artifactId>nojpa_spring</artifactId>
<version>0.1-SNAPSHOT</version>
<exclusions>
<exclusion>
<groupId>org.apache.solr</groupId>
<artifactId>solr-solrj</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.solr</groupId>
<artifactId>solr-core</artifactId>
</exclusion>
</exclusions>
</dependency> <repository>
<id>nojpaRepository</id>
<name>NoJPA repository</name>
<url>http://maven.less-is-more.dk</url>
<snapshots><enabled>true</enabled></snapshots>
</repository>In order to use a Solr database along with NoJPA, you'll have to configure it first.
First of all start from updating your pom.xml with these dependencies:
<!-- solrj dependencies -->
<dependency>
<groupId>org.apache.solr</groupId>
<artifactId>solr-core</artifactId>
<version>${solr.version}</version>
</dependency>
<!-- properties -->
<properties>
<solr.version>4.9.0</solr.version>
</properties>You'll also need to place your solr core config directory inside a resources folder.
Inside this config directory (lets say core name: nojpa and we placed it in frontend/src/main/resources/solr)
at src/main/resources/solr/nojpa/conf/ you'll need to edit your solrconfig.xml.
Make sure your version matches the one you downloaded from Solr download page
<luceneMatchVersion>4.9</luceneMatchVersion>Change the lib dir paths to match the place you have saved solr to.
<lib dir="../../../contrib/extraction/lib" regex=".*\.jar" />
<lib dir="../../../dist/" regex="solr-cell-\d.*\.jar" />
<lib dir="../../../contrib/clustering/lib/" regex=".*\.jar" />
<lib dir="../../../dist/" regex="solr-clustering-\d.*\.jar" />
<lib dir="../../../contrib/langid/lib/" regex=".*\.jar" />
<lib dir="../../../dist/" regex="solr-langid-\d.*\.jar" />
<lib dir="../../../contrib/velocity/lib" regex=".*\.jar" />
<lib dir="../../../dist/" regex="solr-velocity-\d.*\.jar" />Also choose where you will store your solr index
<dataDir>/raid0/reqxl_index</dataDir>Next create a NoJPA.properties file inside your resources folder, here you'll link your ModelObjects to a solr core. It's also comfortable to create parameters here for database and test data creation during startup or set the option to alter the database that is perhaps already running in production.
createDatabase = true
alterDatabase = false
createTestdata = true
# solr server settings
userSolr.coreName = nojpa
Afterwards create a SolrConfig.class in your config folder (lets say at backend/src/main/java/test/config/)
Here you'll have to specify each of your ModelObjects a solr core. So if we had just one User ModelObject it would look like this.
@PropertySource("classpath:/NoJpa.properties")
@Configuration
public class SolrConfig {
@Bean
public SolrService userSolrService(@Value("${userSolr.coreName}") String coreName,
@Value("${createDatabase}") boolean cleanOnStartup) {
return getSolrService(User.class, coreName, cleanOnStartup);
}
private SolrService getSolrService(Class<? extends ModelObjectInterface> clazz, String coreName, boolean cleanOnStartup) {
SolrServiceImpl solrService = new SolrServiceImpl();
solrService.setCoreName(coreName);
solrService.setCleanOnStartup(cleanOnStartup);
ModelObjectSearchService.addSolrServer(clazz, solrService.getServer());
return solrService;
}
}Finally all that left is to create a place where your database will be created. Look in the Getting Started section How to create a database for an example of this.
To create a ModelObject, you simply create an interface which extends ModelObjectInterface. Afterwards you create getters and setters for all the fields you want. Make sure to follow standard Java naming convention. All ModelObject's get two fields automatically: objectID(String) and creationDate(Calendar).
The example below will result in the creation of a ModelObject with the the fields: name, lastName, address, objectID, creationDate.
public interface User extends ModelObjectInterface {
@SearchField
String getName();
void setName(String name);
@SearchField
String getLastName();
void setLastName(String lastName);
@SearchField
String getAddress();
void setAddress(String address);
}Allowed types inside a ModelObject are String, Enum, long, double, float, int and arrays of ModelObjects. (In the future also arrays of primitives).
Finally you can have a service class method for creating your model objects.
Example:
@Service
public class AddressServiceImpl implements AddressService {
@Override
public Address create(String country, String city, String streetAndHouse, String zipCode) {
Address address = ModelObjectService.create(Address.class);
address.setCountry(country);
address.setCity(city);
address.setStreetAndHouse(streetAndHouse);
address.setZipCode(zipCode);
ModelObjectService.save(address);
ModelObjectSearchService.put(address);
return address;
}addressService.create("Lithuania", "Vilnius", "street 5", "11111");The creation of the databases is pretty much done with just one line of code:
DatabaseCreator.createDatabase("dk.lessismore.test.model");
Usually we do it inside a InitDatabaseServiceImpl.class, here's an example:
@PropertySource("classpath:/NoJpa.properties")
@Service
public class InitDatabaseServiceImpl {
private static final Logger log = LoggerFactory.getLogger(InitDatabaseServiceImpl.class);
@Autowired
private Environment environment;
@Value("${createDatabase}")
private boolean createDatabase;
@Value("${alterDatabase}")
private boolean alterDatabase;
@Value("${createTestData}")
private boolean createTestData;
@PostConstruct
public void preInit() {
try {
ConnectionPoolFactory.configure(NoJpaDatabaseProperties.from(environment));
} catch (Exception e) {
}
}
public void init() throws Exception {
try {
if (createDatabase) {
createDatabase();
}
if (alterDatabase) {
alterDatabase();
}
if (createTestData) {
createTestData();
}
} catch (Exception e) {
log.error("InitDatabaseServiceImpl init(): e.getMessage() = " + e.getMessage(), e);
}
}
public void createDatabase() {
log.debug("------------------- Will add database");
DatabaseCreator.createDatabase("dk.lessismore.proxytranslate.model");
DatabaseCreator.createDatabase("dk.lessismore.nojpa.webservicelog");
}
public void alterDatabase() throws Exception {
log.debug("------------------- Will alter database");
DatabaseCreator.alterDatabase("dk.lessismore.proxytranslate.model");
}
private void createTestData() throws FileNotFoundException {
//Create test data here
}
}As seen above you can alter the database also with just one line of code:
DatabaseCreator.alterDatabase("dk.lessismore.proxytranslate.model");
Saving (and Updating) in NoJPA is pretty straightforward.
- To save your object inside a SQL table:
ModelObjectService.save(modelObject); - To put your object to Solr:
ModelObjectSearchService.put(modelObject);
So you could use a method like this to save changes to your model objects:
private void save(User user) {
ModelObjectService.save(user);
ModelObjectSearchService.put(user);
}First of all, you need to "mock" the ModelObject class of the table you want to search in:
User mUser = MQL.mock(User.class);
Afterwards you write the query. The simplest query to find a user by name would look like this:
MQL.select(mUser).where(mUser.getName(), MQL.Comp.EQUAL, "Bob")
This query is pretty self-explanatory. You .select from the user table, and use the .where to write your constraint, to get only user with the name Bob. And finally with .getList() you specify that you want to receive a list of users.
When writing SQL queries you can use the following to compare values.
- EQUAL
- NOT_EQUAL
- LESS
- GREATER
- EQUAL_OR_LESS
- EQUAL_OR_GREATER
- LIKE
- NOT_LIKE
Examples:
MQL.select(mUser).where(mUser.getName(), MQL.Comp.EQUAL, "Bob").getList();
MQL.select(mUser).where(mUser.getName(), MQL.Comp.LIKE, variable + "%").getList();
MQL.select(mUser).where(mUser.getAge(), MQL.Comp.GREATER, 18).getList();The usage is pretty intuitive .where(mock value, MQL Comp, value)
the mock value argument is your modelObjects field, MQL Comp is a compare instruction, and value is whatever you're filtering by.
This allows you to specify multiple values in a WHERE clause.
MQL.select(mUser).whereIn(address.getCity(), "Vilnius", "Copenhagen").getList(); // WHERE city = "Vilnius" OR city = "Copenhagen"
.whereIsNull(mock value)
Returns all objects with null values inside the field passed as the argument
MQL.select(mUser).whereIsNull(mUser.getName()).getList();
.whereIsNotNull(mock value)
Returns all objects with not null values inside the field passed as the argument
MQL.select(mUser).whereIsNotNull(mUser.getName()).getList();
Pass an int as an argument to limit the number of objects returned.
.limit(int n)
Returns the first n objects
Or pass the start and end numbers as int arguments to return a custom range of objects
.limit(int n, int j)
Returns the objects from n to j.
List<User> userList = MQL.select(mUser).
where(MQL.any(constraints)).
limit(5).
getList();.orderBy(mock value, Order direction)
If you want to sort by a certain field pass the field as the first argument, and order direction as the second.
Order directions are:
MQL.ORDER.ASCMQL.ORDER.DESC
List<User> userList = MQL.select(mUser).
where(MQL.any(constraints)).
orderBy(mUser.getCity(), MQL.Order.DESC).
getList();Works just like SQL HAVING.
MQL.select(mTag).having("Insert RAW SQL here").getList();
You must finish your query chain with one of these methods:
Returns a list of your ModelObjects.
Returns the first hit, useful if you expect to have an unique entry or always want to get the newest entry (default sorting by creationDate).
Returns an array of your ModelObjects.
the method returns the SUM of the specified field in double, float, long or int depending on the argument value type.
the method returns the COUNT of the specified field in double, float, long or int depending on the argument value type.
the method returns the largest value of the specified field in double, float, long or int depending on the argument value type.
Returns a LimResultSet. The LimResultSet is a ResultSet wrapper.
the method returns the jdbc SelectSQLStatement used to query the database.
Example: MQL.select(mTag).where(MQL.all(constraints)).getSelectSQLStatement();
In NoJPA AND = MQL.all, OR = MQL.any
You create a constraint in a similar fashion as writing where clauses.
MQL.has(mock value, MQL.Comp, value)
Examples:
MQL.Constraint constraint = MQL.has(address.getCity(), MQL.Comp.EQUAL, "Vilnius"); // WHERE = Vilnius
MQL.Constraint constraint = MQL.any(MQL.has(address.getCity(), MQL.Comp.EQUAL, "Copenhagen"), MQL.has(address.getCity(), MQL.Comp.EQUAL, "Paris"));` // (Copenhagen || Paris)
List<MQL.Constraint> constraints = new ArrayList<>();
constraints.add(MQL.has(address.getPerson(), MQL.Comp.EQUAL, "A")));
constraints.add(MQL.has(address.getAge(), MQL.Comp.EQUAL, "23")));
constraints.add(MQL.has(constraint));
MQL.Constraint funConstraints = MQL.all(constraints); // (A && 23 && ((Paris || Copenhagen)))
MQL.select(address).where(funConstraints).getList();First of all, you need to "mock" the ModelObject class of the table you want to search in:
User mUser = NQL.mock(User.class);
Afterwards you write the query. The simplest query to find a user by name would look like this:
NQL.search(mUser).search(mUser.getName(), NQL.Comp.EQUAL, "Bob")
This query is pretty self-explanatory. You .select from the user table, and use the .where to write your constraint, to get only user with the name Bob. And finally with .getList() you specify that you want to receive a list of users.
When writing NoSQL queries you can use the following to compare values.
- EQUAL
- NOT_EQUAL
- LESS
- GREATER
- EQUAL_OR_LESS
- EQUAL_OR_GREATER
- LIKE
- NOT_LIKE
NQL.search(mUser).search(mUser.getName(), NQL.Comp.EQUAL, "Bob").getList();
NQL.search(mUser).search(mUser.getName(), NQL.Comp.LIKE, variable + "%").getList();
NQL.search(mUser).search(mUser.getAge(), NQL.Comp.GREATER, 18).getList();The usage is pretty intuitive .where(mock value, MQL Comp, value)
the mock value argument is your modelObjects field, MQL Comp is a compare instruction, and value is whatever you're filtering by.
.searchIsNull(mock value)
Returns all objects with null values inside the field passed as the argument
NQL.search(mUser).searchIsNull(mUser.getName()).getList();
.searchIsNotNull(mock value)
Returns all objects with not null values inside the field passed as the argument
NQL.search(mUser).searchIsNotNull(mUser.getName()).getList();
Pass an int as an argument to limit the number of objects returned.
.limit(int n)
Returns the first n objects
Or pass the start and end numbers as int arguments to return a custom range of objects
.limit(int n, int j)
Returns the objects from n to j.
NQL.search(mUser).searchIsNull(mUser.getName()).limit(Integer.MAX_VALUE).getList();
.orderBy(mock value, Order direction)
If you want to sort by a certain field pass the field as the first argument, and order direction as the second.
Order directions are:
NQL.ORDER.ASCNQL.ORDER.DESC
You must finish your query chain with one of these methods:
Returns a list of your ModelObjects.
Returns the first hit, useful if you expect to have an unique entry or always want to get the newest entry (default sorting by creationDate).
Returns an array of your ModelObjects.
The method returns the count of all the selected objects in long.
The method returns the sum value of the selected field in double.
The method returns the largest value of the selected field in double.
The method returns the smallest value of the selected field in double.
The method returns the mean value of the selected field in double.
The method returns the standard deviatian value of the selected field in double.
The method returns the count of all the selected objects in long.
In NoJPA AND = NQL.all, OR = NQL.any
You create a constraint in a similar fashion as writing where clauses.
NQL.has(mock value, NQL.Comp, value)
Examples:
NQL.Constraint constraint = NQL.has(address.getCity(), NQL.Comp.EQUAL, "Vilnius"); // WHERE = Vilnius
NQL.Constraint constraint = NQL.any(NQL.has(address.getCity(), MQL.Comp.EQUAL, "Copenhagen"), NQL.has(address.getCity(), NQL.Comp.EQUAL, "Paris"));` // (Copenhagen || Paris)
List<NQL.Constraint> constraints = new ArrayList<>();
constraints.add(NQL.has(address.getPerson(), NQL.Comp.EQUAL, "A")));
constraints.add(NQL.has(address.getAge(), NQL.Comp.EQUAL, "23")));
constraints.add(NQL.has(constraint));
NQL.Constraint funConstraints = NQL.all(constraints); // (A && 23 && ((Paris || Copenhagen)))
NQL.search(address).search(funConstraints).getList();Sometimes you may want to influence Solr's relevancy scoring to bump up some results higher than others.
This is done by adding boosting. Simply create a NQL.Boost object and pass an int to the constructor, this integer will be the boost factor. The higher the number, the higher relevancy score will be granted to objects fulfilling search criteria.
NQL.SolrFunction titleBoost = new NQL.Boost(12);
NQL.SolrFunction descriptionBoost = new NQL.Boost(5);Afterwards simply add .addFunction(boost object) right after your relevent .search().
return NQL.search(mArticle).
search(NQL.any(titleConstraints)).addFunction(titleBoost).
search(NQL.any(descriptionConstraints)).addFunction(descriptionBoost).
limit(numberOfArticles).
getList();You can also create a SolrMathFunction, which will be added to the end of your solr query. For example
NQL.SolrMathFunction solrMathFunction = new SolrMathFunction("&bf=log(relevancy_score)");
return NQL.search(mArticle).
search(NQL.any(constraints)).addFunction(solrMathFunction).getList();If your ModelObject has an array and you want to search in it simply write [MQL.ANY] after your mock value getter for SQL and [NQL.ANY] for NoSQL.
MQL.select(mTag).where(mTag.getSynonyms()[MQL.ANY].getName(), NQL.Comp.EQUAL, "tag"));
NQL.search(mTag).search(NQL.has(mTag.getSynonyms()[NQL.ANY].getName(), NQL.Comp.EQUAL, "tag"));Search by ID is almost exactly the same as searching by any other field. The only requirement is that the model object has to have atleast one method call.
Examples:
NQL.search(mPerson).search(mPerson.getObjectID(), NQL.Comp.NOT_EQUAL, prev.getObjectID()).getFirst();
NQL.search(mPerson).search(mPerson.getAddress(), NQL.Comp.EQUAL, prev.getAddress()).getFirst();Here the required method calls are .getObjectID() and .getAddress()
First of all create an inner class in the place you want to use your visitor
Example:
private class SearchAgentNotificationVisitor implements DbObjectVisitor<SearchAgent> {
@Override
public void visit(SearchAgent searchAgent) {
//Do stuff with searchAgents here.
//For example send notifications to a large amount of people.
//Here you will have access to every individual object returned by your query.
}
@Override
public void setDone(boolean b) {
}
@Override
public boolean getDone() {
return false;
}
}Then create a new service, or modify an existing one to accept your visitor class as an input argument and return void.
Example:
void getAllActiveAgents(DbObjectVisitor<SearchAgent> visitor);
}Finally, inside the service implementation write your query as you would normally, but finish it with a .visit(visitor).
Example:
public void getAllActiveAgents(DbObjectVisitor<SearchAgent> visitor) {
SearchAgent mSearchAgent = MQL.mock(SearchAgent.class);
MQL.select(mSearchAgent).where(mSearchAgent.getIsActive(), MQL.Comp.EQUAL, true).where(mSearchAgent.getSearchType(), MQL.Comp.EQUAL, SearchType.DAILYMAIL).orderBy(mSearchAgent.getCreationDate(), MQL.Order.DESC).visit(visitor);
}And now you can call your visitor like this:
searchAgentService.getAllActiveAgents(new SearchAgentNotificationVisitor());
This annotation will change the default length of a column. (Default = 32 chars for ID's and 255 for Strings)
Simply write how many chars would you like the column to store:
@Column(length = 20000)
This annotation is placed inside your model object interface
public interface User extends ModelObjectInterface {
@SearchField
@Column(length = 20000)
String getDescription();
void setDescription(String description);
}stripItHard = false + stripItSoft = false : then no strip at all
stripItHard = false + stripItSoft = true : only replace ' and " to -> `
stripItHard = true || no DbStrip over the set method : replaceAll('|"", "`").replaceAll("/|&|'|<|>|;|\\", "")) + first char to upper case
Example:
public interface AppSettings extends `ModelObjectInterface {
String getAppId();
@DbStrip(stripItSoft = true, stripItHard = false)
void setAppId(String appId);
String getAppSecret();
@DbStrip(stripItSoft = true, stripItHard = false)
void setAppSecret(String appSecret);
}Mark fields which you want to search by in Solr with this annotation.
Mark fields which you want to be indexed in Solr with this annotation.
Allows you to find objects by URL, name or something else instead of ID. Useful when you don't want to display objectID's in the URL.
The opposite of the @Locator
Allows you to add additional functionallity to a model objects lifecycle. -onNew() -onDelete() -preUpdate() -postUpdate()
Example of @Locator, @Printer, @ModelObjectLifeCycleListener annotations in use:
@Locator(locator = Article.Locator.class)
@Printer(printer = Article.Printer.class)
@ModelObjectLifeCycleListener(lifeCycleListener = Article.ArticleUrlSetter.class)
public interface Article extends ModelObjectInterface {
@SearchField
String getTitle();
void setTitle(String title);
public static class Locator implements ObjectLocator<Article> {
@Override
public Article get(String url) {
Article mArticle = MQL.mock(Article.class);
Article article = MQL.select(mArticle).where(mArticle.getUrlTitle(), MQL.Comp.EQUAL, url).getFirst();
if (article == null) {
article = MQL.selectByID(Article.class, url);
}
return article;
}
}
public static class Printer implements ObjectPrinter<Article> {
@Override
public String put(Article article) {
return article.getUrlTitle();
}
}
public static class ArticleUrlSetter implements ModelObjectLifeCycleListener.LifeCycleListener {
protected Log log = LogFactory.getLog(getClass());
@Override
public void onNew(Object mother) { }
@Override
public void onDelete(Object mother) { }
@Override
public void preUpdate(Object mother) {
// TODO synchronize
try {
Article article = (Article)mother;
if (article.getUrlTitle() == null) {
String originalUrlPart = getUrlPart(article.getTitle());
String urlPart = originalUrlPart;
int uniqueCount = 1;
Article mArticle = MQL.mock(Article.class);
while (MQL.select(mArticle).where(mArticle.getObjectID(), MQL.Comp.NOT_EQUAL, article.getObjectID()).where(mArticle.getUrlTitle(), MQL.Comp.EQUAL, urlPart).getCount() > 0) {
urlPart = originalUrlPart + "-" + uniqueCount;
uniqueCount++;
}
article.setUrlTitle(urlPart);
//log.debug("setting url [" + urlPart + "] to Article: " + article);
}
} catch (Exception e) {
log.error("cannot set url title for article: " + mother);
}
}
@Override
public void postUpdate(Object mother) { }
public static String getUrlPart(String name) {
if (name == null) {
return "";
}
name = name.toLowerCase();
return Normalizer.normalize(name, Normalizer.Form.NFD)
.replaceAll("\\p{InCombiningDiacriticalMarks}+", "")
.toLowerCase()
.replaceAll("[^a-z0-9\\- ]", "").trim()
.replace(' ', '-')
.replaceAll("\\-+", "-");
}
}
}It's also possible to add a method listener in order to add additional pre or post run functionality
-preRun()
-postRun()
Simple Address-book
Complex Address-book
//TODO
//TODO
//TODO
- Why is it interfaces
