Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions spring-batch-notion/.mvn/jvm.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
--add-exports jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.model=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.processing=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED
--add-opens jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED
--add-opens jdk.compiler/com.sun.tools.javac.comp=ALL-UNNAMED
24 changes: 14 additions & 10 deletions spring-batch-notion/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This project provides a [Spring Batch][] extension module that adds support for

## Compatibility

Spring Batch Notion is based on Spring Batch 5 and tested on Spring Boot 3, thus requiring at least Java 17.
Spring Batch Notion is based on Spring Batch 6 and tested on Spring Boot 4, thus requiring at least Java 17.

Compatibility is guaranteed only with the Spring Batch versions under [OSS support](https://spring.io/projects/spring-batch#support).

Expand Down Expand Up @@ -36,25 +36,29 @@ A minimal configuration of the item reader is as follows:

```java
NotionDatabaseItemReader<Item> itemReader() {
NotionDatabaseItemReader<Item> reader = new NotionDatabaseItemReader<>();
reader.setToken(System.getenv("NOTION_TOKEN"));
reader.setDatabaseId("XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"); // UUID
reader.setPropertiesMapper(new CustomPropertyMapper());
return reader;
String token = System.getenv("NOTION_TOKEN");
String databaseId = "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"; // UUID
PropertyMapper<Item> propertyMapper = new CustomPropertyMapper();
return new NotionDatabaseItemReader<>(token, databaseId, propertyMapper);
}
```

The following configuration options are available:
The following constructor parameters should be provided:

| Property | Description |
|------------------|---------------------------------------------------------------------------------------------------------------------------|
| `token` | The Notion integration token. |
| `databaseId` | UUID of the database to read from. |
| `propertyMapper` | The `PropertyMapper` responsible for mapping properties of a Notion item into a Java object. |

and the following configuration options are available:

| Property | Required | Default | Description |
|------------------|----------|-----------------------------|---------------------------------------------------------------------------------------------------------------------------|
| `baseUrl` | no | `https://api.notion.com/v1` | Base URL of the Notion API. A custom value can be provided for testing purposes (e.g., the URL of a [WireMock][] server). |
| `databaseId` | yes | - | UUID of the database to read from. |
| `filter` | no | `null` | `Filter` condition to limit the returned items. |
| `pageSize` | no | `100` | Number of items to be read with each page. Must be greater than zero and less than or equal to 100. |
| `propertyMapper` | yes | - | The `PropertyMapper` responsible for mapping properties of a Notion item into a Java object. |
| `sorts` | no | `null` | `Sort` conditions to order the returned items. Each condition is applied following the declaration order. |
| `token` | yes | - | The Notion integration token. |

In addition to the Notion-specific configuration, all the configuration options of the Spring Batch
[`AbstractPaginatedDataItemReader`](https://docs.spring.io/spring-batch/docs/current/api/org/springframework/batch/item/data/AbstractPaginatedDataItemReader.html)
Expand Down
29 changes: 26 additions & 3 deletions spring-batch-notion/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>3.5.6</version>
<version>4.0.0</version>
<relativePath/>
</parent>

Expand Down Expand Up @@ -160,8 +160,31 @@ limitations under the License.
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<compilerArgs>
<arg>-Xlint:all,-varargs</arg>
<compilerArg>-Xlint:all,-varargs</compilerArg>
<!-- https://errorprone.info/docs/installation#maven -->
<compilerArg>-XDcompilePolicy=simple</compilerArg>
<compilerArg>--should-stop=ifError=FLOW</compilerArg>
<compilerArg>
-Xplugin:ErrorProne
-XepDisableAllChecks
<!-- Check JSpecify annotations -->
-Xep:NullAway
-XepOpt:NullAway:OnlyNullMarked
-XepOpt:NullAway:SuppressionNameAliases=DataFlowIssue
</compilerArg>
</compilerArgs>
<annotationProcessorPaths>
<path>
<groupId>com.google.errorprone</groupId>
<artifactId>error_prone_core</artifactId>
<version>2.43.0</version>
</path>
<path>
<groupId>com.uber.nullaway</groupId>
<artifactId>nullaway</artifactId>
<version>0.12.11</version>
</path>
</annotationProcessorPaths>
<failOnWarning>true</failOnWarning>
</configuration>
</plugin>
Expand All @@ -178,8 +201,8 @@ limitations under the License.
</additionalDependencies>
<failOnWarnings>true</failOnWarnings>
<links>
<link>https://docs.spring.io/spring-batch/docs/${spring-batch.version}/api/</link>
<link>https://docs.spring.io/spring-framework/docs/${spring-framework.version}/javadoc-api/</link>
<link>https://javadoc.io/static/org.springframework.batch/spring-batch-core/${spring-batch.version}/</link>
</links>
</configuration>
<executions>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,11 @@
import notion.api.v1.model.pages.PageProperty;
import notion.api.v1.model.pages.PageProperty.RichText;
import notion.api.v1.request.databases.QueryDatabaseRequest;
import org.jspecify.annotations.Nullable;
import org.springframework.batch.extensions.notion.mapping.PropertyMapper;
import org.springframework.batch.item.ExecutionContext;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.data.AbstractPaginatedDataItemReader;
import org.springframework.beans.factory.InitializingBean;
import org.springframework.batch.infrastructure.item.ExecutionContext;
import org.springframework.batch.infrastructure.item.ItemReader;
import org.springframework.batch.infrastructure.item.data.AbstractPaginatedDataItemReader;
import org.springframework.util.Assert;

import java.util.Collections;
Expand Down Expand Up @@ -57,39 +57,41 @@
* @author Stefano Cordio
* @param <T> Type of item to be read
*/
public class NotionDatabaseItemReader<T> extends AbstractPaginatedDataItemReader<T> implements InitializingBean {

private static final String DEFAULT_BASE_URL = "https://api.notion.com/v1";
public class NotionDatabaseItemReader<T> extends AbstractPaginatedDataItemReader<T> {

private static final int DEFAULT_PAGE_SIZE = 100;

private String baseUrl;
private static final String DEFAULT_BASE_URL = "https://api.notion.com/v1";

private final String token;

private String token;
private final String databaseId;

private String databaseId;
private final PropertyMapper<T> propertyMapper;

private PropertyMapper<T> propertyMapper;
private String baseUrl = DEFAULT_BASE_URL;

private QueryTopLevelFilter filter;
private @Nullable QueryTopLevelFilter filter;

private List<QuerySort> sorts;
private @Nullable List<QuerySort> sorts;

private NotionClient client;
private @Nullable NotionClient client;

private boolean hasMore;

private String nextCursor;
private @Nullable String nextCursor;

/**
* Create a new {@link NotionDatabaseItemReader} with the following defaults:
* <ul>
* <li>{@code baseUrl} = {@value #DEFAULT_BASE_URL}</li>
* <li>{@code pageSize} = {@value #DEFAULT_PAGE_SIZE}</li>
* </ul>
* Create a new {@link NotionDatabaseItemReader}.
* @param token the Notion integration token
* @param databaseId UUID of the database to read from
* @param propertyMapper the {@link PropertyMapper} responsible for mapping properties
* of a Notion item into a Java object
*/
public NotionDatabaseItemReader() {
this.baseUrl = DEFAULT_BASE_URL;
public NotionDatabaseItemReader(String token, String databaseId, PropertyMapper<T> propertyMapper) {
this.token = Objects.requireNonNull(token);
this.databaseId = Objects.requireNonNull(databaseId);
this.propertyMapper = Objects.requireNonNull(propertyMapper);
this.pageSize = DEFAULT_PAGE_SIZE;
}

Expand All @@ -106,37 +108,6 @@ public void setBaseUrl(String baseUrl) {
this.baseUrl = Objects.requireNonNull(baseUrl);
}

/**
* The Notion integration token.
* <p>
* Always required.
* @param token the token
*/
public void setToken(String token) {
this.token = Objects.requireNonNull(token);
}

/**
* UUID of the database to read from.
* <p>
* Always required.
* @param databaseId the database UUID
*/
public void setDatabaseId(String databaseId) {
this.databaseId = Objects.requireNonNull(databaseId);
}

/**
* The {@link PropertyMapper} responsible for mapping Notion item properties into a
* Java object.
* <p>
* Always required.
* @param propertyMapper the property mapper
*/
public void setPropertyMapper(PropertyMapper<T> propertyMapper) {
this.propertyMapper = Objects.requireNonNull(propertyMapper);
}

/**
* {@link Filter} condition to limit the returned items.
* <p>
Expand Down Expand Up @@ -175,6 +146,19 @@ public void setPageSize(int pageSize) {
super.setPageSize(pageSize);
}

/**
* {@inheritDoc}
*/
@Override
protected void doOpen() {
client = new NotionClient(token);
client.setHttpClient(new JavaNetHttpClient());
client.setLogger(new Slf4jLogger());
client.setBaseUrl(baseUrl);

hasMore = true;
}

/**
* {@inheritDoc}
*/
Expand All @@ -190,6 +174,7 @@ protected Iterator<T> doPageRead() {
request.setStartCursor(nextCursor);
request.setPageSize(pageSize);

@SuppressWarnings("DataFlowIssue")
QueryResults queryResults = client.queryDatabase(request);

hasMore = queryResults.getHasMore();
Expand All @@ -198,7 +183,7 @@ protected Iterator<T> doPageRead() {
return queryResults.getResults()
.stream()
.map(NotionDatabaseItemReader::getProperties)
.map(properties -> propertyMapper.map(properties))
.map(propertyMapper::map)
.iterator();
}

Expand All @@ -224,19 +209,7 @@ private static String getPlainText(List<RichText> texts) {
/**
* {@inheritDoc}
*/
@Override
protected void doOpen() {
client = new NotionClient(token);
client.setHttpClient(new JavaNetHttpClient());
client.setLogger(new Slf4jLogger());
client.setBaseUrl(baseUrl);

hasMore = true;
}

/**
* {@inheritDoc}
*/
@SuppressWarnings("DataFlowIssue")
@Override
protected void doClose() {
client.close();
Expand All @@ -255,14 +228,4 @@ protected void jumpToItem(int itemIndex) throws Exception {
}
}

/**
* {@inheritDoc}
*/
@Override
public void afterPropertiesSet() {
Assert.state(token != null, "'token' must be set");
Assert.state(databaseId != null, "'databaseId' must be set");
Assert.state(propertyMapper != null, "'propertyMapper' must be set");
}

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
/*
* Copyright 2024-2025 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

/**
* Infrastructure to map the properties of a Notion item into a Java object.
*/
@NullMarked
package org.springframework.batch.extensions.notion.mapping;

import org.jspecify.annotations.NullMarked;
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
/*
* Copyright 2024-2025 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

/**
* Spring Batch extension for Notion.
*/
@NullMarked
package org.springframework.batch.extensions.notion;

import org.jspecify.annotations.NullMarked;
Loading