Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Application data cleanup #196

Open
wants to merge 45 commits into
base: develop
from
Open
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
53d305e
Add CleanUp controller
miguelruizdev Feb 26, 2019
d53bd8f
Add test covering clean-up controller
miguelruizdev Feb 26, 2019
189aff1
Refactor name from delete to export
miguelruizdev Feb 26, 2019
3a89bfd
Remove entity finder
miguelruizdev Feb 26, 2019
ca490a8
Add delete flag to clean up endpoint
miguelruizdev Feb 27, 2019
0b893d7
Add conditional property features
miguelruizdev Feb 27, 2019
a279d0c
Add task clean up endpoint and test coverage
miguelruizdev Feb 27, 2019
fc8c16b
Refactor taskCleanUpController
miguelruizdev Feb 28, 2019
d37ef9e
Add test coverage of TaskCleanUpController after refactoring
miguelruizdev Feb 28, 2019
2182b2b
Refactor ProcessInstanceCleanUpController
miguelruizdev Feb 28, 2019
e99cbd4
Add coverage for ProcessInstanceEntityCleanUpControllerIT
miguelruizdev Feb 28, 2019
dfaea6e
Optimize imports of ProcessInstanceEntityCleanUpControllerIT
miguelruizdev Feb 28, 2019
2b892ee
Enable clean-up endpoints by default
miguelruizdev Feb 28, 2019
a27d6f5
Optimize imports
miguelruizdev Mar 1, 2019
7131bd5
fix(version): update org.activiti.cloud.common:activiti-cloud-service…
jx-activiti-cloud Mar 1, 2019
79aae3f
Merge pull request #199 from Activiti/updatebot-e9ed86e4-f652-4bf9-b3…
mergify Mar 1, 2019
2fe5e4e
fix(version): update org.activiti.cloud.common:activiti-cloud-service…
jx-activiti-cloud Mar 4, 2019
b330254
Merge pull request #200 from Activiti/updatebot-384b8212-851e-43cb-a3…
mergify Mar 4, 2019
79c5a6a
Update pom.xml (#188)
ryandawsonuk Mar 5, 2019
da84290
fix(version): update org.activiti.cloud.build:activiti-cloud-parent t…
jx-activiti-cloud Mar 5, 2019
91d84fd
fix(version): update org.activiti.cloud.common:activiti-cloud-service…
jx-activiti-cloud Mar 5, 2019
0fb95a9
Merge pull request #202 from Activiti/updatebot-20a6d9da-0cb8-4084-bb…
mergify Mar 5, 2019
7a99696
Update Jenkinsfile (#203)
ryandawsonuk Mar 6, 2019
bfc6f66
fix(version): update org.activiti.cloud.common:activiti-cloud-service…
jx-activiti-cloud Mar 6, 2019
23dc0ff
Merge pull request #204 from Activiti/updatebot-6b5c4a58-b951-4232-ae…
mergify Mar 6, 2019
c90293f
fix(version): update org.activiti.cloud.common:activiti-cloud-service…
jx-activiti-cloud Mar 6, 2019
4bf94a0
Merge pull request #205 from Activiti/updatebot-b1852e05-99b9-4a43-af…
mergify Mar 6, 2019
c59c202
Add CleanUp controller
miguelruizdev Feb 26, 2019
87da737
Add test covering clean-up controller
miguelruizdev Feb 26, 2019
b0b9e9c
Refactor name from delete to export
miguelruizdev Feb 26, 2019
b17b7f5
Remove entity finder
miguelruizdev Feb 26, 2019
c9c72d0
Add delete flag to clean up endpoint
miguelruizdev Feb 27, 2019
a051915
Add conditional property features
miguelruizdev Feb 27, 2019
3432678
Add task clean up endpoint and test coverage
miguelruizdev Feb 27, 2019
23bb4ce
Refactor taskCleanUpController
miguelruizdev Feb 28, 2019
9ccab9c
Add test coverage of TaskCleanUpController after refactoring
miguelruizdev Feb 28, 2019
fda87ae
Refactor ProcessInstanceCleanUpController
miguelruizdev Feb 28, 2019
34e614b
Add coverage for ProcessInstanceEntityCleanUpControllerIT
miguelruizdev Feb 28, 2019
8cc27b5
Optimize imports of ProcessInstanceEntityCleanUpControllerIT
miguelruizdev Feb 28, 2019
f9d7f6b
Enable clean-up endpoints by default
miguelruizdev Feb 28, 2019
b8b8d87
Optimize imports
miguelruizdev Mar 1, 2019
1936085
Add common-properties module dependecy to starter
miguelruizdev Mar 8, 2019
5564d16
Change enable clean-up endpoints property name
miguelruizdev Mar 8, 2019
61abb02
Merge branch 'miguel-2173-application-data-cleanup-refactor' of https…
miguelruizdev Mar 8, 2019
aee1eac
Change dep in starter from common-properties to common-config
miguelruizdev Mar 12, 2019
File filter...
Filter file types
Jump to…
Jump to file or symbol
Failed to load files and symbols.
+407 −12
Diff settings

Always

Just for now

Copy path View file
@@ -40,19 +40,25 @@ pipeline {
sh "mvn versions:set -DnewVersion=\$(cat VERSION)"
sh 'mvn clean verify'

sh "git add --all"
sh "git commit -m \"Release \$(cat VERSION)\" --allow-empty"
sh "git tag -fa v\$(cat VERSION) -m \"Release version \$(cat VERSION)\""
sh "git push origin v\$(cat VERSION)"
retry(5){
sh "git add --all"
sh "git commit -m \"Release \$(cat VERSION)\" --allow-empty"
sh "git tag -fa v\$(cat VERSION) -m \"Release version \$(cat VERSION)\""
sh "git push origin v\$(cat VERSION)"
}
}
container('maven') {
sh 'mvn clean deploy -DskipTests'

sh 'export VERSION=`cat VERSION`'

sh "jx step git credentials"
sh "updatebot push-version --kind maven org.activiti.cloud.query:activiti-cloud-query-dependencies \$(cat VERSION)"
sh "updatebot update --merge false"
retry(2){
sh "updatebot push-version --kind maven org.activiti.cloud.query:activiti-cloud-query-dependencies \$(cat VERSION)"
sh "rm -rf .updatebot-repos/"
sh "sleep \$((RANDOM % 10))"
sh "updatebot push-version --kind maven org.activiti.cloud.query:activiti-cloud-query-dependencies \$(cat VERSION)"
}
}
}
}
@@ -0,0 +1,63 @@
package org.activiti.cloud.services.query.rest;

import com.querydsl.core.types.Predicate;
import org.activiti.cloud.services.query.app.repository.ProcessInstanceRepository;
import org.activiti.cloud.services.query.model.ProcessInstanceEntity;
import org.activiti.cloud.services.query.resources.ProcessInstanceResource;
import org.activiti.cloud.services.query.rest.assembler.ProcessInstanceResourceAssembler;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.data.querydsl.binding.QuerydslPredicate;
import org.springframework.hateoas.MediaTypes;
import org.springframework.hateoas.Resources;
import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;

import java.util.ArrayList;
import java.util.Collection;

@ConditionalOnProperty(name = "activiti.rest.enable-clean-up", havingValue = "true")
@RestController
@RequestMapping(
value = "/admin/clean-up/v1/process-instances",
produces = {
MediaTypes.HAL_JSON_VALUE,
MediaType.APPLICATION_JSON_VALUE
})
public class ProcessInstanceCleanUpController {

private final ProcessInstanceRepository processInstanceRepository;

private ProcessInstanceResourceAssembler processInstanceResourceAssembler;

@Autowired
public ProcessInstanceCleanUpController(ProcessInstanceRepository processInstanceRepository,
ProcessInstanceResourceAssembler processInstanceResourceAssembler) {
this.processInstanceRepository = processInstanceRepository;
this.processInstanceResourceAssembler = processInstanceResourceAssembler;
}

@RequestMapping(method = RequestMethod.DELETE)
public Resources<ProcessInstanceResource> deleteProcessInstances (@QuerydslPredicate(root = ProcessInstanceEntity.class) Predicate predicate) {

Collection<ProcessInstanceResource> result = new ArrayList<>();
Iterable <ProcessInstanceEntity> iterable = processInstanceRepository.findAll(predicate);

for(ProcessInstanceEntity entity : iterable){
result.add(processInstanceResourceAssembler.toResource(entity));
}

processInstanceRepository.deleteAll(iterable);

return new Resources<>(result);
}







}
@@ -0,0 +1,57 @@
package org.activiti.cloud.services.query.rest;

import com.querydsl.core.types.Predicate;
import org.activiti.cloud.services.query.app.repository.TaskRepository;
import org.activiti.cloud.services.query.model.TaskEntity;
import org.activiti.cloud.services.query.resources.TaskResource;
import org.activiti.cloud.services.query.rest.assembler.TaskResourceAssembler;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.data.querydsl.binding.QuerydslPredicate;
import org.springframework.hateoas.MediaTypes;
import org.springframework.hateoas.Resources;
import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;

import java.util.ArrayList;
import java.util.Collection;

@ConditionalOnProperty(name = "activiti.rest.enable-clean-up", havingValue = "true")
@RestController
@RequestMapping(
value = "/admin/clean-up/v1/tasks",
produces = {
MediaTypes.HAL_JSON_VALUE,
MediaType.APPLICATION_JSON_VALUE
})
public class TaskCleanUpController {

private final TaskRepository taskRepository;

private TaskResourceAssembler taskResourceAssembler;

@Autowired
public TaskCleanUpController(TaskRepository taskRepository,
TaskResourceAssembler taskResourceAssembler) {
this.taskRepository = taskRepository;
this.taskResourceAssembler = taskResourceAssembler;
}

@RequestMapping(method = RequestMethod.DELETE)
public Resources<TaskResource> deleteTasks (@QuerydslPredicate(root = TaskEntity.class) Predicate predicate) {

Collection <TaskResource> result = new ArrayList<>();
Iterable <TaskEntity> iterable = taskRepository.findAll(predicate);

for(TaskEntity entity : iterable){
result.add(taskResourceAssembler.toResource(entity));
}

taskRepository.deleteAll(iterable);

return new Resources<>(result);
}

}
@@ -19,4 +19,7 @@ keycloak.principal-attribute=preferred-username

activiti.keycloak.admin-client-app=admin-cli
activiti.keycloak.test-user=client
activiti.keycloak.test-password=client
activiti.keycloak.test-password=client

# enable clean up endpoints
activiti.enable-clean-up=true
This conversation was marked as resolved by miguelruizdev

This comment has been minimized.

@erdemedeiros

erdemedeiros Mar 5, 2019

Member

Actually, I think this is not going to work. As far as I remember this application.properties will be erased by the one coming from the end application, i.e. activiti-cloud-query. We need to remove this application.properties from here. Can you create an issue for it, please?

Currently, the way we are using to load properties in the end application is having them in a dedicated properties file (similar to this one) and the property file should be referenced by and auto-configuration (as here)

@@ -0,0 +1,125 @@
package org.activiti.cloud.services.query.rest;


import com.querydsl.core.types.Predicate;
import org.activiti.api.process.model.ProcessInstance;
import org.activiti.api.runtime.conf.impl.CommonModelAutoConfiguration;
import org.activiti.api.runtime.shared.security.SecurityManager;
import org.activiti.cloud.conf.QueryRestAutoConfiguration;
import org.activiti.cloud.services.query.app.repository.EntityFinder;
import org.activiti.cloud.services.query.app.repository.ProcessDefinitionRepository;
import org.activiti.cloud.services.query.app.repository.ProcessInstanceRepository;
import org.activiti.cloud.services.query.model.ProcessInstanceEntity;
import org.activiti.cloud.services.security.TaskLookupRestrictionService;
import org.activiti.core.common.spring.security.policies.SecurityPoliciesManager;
import org.activiti.core.common.spring.security.policies.conf.SecurityPoliciesProperties;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.ArgumentMatchers;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.restdocs.AutoConfigureRestDocs;
import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest;
import org.springframework.boot.test.mock.mockito.MockBean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Import;
import org.springframework.data.web.config.EnableSpringDataWebSupport;
import org.springframework.http.MediaType;
import org.springframework.test.context.TestPropertySource;
import org.springframework.test.context.junit4.SpringRunner;
import org.springframework.test.web.servlet.MockMvc;

import java.util.Collections;
import java.util.Date;
import java.util.List;
import java.util.UUID;

import static org.activiti.alfresco.rest.docs.AlfrescoDocumentation.resourcesResponseFields;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.BDDMockito.given;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import static org.springframework.restdocs.mockmvc.MockMvcRestDocumentation.document;
import static org.springframework.restdocs.mockmvc.RestDocumentationRequestBuilders.delete;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;

@TestPropertySource(properties="activiti.rest.enable-clean-up=true")
@RunWith(SpringRunner.class)
@WebMvcTest(ProcessInstanceCleanUpController.class)
@Import({
QueryRestAutoConfiguration.class,
CommonModelAutoConfiguration.class,
})
@EnableSpringDataWebSupport
@AutoConfigureMockMvc(secure = false)
@AutoConfigureRestDocs(outputDir = "target/snippets")
@ComponentScan(basePackages = {"org.activiti.cloud.services.query.rest.assembler", "org.activiti.cloud.alfresco"})
public class ProcessInstanceEntityCleanUpControllerIT {

private static final String PROCESS_INSTANCE_ALFRESCO_IDENTIFIER = "process-instance-alfresco";

@Autowired
private MockMvc mockMvc;

@MockBean
private ProcessInstanceRepository processInstanceRepository;

@MockBean
private SecurityManager securityManager;

@MockBean
private EntityFinder entityFinder;

@MockBean
private SecurityPoliciesManager securityPoliciesManager;

@MockBean
private ProcessDefinitionRepository processDefinitionRepository;

@MockBean
private SecurityPoliciesProperties securityPoliciesProperties;

@MockBean
private TaskLookupRestrictionService taskLookupRestrictionService;

@Before
public void setUp() {
when(securityManager.getAuthenticatedUserId()).thenReturn("admin");
assertThat(entityFinder).isNotNull();
assertThat(securityPoliciesManager).isNotNull();
assertThat(processDefinitionRepository).isNotNull();
assertThat(securityPoliciesProperties).isNotNull();
assertThat(taskLookupRestrictionService).isNotNull();
}

@Test
public void deleteProcessInstancesShouldReturnAllProcessInstancesAndDeleteThem() throws Exception{

//given
List<ProcessInstanceEntity> processInstanceEntities = Collections.singletonList(buildDefaultProcessInstance());
given(processInstanceRepository.findAll(ArgumentMatchers.<Predicate>any()))
.willReturn(processInstanceEntities);

//when
mockMvc.perform(delete("/admin/clean-up/v1/process-instances")
.accept(MediaType.APPLICATION_JSON))
//then
.andExpect(status().isOk())
.andDo(document(PROCESS_INSTANCE_ALFRESCO_IDENTIFIER + "/list",
resourcesResponseFields()

));

verify(processInstanceRepository).deleteAll(processInstanceEntities);

}

private ProcessInstanceEntity buildDefaultProcessInstance() {
return new ProcessInstanceEntity("My-app", "My-app", "1", null, null,
UUID.randomUUID().toString(),
UUID.randomUUID().toString(),
ProcessInstance.ProcessInstanceStatus.RUNNING,
new Date());
}
}
Oops, something went wrong.
ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.