Skip to content

feat: enhance Docker configuration with additional services and profiles #231

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Dallas98 merged 2 commits into from
Jan 12, 2026
Merged

feat: enhance Docker configuration with additional services and profiles #231

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
380873e
feat: update error handling in RagEtlService and add commons-io depen…
Dallas98 Jan 12, 2026
751b97a
feat: enhance Docker configuration with additional services and profiles
Dallas98 Jan 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -189,4 +189,4 @@ Thumbs.db
*.sublime-workspace

# Milvus
deployment/docker/milvus/volumes/
**/volumes/
52 changes: 24 additions & 28 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -238,7 +238,7 @@ endif
# ========== Docker Install/Uninstall Targets ==========

# Valid service targets for docker install/uninstall
VALID_SERVICE_TARGETS := datamate backend frontend runtime mineru "deer-flow" milvus "label-studio" "data-juicer" dj
VALID_SERVICE_TARGETS := datamate backend frontend runtime backend-python database gateway redis mineru deer-flow milvus label-studio data-juicer dj

# Generic docker service install target
.PHONY: %-docker-install
Expand All @@ -252,21 +252,23 @@ VALID_SERVICE_TARGETS := datamate backend frontend runtime mineru "deer-flow" mi
exit 1; \
fi
@if [ "$*" = "label-studio" ]; then \
$(call docker-compose-service,label-studio,up -d,deployment/docker/label-studio); \
elif [ "$*" = "mineru" ]; then \
REGISTRY=$(REGISTRY) && docker compose -f deployment/docker/datamate/docker-compose.yml up -d datamate-mineru; \
REGISTRY=$(REGISTRY) docker compose -f deployment/docker/datamate/docker-compose.yml --profile label-studio up -d; \
elif [ "$*" = "datamate" ]; then \
REGISTRY=$(REGISTRY) docker compose -f deployment/docker/datamate/docker-compose.yml up -d; \
elif [ "$*" = "deer-flow" ]; then \
cp runtime/deer-flow/.env deployment/docker/deer-flow/.env; \
cp runtime/deer-flow/conf.yaml deployment/docker/deer-flow/conf.yaml; \
REGISTRY=$(REGISTRY) docker compose -f deployment/docker/deer-flow/docker-compose.yml up -d; \
elif [ "$*" = "milvus" ]; then \
docker compose -f deployment/docker/milvus/docker-compose.yml up -d; \
elif [ "$*" = "mineru" ]; then \
REGISTRY=$(REGISTRY) docker compose -f deployment/docker/datamate/docker-compose.yml --profile mineru up -d datamate-mineru; \
elif [ "$*" = "data-juicer" ] || [ "$*" = "dj" ]; then \
REGISTRY=$(REGISTRY) && docker compose -f deployment/docker/datamate/docker-compose.yml up -d datamate-data-juicer; \
REGISTRY=$(REGISTRY) docker compose -f deployment/docker/datamate/docker-compose.yml --profile data-juicer up -d datamate-data-juicer; \
elif [ "$*" = "redis" ]; then \
REGISTRY=$(REGISTRY) docker compose -f deployment/docker/datamate/docker-compose.yml --profile redis up -d datamate-redis; \
elif [ "$*" = "milvus" ]; then \
REGISTRY=$(REGISTRY) docker compose -f deployment/docker/datamate/docker-compose.yml --profile milvus up -d; \
elif [ "$*" = "deer-flow" ]; then \
cp runtime/deer-flow/.env deployment/docker/datamate/.env; \
cp runtime/deer-flow/conf.yaml deployment/docker/datamate/conf.yaml; \
REGISTRY=$(REGISTRY) docker compose -f deployment/docker/datamate/docker-compose.yml --profile deer-flow up -d; \
else \
$(call docker-compose-service,$*,up -d,deployment/docker/datamate); \
REGISTRY=$(REGISTRY) docker compose -f deployment/docker/datamate/docker-compose.yml up -d datamate-$*; \
fi

# Generic docker service uninstall target
Expand All @@ -281,29 +283,23 @@ VALID_SERVICE_TARGETS := datamate backend frontend runtime mineru "deer-flow" mi
exit 1; \
fi
@if [ "$*" = "label-studio" ]; then \
if [ "$(DELETE_VOLUMES_CHOICE)" = "1" ]; then \
cd deployment/docker/label-studio && docker compose down -v && cd - >/dev/null; \
else \
cd deployment/docker/label-studio && docker compose down && cd - >/dev/null; \
fi; \
docker compose -f deployment/docker/datamate/docker-compose.yml rm -f -s label-studio pg-db; \
elif [ "$*" = "mineru" ]; then \
$(call docker-compose-service,datamate-mineru,down,deployment/docker/datamate); \
docker compose -f deployment/docker/datamate/docker-compose.yml rm -f -s datamate-mineru; \
elif [ "$*" = "data-juicer" ] || [ "$*" = "dj" ]; then \
docker compose -f deployment/docker/datamate/docker-compose.yml rm -f -s datamate-data-juicer; \
elif [ "$*" = "redis" ]; then \
docker compose -f deployment/docker/datamate/docker-compose.yml rm -f -s datamate-redis; \
elif [ "$*" = "datamate" ]; then \
if [ "$(DELETE_VOLUMES_CHOICE)" = "1" ]; then \
docker compose -f deployment/docker/datamate/docker-compose.yml --profile mineru down -v; \
docker compose -f deployment/docker/datamate/docker-compose.yml --profile mineru --profile redis --profile data-juicer --profile deer-flow --profile label-studio --profile milvus down -v; \
else \
docker compose -f deployment/docker/datamate/docker-compose.yml --profile mineru down; \
docker compose -f deployment/docker/datamate/docker-compose.yml --profile mineru --profile redis --profile data-juicer --profile deer-flow --profile label-studio --profile milvus down; \
fi; \
elif [ "$*" = "deer-flow" ]; then \
docker compose -f deployment/docker/deer-flow/docker-compose.yml down; \
docker compose -f deployment/docker/datamate/docker-compose.yml rm -f -s deer-flow-backend deer-flow-frontend; \
elif [ "$*" = "milvus" ]; then \
if [ "$(DELETE_VOLUMES_CHOICE)" = "1" ]; then \
docker compose -f deployment/docker/milvus/docker-compose.yml down -v; \
else \
docker compose -f deployment/docker/milvus/docker-compose.yml down; \
fi; \
elif [ "$*" = "data-juicer" ] || [ "$*" = "dj" ]; then \
$(call docker-compose-service,datamate-data-juicer,down,deployment/docker/datamate); \
docker compose -f deployment/docker/datamate/docker-compose.yml rm -f -s milvus etcd minio; \
else \
$(call docker-compose-service,$*,down,deployment/docker/datamate); \
fi
Expand Down
6 changes: 6 additions & 0 deletions backend/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,12 @@
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.16.1</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-bom</artifactId>
Expand Down
6 changes: 6 additions & 0 deletions backend/services/rag-indexer-service/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,12 @@
<dependency>
<groupId>dev.langchain4j</groupId>
<artifactId>langchain4j-document-parser-apache-poi</artifactId>
<exclusions>
<exclusion>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>dev.langchain4j</groupId>
Expand Down
2 changes: 1 addition & 1 deletion ...er-service/src/main/java/com/datamate/rag/indexer/infrastructure/event/RagEtlService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ public void processAfterCommit(DataInsertedEvent event) {
// 更新文件状态为已处理
ragFile.setStatus(FileStatus.PROCESSED);
ragFileRepository.updateById(ragFile);
} catch (Exception e) {
} catch (Throwable e) {
// 处理异常
log.error("Error processing RAG file: {}", ragFile.getFileId(), e);
ragFile.setStatus(FileStatus.PROCESS_FAILED);
Expand Down
Loading
Loading