Skip to content

Commit

Permalink
fix(ai): 修复AI(beta)系统中无法删除创建失败的镜像的问题 (#1118)
Browse files Browse the repository at this point in the history
### 问题原因:
Harbor中找不到对应镜像时会直接抛出错误,终止删除数据
导致就算是创建失败的镜像,因为在Harbor中没有找到对应镜像而无法删除本地数据
关联bug
[创建失败的镜像无法删除](PKUHPC/scow-ai-internal-dev#20)

### 解决方案:
由于在前面的代码中已经处理了调用Harbor删除API无法正常返回的情况
在这之后Harbor中找不到对应镜像时应该无论镜像是什么状态直接删除本地数据,以保证数据的一致性

同时此PR还修复了删除镜像api调用时协议引入的错误,不应该引入scow访问协议,在HarborConfig中增加了Harbor访问协议的配置,默认值为`http`
当前ai-beta只有支持`http`协议访问的测试环境,后续完成https服务的测试后再继续补充该配置项在文档中的说明

### 修改后:
删除创建失败的镜像

![image](https://github.com/PKUHPC/SCOW/assets/43978285/ce158cfa-8742-4138-a325-cd6cc204a58f)
执行删除操作,删除成功

![image](https://github.com/PKUHPC/SCOW/assets/43978285/6917cf7d-8745-4ef5-bf11-fa7eb17685c9)
后台打印了执行删除时的日志

![image](https://github.com/PKUHPC/SCOW/assets/43978285/85e71f18-fe4c-4e78-8296-664eced30ba7)

---------

Co-authored-by: ZihanChen821 <827625357@qq.com>
  • Loading branch information
piccaSun and ZihanChen821 committed Feb 5, 2024
1 parent 2f7590a commit 3242957
Show file tree
Hide file tree
Showing 4 changed files with 20 additions and 9 deletions.
5 changes: 5 additions & 0 deletions .changeset/hungry-hornets-dress.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@scow/ai": patch
---

修复创建失败的镜像无法删除的问题
5 changes: 5 additions & 0 deletions .changeset/mean-colts-report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@scow/config": patch
---

在 aiConfig 下的 harborConfig 配置中增加 protocol 配置,默认值为 "http"
18 changes: 9 additions & 9 deletions apps/ai/src/server/trpc/route/image/image.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ import { getSortedClusterIds } from "@scow/config/build/cluster";
import { sshConnect as libConnect } from "@scow/lib-ssh";
import { TRPCError } from "@trpc/server";
import { aiConfig } from "src/server/config/ai";
import { config, rootKeyPair } from "src/server/config/env";
import { rootKeyPair } from "src/server/config/env";
import { Image, Source, Status } from "src/server/entities/Image";
import { procedure } from "src/server/trpc/procedure/base";
import { clusterNotFound } from "src/server/utils/errors";
Expand Down Expand Up @@ -340,7 +340,7 @@ export const deleteImage = procedure
}

// 获取harrbor中的reference以删除镜像
const getReferenceUrl = `${ config.PROTOCOL || "http"}://${aiConfig.harborConfig.url}/api/v2.0/projects`
const getReferenceUrl = `${aiConfig.harborConfig.protocol}://${aiConfig.harborConfig.url}/api/v2.0/projects`
+ `/${aiConfig.harborConfig.project}/repositories/${user.identityId}%252F${image.name}/artifacts`;
const getReferenceRes = await fetch(getReferenceUrl, {
method: "GET",
Expand All @@ -354,7 +354,7 @@ export const deleteImage = procedure

// 没有返回值且镜像本身状态就是 failure 的,不需要去 harbor 删除了
if (errorText === "" && image.status === Status.FAILURE) {
logger.error(`Maybe image ${input.id} not exist on harbor`);
logger.error(`Maybe image(${input.id}) ${image.name}:${image.tag} not exist on harbor`);
await em.removeAndFlush(image);
return;
}
Expand Down Expand Up @@ -388,18 +388,18 @@ export const deleteImage = procedure
}

if (!reference) {
throw new TRPCError({
code: "INTERNAL_SERVER_ERROR",
message: "Failed to find image tag in harbor! Please contact the administrator! ",
});
// Harbor API 请求接收到正常返回值,但是在Harbor中没有找到对应镜像,则直接删除本地数据库镜像信息
logger.error(`Maybe image(${input.id}) ${image.name}:${image.tag} not exist on harbor`);
await em.removeAndFlush(image);
return;
}

const authInfo = Buffer.from(`${aiConfig.harborConfig.user}:${aiConfig.harborConfig.password}`).toString("base64");

// 如果上面的tag是最相同imageName下相同镜像的最后一个标签,则删除整个Artifact
if (needDeleteArtifact) {

const deleteArtifactUrl = `${ config.PROTOCOL || "http"}://${aiConfig.harborConfig.url}/api/v2.0/projects`
const deleteArtifactUrl = `${aiConfig.harborConfig.protocol}://${aiConfig.harborConfig.url}/api/v2.0/projects`
+ `/${aiConfig.harborConfig.project}/repositories/${user.identityId}%252F${image.name}`
+ `/artifacts/${reference}`;

Expand All @@ -425,7 +425,7 @@ export const deleteImage = procedure

// 如果上面的tag不是最相同imageName下相同镜像的最后一个标签,则只删除该标签
} else {
const deleteUrl = `${ config.PROTOCOL || "http"}://${aiConfig.harborConfig.url}/api/v2.0/projects`
const deleteUrl = `${aiConfig.harborConfig.protocol}://${aiConfig.harborConfig.url}/api/v2.0/projects`
+ `/${aiConfig.harborConfig.project}/repositories/${user.identityId}%252F${image.name}`
+ `/artifacts/${reference}/tags/${image.tag}`;

Expand Down
1 change: 1 addition & 0 deletions libs/config/src/ai.ts
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ export const AiConfigSchema = Type.Object({
project: Type.String({ description: "镜像存储用的Harbor仓库地址下项目名,会作为镜像存储的上级路径" }),
user: Type.String({ description: "Harbor仓库地址登录时使用的用户名" }),
password: Type.String({ description: "Harbor仓库地址登录时使用的登录密码" }),
protocol: Type.String({ description: "Harbor API 的访问协议", default: "http" }),
}),

});
Expand Down

0 comments on commit 3242957

Please sign in to comment.