Skip to content

feat: 新增csv文件导入导出功能#6

Closed
byk11 wants to merge 9 commits intolabring:dev2.7from
byk11:dev2.7
Closed

feat: 新增csv文件导入导出功能#6
byk11 wants to merge 9 commits intolabring:dev2.7from
byk11:dev2.7

Conversation

@byk11
Copy link
Copy Markdown

@byk11 byk11 commented Apr 10, 2023

新增csv文件导入导出功能

src/api/model.ts Outdated
modelId: string,
jsonData: { prompt: string; completion: string; vector?: number[] }[]
) => POST(`/model/data/pushModelDataJson`, { modelId, data: jsonData });
) => POST(`/model/data/pushModelDataJson`, { modelId, data: jsonData }, { timeout: 300000 });
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

参数名改下

suffix: 'csv',
fileType: 'text/csv'
},
json: {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

json直接都去掉

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个方法不要写成hook,不是每个地方都用button点击导出的。写成一个普通的方法就可以了

return false;
}
return true;
});
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这两个代码直接和成一个就好了,不要分个方法。
const redisData = await Promise.all(data.map(item => ****))
const filterData =

export const config = {
api: {
bodyParser: {
sizeLimit: '8mb'
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

改成20m

} catch (error: any) {
console.log(error);
toast({
title: error?.message || 'Csv文件格式有误',
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

小写

completion: ''
};
header.forEach((item, index) => {
// @ts-ignore
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里需要校验是否有prompt和completion,没有的话就跳过。

* 读取 csv 文件内容
*/
export const readCsvContent = (file: File) => {
return new Promise((resolve: (_: string) => void, reject) => {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个方法是不是可以先调用 readRawText的方法直接获取到文本?

const rows = data.map(function (obj) {
const values = Object.keys(obj).map(function (key) {
return obj[key];
});
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

直接Object.values()就可以了

export const objectToCsv = (data: CsvData[]) => {
console.log(data, 'data');
const header = Object.keys(data[0]).join(',');
const rows = data.map(function (obj) {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

data[0]是header?header的话还需要遍历么

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

data里面的是纯数据,这里是遍历data[0]的key取出并拼接成header

let data = JSON.parse(res);
console.log(res, 'rse');
if (progressData) {
data = progressData(data);
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个地方可能不能用object,csv是有顺序的

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这是一个对象数组,数组是有顺序的,对象的键也有顺序,最后拿到的顺序和redis里给出的顺序是一样的

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

理论上键值对是没有顺序的,只不过大多数情况是一样的

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

那还是得用Object.key(header[i]),这样的顺序肯定和header的顺序一致

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants