Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CSVパース処理の例を追加して整理 #197

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

lempiji
Copy link
Member

@lempiji lempiji commented Sep 23, 2023

No description provided.

Copy link
Member

@shoo shoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dateのところだけ必須で、ほかはお任せします。

Comment on lines +157 to +159
mkdir("temp");
scope (exit) rmdirRecurse("temp");
write("temp/test.csv", "Name,Hoge,Value\nA,Fuga,10.0\nB,Foo,1.5\nC,Bar,-12.5");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

これはただのコメントですが、一時ディレクトリを作って最後に消去するやつは、並列テストした時に競合を起こしそうなので、名前かぶりを防ぐため別途共通化した関数とかを作ったほうが良さそう。

↓みたいなやつ

struct DisposableDir
{
    string path;
    ~this()
    {
        if (path.length && path.exists())
	        rmdirRecurse(path);
    }
    alias path this;
}

DisposableDir createDisposableDirectory()
{
    DisposableDir dir;
    dir.path = buildPath(tempDir(), randomUUID.toString());
    mkdir(dir.path);
    return dir.move();
}

Comment on lines +223 to +229
auto records = readText("temp/test.csv").csvReader(null);

// バッファを用意して、1行ずつ処理します。
import std.array : appender;

auto results = appender!(Record[]);
foreach (record; records)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

この部分、せっかくなら公式の例に倣って

import std.array : appender;
auto results = appender!(Record[]);
foreach (record; csvReader(File("temp/test.csv").byLine(KeepTerminator.yes), null))

こうしてみるのはどうでしょうか。
csvReaderはpopFrontのたびに先頭から順に1文字ずつパースしていく仕組みみたいなので、こうすることでファイルの内容を一度に読み込まなくても済みそうです。
数十GBのCSVファイルを読み込む想定だとこのほうが便利かもしれません。

write("temp/test.csv", "Name,Date\nA,20200101\nB,20210101\nC,20220101");

// あらかじめDate型に変換する簡単な処理を用意します。異常値は考慮していません。
Date parseDate(string text8Digit)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dateを事前にimportする必要がありますね。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants