CVS Challenge

• CVS Challenge
  • 项目介绍
  • 步骤
    • 1. 创建项目
    • 2. 使用structopt解析命令行参数
    • 3. 添加opt模块
    • 4. 添加对csv的核心处理模块
    • 5. 添加单元测试
    • 6. 添加集成测试
      • 6.1. 新增src/lib.rs，将所有模块引入其中，并暴露对外可以调用的函数。
      • 6.2. 修改main.rs
      • 6.3. 创建tests/integration_test.rs文件
    • 7. 添加性能基准测试
      • 7.1. 创建benches/file_op_bench.rs
      • 7.2. 安装并使用rust nightly版本进行基准测试

项目介绍

本项目主要是用来验证Rust模块化编程。

步骤

1. 创建项目

cargo new --bin csv_challenge

创建二进制项目。cargo默认使用了--bin参数。

默认目录结构

.
├── Cargo.toml
├── src
│  └── main.rs

2. 使用structopt解析命令行参数

structopt是基于clap的基础上构建而成，简化了操作。

添加依赖

[dependencies]
structopt = "0.2"
structopt-derive = "0.2"

因为structopt是基于过程宏(Procedural Macro)的，所以它依赖sructopt-derive包。

3. 添加opt模块

创建src/opt.rs

use structopt_derive::*;
#[derive(StructOpt, Debug)]
#[structopt(name = "csv_challenge", about = "Usage")]
pub struct Opt {
    #[structopt(help = "Input file")]
    pub input : String,
    #[structopt(help = "Column Name")]
    pub column_name: String,
    #[structopt(help = "Replcaement Column Name")]
    pub replacement:String,
    #[structopt(help ="Output file, stdout if not present")]
    pub output: Option<String>, 
}

在main.rs中来引用Opt。

mod opt;
use self::opt::Opt;

fn main(){
    let opt = Opt::from_args();
}

4. 添加对csv的核心处理模块

添加src/core/read.rs和src/core/write.rs两个文件

• read.rs

use super::{Error, PathBuf,File,Read,Write};

pub fn load_csv(csv_file: PathBuf) -> Result<String,Error> {
    let file = read(csv_file)?;
    Ok(file)
}

pub fn write_csv(csv_data: &str, filename: &str) -> Result<(),Error> {
    write(csv_data,filename)?;
    Ok(())
}

fn read(path: PathBuf) -> Result<String,Error> {
    let mut buffer = String::new();
    let mut file = open(path)?;
    file.read_to_string(&mut buffer)?;

    if buffer.is_empty() {
        return Err("input file missing")?
    }

    Ok(buffer)
}

fn open(path: PathBuf) -> Result<File, Error> {
    let file = File::open(path)?;
    Ok(file)
}

fn write(data: &str, filename : &str) -> Result<(), Error> {
    let mut buffer = File::create(filename)?;
    buffer.write_all(data.as_bytes())?;
    Ok(())
}

• write.rs

use super::*;

pub fn replace_column(data: String, column: &str, replacement: &str) -> Result<String, Error> {
    let mut lines = data.lines();
    let headers = lines.next().unwrap();

    let columns : Vec<&str> = headers.split(',').collect();

    let column_number = columns.iter().position(|&e| e == column);
    let column_number = match column_number {
        Some(column) => column,
        None => Err("column name doesn't exist in the input file")?,
    };

    let mut result = String::with_capacity(data.capacity());

    result.push_str(&columns.join(","));
    result.push('\n');

    for line in lines {
        let mut records : Vec<&str> = line.split(',').collect();
        records[column_number] = replacement;
        result.push_str(&records.join(","));
        result.push('\n');
    }

    Ok(result)
}

5. 添加单元测试

#[cfg(test)]
mod test {
    use std::path::PathBuf;
    use super::load_csv;

    #[test]
    fn test_valid_load_csv() {
        let filename = PathBuf::from("./input/challenge.csv");
        let csv_data =load_csv(filename);
        assert!(csv_data.is_ok());
    }
}

6. 添加集成测试

Rust对于二进制是不能添加集成测试的，因为二进制包只能独立使用，并不能对外提供可调用的函数，需要进行改造。

6.1. 新增src/lib.rs，将所有模块引入其中，并暴露对外可以调用的函数。

mod opt;
mod err;
mod core;

pub use self::opt::Opt;
pub use self::core::{
    read::{load_csv,write_csv},
    write::replace_column,
};

6.2. 修改main.rs

use csv_challenge::{
    Opt,
    {load_csv,write_csv},
    replace_column,
};

这种main.rs配合lib.rs的形式，是二进制包的最佳实践

6.3. 创建tests/integration_test.rs文件

#[cfg(test)]

mod test{
    use std::path::PathBuf;
    use std::fs;

    use csv_challenge::{
        {load_csv,write_csv},
        replace_column,
    };

    #[test]
    fn test_csv_challenge() {
        let filename = PathBuf::from("./input/challenge.csv");
        let csv_data = load_csv(filename).unwrap();
        let modified_data = replace_column(csv_data,"City","Beijing").unwrap();
        let output_file = write_csv(&modified_data, "output/test.csv");

        assert!(output_file.is_ok());

        fs::remove_file("output/test.csv");
    }
}

7. 添加性能基准测试

7.1. 创建benches/file_op_bench.rs

#![feature(test)]
extern crate test;
use test::Bencher;
use std::path::PathBuf;
use csv_challenge::{
    Opt,
    {load_csv, write_csv},
    replace_column,
};

#[bench]
fn bench_read_100times(b: &mut test::Bencher) {
    b.iter(|| {
        let n = test::black_box(100);
        (0..n).fold(0,  |_,_|{test_load_csv();0})
    });
}

fn test_load_csv() {
    let filename = PathBuf::from("./input/challenge.csv");
    load_csv(filename);
}

注意：要使用基准测试，必须启用#![feature(test)]。但是只能在rust nightly版本中使用。

7.2. 安装并使用rust nightly版本进行基准测试

• 安装nightly版本

rustup toolchian install nightly
rustup default nightly

• 运行基准测试

cargo bench

基准测试运行结果

     Running target/release/deps/file_op_bench-4a13631faeeecf0a

running 1 test
test bench_read_100times ... bench:   2,019,512 ns/iter (+/- 1,480,889)

test result: ok. 0 passed; 0 failed; 0 ignored; 1 measured; 0 filtered out