Flush doesn't work

[Ilqjx]

Neither file.flush() nor buffer.flush() work.

use std::{
    io::{BufWriter, Read, Write},
    time::Duration,
};

use hdrs::Client;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let uri = "hdfs://localhost:9000".to_owned();
    let path = "/tmp/hello.txt".to_owned();
    let mut append = true;

    let client = Client::connect(uri.as_str()).expect("Failed to create hdfs client");

    if client.metadata(path.as_str()).is_err() {
        append = false;
    }

    let file = client
        .open_file()
        .write(true)
        .create(true)
        .append(append)
        .truncate(!append)
        .open(path.as_str())
        .expect(format!("Failed to open file: {}", path).as_str());

    // file.write_all("hello".as_bytes())
    //     .expect(format!("Failed to write file: {}", path).as_str());
    // file.flush()?;

    let mut buffer = BufWriter::new(file);
    buffer.write_all(b"hello")?;
    buffer.flush()?;

    // let mut file = client.open_file().read(true).open(path.as_str())?;
    // let mut buf = vec![0; 1024];
    // let n = file.read(&mut buf)?;
    // println!("n: {:?}", n);

    println!("sleep 8 secs start");
    std::thread::sleep(Duration::from_secs(8));
    println!("sleep end");

    // file.write("WORLD".as_bytes())
    //     .expect(format!("Failed to write file: {}", path).as_str());
    // file.flush()?;

    Ok(())
}

[Xuanwo]

Hi, can I help you?

[Ilqjx]

Thanks. I solved it. Because my data is too small, the cache is not full.

Calling flush doesn't work when there is only one hello, is this intentional?

[Xuanwo]

Calling flush doesn't work when there is only one hello, is this intentional?

What does Calling flush doesn't work mean? What are you expected to see? And what's the actual behavior?

[Ilqjx]

After file.write(b"hello"), file.flush() is called. It is expected that hello should be written to the file immediately. In fact, it should be written to the file after the buffer is full.

[Xuanwo]

After file.write(b"hello"), file.flush() is called. It is expected that hello should be written to the file immediately. In fact, it should be written to the file after the buffer is full.

HDFS has its internal logic. It's only making sure the file is written after close (A.k.a, reaching the consistent view for this file).

[Ilqjx]

Thank you very much, learned.