Skip to content

datafusion cannot recognize chinese charactors. #3203

@CheersGrant

Description

@CheersGrant

Describe the bug
datafusion cannot recognize chinese charactors.

To Reproduce

  1. simply create a csv file, put the title as name in chinese charactor: 扫描人
  2. run the rust code as below,will get the parse error: Error: SQL(ParserError("Expected an expression:, found: 扫"))

`use datafusion::prelude::*;

#[tokio::main]
async fn main() -> datafusion::error::Result<()> {
// register the table
let ctx = SessionContext::new();
ctx.register_csv("example", "lite.csv", CsvReadOptions::new()).await?;

// create a plan to run a SQL query
let df = ctx.sql("SELECT distinct 扫描人 FROM example").await?;

// execute and print results
df.show().await?;
Ok(())
}`

Expected behavior
make datafusion recognize the chinese charactors.

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions