Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TextDecoderStream ignores empty lines #13809

Closed
aheissenberger opened this issue Mar 1, 2022 · 2 comments
Closed

TextDecoderStream ignores empty lines #13809

aheissenberger opened this issue Mar 1, 2022 · 2 comments

Comments

@aheissenberger
Copy link

Describe the bug
controller.enque will remove empty chunks from stream
When using e.g. LineStream, all empty lines are removed from the stream.

Steps to Reproduce

import { LineStream, DelimiterStream } from "https://deno.land/std@0.127.0/streams/mod.ts"

const encoder = new TextEncoder();
const textStream = new ReadableStream({
    start(controller) {
        controller.enqueue(encoder.encode("Linie 1\n\nLine 3\n"));
        controller.close();
    },
});
let countLines = 0;
const lines = textStream
    .pipeThrough(new LineStream())
    .pipeThrough(new TextDecoderStream())
    .pipeThrough(new TransformStream({
        transform(chunk, controller) {
            countLines++
            console.log(`${countLines} LINE: "${chunk}" - ${chunk.length}`)
            controller.enqueue(chunk)
        }
    }))
    .pipeThrough(new TextEncoderStream())

const buf = [];
for await (const s of lines) {
    buf.push(s);
}

wrong result

1 LINE: "Linie 1" - 7
2 LINE: "Line 3" - 6

Expected behavior

empty Lines should not be suppressed

correct result

1 LINE: "Linie 1" - 7
2 LINE: ""- 0
3 LINE: "Line 3" - 6

Environment

  • OS: MacOS 12.2.1
  • deno version: deno 1.19.1 (release, aarch64-apple-darwin); v8 9.9.115.7; typescript 4.5.2
  • std version: 0.127.0
@aheissenberger aheissenberger added bug Something isn't working needs triage labels Mar 1, 2022
@crowlKats crowlKats changed the title std/streams - LineStream,DelimiterStream remove empty lines, records TextDecoderStream ignores empty lines Mar 1, 2022
@crowlKats
Copy link
Member

upstream whatwg/encoding#283

@crowlKats crowlKats transferred this issue from denoland/deno_std Mar 1, 2022
@crowlKats crowlKats added upstream Changes in upstream are required to solve these issues and removed bug Something isn't working needs triage labels Mar 1, 2022
@crowlKats
Copy link
Member

This has been addressed in denoland/deno_std#1978. This problem will now not occur with TextLineStream.

@crowlKats crowlKats removed the upstream Changes in upstream are required to solve these issues label Mar 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants