Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New log streams not showing up in log streams list #51

Closed
123vivekr opened this issue Aug 22, 2022 · 5 comments · Fixed by #69
Closed

New log streams not showing up in log streams list #51

123vivekr opened this issue Aug 22, 2022 · 5 comments · Fixed by #69
Assignees

Comments

@123vivekr
Copy link
Contributor

I was able to create a log stream but it wont show up when I list the streams, even when I have successfully pushed logs into the stream.

How to reproduce?

  1. Delete log stream (in-case it exists already)
curl --request DELETE "http://0.0.0.0:8000/api/v1/logstream/testnewlogbug" --header 'Authorization: Basic cGFyc2VhYmxlOnBhcnNlYWJsZQ=='
  1. Create a new log stream testnewlogbug
curl --request PUT "http://0.0.0.0:8000/api/v1/logstream/testnewlogbug" --header 'Authorization: Basic cGFyc2VhYmxlOnBhcnNlYWJsZQ=='
  1. List log stream
curl --request GET "http://0.0.0.0:8000/api/v1/logstream" --header 'Authorization: Basic cGFyc2VhYmxlOnBhcnNlYWJsZQ=='

The stream testnewlogbug does not show up.
Double check if testnewlogbug folder is present in s3

aws --endpoint=https://sgp1.digitaloceanspaces.com --region="sgp1" s3 ls s3://parseable/
@123vivekr
Copy link
Contributor Author

@nitisht Can you assign this to me?

@nitisht
Copy link
Member

nitisht commented Aug 22, 2022

yes..done

@123vivekr
Copy link
Contributor Author

Update: Seems like only stream names starting with test aren't getting listed.

image

NOTE: I'm testing on the "live demo" given in the README.md

@123vivekr
Copy link
Contributor Author

The problem is with either the AWS SDK or DO Spaces API

@nitisht
Copy link
Member

nitisht commented Aug 27, 2022

we're moving to MinIO as the default demo storage with #65 we can test after that PR is merged

nitisht pushed a commit that referenced this issue Aug 27, 2022
Fixes log stream listing by using common prefixes.
https://docs.rs/aws-sdk-s3/latest/aws_sdk_s3/output/struct.ListObjectsV2Output.html#method.common_prefixes

This lists all the root level objects which in case of 
parseable are log stream prefixes.

Fixes #51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants