Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Vineyard storage backend changes the arrow type from string to large_string. #661

Open
codingl2k1 opened this issue Aug 16, 2023 · 0 comments
Labels
bug Something isn't working
Milestone

Comments

@codingl2k1
Copy link
Contributor

Describe the bug

A clear and concise description of what the bug is.

=================================== FAILURES ===================================
_________________________ test_put_get_arrow[vineyard] _________________________

storage_context = <xorbits._mars.storage.vineyard.VineyardStorage object at 0x7fdd9a401940>

    @pytest.mark.asyncio
    @require_lib
    async def test_put_get_arrow(storage_context):
        storage = storage_context
    
        data = [
            pa.Table.from_pydict({"a": [1, 2, 3], "b": list("abc")}),
            pa.RecordBatch.from_pydict({"a": [1, 2, 3], "b": list("abc")}),
        ]
        for d in data:
            put_info = await storage.put(d)
            get_data = await storage.get(put_info.object_id)
>           assert d == get_data
E           assert pyarrow.Table...["a","b","c"]] == pyarrow.Table...["a","b","c"]]
E             Full diff:
E               pyarrow.Table
E               a: int64
E             - b: large_string
E             ?    ------
E             + b: string
E               ----
E               a: [[1,2,3]]
E               b: [["a","b","c"],
E               ]

xorbits/_mars/storage/tests/test_libs.py:400: AssertionError

To Reproduce

To help us to reproduce this bug, please provide information below:

  1. Your Python version
  2. The version of Xorbits you use
  3. Versions of crucial packages, such as numpy, scipy and pandas
  4. Full stack of the error.
  5. Minimized code to reproduce the error.

Expected behavior

A clear and concise description of what you expected to happen.

Additional context

Add any other context about the problem here.

@XprobeBot XprobeBot added the bug Something isn't working label Aug 16, 2023
@XprobeBot XprobeBot added this to the v0.5.2 milestone Aug 16, 2023
@XprobeBot XprobeBot modified the milestones: v0.5.2, Temp, v0.6.0, v0.6.1 Sep 8, 2023
@XprobeBot XprobeBot modified the milestones: v0.6.1, v0.6.2, v0.6.3 Sep 15, 2023
@XprobeBot XprobeBot modified the milestones: v0.6.3, v0.7.0 Sep 25, 2023
@XprobeBot XprobeBot modified the milestones: v0.7.0, v0.7.1 Oct 23, 2023
@XprobeBot XprobeBot modified the milestones: v0.7.1, v0.7.2 Nov 21, 2023
@XprobeBot XprobeBot modified the milestones: v0.7.2, v0.7.3 Jan 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants