Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update tabular data examples #611

Merged
merged 8 commits into from
May 15, 2024

Conversation

npentrel
Copy link
Contributor

No description provided.

@npentrel npentrel requested a review from a team as a code owner May 13, 2024 15:54
@npentrel npentrel requested review from njooma and stuqdog May 13, 2024 15:54
src/viam/app/data_client.py Outdated Show resolved Hide resolved
@@ -255,10 +255,14 @@ async def tabular_data_by_sql(self, organization_id: str, sql_query: str) -> Lis

async def tabular_data_by_mql(self, organization_id: str, mql_binary: List[bytes]) -> List[Dict[str, ValueTypes]]:
"""Obtain unified tabular data and metadata, queried with MQL.
You must install the `pymongo` package and `import bson` to use the `tabular_data_by_mql` method.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this accurate? I don't think this is strictly necessary -- you can use any bson serialization library. I would change this to be more of a suggestion, something like:

mql_binary is an array of encoded bson queries. You can encode your bson queries using a library like pymongo.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well you cannot use the regular bson library that I first had installed. hyperopt/hyperopt#547

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens when you try to use the bson library? I was just able to do this:

import bson

bson.dumps({ '$match': { 'location_id': '<location-id>' }})

And that outputs a valid bson binary. I'm curious as to the functionality you are seeing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that does indeed work. You can't call encode on that. I think that's what I was trying before and it has just a varietey of encode_* methods

src/viam/app/data_client.py Outdated Show resolved Hide resolved
Copy link
Member

@stuqdog stuqdog left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks reasonable to me though I don't have a ton of experience with bson. I assume this was all tested and works?

@npentrel
Copy link
Contributor Author

tested both, yes

@@ -255,10 +255,14 @@ async def tabular_data_by_sql(self, organization_id: str, sql_query: str) -> Lis

async def tabular_data_by_mql(self, organization_id: str, mql_binary: List[bytes]) -> List[Dict[str, ValueTypes]]:
"""Obtain unified tabular data and metadata, queried with MQL.
You must `import bson` to create valid bson binary objects as input for the `tabular_data_by_mql` method.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, I don't want to prescribe a library, particularly given that the bson library hasn't been updated in 3 years. Further, a library isn't even technically required -- you could encode the bson by hand.

My suggestion would be to remove the must, and then provide an example of using a library:

async def tabular_data_by_mql(...):
    """Obtain unified tabular data and metadata, queried with MQL.
    `mql_binary` is an array of encoded `bson` queries. You can encode your bson queries using a library like `pymongo` or `bson`.

    ::
        # using bson 
        import bson
        data = ...(..., mql_binary=[bosn.dumps(...)])

        # using pymongo 
        import bson
        data = ...(..., mql_binary=[bosn.encode(...)])
"""

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You didn't actually say that before. I'm ok using your proposal.

src/viam/app/data_client.py Outdated Show resolved Hide resolved
src/viam/app/data_client.py Outdated Show resolved Hide resolved
src/viam/app/data_client.py Outdated Show resolved Hide resolved
src/viam/app/data_client.py Outdated Show resolved Hide resolved
src/viam/app/data_client.py Outdated Show resolved Hide resolved
@npentrel npentrel merged commit 432f185 into viamrobotics:main May 15, 2024
12 checks passed
njooma pushed a commit that referenced this pull request May 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants