Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feast UI: URL scheme "postgresql" is not supported when specifying SQL registry in projects-list.json #3974

Open
TomSteenbergen opened this issue Feb 28, 2024 · 5 comments

Comments

@TomSteenbergen
Copy link
Contributor

TomSteenbergen commented Feb 28, 2024

Expected Behavior

The UI created using a custom React app with a SQL registry shows the entities, feature views, etc. per project.

Current Behavior

It is unclear how to specify the registryPath in your projects-list.json file. The structure of this file could definitely benefit from some more documentation.

While the Feast UI does spin up and shows the project names and descriptions from the projects-list.json on the homepage, it does not load and visualize the corresponding Feast objects when clicking on one of these projects, even though the registry does contain that data (see screenshot).

image

The browser console shows the following error:

Fetch API cannot load postgresql://postgres:test@db:5432/feast. URL scheme "postgresql" is not supported.

The part postgresql://postgres:test@db:5432/feast is the connection string to my SQL registry that I created through docker-compose and that I pass in projects-list.json under the registryPath key.

Is a SQL registry is not supported yet by the UI? Or did I format registryPath incorrectly?

Steps to reproduce

  1. Follow steps from the docs to create a React app.
  2. Add your projects-list.json to the public directory. My file looked like this:
{
  "projects": [
    {
      "name": "Project 1",
      "description": "Description 1.",
      "id": "project_1",
      "registryPath": "postgresql://postgres:test@db:5432/feast"
    },
    {
      "name": "Project 2",
      "description": "Description 2.",
      "id": "project_2",
      "registryPath": "postgresql://postgres:test@db:5432/feast"
    }
  ]
}
  1. Start a Postgres database using the following docker-compose.yml
version: "3"
services:
  db:
    restart: always
    image: postgres:16-alpine
    ports:
      - "5432:5432"
    networks:
      - backend
    volumes:
      - ~/feast_postgres_data:/var/lib/postgresql/data
    environment:
      - POSTGRES_DB=feast
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=test
networks:
  backend:
    driver: bridge

volumes:
  feast_postgres_data: null
  1. Run yarn start.

Specifications

  • Version: 0.35.0
  • Platform: MacOS
  • Subsystem: Sonoma 14.2

Possible Solution

More documentation on setting up the UI would be very much welcome.

@TomSteenbergen TomSteenbergen changed the title Feast UI: Unclear how to host for multiple projects using single SQL registry Feast UI: URL scheme "postgresql" is not supported when specifying SQL registry in projects-list.json Feb 28, 2024
@jeremyary
Copy link
Collaborator

Hey @TomSteenbergen thanks for raising your issue. My initial suspicion is that the postgres protocol isn't supported for UI's registryPath, so we're looking into this to confirm. Should this be the case, we'll take a look at a docs update to better specify possible values there &, if appropriate based on findings, potentially open another issue to consider supporting the protocol as a feature request. We'll keep you posted here.

@jeremyary jeremyary self-assigned this Mar 4, 2024
tmihalac added a commit to tmihalac/feast that referenced this issue Mar 5, 2024
Added a note in Webui reference in regard to supported registryPath values

Related to feast-dev#3974

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
@tmihalac
Copy link
Contributor

tmihalac commented Mar 5, 2024

Currently registryPath parameters supports only a file location or url value.

Will add a note in the reference documentation mentioning that.

tmihalac added a commit to tmihalac/feast that referenced this issue Mar 5, 2024
Added a note in Webui reference in regard to supported registryPath values

Related to feast-dev#3974

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
HaoXuAI pushed a commit that referenced this issue Mar 5, 2024
…3983)

* fix: Added registryPath parameter documentation
Added a note in Webui reference in regard to supported registryPath values

Related to #3974

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>

* fix: Added registryPath parameter documentation
Added a note in Webui reference in regard to supported registryPath values

Related to #3974

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>

---------

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
@TomSteenbergen
Copy link
Contributor Author

TomSteenbergen commented Mar 7, 2024

Currently registryPath parameters supports only a file location or url value.

@tmihalac Thanks for confirming! Do you happen to know whether support for a SQL registry is somewhere on the roadmap?

@jeremyary
Copy link
Collaborator

jeremyary commented Mar 7, 2024

@TomSteenbergen I feel we're still working on our community roadmap & I can't unilaterally say with certainty, but I can share that there's been interest outside of your own expressed in seeing SQL-based registry/offline/online followed up so I'd think it's likely to be explored.

lokeshrangineni pushed a commit to lokeshrangineni/feast that referenced this issue Mar 7, 2024
…east-dev#3983)

* fix: Added registryPath parameter documentation
Added a note in Webui reference in regard to supported registryPath values

Related to feast-dev#3974

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>

* fix: Added registryPath parameter documentation
Added a note in Webui reference in regard to supported registryPath values

Related to feast-dev#3974

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>

---------

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: Lokesh Rangineni <lokeshforjava@gmail.com>
HaoXuAI pushed a commit that referenced this issue Mar 9, 2024
…3983)

* fix: Added registryPath parameter documentation
Added a note in Webui reference in regard to supported registryPath values

Related to #3974

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>

* fix: Added registryPath parameter documentation
Added a note in Webui reference in regard to supported registryPath values

Related to #3974

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>

---------

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
HaoXuAI added a commit that referenced this issue Mar 9, 2024
* fix: remove not use input parameter in spark source (#3980)

remove unused parameter when init sparksource

Signed-off-by: tanlocnguyen <tanlocnguyen296@gmail.com>
Co-authored-by: tanlocnguyen <tanlocnguyen296@gmail.com>

* fix: Added registryPath parameter documentation in WebUI reference (#3983)

* fix: Added registryPath parameter documentation
Added a note in Webui reference in regard to supported registryPath values

Related to #3974

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>

* fix: Added registryPath parameter documentation
Added a note in Webui reference in regard to supported registryPath values

Related to #3974

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>

---------

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>

* fix: Get container host addresses from testcontainers (#3946)

* fix: get container host addresses from testcontainers

Signed-off-by: tokoko <togurg14@freeuni.edu.ge>

* resolve trino container host with testcontainers

Signed-off-by: tokoko <togurg14@freeuni.edu.ge>

---------

Signed-off-by: tokoko <togurg14@freeuni.edu.ge>

* fix: Swap security label check on the PR title validation job to explicit permissions instead (#3987)

revert security label check for PR title validation & add explicit read-only permission instead

Signed-off-by: Jeremy Ary <jary@redhat.com>

* feat: Add Entity df in format of a Spark Dataframe instead of just pd.DataFrame or string for SparkOfflineStore (#3988)

* remove unused parameter when init sparksource

Signed-off-by: tanlocnguyen <tanlocnguyen296@gmail.com>

* feat: add entity df to SparkOfflineStore when get_historical_features

Signed-off-by: tanlocnguyen <tanlocnguyen296@gmail.com>

* fix: lint error

Signed-off-by: tanlocnguyen <tanlocnguyen296@gmail.com>

---------

Signed-off-by: tanlocnguyen <tanlocnguyen296@gmail.com>
Co-authored-by: tanlocnguyen <tanlocnguyen296@gmail.com>

* feat: Dropping unit tests for Python 3.8 (#3989)

feat: dropping unit tests for Python 3.8

Update unit_tests.yml to no longer run for Python 3.8

Signed-off-by: franciscojavierarceo <francisco.arceo@affirm.com>

* fix: Move gRPC dependencies to an extra (#3900)

* chore: Bump ip from 1.1.5 to 1.1.9 in /ui (#3959)

* chore: Update readme spark.md (#3992)

update readme spark.md

Signed-off-by: tanlocnguyen <tanlocnguyen296@gmail.com>
Co-authored-by: tanlocnguyen <tanlocnguyen296@gmail.com>

* feat: Dropping Python 3.8 from local integration tests and integration tests (#3994)

* fix: Handle ComplexFeastType to None comparison (#3876)

Signed-off-by: Aliaksandr Sasnouskikh <jahstreetlove@gmail.com>

* feat: Add python client for remote registry server (#3941)

* add remote registry

Signed-off-by: tokoko <togurg14@freeuni.edu.ge>

* format and lint remote registry code

Signed-off-by: tokoko <togurg14@freeuni.edu.ge>

* add read-only registry exception

Signed-off-by: tokoko <togurg14@freeuni.edu.ge>

---------

Signed-off-by: tokoko <togurg14@freeuni.edu.ge>

---------

Signed-off-by: tanlocnguyen <tanlocnguyen296@gmail.com>
Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
Signed-off-by: tokoko <togurg14@freeuni.edu.ge>
Signed-off-by: Jeremy Ary <jary@redhat.com>
Signed-off-by: franciscojavierarceo <francisco.arceo@affirm.com>
Signed-off-by: Aliaksandr Sasnouskikh <jahstreetlove@gmail.com>
Co-authored-by: locnt241 <73770977+ElliotNguyen68@users.noreply.github.com>
Co-authored-by: tanlocnguyen <tanlocnguyen296@gmail.com>
Co-authored-by: Theodor Mihalache <84387487+tmihalac@users.noreply.github.com>
Co-authored-by: Tornike Gurgenidze <togurg14@freeuni.edu.ge>
Co-authored-by: Jeremy Ary <jary@redhat.com>
Co-authored-by: Francisco Javier Arceo <franciscojavierarceo@users.noreply.github.com>
Co-authored-by: cburroughs <chris.burroughs@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Alex Sasnouskikh <jahstreetlove@gmail.com>
tqtensor pushed a commit to tqtensor/feast that referenced this issue Mar 11, 2024
…east-dev#3983)

* fix: Added registryPath parameter documentation
Added a note in Webui reference in regard to supported registryPath values

Related to feast-dev#3974

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>

* fix: Added registryPath parameter documentation
Added a note in Webui reference in regard to supported registryPath values

Related to feast-dev#3974

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>

---------

Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
@tokoko
Copy link
Contributor

tokoko commented Mar 19, 2024

We recently added grpc registry server in #3924. The intention for it was to be used from go and java servers, but i guess it could be a solution in here as well... if someone figures out a way to call grpc from the browser :) and get the registry proto from there.

franciscojavierarceo pushed a commit that referenced this issue Apr 16, 2024
# [0.36.0](v0.35.0...v0.36.0) (2024-04-16)

### Bug Fixes

* Add __eq__, __hash__ to SparkSource for correct comparison ([#4028](#4028)) ([e703b40](e703b40))
* Add conn.commit() to Postgresonline_write_batch.online_write_batch ([#3904](#3904)) ([7d75fc5](7d75fc5))
* Add missing __init__.py to embedded_go ([#4051](#4051)) ([6bb4c73](6bb4c73))
* Add missing init files in infra utils ([#4067](#4067)) ([54910a1](54910a1))
* Added registryPath parameter documentation in WebUI reference ([#3983](#3983)) ([5e0af8f](5e0af8f)), closes [#3974](#3974) [#3974](#3974)
* Adding missing init files in materialization modules ([#4052](#4052)) ([df05253](df05253))
* Allow trancated timestamps when converting ([#3861](#3861)) ([bdd7dfb](bdd7dfb))
* Azure blob storage support in Java feature server ([#2319](#2319)) ([#4014](#4014)) ([b9aabbd](b9aabbd))
* Bugfix for grabbing historical data from Snowflake with array type features. ([#3964](#3964)) ([1cc94f2](1cc94f2))
* Bytewax materialization engine fails when loading feature_store.yaml ([#3912](#3912)) ([987f0fd](987f0fd))
* CI unittest warnings ([#4006](#4006)) ([0441b8b](0441b8b))
* Correct the returning class proto type of StreamFeatureView to StreamFeatureViewProto instead of FeatureViewProto. ([#3843](#3843)) ([86d6221](86d6221))
* Create index only if not exists during MySQL online store update ([#3905](#3905)) ([2f99a61](2f99a61))
* Disable minio tests in workflows on master and nightly ([#4072](#4072)) ([c06dda8](c06dda8))
* Disable the Feast Usage feature by default. ([#4090](#4090)) ([b5a7013](b5a7013))
* Dump repo_config by alias ([#4063](#4063)) ([e4bef67](e4bef67))
* Extend SQL registry config with a sqlalchemy_config_kwargs key ([#3997](#3997)) ([21931d5](21931d5))
* Feature Server image startup in OpenShift clusters ([#4096](#4096)) ([9efb243](9efb243))
* Fix copy method for StreamFeatureView ([#3951](#3951)) ([cf06704](cf06704))
* Fix for materializing entityless feature views in Snowflake ([#3961](#3961)) ([1e64c77](1e64c77))
* Fix type mapping spark ([#4071](#4071)) ([3afa78e](3afa78e))
* Fix typo as the cli does not support shortcut-f option. ([#3954](#3954)) ([dd79dbb](dd79dbb))
* Get container host addresses from testcontainers ([#3946](#3946)) ([2cf1a0f](2cf1a0f))
* Handle ComplexFeastType to None comparison ([#3876](#3876)) ([fa8492d](fa8492d))
* Hashlib md5 errors in FIPS for python 3.9+ ([#4019](#4019)) ([6d9156b](6d9156b))
* Making the query_timeout variable as optional int because upstream is considered to be optional ([#4092](#4092)) ([fd5b620](fd5b620))
* Move gRPC dependencies to an extra ([#3900](#3900)) ([f93c5fd](f93c5fd))
* Prevent spamming pull busybox from dockerhub ([#3923](#3923)) ([7153cad](7153cad))
* Quickstart notebook example ([#3976](#3976)) ([b023aa5](b023aa5))
* Raise error when not able read of file source spark source ([#4005](#4005)) ([34cabfb](34cabfb))
* remove not use input parameter in spark source ([#3980](#3980)) ([7c90882](7c90882))
* Remove parentheses in pull_latest_from_table_or_query ([#4026](#4026)) ([dc4671e](dc4671e))
* Remove proto-plus imports ([#4044](#4044)) ([ad8f572](ad8f572))
* Remove unnecessary dependency on mysqlclient ([#3925](#3925)) ([f494f02](f494f02))
* Restore label check for all actions using pull_request_target ([#3978](#3978)) ([591ba4e](591ba4e))
* Revert mypy config ([#3952](#3952)) ([6b8e96c](6b8e96c))
* Rewrite Spark materialization engine to use mapInPandas ([#3936](#3936)) ([dbb59ba](dbb59ba))
* Run feature server w/o gunicorn on windows ([#4024](#4024)) ([584e9b1](584e9b1))
* SqlRegistry _apply_object update statement ([#4042](#4042)) ([ef62def](ef62def))
* Substrait ODFVs for online ([#4064](#4064)) ([26391b0](26391b0))
* Swap security label check on the PR title validation job to explicit permissions instead ([#3987](#3987)) ([f604af9](f604af9))
* Transformation server doesn't generate files from proto ([#3902](#3902)) ([d3a2a45](d3a2a45))
* Trino as an OfflineStore Access Denied when BasicAuthenticaion ([#3898](#3898)) ([49d2988](49d2988))
* Trying to import pyspark lazily to avoid the dependency on the library ([#4091](#4091)) ([a05cdbc](a05cdbc))
* Typo Correction in Feast UI Readme ([#3939](#3939)) ([c16e5af](c16e5af))
* Update actions/setup-python from v3 to v4 ([#4003](#4003)) ([ee4c4f1](ee4c4f1))
* Update typeguard version to >=4.0.0 ([#3837](#3837)) ([dd96150](dd96150))
* Upgrade sqlalchemy from 1.x to 2.x regarding PVE-2022-51668. ([#4065](#4065)) ([ec4c15c](ec4c15c))
* Use CopyFrom() instead of __deepycopy__() for creating a copy of protobuf object. ([#3999](#3999)) ([5561b30](5561b30))
* Using version args to install the correct feast version ([#3953](#3953)) ([b83a702](b83a702))
* Verify the existence of Registry tables in snowflake before calling CREATE sql command. Allow read-only user to call feast apply. ([#3851](#3851)) ([9a3590e](9a3590e))

### Features

* Add duckdb offline store ([#3981](#3981)) ([161547b](161547b))
* Add Entity df in format of a Spark Dataframe instead of just pd.DataFrame or string for SparkOfflineStore ([#3988](#3988)) ([43b2c28](43b2c28))
* Add gRPC Registry Server ([#3924](#3924)) ([373e624](373e624))
* Add local tests for s3 registry using minio ([#4029](#4029)) ([d82d1ec](d82d1ec))
* Add python bytes to array type conversion support proto ([#3874](#3874)) ([8688acd](8688acd))
* Add python client for remote registry server ([#3941](#3941)) ([42a7b81](42a7b81))
* Add Substrait-based ODFV transformation ([#3969](#3969)) ([9e58bd4](9e58bd4))
* Add support for arrays in snowflake ([#3769](#3769)) ([8d6bec8](8d6bec8))
* Added delete_table to redis online store ([#3857](#3857)) ([03dae13](03dae13))
* Adding support for Native Python feature transformations for ODFVs ([#4045](#4045)) ([73bc853](73bc853))
* Bumping requirements ([#4079](#4079)) ([1943056](1943056))
* Decouple transformation types from ODFVs ([#3949](#3949)) ([0a9fae8](0a9fae8))
* Dropping Python 3.8 from local integration tests and integration tests ([#3994](#3994)) ([817995c](817995c))
* Dropping python 3.8 requirements files from the project. ([#4021](#4021)) ([f09c612](f09c612))
* Dropping the support for python 3.8 version from feast ([#4010](#4010)) ([a0f7472](a0f7472))
* Dropping unit tests for Python 3.8 ([#3989](#3989)) ([60f24f9](60f24f9))
* Enable Arrow-based columnar data transfers  ([#3996](#3996)) ([d8d7567](d8d7567))
* Enable Vector database and retrieve_online_documents API ([#4061](#4061)) ([ec19036](ec19036))
* Kubernetes materialization engine written based on bytewax ([#4087](#4087)) ([7617bdb](7617bdb))
* Lint with ruff ([#4043](#4043)) ([7f1557b](7f1557b))
* Make arrow primary interchange for offline ODFV execution ([#4083](#4083)) ([9ed0a09](9ed0a09))
* Pandas v2 compatibility ([#3957](#3957)) ([64459ad](64459ad))
* Pull duckdb from contribs, add to CI ([#4059](#4059)) ([318a2b8](318a2b8))
* Refactor ODFV schema inference ([#4076](#4076)) ([c50a9ff](c50a9ff))
* Refactor registry caching logic into a separate class ([#3943](#3943)) ([924f944](924f944))
* Rename OnDemandTransformations to Transformations ([#4038](#4038)) ([9b98eaf](9b98eaf))
* Revert updating dependencies so that feast can be run on 3.11. ([#3968](#3968)) ([d3c68fb](d3c68fb)), closes [#3958](#3958)
* Rewrite ibis point-in-time-join w/o feast abstractions ([#4023](#4023)) ([3980e0c](3980e0c))
* Support s3gov schema by snowflake offline store during materialization ([#3891](#3891)) ([ea8ad17](ea8ad17))
* Update odfv test ([#4054](#4054)) ([afd52b8](afd52b8))
* Update pyproject.toml to use Python 3.9 as default ([#4011](#4011)) ([277b891](277b891))
* Update the Pydantic from v1 to v2 ([#3948](#3948)) ([ec11a7c](ec11a7c))
* Updating dependencies so that feast can be run on 3.11. ([#3958](#3958)) ([59639db](59639db))
* Updating protos to separate transformation ([#4018](#4018)) ([c58ef74](c58ef74))

### Reverts

* Reverting bumping requirements ([#4081](#4081)) ([1ba65b4](1ba65b4)), closes [#4079](#4079)
* Verify the existence of Registry tables in snowflake… ([#3907](#3907)) ([c0d358a](c0d358a)), closes [#3851](#3851)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants