Skip to content

Remove pub/sub code - #232

Merged
jdavid merged 8 commits into
mainfrom
delpubsub
Jul 31, 2025
Merged

Remove pub/sub code#232
jdavid merged 8 commits into
mainfrom
delpubsub

Conversation

@jdavid

@jdavid jdavid commented Jul 17, 2025

Copy link
Copy Markdown
Collaborator

No description provided.

@jdavid
jdavid marked this pull request as ready for review July 29, 2025 17:22
@jdavid
jdavid requested a review from FrancescAlted July 29, 2025 17:22

@FrancescAlted FrancescAlted left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent pruning work! I see that you keep your shears sharp and clean :-)

FWIW, I have some (light) comments. Also, I have run the new version locally and it runs flawlessly for what I have experienced (including HDF5 support). I am approving this, so feel free to merge when you consider this finished.

Comment thread caterva2.sample.toml
urlbase = "https://cat2.example.com"
# ... other settings as above ...
statedir = "_caterva2/sub"
#http = "_caterva2/sub/uvicorn.socket"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that still working? If so, maybe it is worth a simple line what is that.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, they are described above:

# - statedir: the directory where the subcriber's data will be stored (default: _caterva2/sub)
# - http: where the subscriber listens to (a unix socket or a host/port) (default: localhost:8002)
# - urlbase: the base url users will use to reach the subscriber (default: http://localhost:8002)

Comment thread caterva2/services/sub.py
# src should start with a special root or known root
for src in srcs:
if not src.startswith(("@personal", "@shared", "@public")) and src not in settings.database.roots:
if not src.startswith(("@personal", "@shared", "@public")):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK, there is no support for other roots than these, so I wonder if this check is necessary.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this specific case if the src parameter has an unexpected value we send a "400 Bad Request" response, so the check is still relevant.
But there are many other places that still need clean-up, either in this PR or in a follow up. For example just below for the dst argument, there is still code for supporting the old roots, this should be change to instead return an error.

[1, slice(None, 1), slice(0, 10), slice(10, 20), slice(None), slice(10, 20, 1)],
)
def test_dataset_getitem_fetch(slice_, examples_dir, client):
def test_dataset_getitem_fetch(slice_, examples_dir, client, fill_public):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fill_public does not seem to be used here.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The return value of fill_public is not used, but this fixture copies data (to the @public root) that is required for the test to pass.

def test_download_public_file(examples_dir, fill_public, tmp_path):
fnames, mypublic = fill_public
for fname in fnames:
# TODO fetch (and download) of HDF5 files is not supported (gives a 500 error)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. Added ticket #233 .

@jdavid

jdavid commented Jul 31, 2025

Copy link
Copy Markdown
Collaborator Author

I will merge this PR and create an issue with follow-up actions.

@jdavid
jdavid merged commit ebd3909 into main Jul 31, 2025
3 of 4 checks passed
@jdavid jdavid mentioned this pull request Jul 31, 2025
@jdavid
jdavid deleted the delpubsub branch July 31, 2025 10:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants