Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Address how to treat identical corpora in different servers #9

Open
engisalor opened this issue Mar 1, 2023 · 0 comments
Open

Address how to treat identical corpora in different servers #9

engisalor opened this issue Mar 1, 2023 · 0 comments
Labels
enhancement New feature or request

Comments

@engisalor
Copy link
Owner

Currently, caching uses a custom key creation function, which can still be refined #8. This function excludes the host URL and instead uses the API call type (view) and parameters dict ({"corpname": ... etc.}) to determine whether making a call is necessary.

This is intentional: it's assumed that two corpora of the same name are identical even if they're on different servers. For instance, querying the Susanne corpus locally and on Sketch Engine retrieves the same results, so SGEX shouldn't duplicate all those calls.

However, this is a narrow use case that won't fit every circumstance. SGEX should expose a means to manage identical/different corpora with identical/different names on multiple servers. Sketch Engine also uses the preloaded/ prefix for some corpora, meaning that local corpora should use the same naming conventions.

Part of this discussion may include adding additional headers to cached content to streamline how duplicate data, out of date data (e.g., for diachronic corpora), and so on can be identified.

@engisalor engisalor added the enhancement New feature or request label Mar 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant