Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Query results caching #2881

Closed
RayetSena opened this issue Feb 16, 2024 · 2 comments
Closed

Query results caching #2881

RayetSena opened this issue Feb 16, 2024 · 2 comments
Labels

Comments

@RayetSena
Copy link

Hello,

I am using http storage plugin and I want to activate the caching. In the documentation it says that this requires adding cacheResults. So I added this to my config. But when I run the same query again, there is almost no difference in query duration. You can see the configuration and query duration below.

image

{
  "type": "http",
  "cacheResults": true,
  "connections": {
    "opendata": {
      "url": "https://opendata.cbs.nl/ODataApi/odata/",
      "requireTail": true,
      "method": "GET",
      "authType": "none",
      "inputType": "json",
      "xmlDataLevel": 1,
      "postParameterLocation": "QUERY_STRING",
      "verifySSLCert": false
    }
  },
  "timeout": 4000,
  "retryDelay": 5000,
  "proxyType": "direct",
  "authMode": "USER_TRANSLATION",
  "enabled": true
}

image

Also I tried to configure the caching properties in the drill-override.conf file. Here is the configuration that I added:

drill.exec: {
    caching: {
	enabled: true
	storage:{
           type: “local”
           path:  “cache/directory”
        }
    }
}

Drill version
1.21.1

@RayetSena RayetSena added the bug label Feb 16, 2024
@RayetSena
Copy link
Author

When I test using an older version(1.20.1), I can see the query result files under the tmp/http-cache directory, but when I test using a newer version(1.21.1), there are no query result files in that directory, it only contains the journal.

@jnturton
Copy link
Contributor

Promoted to DRILL-8487.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants