New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
New feature: local cache #35
Merged
Merged
Changes from 1 commit
Commits
Show all changes
21 commits
Select commit
Hold shift + click to select a range
2eee140
feat(cache): add simple local cache support
icyleaf 037cc11
Refactor(feature): remove Halite::Features module and move logger fea…
icyleaf fd1402f
refactor(logger): Renamed feature logger to logging to better accept …
icyleaf 8f06c87
refactor(mime_type): Rename Halite::MimeTypes to Halite::MimeType, an…
icyleaf c2d997d
refactor(mime_type): Rename Halite::MimeTypes to Halite::MimeType, an…
icyleaf f62437e
fix(logging): Remove instance Logging with format
icyleaf 44c8f44
feat(logging): apply skip_request_body, skip_response_body, skip_benc…
icyleaf 478f087
test: fix ci
icyleaf 72880e1
doc: update new changes in README
icyleaf 655ad4d
chore: clean up
icyleaf 8842c48
fix(cache): keep with new feature spec
icyleaf 7f99953
feat(cache): store cache into a directory with metdata file and body …
icyleaf 195d5ac
doc: update cache comment
icyleaf 7e17d02
fix merge conflict
icyleaf 990b3bd
test: add cache feature spec
icyleaf 229486f
test(cache): add more feature cache tests
icyleaf c1a4555
test(cache): remove path writable test
icyleaf f220ba4
feat(cache): load cache from file support
icyleaf 266adb4
Merge branch 'fix/reduce-response-when-use-intercept' into feature/lo…
icyleaf 35563f2
fix(cache): expires compare use utc time
icyleaf fe4b5b7
refactor(cache): cache default path change to /tmp/halite/cache
icyleaf File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -11,3 +11,4 @@ main.cr | |
# Dependencies will be locked in application that uses them | ||
shard.lock | ||
.history/ | ||
cache/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,109 @@ | ||
require "json" | ||
require "digest" | ||
require "file_utils" | ||
|
||
module Halite::Features | ||
# Cache feature use for caching HTTP response to local storage to speed up in developing stage. | ||
# | ||
# It has the following options: | ||
# | ||
# - `path`: The path of cache, default is "cache/" | ||
# - `expires`: The expires time of cache, default is nerver expires. | ||
# - `debug`: The debug mode of cache, default is `true` | ||
# | ||
# With debug mode, cached response it always included some headers information: | ||
# | ||
# - `X-Cached-Key`: Cache key with verb, uri and body | ||
# - `X-Cached-At`: Cache created time | ||
# - `X-Cached-Expires-At`: Cache expired time | ||
# - `X-Cached-By`: Always return "Halite" | ||
# | ||
# ``` | ||
# Halite.use("cache").get "http://httpbin.org/anything" # request a HTTP | ||
# r = Halite.use("cache").get "http://httpbin.org/anything" # request from local storage | ||
# r.headers # => {..., "X-Cached-At" => "2018-08-30 10:41:14 UTC", "X-Cached-By" => "Halite", "X-Cached-Expires-At" => "2018-08-30 10:41:19 UTC", "X-Cached-Key" => "2bb155e6c8c47627da3d91834eb4249a"}} | ||
# ``` | ||
class Cache < Feature | ||
DEFAULT_PATH = "cache/" | ||
|
||
@path : String | ||
@expires : Time::Span? | ||
@debug : Bool | ||
|
||
def initialize(**options) | ||
@path = options.fetch(:path, DEFAULT_PATH).as(String) | ||
@expires = options[:expires]?.as(Time::Span?) | ||
@debug = options.fetch(:debug, true).as(Bool) | ||
end | ||
|
||
def intercept(chain) | ||
response = cache(chain) do | ||
chain.perform | ||
end | ||
|
||
chain.return(response) | ||
end | ||
|
||
private def cache(chain : Interceptor::Chain, &block : -> Response) | ||
if response = find_cache(chain.request) | ||
return response | ||
end | ||
|
||
response = yield | ||
write_cache(chain.request, response) | ||
response | ||
end | ||
|
||
private def find_cache(request : Request) : Response? | ||
key = generate_cache_key(request) | ||
file = File.join(@path, key) | ||
|
||
if File.exists?(file) && !cache_expired?(file) | ||
cache = JSON.parse(File.open(file)).as_h | ||
status_code = cache["status_code"].as_i | ||
body = cache["body"].as_s | ||
headers = cache["headers"].as_h.each_with_object(HTTP::Headers.new) do |(key, value), obj| | ||
obj[key] = value.as_s | ||
end | ||
|
||
if @debug | ||
headers["X-Cached-Key"] = key | ||
headers["X-Cached-At"] = cache_created_time(file).to_s | ||
headers["X-Cached-Expires-At"] = @expires ? (cache_created_time(file) + @expires.not_nil!).to_s : "None" | ||
headers["X-Cached-By"] = "Halite" | ||
end | ||
|
||
return Response.new(request.uri, status_code, body, headers) | ||
end | ||
end | ||
|
||
private def cache_expired?(file) | ||
return false unless expires = @expires | ||
file_modified_time = cache_created_time(file) | ||
Time.now >= (file_modified_time + expires) | ||
end | ||
|
||
private def cache_created_time(file) | ||
File.info(file).modification_time | ||
end | ||
|
||
private def generate_cache_key(request : Request) : String | ||
Digest::MD5.hexdigest("#{request.verb}-#{request.uri}-#{request.body}") | ||
end | ||
|
||
private def write_cache(request, response) | ||
FileUtils.mkdir_p(@path) unless Dir.exists?(@path) | ||
|
||
key = generate_cache_key(request) | ||
File.open(File.join(@path, key), "w") do |f| | ||
f.puts({ | ||
"status_code" => response.status_code, | ||
"headers" => response.headers.to_h, | ||
"body" => response.body | ||
}.to_json) | ||
end | ||
end | ||
|
||
Halite::Features.register "cache", self | ||
end | ||
end |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's not a good idea to create a directory in the root directory, at least create in
/tmp
Maybe a better approach would be:
Cache
object creation, a temporary directory will be created with https://crystal-lang.org/api/master/Tempfile.html (Tempfile.new("halite-cache").path
)finalize
method that will delete the directory when the Object is destructed (#delete
)Note: the API will likely change in the next release crystal-lang/crystal#6485
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This commit just a simple test, it was not following my spec. May be push it early.