Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New feature: local cache #35

Merged
merged 21 commits into from Sep 3, 2018
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
2eee140
feat(cache): add simple local cache support
icyleaf Aug 30, 2018
037cc11
Refactor(feature): remove Halite::Features module and move logger fea…
icyleaf Aug 31, 2018
fd1402f
refactor(logger): Renamed feature logger to logging to better accept …
icyleaf Aug 31, 2018
8f06c87
refactor(mime_type): Rename Halite::MimeTypes to Halite::MimeType, an…
icyleaf Aug 31, 2018
c2d997d
refactor(mime_type): Rename Halite::MimeTypes to Halite::MimeType, an…
icyleaf Aug 31, 2018
f62437e
fix(logging): Remove instance Logging with format
icyleaf Aug 31, 2018
44c8f44
feat(logging): apply skip_request_body, skip_response_body, skip_benc…
icyleaf Aug 31, 2018
478f087
test: fix ci
icyleaf Aug 31, 2018
72880e1
doc: update new changes in README
icyleaf Aug 31, 2018
655ad4d
chore: clean up
icyleaf Aug 31, 2018
8842c48
fix(cache): keep with new feature spec
icyleaf Aug 31, 2018
7f99953
feat(cache): store cache into a directory with metdata file and body …
icyleaf Aug 31, 2018
195d5ac
doc: update cache comment
icyleaf Aug 31, 2018
7e17d02
fix merge conflict
icyleaf Aug 31, 2018
990b3bd
test: add cache feature spec
icyleaf Aug 31, 2018
229486f
test(cache): add more feature cache tests
icyleaf Sep 3, 2018
c1a4555
test(cache): remove path writable test
icyleaf Sep 3, 2018
f220ba4
feat(cache): load cache from file support
icyleaf Sep 3, 2018
266adb4
Merge branch 'fix/reduce-response-when-use-intercept' into feature/lo…
icyleaf Sep 3, 2018
35563f2
fix(cache): expires compare use utc time
icyleaf Sep 3, 2018
fe4b5b7
refactor(cache): cache default path change to /tmp/halite/cache
icyleaf Sep 3, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Expand Up @@ -11,3 +11,4 @@ main.cr
# Dependencies will be locked in application that uses them
shard.lock
.history/
cache/
109 changes: 109 additions & 0 deletions src/halite/features/cache.cr
@@ -0,0 +1,109 @@
require "json"
require "digest"
require "file_utils"

module Halite::Features
# Cache feature use for caching HTTP response to local storage to speed up in developing stage.
#
# It has the following options:
#
# - `path`: The path of cache, default is "cache/"
# - `expires`: The expires time of cache, default is nerver expires.
# - `debug`: The debug mode of cache, default is `true`
#
# With debug mode, cached response it always included some headers information:
#
# - `X-Cached-Key`: Cache key with verb, uri and body
# - `X-Cached-At`: Cache created time
# - `X-Cached-Expires-At`: Cache expired time
# - `X-Cached-By`: Always return "Halite"
#
# ```
# Halite.use("cache").get "http://httpbin.org/anything" # request a HTTP
# r = Halite.use("cache").get "http://httpbin.org/anything" # request from local storage
# r.headers # => {..., "X-Cached-At" => "2018-08-30 10:41:14 UTC", "X-Cached-By" => "Halite", "X-Cached-Expires-At" => "2018-08-30 10:41:19 UTC", "X-Cached-Key" => "2bb155e6c8c47627da3d91834eb4249a"}}
# ```
class Cache < Feature
DEFAULT_PATH = "cache/"
Copy link

@j8r j8r Aug 30, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not a good idea to create a directory in the root directory, at least create in /tmp
Maybe a better approach would be:

  1. On a new Cache object creation, a temporary directory will be created with https://crystal-lang.org/api/master/Tempfile.html (Tempfile.new("halite-cache").path)
  2. Add a finalize method that will delete the directory when the Object is destructed (#delete)

Note: the API will likely change in the next release crystal-lang/crystal#6485

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This commit just a simple test, it was not following my spec. May be push it early.


@path : String
@expires : Time::Span?
@debug : Bool

def initialize(**options)
@path = options.fetch(:path, DEFAULT_PATH).as(String)
@expires = options[:expires]?.as(Time::Span?)
@debug = options.fetch(:debug, true).as(Bool)
end

def intercept(chain)
response = cache(chain) do
chain.perform
end

chain.return(response)
end

private def cache(chain : Interceptor::Chain, &block : -> Response)
if response = find_cache(chain.request)
return response
end

response = yield
write_cache(chain.request, response)
response
end

private def find_cache(request : Request) : Response?
key = generate_cache_key(request)
file = File.join(@path, key)

if File.exists?(file) && !cache_expired?(file)
cache = JSON.parse(File.open(file)).as_h
status_code = cache["status_code"].as_i
body = cache["body"].as_s
headers = cache["headers"].as_h.each_with_object(HTTP::Headers.new) do |(key, value), obj|
obj[key] = value.as_s
end

if @debug
headers["X-Cached-Key"] = key
headers["X-Cached-At"] = cache_created_time(file).to_s
headers["X-Cached-Expires-At"] = @expires ? (cache_created_time(file) + @expires.not_nil!).to_s : "None"
headers["X-Cached-By"] = "Halite"
end

return Response.new(request.uri, status_code, body, headers)
end
end

private def cache_expired?(file)
return false unless expires = @expires
file_modified_time = cache_created_time(file)
Time.now >= (file_modified_time + expires)
end

private def cache_created_time(file)
File.info(file).modification_time
end

private def generate_cache_key(request : Request) : String
Digest::MD5.hexdigest("#{request.verb}-#{request.uri}-#{request.body}")
end

private def write_cache(request, response)
FileUtils.mkdir_p(@path) unless Dir.exists?(@path)

key = generate_cache_key(request)
File.open(File.join(@path, key), "w") do |f|
f.puts({
"status_code" => response.status_code,
"headers" => response.headers.to_h,
"body" => response.body
}.to_json)
end
end

Halite::Features.register "cache", self
end
end