Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementation/45801 api with subset representer #12107

Merged
merged 86 commits into from
Mar 1, 2023
Merged
Show file tree
Hide file tree
Changes from 61 commits
Commits
Show all changes
86 commits
Select commit Hold shift + click to select a range
151adf5
baseline api: parse timestamps in ParseQueryParamsService
fiedl Dec 8, 2022
ffca847
baseline api: pass timestamps to query
fiedl Dec 8, 2022
3231f1d
baseline api: handle special-value timestamps like "now" or "2y"
fiedl Dec 8, 2022
6dcfea8
baseline api: provide wrapper `Journable::WithAttributesAtTimestamps`
fiedl Dec 10, 2022
efe70c2
baseline api: rubocop refactoring
fiedl Dec 10, 2022
18339f3
baseline api: rename wrapper to `Journable::WithHistoricAttributes`
fiedl Dec 18, 2022
e9c6bce
Journable::Timestamps: columns missing in the journal-data table are …
fiedl Dec 18, 2022
926f6dc
baseline api: rename accessor to `Journable::WithHistoricAttributes#a…
fiedl Dec 18, 2022
17db827
baseline api: pass `query` and `timestamps` to the `WorkPackageCollec…
fiedl Dec 18, 2022
a115aa7
baseline api: add `Journable::WithHistoricTimestamps` attributes thro…
fiedl Dec 18, 2022
9a1ffea
baseline api: redefine `inspect` on wrapper objects to help debugging
fiedl Dec 18, 2022
6f28f1e
baseline api: fixing accessor name in specs
fiedl Dec 18, 2022
edb3cd0
baseline api: add `attributesByTimestamp` and `baselineAttributes` to…
fiedl Dec 18, 2022
2acabed
baseline api: have the representer include the baseine-attributes pro…
fiedl Dec 19, 2022
e1ae8ea
baseline api: the work-package-collection self link includes the abso…
fiedl Dec 19, 2022
00958b8
baseline api: add `timestamps` only to `query` hash if present
fiedl Dec 19, 2022
72fe579
baseline api: add self link to embedded baseline attributes
fiedl Dec 19, 2022
9fdc2ad
baseline api: don't include timestamps in links when only `Timestamp.…
fiedl Dec 19, 2022
c4bb709
baseline api: don't mock the eager-loading wrapper
fiedl Dec 19, 2022
4e8a84f
baseline api: include as meta information at which timestamps the wor…
fiedl Dec 19, 2022
8dce4df
baseline api: pass timestamps to representer on show endpoint
fiedl Dec 20, 2022
d172a4f
baseline api: include `matchesFilters` meta info only if a query is g…
fiedl Dec 20, 2022
311d658
baseline api: make `Journable::WithHistoricAttributes.wrap` detect si…
fiedl Dec 20, 2022
969f50f
baseline api: rename `matches_query_at_timestamps` to `matches_query_…
fiedl Dec 20, 2022
f14f64e
baseline api: pass timestamps separately to representer rather than w…
fiedl Dec 20, 2022
8bbbeba
baseline api: rubocop refactoring
fiedl Dec 20, 2022
d426ee1
baseline api: use `Journable::WithHistoricAttributes include_only_cha…
fiedl Dec 20, 2022
32d92a0
baseline api: fix naming of `matches_query_filters_at_timestamp`
fiedl Dec 20, 2022
79716e5
baseline api: include `_meta/matchesFilters` in the base work package
fiedl Dec 20, 2022
2b86510
baseline api: rubocop refactoring
fiedl Dec 20, 2022
d5bea80
baseline api: adding request specs for `/work_packages?timestamps=...`
fiedl Dec 21, 2022
26c7b91
baseline api: rubocop refactoring
fiedl Dec 21, 2022
f7b5734
baseline api: track existence as meta attribute
fiedl Jan 11, 2023
2611fc9
baseline api: make `attributesByTimestamp` an array rather than a hash
fiedl Jan 12, 2023
a6ff6bf
Merge 'dev' into https://github.com/opf/openproject/pull/11783
fiedl Feb 20, 2023
d9bd507
Merge 'dev' into https://github.com/opf/openproject/pull/11783
fiedl Feb 20, 2023
1219ac3
baseline api: adding spec with a work package that has not changed at…
fiedl Feb 20, 2023
f4585ba
baseline api: adding api documentation
fiedl Feb 20, 2023
7699eb7
Merge 'dev' into https://github.com/opf/openproject/pull/11783
fiedl Feb 20, 2023
acb0526
baseline api: add timestamps to cache key
fiedl Feb 21, 2023
671de40
baseline api: show `attributesByTimestamp` and `_meta` when using `ti…
fiedl Feb 21, 2023
84bfe21
baseline api: fixing `exists` by avoiding subquery for now
fiedl Feb 21, 2023
22c95c4
baseline api: removing redundant `baselineAttributes`
fiedl Feb 21, 2023
18dbf27
baseline api: adding spec for backwards compatibility
fiedl Feb 21, 2023
15be749
baseline api: fixing open-api spec:
fiedl Feb 22, 2023
1a0d02d
baseline api: show "_meta" "matchesFilters" only when a query is passed
fiedl Feb 22, 2023
547711b
baseline api: rubocop refactoring
fiedl Feb 22, 2023
5ebacf4
introduce a WP representer only rendering a subset of properties
ulferts Feb 7, 2023
b61ef7f
fix typo
ulferts Feb 7, 2023
707fdcc
adapt eager loading for wp collection to attributes_at_timestamp changes
ulferts Feb 10, 2023
bde8741
fix wrapping multiple work packages WithHistoricAttributes
ulferts Feb 13, 2023
8339975
simplify timestamp handling in url query
ulferts Feb 16, 2023
e399e81
move timestamp info on wrapped work package to decorator
ulferts Feb 16, 2023
3d033a5
introduce changed_at_timestamp on WithHistoricAttributes
ulferts Feb 17, 2023
f2251ba
introduce loader object for historic_attributes
ulferts Feb 20, 2023
c5ccdc6
enable Timestamp to be used as a hash key
ulferts Feb 21, 2023
8845771
avoid loading custom field data for attributesByTimestamp WP
ulferts Feb 21, 2023
40b9b0f
fix wp index without timestamps
ulferts Feb 23, 2023
9e534b1
fix loading only at single historical timestamp
ulferts Feb 23, 2023
d4b81c7
avoid AR dirty checking showing changes on historic attributes eager …
ulferts Feb 24, 2023
ca406b7
extract method for readability
ulferts Feb 24, 2023
debc37d
Timestamp: extracting redundant code into method
fiedl Feb 27, 2023
0861cb9
baseline api: minor corrections after review
fiedl Feb 27, 2023
a24a06f
baseline api: refactoring nomenclature in collection representer
fiedl Feb 27, 2023
043e13a
baseline api: move eager-loading-wrapper options assignment into base…
fiedl Feb 28, 2023
95861d5
Merge 'dev' into https://github.com/opf/openproject/pull/11783
fiedl Feb 28, 2023
f4831b1
baseline api: rubocop refactoring
fiedl Feb 28, 2023
9afa928
baseline api: fixing scenario in which only a historic timestamp is r…
fiedl Feb 28, 2023
12a73fd
baseline api: adding missing specs for wrapping multiple work packages
fiedl Feb 28, 2023
a18925e
baseline api: adding specs for caching
fiedl Feb 28, 2023
222873c
baseline api: remove redundant `timestamp` attribute
fiedl Feb 28, 2023
f1df4a3
baseline api: fixing test issue with relative timestamps
fiedl Feb 28, 2023
b35df5b
baseline api: removing `timestamps` from `json_cache_key` for now.
fiedl Mar 1, 2023
5312e46
empty commit to re-trigger CI
fiedl Mar 1, 2023
5a33ab6
baseline api: improving documentation
fiedl Mar 1, 2023
45dc3a7
Merge 'dev' into https://github.com/opf/openproject/pull/11783
fiedl Mar 1, 2023
4ea8fe4
baseline api: adding missing specs for `Journable::WithHistoricAttrib…
fiedl Mar 1, 2023
fda05d7
Merge remote-tracking branch 'origin/pr/11783' into implementation/45…
ulferts Mar 1, 2023
2830240
linting
ulferts Mar 1, 2023
c028d13
salvage specs for fetching for a timestamp where the wp did not exist…
ulferts Mar 1, 2023
abed488
fix typo
ulferts Mar 1, 2023
5431eb0
fix spec method reference
ulferts Mar 1, 2023
fb5faab
extract method
ulferts Mar 1, 2023
74fbac0
rename method
ulferts Mar 1, 2023
8495bff
document #changed_at_timestamp
ulferts Mar 1, 2023
23c9ac5
return nil on matches_query_fitlers_at_*_timestamp? without query
ulferts Mar 1, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
212 changes: 212 additions & 0 deletions app/models/journable/with_historic_attributes.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,212 @@
#-- copyright
# OpenProject is an open source project management software.
# Copyright (C) 2012-2022 the OpenProject GmbH
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License version 3.
#
# OpenProject is a fork of ChiliProject, which is a fork of Redmine. The copyright follows:
# Copyright (C) 2006-2013 Jean-Philippe Lang
# Copyright (C) 2010-2013 the ChiliProject Team
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
#
# See COPYRIGHT and LICENSE files for more details.
#++

# This class is used to wrap a Journable and provide access to its attributes at given timestamps.
# It is used to provide the old and new values of a journable in the journables's payload.
# https://github.com/opf/openproject/pull/11783
#
# Usage:
#
# # Wrap single work package
# timestamps = [Timestamp.parse("2022-01-01T00:00:00Z"), Timestamp.parse("PT0S")]
# work_package = WorkPackage.find(1)
# work_package = Journable::WithHistoricAttributes.wrap(work_package, timestamps:)
#
# # Wrap multiple work packages
# timestamps = query.timestamps
# work_packages = query.results.work_packages
# work_packages = Journable::WithHistoricAttributes.wrap_multiple(work_packages, timestamps:)
#
# # Access historic attributes at timestamps after wrapping
# work_package = Journable::WithHistoricAttributes.wrap(work_package, timestamps:)
# work_package.subject # => "Subject at PT0S (current time)"
# work_package.attributes_by_timestamp["2022-01-01T00:00:00Z"].subject # => "Subject at 2022-01-01 (baseline time)"
#
# # Check at which timestamps the work package matches query filters after wrapping
# query.timestamps # => [<Timestamp 2022-01-01T00:00:00Z>, <Timestamp PT0S>]
# work_package = Journable::WithHistoricAttributes.wrap(work_package, query:)
# work_package.matches_query_filters_at_timestamps # => [<Timestamp 2022-01-01T00:00:00Z>]
#
# # Include only changed attributes in payload
# # i.e. only historic attributes that differ from the work_package's attributes
# timestamps = [Timestamp.parse("2022-01-01T00:00:00Z"), Timestamp.parse("PT0S")]
# work_package = Journable::WithHistoricAttributes.wrap(work_package, timestamps:, include_only_changed_attributes: true)
# work_package.attributes_by_timestamp["2022-01-01T00:00:00Z"].subject # => "Subject at 2022-01-01 (baseline time)"
# work_package.attributes_by_timestamp["PT0S"].subject # => nil
#
# # Simplified interface for two timestamps
# query.timestamps # => [<Timestamp 2022-01-01T00:00:00Z>, <Timestamp PT0S>]
# work_package = Journable::WithHistoricAttributes.wrap(work_package, query:)
# work_package.baseline_timestamp # => [<Timestamp 2022-01-01T00:00:00Z>]
# work_package.current_timestamp # => [<Timestamp PT0S>]
# work_package.matches_query_filters_at_baseline_timestamp?
# work_package.matches_query_filters_at_current_timestamp?
# work_package.baseline_attributes.subject # => "Subject at 2022-01-01 (baseline time)"
# work_package.subject # => "Subject at PT0S (current time)"
#
class Journable::WithHistoricAttributes < SimpleDelegator
attr_accessor :timestamps,
:query,
:include_only_changed_attributes,
:loader

def initialize(journable,
timestamps: nil,
query: nil,
include_only_changed_attributes: false,
loader: Loader.new(journable))
super(journable)

if query and not journable.is_a? WorkPackage
raise Journable::NotImplementedError, "Journable::WithHistoricAttributes with query " \
"is only implemented for WorkPackages at the moment " \
"because Query objects currently only support work packages."
end

self.query = query
self.timestamps = timestamps || query.try(:timestamps) || []
self.include_only_changed_attributes = include_only_changed_attributes

self.loader = loader
end
private_class_method :new

class << self
def wrap(journable_or_journables,
timestamps: query.try(:timestamps) || [],
query: nil,
include_only_changed_attributes: false)
wrapped = wrap_each_journable(Array(journable_or_journables), timestamps:, query:, include_only_changed_attributes:)

case journable_or_journables
when Array, ActiveRecord::Relation
wrapped
else
wrapped.first
end
end

private

def wrap_each_journable(journables, timestamps:, query:, include_only_changed_attributes:)
loader = Loader.new(journables)
journables = loader.at_timestamp(timestamps.last).values if timestamps.last.try(:historic?)

journables.map { |j| new(j, timestamps:, query:, include_only_changed_attributes:, loader:) }
end
end

def attributes_by_timestamp
@attributes_by_timestamp ||= Hash.new do |h, t|
attributes = if include_only_changed_attributes
changes_at_timestamp(t)&.transform_values(&:last)
ulferts marked this conversation as resolved.
Show resolved Hide resolved
else
historic_attributes_at(t)
ulferts marked this conversation as resolved.
Show resolved Hide resolved
end

h[t] = attributes ? Hashie::Mash.new(attributes) : nil
end
end

def changed_at_timestamp(timestamp)
changes_at_timestamp(timestamp)&.keys || []
end
ulferts marked this conversation as resolved.
Show resolved Hide resolved

def matches_query_filters_at_timestamps
if query.present?
timestamps.select { |timestamp| loader.work_package_ids_of_query_at_timestamp(query:, timestamp:).include?(__getobj__.id) }
else
[]
end
end

def exists_at_timestamps
timestamps.select { |t| at_timestamp(t).present? }
end

def baseline_timestamp
timestamps.first
end

def baseline_attributes
attributes_by_timestamp[baseline_timestamp.to_s]
end

def matches_query_filters_at_baseline_timestamp?
matches_query_filters_at_timestamps.include?(baseline_timestamp)
end

def current_timestamp
timestamps.last
end

def matches_query_filters_at_current_timestamp?
matches_query_filters_at_timestamps.include?(current_timestamp)
end
ulferts marked this conversation as resolved.
Show resolved Hide resolved

def matches_query_filters_at_timestamp?(timestamp)
matches_query_filters_at_timestamps.include?(timestamp)
end

def at_timestamp(timestamp)
loader.journable_at_timestamp(__getobj__, timestamp)
end

def to_ary
__getobj__.send(:to_ary)
end

def inspect
__getobj__.inspect.gsub(/#<(.+)>/m, "#<#{self.class.name} \\1>")
end

private

def historic_attributes_at(timestamp)
historic_journable = at_timestamp(Timestamp.parse(timestamp))

return unless historic_journable

historic_journable
.attributes
.select do |key, _|
respond_to?(key)
end
end

def changes_at_timestamp(timestamp)
historic_journable = at_timestamp(Timestamp.parse(timestamp))

return unless historic_journable

::Acts::Journalized::JournableDiffer
.changes(__getobj__, historic_journable)
end

class NotImplemented < StandardError; end
end
69 changes: 69 additions & 0 deletions app/models/journable/with_historic_attributes/loader.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# -- copyright
# OpenProject is an open source project management software.
# Copyright (C) 2010-2023 the OpenProject GmbH
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License version 3.
#
# OpenProject is a fork of ChiliProject, which is a fork of Redmine. The copyright follows:
# Copyright (C) 2006-2013 Jean-Philippe Lang
# Copyright (C) 2010-2013 the ChiliProject Team
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
#
# See COPYRIGHT and LICENSE files for more details.
# ++

class Journable::WithHistoricAttributes
class Loader
def initialize(journables)
@journables = Array(journables)
end

def journable_at_timestamp(journable, timestamp)
at_timestamp(timestamp)[journable.id]
end

def at_timestamp(timestamp)
@at_timestamp ||= Hash.new do |h, t|
h[t] = journables.first.class.at_timestamp(t).where(id: journables.map(&:id)).index_by(&:id)
end

@at_timestamp[timestamp]
end
ulferts marked this conversation as resolved.
Show resolved Hide resolved

def work_package_ids_of_query_at_timestamp(query:, timestamp: nil)
@work_package_ids_of_query_at_timestamp ||= Hash.new do |qh, q|
qh[q] = Hash.new do |ht, t|
ht[t] = work_package_ids_of_query_at_timestamp_calculation(q, t)
end
end

@work_package_ids_of_query_at_timestamp[query][timestamp]
end

private

def work_package_ids_of_query_at_timestamp_calculation(query, timestamp)
query = query.dup
query.timestamps = [timestamp] if timestamp

query.results.work_packages.where(id: journables.map(&:id)).pluck(:id)
end

attr_accessor :journables
end
private_constant :Loader
end
79 changes: 74 additions & 5 deletions app/models/timestamp.rb
Original file line number Diff line number Diff line change
Expand Up @@ -40,14 +40,29 @@ def initialize(arg = Timestamp.now.to_s)
end

def self.parse(iso8601_string)
return iso8601_string if iso8601_string.is_a?(Timestamp)

iso8601_string.strip!
iso8601_string = substitute_special_shortcut_values(iso8601_string)
if iso8601_string.start_with? "P" # ISO8601 "Period"
ActiveSupport::Duration.parse(iso8601_string)
elsif Time.zone.parse(iso8601_string).blank?
raise ArgumentError, "The string \"#{iso8601_string}\" cannot be parsed to a Time."
iso8601_string = ActiveSupport::Duration.parse(iso8601_string).iso8601
elsif (time = Time.zone.parse(iso8601_string)).present?
iso8601_string = time.iso8601
else
raise ArgumentError, "The string \"#{iso8601_string}\" cannot be parsed to Time or ActiveSupport::Duration."
end
Timestamp.new(iso8601_string)
end

# Take a comma-separated string of ISO-8601 timestamps and convert it
# into an array of Timestamp objects.
#
def self.parse_multiple(comma_separated_iso8601_string)
comma_separated_iso8601_string.to_s.split(",").compact_blank.collect do |iso8601_string|
Timestamp.parse(iso8601_string)
end
end

def self.now
new(ActiveSupport::Duration.build(0).iso8601)
end
Expand All @@ -68,10 +83,18 @@ def iso8601
@timestamp_iso8601_string.to_s
end

def to_iso8601
iso8601
end

def inspect
"#<Timestamp \"#{iso8601}\">"
end

def absolute
Timestamp.new(to_time)
end

def to_time
if relative?
Time.zone.now - (to_duration * (to_duration.to_i.positive? ? 1 : -1))
Expand All @@ -88,7 +111,7 @@ def to_duration
end
end

def as_json
def as_json(*_args)
to_s
end

Expand All @@ -97,8 +120,54 @@ def to_json(*_args)
end

def ==(other)
iso8601 == other.iso8601
case other
when String
iso8601 == other or to_s == other
when Timestamp
iso8601 == other.iso8601
when NilClass
to_s.blank?
else
raise Timestamp::Exception, "Comparison to #{other.class.name} not implemented, yet."
end
end

def eql?(other)
self == other
end

def historic?
self != Timestamp.now
end

delegate :hash, to: :iso8601

class Exception < StandardError; end

# rubocop:disable Metrics/AbcSize
# rubocop:disable Metrics/PerceivedComplexity
def self.substitute_special_shortcut_values(string)
# map now to PT0S
string = "PT0S" if string == "now"

# map 1y to P1Y, 1m to P1M, 1w to P1W, 1d to P1D
# map -1y to P-1Y, -1m to P-1M, -1w to P-1W, -1d to P-1D
# map -1y1d to P-1Y-1D
sign = "-" if string.start_with? "-"
years = string.scan(/(\d+)y/).flatten.first
months = string.scan(/(\d+)m/).flatten.first
weeks = string.scan(/(\d+)w/).flatten.first
days = string.scan(/(\d+)d/).flatten.first
if years || months || weeks || days
string = "P" \
"#{sign if years}#{years}#{'Y' if years}" \
"#{sign if months}#{months}#{'M' if months}" \
"#{sign if weeks}#{weeks}#{'W' if weeks}" \
"#{sign if days}#{days}#{'D' if days}"
end

string
end
# rubocop:enable Metrics/AbcSize
# rubocop:enable Metrics/PerceivedComplexity
end