Skip to content

feat(influx_publisher): allow overriding the timestamp #53

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
stakach merged 3 commits into from
Feb 15, 2023
Merged

feat(influx_publisher): allow overriding the timestamp #53

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ec73102
feat(influx_publisher): allow overriding the timestamp
stakach Feb 15, 2023
d764564
fix unix time
stakach Feb 15, 2023
320d05a
update specs
stakach Feb 15, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion shard.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ shards:

action-controller:
git: https://github.com/spider-gazelle/action-controller.git
version: 5.6.0
version: 5.6.2

active-model:
git: https://github.com/spider-gazelle/active-model.git
Expand Down
4 changes: 2 additions & 2 deletions spec/publishing/influx_publisher_spec.cr
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,7 @@ module PlaceOS::Source
ts_tags: {
pos_building: "pack",
},
ts_timestamp: "last_seen",
}.to_json)

points = InfluxPublisher.transform(message)
Expand All @@ -167,7 +168,7 @@ module PlaceOS::Source

point.measurement.should eq "M'Odule"

point.timestamp.should eq Time::UNIX_EPOCH
point.timestamp.not_nil!.to_unix.should eq 1601555879_i64

point.tags.should eq({
"pos_org" => "org-donor",
Expand All @@ -193,7 +194,6 @@ module PlaceOS::Source
"lat" => 25.20090608906493,
"mac" => "66e0fd1279ce",
"variance" => 4.5194575835650745,
"last_seen" => 1601555879,
"building" => "zone-EmWLJNm0i~6",
"level" => "zone-Epaq-dE1DaH",
"map_width" => 1234.2,
Expand Down
12 changes: 11 additions & 1 deletion src/source/publishing/influx_publisher.cr
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,9 @@ module PlaceOS::Source
property value : Array(Hash(String, FieldTypes?))
property ts_tag_keys : Array(String)?
property ts_map : Hash(String, String)?

# allow for a custom timestamp field
property ts_timestamp : String?
end

alias Value = FieldTypes | Hash(String, FieldTypes?) | Hash(String, Hash(String, FieldTypes?)) | Array(Hash(String, FieldTypes?)) | CustomMetrics
Expand Down Expand Up @@ -173,12 +176,19 @@ module PlaceOS::Source
next if compacted.empty?
measurement = default_measurement || data.module_name

override_timestamp = nil
if time_key = raw.ts_timestamp
if time = compacted.delete(time_key).as?(Float64)
override_timestamp = Time.unix time.to_i64
end
end

# Must include a `pos_uniq` tag for seperating points
# as per: https://docs.influxdata.com/influxdb/v2.0/write-data/best-practices/duplicate-points/#add-an-arbitrary-tag
local_tags = tags.dup
local_tags["pos_uniq"] = index.to_s

points << build_custom_point(measurement, data, fields, local_tags, compacted, timestamp, ts_map, raw.ts_tag_keys)
points << build_custom_point(measurement, data, fields, local_tags, compacted, override_timestamp || timestamp, ts_map, raw.ts_tag_keys)
end

points
Expand Down