Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SWATCH-654: bugfix: only a single set of measurements kept per day #1560

Merged
merged 1 commit into from
Oct 25, 2022

Conversation

kahowell
Copy link
Contributor

This corrects a bug where we were clearing hourly measurements on processing each successive hour in a given day. This was happening because we adjust the range of fetched existing snapshots to include enough to aggregate the next granularity (hourly -> daily), and we were treating all fetched snapshots as affected by the gathered usage (rather than just the given hour).

Testing

Run through these steps once on hotfix or main, and then on this branch.

Run mock prometheus:

bin/prometheus-mock-data.sh

Clear state:

psql -h localhost -U rhsm-subscriptions <<EOF
delete from tally_snapshots cascade;
delete from hosts cascade;
EOF

Run the application:

DEV_MODE=true \
  USER_USE_STUB=true \
  PROM_URL=http://localhost:9090/api/v1 \
  ./gradlew :bootRun

Gather metrics (adjust date/time as needed; the mock prometheus generates data for the past 24 hours):

http :9000/hawtio/jolokia \
  type=exec \
  mbean=org.candlepin.subscriptions.metering.jmx:name=meteringJmxBean,type=MeteringJmxBean \
  operation='performCustomMetering(java.lang.String,java.lang.String,java.lang.Integer)' \
  arguments:='["rhosak","2022-10-25T12:00:00Z",240]'

Run tally against several successive hours:

http :9000/hawtio/jolokia \
  type=exec \
  mbean=org.candlepin.subscriptions.jmx:name=tallyJmxBean,type=TallyJmxBean \
  operation='tallyAllAccountsByHourly(java.lang.String,java.lang.String)' \
  arguments:='["2022-10-25T09:00:00Z","2022-10-25T10:00Z"]'
sleep 2
http :9000/hawtio/jolokia \
  type=exec \
  mbean=org.candlepin.subscriptions.jmx:name=tallyJmxBean,type=TallyJmxBean \
  operation='tallyAllAccountsByHourly(java.lang.String,java.lang.String)' \
  arguments:='["2022-10-25T10:00:00Z","2022-10-25T11:00Z"]'
sleep 2
http :9000/hawtio/jolokia \
  type=exec \
  mbean=org.candlepin.subscriptions.jmx:name=tallyJmxBean,type=TallyJmxBean \
  operation='tallyAllAccountsByHourly(java.lang.String,java.lang.String)' \
  arguments:='["2022-10-25T11:00:00Z","2022-10-25T12:00Z"]'

Check the tally snapshots measurements:

  • on main/hotfix notice that the measurements were cleared
  • on this branch, notice they are intact
psql -h localhost -U rhsm-subscriptions <<EOF
select id, value, snapshot_date
from tally_snapshots s
         left join tally_measurements m on m.snapshot_id = s.id
where sla = '_ANY'
  and usage ='_ANY'
  and billing_account_id='_ANY'
  and granularity='HOURLY'
  and billing_provider='_ANY'
  and (measurement_type='PHYSICAL'
   or measurement_type is null)
order by snapshot_date asc
EOF

This corrects a bug where we were clearing hourly measurements on processing each successive hour in
a given day. This was happening because we adjust the range of fetched existing snapshots to include
enough to aggregate the next granularity (hourly -> daily), and we were treating all fetched
snapshots as affected by the gathered usage (rather than just the given hour).

Testing
=======

Run through these steps once on `hotfix` or `main`, and then on this branch.

Run mock prometheus:

```shell
bin/prometheus-mock-data.sh
```

Clear state:

```shell
psql -h localhost -U rhsm-subscriptions <<EOF
delete from tally_snapshots cascade;
delete from hosts cascade;
EOF
```

Run the application:

```shell
DEV_MODE=true \
  USER_USE_STUB=true \
  PROM_URL=http://localhost:9090/api/v1 \
  ./gradlew :bootRun
```

Gather metrics (adjust date/time as needed; the mock prometheus generates data for the past 24 hours):

```shell
http :9000/hawtio/jolokia \
  type=exec \
  mbean=org.candlepin.subscriptions.metering.jmx:name=meteringJmxBean,type=MeteringJmxBean \
  operation='performCustomMetering(java.lang.String,java.lang.String,java.lang.Integer)' \
  arguments:='["rhosak","2022-10-25T12:00:00Z",240]'
```

Run tally against several successive hours:

```shell
http :9000/hawtio/jolokia \
  type=exec \
  mbean=org.candlepin.subscriptions.jmx:name=tallyJmxBean,type=TallyJmxBean \
  operation='tallyAllAccountsByHourly(java.lang.String,java.lang.String)' \
  arguments:='["2022-10-25T09:00:00Z","2022-10-25T10:00Z"]'
sleep 2
http :9000/hawtio/jolokia \
  type=exec \
  mbean=org.candlepin.subscriptions.jmx:name=tallyJmxBean,type=TallyJmxBean \
  operation='tallyAllAccountsByHourly(java.lang.String,java.lang.String)' \
  arguments:='["2022-10-25T10:00:00Z","2022-10-25T11:00Z"]'
sleep 2
http :9000/hawtio/jolokia \
  type=exec \
  mbean=org.candlepin.subscriptions.jmx:name=tallyJmxBean,type=TallyJmxBean \
  operation='tallyAllAccountsByHourly(java.lang.String,java.lang.String)' \
  arguments:='["2022-10-25T11:00:00Z","2022-10-25T12:00Z"]'
```

Check the tally snapshots measurements:
* on `main`/`hotfix` notice that the measurements were cleared
* on this branch, notice they are intact

```shell
psql -h localhost -U rhsm-subscriptions <<EOF
select id, value, snapshot_date
from tally_snapshots s
         left join tally_measurements m on m.snapshot_id = s.id
where sla = '_ANY'
  and usage ='_ANY'
  and billing_account_id='_ANY'
  and granularity='HOURLY'
  and billing_provider='_ANY'
  and (measurement_type='PHYSICAL'
   or measurement_type is null)
order by snapshot_date asc
EOF
```
@barnabycourt barnabycourt changed the title bugfix: only a single set of measurements kept per day SWATCH-654: bugfix: only a single set of measurements kept per day Oct 25, 2022
@kflahert kflahert self-assigned this Oct 25, 2022
@kahowell kahowell merged commit a5175cd into main Oct 25, 2022
@kahowell kahowell deleted the khowell/single_measurement_per_day_bug branch October 25, 2022 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants