Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GPRINT text_format does not replace Data Query and Host Fields #2656

Closed
eschoeller opened this issue Apr 29, 2019 · 29 comments
Closed

GPRINT text_format does not replace Data Query and Host Fields #2656

eschoeller opened this issue Apr 29, 2019 · 29 comments
Labels
aggregate Aggregate related issue bug Undesired behaviour resolved A fixed issue
Milestone

Comments

@eschoeller
Copy link

eschoeller commented Apr 29, 2019

I have several graphs which have malfunctioning |query_*| issues in the legend. See this example:
index
I haven't done too much debugging, but I'm seeing this in various different places throughout my installation after upgrading to 1.2.3

For one data source I initiated a verbose requery. This is what I got:

Total: 0.360000, Delta: 0.010000, Detected New Sort Field: 'tempHumidSensorID' Old Sort Field 'index'
Total: 0.360000, Delta: 0.000000, New Sort Field validated. Sort Field be updated.
Total: 0.360000, Delta: 0.000000, WARNING: Sort Field Association has Changed. Re-mapping issues may occur!
Total: 0.360000, Delta: 0.000000, Update Data Query Sort Cache complete
Total: 1.040000, Delta: 0.680000, Index Association with Local Data complete
Total: 1.060000, Delta: 0.020000, No Index Changes Detected, Skipping Re-Index and Poller Cache Re-population
Total: 1.060000, Delta: 0.000000, Automation Execution for Data Query complete
Total: 1.060000, Delta: 0.000000, Plugin Hooks complete

I immediately had to go back into the data sources and change the sort field from 'tempHumidSensorID' back to 'index' (and manually restore the indexes), since using the SensorID would break everything. I was seeing this in other places a few months ago as well. Not sure if the two issues are related, but the requery did not fix the broken graph legend.

@eschoeller
Copy link
Author

eschoeller commented Apr 29, 2019

Here's the rrdtool command:

/usr/local/rrdtool/bin/rrdtool graph - \
--imgformat=PNG \
--start='1556424900' \
--end='1556511300' \
--pango-markup  \
--title='Rack Inlet Temperature' \
--vertical-label='Fahrenheit / %RH ' \
--slope-mode \
--base=1000 \
--height=130 \
--width=550 \
--tabwidth '30' \
--alt-autoscale \
COMMENT:"From 2019/04/27 22\:15\:00 To 2019/04/28 22\:15\:00\c" \
COMMENT:"  \n" \
--color BACK#F3F3F3 \
--color CANVAS#FDFDFD \
--color SHADEA#CBCBCB \
--color SHADEB#999999 \
--color FONT#000000 \
--color AXIS#2C4D43 \
--color ARROW#2C4D43 \
--color FRAME#2C4D43 \
--border 1 --font TITLE:10: \
--font AXIS:8: \
--font LEGEND:8: \
--font UNIT:8: \
--font WATERMARK:8: \
--slope-mode \
DEF:a='/cacti/cacti-1.2.3-prod/rra/f19_temperature_15772.rrd':'temperature':MAX \
DEF:b='/cacti/cacti-1.2.3-prod/rra/f24_temperature_16608.rrd':'temperature':MAX \
DEF:c='/cacti/cacti-1.2.3-prod/rra/g19_temperature_15795.rrd':'temperature':MAX \
DEF:d='/cacti/cacti-1.2.3-prod/rra/g24_temperature_17257.rrd':'temperature':MAX \
DEF:e='/cacti/cacti-1.2.3-prod/rra/h24_temperature_14270.rrd':'temperature':MAX \
DEF:f='/cacti/cacti-1.2.3-prod/rra/i19_temperature_30987.rrd':'temperature':MAX \
DEF:g='/cacti/cacti-1.2.3-prod/rra/i24_temperature_14307.rrd':'temperature':MAX \
DEF:h='/cacti/cacti-1.2.3-prod/rra/j19_temperature_18115.rrd':'temperature':MAX \
DEF:i='/cacti/cacti-1.2.3-prod/rra/j24_temperature_14503.rrd':'temperature':MAX \
DEF:j='/cacti/cacti-1.2.3-prod/rra/k24_temperature_14514.rrd':'temperature':MAX \
DEF:ba='/cacti/cacti-1.2.3-prod/rra/l19_temperature_31647.rrd':'temperature':MAX \
DEF:bb='/cacti/cacti-1.2.3-prod/rra/l24_temperature_59337.rrd':'temperature':MAX \
DEF:bc='/cacti/cacti-1.2.3-prod/rra/m24_temperature_14381.rrd':'temperature':MAX \
DEF:bd='/cacti/cacti-1.2.3-prod/rra/m24_temperature_55036.rrd':'temperature':MAX \
DEF:be='/cacti/cacti-1.2.3-prod/rra/n19_temperature_30123.rrd':'temperature':MAX \
DEF:bf='/cacti/cacti-1.2.3-prod/rra/n24_temperature_14418.rrd':'temperature':MAX \
CDEF:cdefa='a,10,/,1.8,*,32,+' \
CDEF:cdeff='b,10,/,1.8,*,32,+' \
CDEF:cdefba='c,10,/,1.8,*,32,+' \
CDEF:cdefbf='d,10,/,1.8,*,32,+' \
CDEF:cdefca='e,10,/,1.8,*,32,+' \
CDEF:cdefzzcf='f,10,/,1.8,*,32,+' \
CDEF:cdefda='g,10,/,1.8,*,32,+' \
CDEF:cdefdf='h,10,/,1.8,*,32,+' \
CDEF:cdefea='i,10,/,1.8,*,32,+' \
CDEF:cdefef='j,10,/,1.8,*,32,+' \
CDEF:cdeffa='ba,10,/,1.8,*,32,+' \
CDEF:cdefff='bb,10,/,1.8,*,32,+' \
CDEF:cdefga='bc,10,/,1.8,*,32,+' \
CDEF:cdefgb='bd,10,/,1.8,*,32,+' \
CDEF:cdefgf='be,10,/,1.8,*,32,+' \
CDEF:cdefha='bf,10,/,1.8,*,32,+' \
LINE1:cdefa#EACC00FF:'|query_tHSensorName|'  \
GPRINT:cdefa:LAST:'Current\:%8.2lf %s'  \
GPRINT:cdefa:MIN:'Min\:%8.2lf %s'  \
GPRINT:cdefa:AVERAGE:'Avg\:%8.2lf %s'  \
GPRINT:cdefa:MAX:'Max\:%8.2lf %s\n'  \
LINE1:cdeff#FFAB00FF:'|query_tHSensorName|'  \
GPRINT:cdeff:LAST:'Current\:%8.2lf %s'  \
GPRINT:cdeff:MIN:'Min\:%8.2lf %s'  \
GPRINT:cdeff:AVERAGE:'Avg\:%8.2lf %s'  \
GPRINT:cdeff:MAX:'Max\:%8.2lf %s\n'  \
LINE1:cdefba#EAAF00FF:'|query_tHSensorName|'  \
GPRINT:cdefba:LAST:'Current\:%8.2lf %s'  \
GPRINT:cdefba:MIN:'Min\:%8.2lf %s'  \
GPRINT:cdefba:AVERAGE:'Avg\:%8.2lf %s'  \
GPRINT:cdefba:MAX:'Max\:%8.2lf %s\n'  \
LINE1:cdefbf#EA8F00FF:'|query_tHSensorName|'  \
GPRINT:cdefbf:LAST:'Current\:%8.2lf %s'  \
GPRINT:cdefbf:MIN:'Min\:%8.2lf %s'  \
GPRINT:cdefbf:AVERAGE:'Avg\:%8.2lf %s'  \
GPRINT:cdefbf:MAX:'Max\:%8.2lf %s\n'  \
LINE1:cdefca#ED7600FF:'|query_tHSensorName|'  \
GPRINT:cdefca:LAST:'Current\:%8.2lf %s'  \
GPRINT:cdefca:MIN:'Min\:%8.2lf %s'  \
GPRINT:cdefca:AVERAGE:'Avg\:%8.2lf %s'  \
GPRINT:cdefca:MAX:'Max\:%8.2lf %s\n'  \
LINE1:cdefzzcf#FF7D00FF:'|query_tHSensorName|'  \
GPRINT:cdefzzcf:LAST:'Current\:%8.2lf %s'  \
GPRINT:cdefzzcf:MIN:'Min\:%8.2lf %s'  \
GPRINT:cdefzzcf:AVERAGE:'Avg\:%8.2lf %s'  \
GPRINT:cdefzzcf:MAX:'Max\:%8.2lf %s\n'  \
LINE1:cdefda#FF5700FF:'|query_tHSensorName|'  \
GPRINT:cdefda:LAST:'Current\:%8.2lf %s'  \
GPRINT:cdefda:MIN:'Min\:%8.2lf %s'  \
GPRINT:cdefda:AVERAGE:'Avg\:%8.2lf %s'  \
GPRINT:cdefda:MAX:'Max\:%8.2lf %s\n'  \
LINE1:cdefdf#FF4105FF:'|query_tHSensorName|'  \
GPRINT:cdefdf:LAST:'Current\:%8.2lf %s'  \
GPRINT:cdefdf:MIN:'Min\:%8.2lf %s'  \
GPRINT:cdefdf:AVERAGE:'Avg\:%8.2lf %s'  \
GPRINT:cdefdf:MAX:'Max\:%8.2lf %s\n'  \
LINE1:cdefea#EE5019FF:'|query_tHSensorName|'  \
GPRINT:cdefea:LAST:'Current\:%8.2lf %s'  \
GPRINT:cdefea:MIN:'Min\:%8.2lf %s'  \
GPRINT:cdefea:AVERAGE:'Avg\:%8.2lf %s'  \
GPRINT:cdefea:MAX:'Max\:%8.2lf %s\n'  \
LINE1:cdefef#DA4725FF:'|query_tHSensorName|'  \
GPRINT:cdefef:LAST:'Current\:%8.2lf %s'  \
GPRINT:cdefef:MIN:'Min\:%8.2lf %s'  \
GPRINT:cdefef:AVERAGE:'Avg\:%8.2lf %s'  \
GPRINT:cdefef:MAX:'Max\:%8.2lf %s\n'  \
LINE1:cdeffa#FF3932FF:'|query_tHSensorName|'  \
GPRINT:cdeffa:LAST:'Current\:%8.2lf %s'  \
GPRINT:cdeffa:MIN:'Min\:%8.2lf %s'  \
GPRINT:cdeffa:AVERAGE:'Avg\:%8.2lf %s'  \
GPRINT:cdeffa:MAX:'Max\:%8.2lf %s\n'  \
LINE1:cdefff#F51D30FF:'|query_tHSensorName|'  \
GPRINT:cdefff:LAST:'Current\:%8.2lf %s'  \
GPRINT:cdefff:MIN:'Min\:%8.2lf %s'  \
GPRINT:cdefff:AVERAGE:'Avg\:%8.2lf %s'  \
GPRINT:cdefff:MAX:'Max\:%8.2lf %s\n'  \
LINE1:cdefga#FF0000FF:'|query_tHSensorName|'  \
GPRINT:cdefgb:LAST:'Current\:%8.2lf %s'  \
GPRINT:cdefgb:MIN:'Min\:%8.2lf %s'  \
GPRINT:cdefgb:AVERAGE:'Avg\:%8.2lf %s'  \
GPRINT:cdefgb:MAX:'Max\:%8.2lf %s\n'  \
LINE1:cdefgf#DE0056FF:'|query_tHSensorName|'  \
GPRINT:cdefgf:LAST:'Current\:%8.2lf %s'  \
GPRINT:cdefgf:MIN:'Min\:%8.2lf %s'  \
GPRINT:cdefgf:AVERAGE:'Avg\:%8.2lf %s'  \
GPRINT:cdefgf:MAX:'Max\:%8.2lf %s\n'  \
LINE1:cdefha#B90054FF:'|query_tHSensorName|'  \
GPRINT:cdefha:LAST:'Current\:%8.2lf %s'  \
GPRINT:cdefha:MIN:'Min\:%8.2lf %s'  \
GPRINT:cdefha:AVERAGE:'Avg\:%8.2lf %s'  \
GPRINT:cdefha:MAX:'Max\:%8.2lf %s\n' 

RRDtool Says:
OK

@eschoeller
Copy link
Author

Actually the re-index sort issue looks like my fault. I've been cheating the system for a long time (using ".1.3.6.1.2.1.2.2.1.1" for the index) and somehow it's been working. I think some more recent changes in the codebase have caught up with me. So I'm going to have to fix that with some OID/REGEX magic in several places (aka do it the "right" way).

@cigamit
Copy link
Member

cigamit commented Apr 29, 2019

Okay, close if you are certain. Otherwise, share your findings.

@cigamit cigamit added the unverified Some days we don't have a clue label Apr 29, 2019
@eschoeller
Copy link
Author

I think these are two separate problems. I muddled with my data query and fixed it, but these graphs are still not resolving the |query_| directives.

@eschoeller
Copy link
Author

eschoeller commented Apr 30, 2019

I have other graphs, using the same exact type of data sources, and they have no problems. So the results are inconsistent which is really baffling to me right now.
I should also mention that I believe this issue may only be isolated to legacy aggregate graphs.

@eschoeller
Copy link
Author

And I was able to just break it on a new-style Aggregate by making an unrelated change to the vertical label in the Aggregate template. So now I have unresolved |query_| text there too.

@netniV
Copy link
Member

netniV commented Apr 30, 2019

Can you post the exact steps when creating a new aggregate and what you do to break it?

@eschoeller
Copy link
Author

I've upgraded to 1.2.4 and I'm still faced with this issue.

@cigamit cigamit added aggregate Aggregate related issue bug Undesired behaviour and removed unverified Some days we don't have a clue labels Jun 22, 2019
@eschoeller
Copy link
Author

@cigamit what do you think about this one? I thought after working through #2663 this might see some improvement, but it's still exhibiting the same behavior.

@cigamit
Copy link
Member

cigamit commented Jun 23, 2019

Let me review the latest dump and see if there is a quick way to repair. I might add this to the repair and upgrade scripts.

@eschoeller
Copy link
Author

Take a look at graph ids 28779, 28780, 28781

@cigamit
Copy link
Member

cigamit commented Jun 23, 2019

Okay, classic aggregates. Digging in.

@eschoeller
Copy link
Author

eschoeller commented Jun 23, 2019

Yep. Eventually some day I will divorce myself from them. But I do face situations where I need to combine data sources from several different places into one graph and the legacy aggregates are the only way to do that right now. Hence #793 and #431. Usually I need to graph temperature data from two different types of devices, or combine temperature and humidity on the same graph. Stuff like that.
I gotta step out for about 30 min but I'll be back

@cigamit
Copy link
Member

cigamit commented Jun 24, 2019

Well, that is odd. Seems we've like you may have had some sort of hack. Just about to upload the fix.

@cigamit cigamit changed the title [1.2.3] Legend queries no longer functioning GPRINT text_format does not replace Data Query and Host Fields Jun 24, 2019
@eschoeller
Copy link
Author

OK, I can hang out for a few more minutes. My ramen place closes in an hour:)

cigamit added a commit that referenced this issue Jun 24, 2019
GPRINT text_format does not replace Data Query and Host Fields
@cigamit
Copy link
Member

cigamit commented Jun 24, 2019

I just uploaded an updated lib/rrd.php. Please test.

@eschoeller
Copy link
Author

And yeah, you could create an aggregate with all the same type of data sources, then go back into it and manually add some other rouge dissimilar data sources ... ;)
In the very beginning you could select whatever data sources you wanted, and there would simply be a warning that you'd just ignore and blow past. But that was a LONG time ago.

@cigamit
Copy link
Member

cigamit commented Jun 24, 2019

Yea, Reinhard wanted to have us add that functionality back. There are just too many outstanding feature requests.

@eschoeller
Copy link
Author

Yeah, understood on that one:)
Just put in the updated rrd.php, still the same problem.

@cigamit
Copy link
Member

cigamit commented Jun 24, 2019

Hmm, let me run a few queries just to ensure nothing is broken. Go eat your romen. There'll be one more update before the nights over.

@eschoeller
Copy link
Author

K, I'll be back in about 30.

cigamit added a commit that referenced this issue Jun 24, 2019
Wront host_id was being passed
@cigamit
Copy link
Member

cigamit commented Jun 24, 2019

Fresh update to lib/rrd.php committed.

@eschoeller
Copy link
Author

Updated. Still no dice there.

@cigamit
Copy link
Member

cigamit commented Jun 24, 2019

Bah, humbug. Digging in deeper then.

cigamit added a commit that referenced this issue Jun 24, 2019
This has finally fixed the issue.  Was specific to Aggregate Graphs.
@cigamit
Copy link
Member

cigamit commented Jun 24, 2019

Okay, I can confirm it's fixed now. Once you confirm, look at the other Aggregate bug again.

@eschoeller
Copy link
Author

Yup, that got it! Which other aggregate bug?

@cigamit
Copy link
Member

cigamit commented Jun 24, 2019

It's #2659, I also noticed that you have like 216 Aggregate Graphs where there are no longer any Data Sources. They are Orphaned. Was this by Accident? Just goto Graph Management and select "Orphaned" checkbox.

@cigamit
Copy link
Member

cigamit commented Jun 24, 2019

The 'Aggregate' total is 109. But there are some other graphs that are showing orphans that are in fact not orphans.

@cigamit cigamit added the resolved A fixed issue label Jun 24, 2019
@eschoeller
Copy link
Author

Ah, yeah. Look at all these orphaned items. Some of this stuff was really cool back in the day. Some of it is very easy to part with. But I suppose since they are 'orphaned' that means there's really no data at all behind them anymore and they're essentially useless (without somehow re-creating the data sources)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
aggregate Aggregate related issue bug Undesired behaviour resolved A fixed issue
Projects
None yet
Development

No branches or pull requests

3 participants