Where to find influxdb.conf ?

[hary66]

Hi.
Where to find influxdb.conf ?

[Slyke]

There is no influxdb.conf, everything is configured with services/influxdb/influxdb.env.

#INFLUXDB_DB=mydb
INFLUXDB_DATA_ENGINE=tsm1
INFLUXDB_REPORTING_DISABLED=false
#INFLUXDB_HTTP_AUTH_ENABLED=true
INFLUXDB_ADMIN_ENABLED=true
INFLUXDB_ADMIN_USER=root
INFLUXDB_ADMIN_PASSWORD=MYP45SW0RDO
INFLUXDB_USER=influxuser
INFLUXDB_USER_PASSWORD=MYOTH3RP45SW0RDO
INFLUXDB_READ_USER=read
INFLUXDB_READ_USER_PASSWORD=AND4NOTH3RP45SW0RDO
#INFLUXDB_WRITE_USER=mywriteuser
#INFLUXDB_WRITE_USER_PASSWORD=mywritepassword

I'm not too familiar with influxdb, you could try putting in your own config.

Directory you're looking for is /volumes/influxdb/data.

[Paraphraser]

Slyke is spot on - use influxdb.env.

But I thought I would expand on Slyke's answer a bit because I'm prety sure putting an influxdb.conf into ~/IOTstack/volumes/influxdb/data won't work. That's because (from the docker-compose.yml file):

- ./volumes/influxdb/data:/var/lib/influxdb

The "data" directory ouside the container maps to "/var/lib/influxdb" inside the container.

Inside the container, influxdb.conf is in:

/etc/influxdb

and those are not the same directory. I hope that makes sense.

If you really want to find influxdb.conf:

$ docker exec -it influxdb bash
# cat etc/influxdb/influxdb.conf 
[meta]
  dir = "/var/lib/influxdb/meta"

[data]
  dir = "/var/lib/influxdb/data"
  engine = "tsm1"
  wal-dir = "/var/lib/influxdb/wal"

You can edit influxdb.conf if you wish. Then, when you restart the container, those changes will be applied.

The problem comes later when Docker next rebuilds the container from the image. Anything you have done inside the old container will be lost and you will wind up having to re-do your changes. That's a maintenance hassle you probably don't need.

As Slyke said, the recommended approach in the Docker context is editing ~/IOTstack/services/influxdb/influxdb.env and the earlier reply contains a listing of that file.

Ignore the "#" characters and notice the pattern where each line begins with "INFLUXDB_".

Now, go to the InfluxDB documentation. That link takes you part way down the page, which is intentional.

The first entry under "Global Settings" is:

reporting-disabled = false

If you were editing influxdb.conf then you would use that syntax. The documentation is also telling you that "false" is the default value so you will only ever need to implement this setting if you want to change it to "true".

When you are working in a Docker environment, you need to keep reading a little bit further down until you find:

Environment variable: INFLUXDB_REPORTING_DISABLED

That tells you that, if you are editing influxdb.env then the syntax you will need is:

INFLUXDB_REPORTING_DISABLED=true

All of the Influx options follow that same pattern:

• A keyword to use in influxdb.conf followed by the default value (from which you can infer the type); then
• The matching keyword to use in influxdb.env, where those keywords always start with "INFLUXDB_"

Once you have made a change and saved influxdb.env, you just:

$ cd ~/IOTstack
$ docker-compose up -d

Docker-compose will notice that the influxdb.env has changed and you will see:

Recreating influxdb ... done

How can you check that a change made in influxdb.env gets through to Influx? Well, that can be tricky.

One thing you can't do is go into the container and list the contents of influxdb.conf because it doesn't work that way. Environment variables are passed directly. There is no preliminary step where they get mapped into influxdb.conf.

Scroll to the near the top of the InfluxDB documentation page and read the "Environment variables" section. This tells you that an option set by an enviroment variable will always override anything in the influxdb.conf.

You can confirm that a setting in influxdb.env is getting into the container. For example, assuming influxdb.env contains this line:

INFLUXDB_REPORTING_DISABLED=true

then you could check that it is being passed into the container like this:

$ docker exec -it influxdb bash
# set | grep "INFLUXDB_REPORTING_DISABLED"
INFLUXDB_REPORTING_DISABLED=true
#

You can also ask Influx to show you its settings via "show diagnostics". I am not sure whether every option shows up but it is a good place to start. In the case of "reporting disabled" it shows up:

# influx
> show diagnostics

...
name: config
bind-address   reporting-disabled
------------   ------------------
127.0.0.1:8088 true
...

> quit
# 

Anyway, while you might want to go into the container and edit influxdb.conf during testing, it's really just as easy to use influxdb.env.

[hary66]

Going that way, as supposed, I got :

pi@raspberrypi:~/IOTstack $ docker-compose up -d
nodered is up-to-date
portainer is up-to-date
mosquitto is up-to-date
grafana is up-to-date
pihole is up-to-date
Recreating influxdb ... done
Recreating telegraf ... done

But, it through ..

from(bucket:"telegraf/autogen")
ERR: error parsing query: found FROM, expected SELECT, DELETE, SHOW, CREATE, DROP, EXPLAIN, GRANT, REVOKE, ALTER, SET, KILL at line 1, char 1`

It seems FLUX isn't working.

[Paraphraser]

I don't use Telegraf so I can't really comment on that part. Sorry.

However, when I see an error message like that one, I will suspect the process generating the query (ie Telegraf) rather than the database engine (Influx).

If I got an error like that and wanted to confirm that Influx was working, I would start by connecting to the Raspberry Pi via ssh.

The next step is to get into the Influx Command-Line Interface. Graham has a service script at:

~/IOTstack/services/influxdb/terminal.sh

but I prefer a scheme of aliases in my .profile

alias INFLUX_DATA='cd ~/IOTstack/volumes/influxdb/data; ls'
alias INFLUX_RESTART='pushd ~/IOTstack;docker-compose restart influxdb; popd'
alias INFLUX_SERVICES='cd ~/IOTstack/services/influxdb; ls'
alias INFLUX_SHELL='docker exec -it influxdb bash'
alias influx='docker exec -it influxdb influx -precision=rfc3339'

The last line is the most-relevant. I just type:

$ influx

Whether you use Graham's script, or an alias like I do, or even by just typing the entire "docker exec" command, you wind up in the CLI:

Connected to http://localhost:8086 version 1.7.9
InfluxDB shell version: 1.7.9
> 

Next, check that your database(s) exist. These are mine:

> show databases
name: databases
name
----
_internal
power
test
weather
airquality
> 

Your list will be different (obviously) but you should at least see "_internal" along with the databases you have created.

Never created a database via "CREATE DATABASE «name»"? That might be the problem. I use Node-Red to add measurements to Influx and I always have to create any new database by hand. Telegraf may be different but I thought I'd mention this as a possibility.

Pick one of your databases:

> use power
Using database power
> 

and then ask what series exist within your chosen database:

> show series
key
---
hiking
solax
> 

Now run a dumb query against one of those:

> select * from hiking limit 1
name: hiking
time                     current frequency importEnergy meterErrors voltage
----                     ------- --------- ------------ ----------- -------
2018-04-10T14:00:06.663Z 0.16    50.11     27.19        0           247.4
> 

All up, those steps demonstrate:

• The InfluxDB container is installed and running.
• The Influx CLI within the container is launchable and can connect to the influxdb process.
• The expected databases are present and I can "use" one of them.
• The expected series are present in the chosen database.
• A query against one of the series returns a sensible response.

I can also replicate your error message by typing a mal-formed query:

> from hiking limit 1
ERR: error parsing query: found FROM, expected SELECT, DELETE, SHOW, CREATE, DROP, EXPLAIN, GRANT, REVOKE, ALTER, SET, KILL at line 1, char 1
> 

That takes me back to my original hypothesis that it's more likely to be the process generating the query (in this case, me, typing bad queries at the CLI) than Influx.

Does that help?

> quit
$ 

[hary66]

Paraphraser , Thank you to that long and detailed response.

I was trying to use FLUX, a new influxdb feature

Otherwise, I think my InfluxDB and influx CLI are working properly as you can see below.

`pi@raspberrypi:~/IOTstack/services/influxdb $ ./terminal.sh
You are about to enter the influxdb console:

to create a db: CREATE DATABASE myname
to show existing a databases: SHOW DATABASES
to use a specific db: USE myname

to exit type: EXIT

Connected to http://localhost:8086 version 1.7.9
InfluxDB shell version: 1.7.9

show databases
name: databases
name

---

_internal
telegraf
myfile_from_csv

use telegraf
Using database telegraf
from(bucket:"telegraf/autogen")
ERR: error parsing query: found FROM, expected SELECT, DELETE, SHOW, CREATE, DROP, EXPLAIN, GRANT, REVOKE, ALTER, SET, KILL at line 1, char 1

`
You're right I'm using Telegraf to get my data in InfluxDB (pulling my data from CSV file into InfluxDB to manage my "real" personal project vs now I'm trying to experiment and follow the documentation's example), but the telegraf you see in my querry is the database example, as shown half way of that page.

In case you haven't noticed, I'm pretty beginner, so forgive me if I don't understand straight away what you try to explain.

[Paraphraser]

I must admit to never having heard of FLUX. I followed the link you provided (thanks) and read through the page but my initial take was that I could not quite see the point. But that’s probably just the natural reaction of someone who has spent much of the last 30 years writing and teaching SQL. It’s probably like everything else in IT - once you have a use case of your own you immediately see the point and it immediately becomes a brilliant idea.

In looking at the overall pattern, though, there must be an engine that parses FLUX code and either converts it to Influx’s idea of SQL or bypasses any SQL interface to interact directly with the database. Because the example FLUX statements begin with “from” and it’s clearly the SQL parser that is grizzling about “from” (else it wouldn’t be complaining about not having seen a SELECT etc), it probably follows that you need to invoke a different parser environment.

This would seem to be the clue:

Flux v0.50 (technical preview) is built into InfluxDB v1.7 and can be used to query data stored in InfluxDB. InfluxDB v1.7’s influx CLI also includes a new -type= option that, when set to flux, will start an interactive Flux Read-Eval-Print-Loop (REPL) allowing you to write and run Flux queries from the command line.

I just did a quick test. Assuming you have my “influx” alias installed, this gets the job done:

$ influx -type=flux

After that I got a bit lost but I did figure out that a Control+D gets you out of the REPL.

[hary66]

Hi Paraphraser .

it seems I was giving -type=FLUX instead of -type=flux, so I get the previous error.
Following your response, but not having your aliases, I've now got the following :

pi@raspberrypi:~/IOTstack $ docker exec -it influxdb influx -precision=rfc3339 -type=flux Connected to http://localhost:8086 version 1.7.9 InfluxDB shell version: 1.7.9 **> from(bucket:"telegraf/autogen") Error: Flux query service disabled. Verify flux-enabled=true in the [http] section of the InfluxDB config. : 403 Forbidden aggregateWindow bool bottom buckets bytes chandeMomentumOscillator

Checking my config.env gives me :
pi@raspberrypi:~/IOTstack/services/influxdb $ cat influxdb.env #INFLUXDB_DB=mydb INFLUXDB_DATA_ENGINE=tsm1 INFLUXDB_REPORTING_DISABLED=false #INFLUXDB_HTTP_AUTH_ENABLED=true INFLUXDB_FLUX_ENABLED=true INFLUXDB_ADMIN_ENABLED=true #INFLUXDB_ADMIN_USER=myadminuser #INFLUXDB_ADMIN_PASSWORD=myadminpassword INFLUXDB_USER=nodered INFLUXDB_USER_PASSWORD=nodered #INFLUXDB_READ_USER=myreaduser #INFLUXDB_READ_USER_PASSWORD=myreadpassword #INFLUXDB_WRITE_USER=mywriteuser #INFLUXDB_WRITE_USER_PASSWORD=mywritepassword

You can see I have the "INFLUXDB_FLUX_ENABLED=true" in my config.env file, but still the error !

[Paraphraser]

I think the basic problem is that you are using the wrong key in influxdb.env. I think the key should be:

INFLUXDB_HTTP_FLUX_ENABLED=true

How did I reach that conclusion? I agree that this is not immediately obvious but there is a clue in the documentation:

Enable Flux by setting the flux-enabled option to true under the [http] section of your influxdb.conf

When you look through Influx's various "this setting = this environment variable" combinations, there is an implicit rule pattern which is:

1. Environment variables all start with "INFLUX_".
2. If the configuration variable is associated with a section (like "meta" or "data" or "http") then the section name is appended (eg "INFLUX_META_").
3. If the variable name contains hyphens, those are replaced with underscores and appended. For example, "logging-enabled" becomes "INFLUX_META_LOGGING_ENABLED".

The rules are probably stated explicitly somewhere but I could not find them.

Applying those rules gets:

INFLUXDB_
         HTTP_
              FLUX_ENABLED

I put this to the test. First, WITHOUT that key added to influxdb.env:

$ influx -database=power -type=flux
Connected to http://localhost:8086 version 1.7.9
InfluxDB shell version: 1.7.9
> from(bucket:"hiking/autogen")
Error: Flux query service disabled. Verify flux-enabled=true in the [http] section of the InfluxDB config.
: 403 Forbidden

That's close to the error in your output.

Next, WITH that key added to influxdb.env and the container restarted:

$ influx -database=power -type=flux
Connected to http://localhost:8086 version 1.7.9
InfluxDB shell version: 1.7.9
> from(bucket:"hiking/autogen")
Error: unknown server error: 500 Internal Server Error

While the second result still returns an error, at least it's a different error, and it's no longer whinging about the Flux query service being disabled. I'd call that progress - yes?

[Paraphraser]

Following your response, but not having your aliases ...

I thought it might help if I wrote a gist explaining the how-to of setting up some useful aliases.

[hary66]

Hi Paraphraser.
My server now seems to be stuck, so I need to get in reach of it to do a hard reboot I guess. I can pung it, but cannot ssh it any more.
I'll try what you say after tomorrow so ... We'll see.

By the way. May I ask you some more question as you seem to be a database "killer" ?

I'm learning database stuff for a near future use that I think it would be useful and practical, but I'm not 100% sure that's the right solution. May I querry my question to you ? Maybe in another post or another place not to disturb that one.

[Paraphraser]

I sent an email to the address on your profile page.