mirror of
https://github.com/bsmithio/OPNsense-Dashboard.git
synced 2026-02-16 14:14:16 +00:00
Create configure.md
This commit is contained in:
bsmithio
GitHub
parent
fe037178af
commit
63c65c6f19
1 changed files with 300 additions and 0 deletions
300
configure.md
Normal file
300
configure.md
Normal file
|
|
@ -0,0 +1,300 @@
|
|||
## Docker
|
||||
|
||||
To simplify everything, we'll use a docker compose file.
|
||||
|
||||
You'll need to have Docker and Docker-Compose installed on the server that will host the dashboard.
|
||||
|
||||
|
||||
[Install Docker](https://docs.docker.com/engine/install/)
|
||||
|
||||
[Install Docker Compose](https://docs.docker.com/compose/install/)
|
||||
|
||||
After you've installed Docker and Docker Compose, copy this code block to a file named docker-compose.yaml.
|
||||
It's important that you change the TZ environment variable to your timezone for everything to work properly.
|
||||
I also recommend you change the passwords in this compose file as well.
|
||||
After you've made the necessary changes, run `docker-compose up -d` in the same directory as your docker-compose.yaml.
|
||||
|
||||
```
|
||||
version: '3'
|
||||
services:
|
||||
mongodb:
|
||||
container_name: mongodb
|
||||
image: mongo:4.4.10
|
||||
volumes:
|
||||
- mongodb_data:/data/db
|
||||
restart: "unless-stopped"
|
||||
environment:
|
||||
# Change this to your time zone, valid time zones can be found here: https://www.joda.org/joda-time/timezones.html
|
||||
- TZ=CST6CDT
|
||||
elasticsearch:
|
||||
container_name: elasticsearch
|
||||
image: docker.elastic.co/elasticsearch/elasticsearch-oss:7.10.2
|
||||
volumes:
|
||||
- es_data:/usr/share/elasticsearch/data
|
||||
environment:
|
||||
# Change this to your time zone, valid time zones can be found here: https://www.joda.org/joda-time/timezones.html
|
||||
- TZ=CST6CDT
|
||||
- http.host=0.0.0.0
|
||||
- transport.host=localhost
|
||||
- network.host=0.0.0.0
|
||||
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
|
||||
ulimits:
|
||||
memlock:
|
||||
soft: -1
|
||||
hard: -1
|
||||
mem_limit: 1g
|
||||
restart: "unless-stopped"
|
||||
graylog:
|
||||
container_name: graylog
|
||||
image: graylog/graylog:4.2.4
|
||||
volumes:
|
||||
- graylog_data:/usr/share/graylog/data
|
||||
environment:
|
||||
# Change this to your time zone, valid time zones can be found here: https://www.joda.org/joda-time/timezones.html
|
||||
- TZ=CST6CDT
|
||||
# CHANGE ME (must be at least 16 characters)!
|
||||
- GRAYLOG_PASSWORD_SECRET=ZDcwMzQ3NTE4ZTIwM
|
||||
# Username is "admin"
|
||||
# Password is "admin", change this to your own hashed password. 'echo -n "password" | sha256sum'
|
||||
- GRAYLOG_ROOT_PASSWORD_SHA2=8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918
|
||||
- GRAYLOG_HTTP_EXTERNAL_URI=http://127.0.0.1:9000/
|
||||
entrypoint: /usr/bin/tini -- wait-for-it elasticsearch:9200 -- /docker-entrypoint.sh
|
||||
links:
|
||||
- mongodb:mongo
|
||||
- elasticsearch
|
||||
depends_on:
|
||||
- mongodb
|
||||
- elasticsearch
|
||||
ports:
|
||||
# Graylog web interface and REST API
|
||||
- 9000:9000
|
||||
# Syslog UDP
|
||||
- 1514:1514/udp
|
||||
# Syslog TCP
|
||||
- 1514:1514
|
||||
restart: "unless-stopped"
|
||||
influxdb:
|
||||
container_name: influxdb
|
||||
image: influxdb:2.1.1
|
||||
ports:
|
||||
- '8086:8086'
|
||||
volumes:
|
||||
- influxdb2_data:/var/lib/influxdb2
|
||||
environment:
|
||||
# Change this to your time zone, valid time zones can be found here: https://www.joda.org/joda-time/timezones.html
|
||||
- TZ=CST6CDT
|
||||
restart: "unless-stopped"
|
||||
grafana:
|
||||
container_name: grafana
|
||||
image: grafana/grafana:8.3.3
|
||||
ports:
|
||||
- '3000:3000'
|
||||
volumes:
|
||||
- grafana_data:/var/lib/grafana
|
||||
depends_on:
|
||||
- influxdb
|
||||
environment:
|
||||
# Change this to your time zone, valid time zones can be found here: https://www.joda.org/joda-time/timezones.html
|
||||
- TZ=CST6CDT
|
||||
# Change these
|
||||
- GF_SECURITY_ADMIN_USER=opnsense
|
||||
- GF_SECURITY_ADMIN_PASSWORD=opnsense
|
||||
- GF_INSTALL_PLUGINS=grafana-worldmap-panel
|
||||
restart: "unless-stopped"
|
||||
volumes:
|
||||
grafana_data:
|
||||
influxdb2_data:
|
||||
graylog_data:
|
||||
es_data:
|
||||
mongodb_data:
|
||||
```
|
||||
|
||||
|
||||
Once you have your docker containers running, follow the steps below.
|
||||
|
||||
## Configuring InfluxDB
|
||||
After InfluxDB is started, go to http://(ip of docker server):8086, you will need to setup your username, password, bucket and organization here. Once that is done navigate to the Data tab, click on Telegraf, and create a configuration for a system. Name it, and copy your API token, you will need this for your telegraf configuration. I recommend generating another API token for Grafana. Click on API tokens -> Generate API Token -> Read/Write Access -> Click on your bucket under Read -> and Save. Copy this somewhere as well, you'll need it for Grafana.
|
||||
|
||||
## Configuring Telegraf
|
||||
|
||||
You must manually install Telegraf on OPNsense, as the OPNsense Telegraf plugin does not currently support custom telegraf configuration. To do so, SSH into your OPNsense router and use the command
|
||||
|
||||
`sudo pkg install telegraf`
|
||||
|
||||
After that, use these commands. The first, enables Telegraf to start on boot, and the second, adds Telegraf to sudoers and restricts nopasswd to only what Telegraf needs to run as root.
|
||||
|
||||
```
|
||||
printf 'telegraf_enable="YES"' > /etc/rc.conf.d/telegraf
|
||||
|
||||
printf 'telegraf ALL=(root) NOPASSWD: /sbin/pfctl -s info,/usr/local/bin/telegraf_pfifgw.php' >> /usr/local/etc/sudoers
|
||||
```
|
||||
You will need this [telegraf config](https://github.com/bsmithio/OPNsense-Dashboard/blob/master/config/telegraf.conf) file.
|
||||
|
||||
You must edit this file and type in your InfluxDB URL, InfluxDB Telegraf API token, organization, and bucket under [[outputs.influxdb_v2]].
|
||||
|
||||
You will need to place this config in /usr/local/etc on your OPNsense system.
|
||||
|
||||
```
|
||||
cd /usr/local/etc
|
||||
curl https://github.com/bsmithio/OPNsense-Dashboard/blob/master/config/telegraf.conf
|
||||
# Make the necessary changes with vi, if you wish to edit with nano you'll need to install nano with pkg install nano
|
||||
vi telegraf.conf
|
||||
```
|
||||
|
||||
### Telegraf Plugins
|
||||
|
||||
**Plugins must be copied to your OPNsense system**
|
||||
|
||||
Place [telegraf_pfifgw.php](https://raw.githubusercontent.com/bsmithio/OPNsense-Dashboard/master/plugins/telegraf_pfifgw.php) and [telegraf_temperature.sh](https://raw.githubusercontent.com/bsmithio/OPNsense-Dashboard/master/plugins/telegraf_temperature.sh) in /usr/local/bin and chmod them to 755.
|
||||
|
||||
```
|
||||
cd /usr/local/bin
|
||||
curl https://raw.githubusercontent.com/bsmithio/OPNsense-Dashboard/master/plugins/telegraf_pfifgw.php
|
||||
curl https://raw.githubusercontent.com/bsmithio/OPNsense-Dashboard/master/plugins/telegraf_temperature.sh
|
||||
chmod 755 telegraf_temperature.sh telegraf_pfifgw.php
|
||||
```
|
||||
|
||||
Test these out before starting the telegraf service by executing them
|
||||
|
||||
`./telegraf_pfifgw.php`
|
||||
|
||||
`./telegraf_temperature.sh`
|
||||
|
||||
The temperature plugin may not work on every system, if you receive `sysctl: unknown oid 'hw.acpi.thermal'` comment out or remove that line from the plugin.
|
||||
|
||||
|
||||
After this is done, use `sudo service telegraf restart` to start telegraf with the new configuration.
|
||||
|
||||
## Configuring Graylog
|
||||
|
||||
### Add GeoIP to Graylog
|
||||
|
||||
To make the map work on Grafana, you must create a MaxMind account here https://www.maxmind.com/en/geolite2/signup. Then generate a license key by going to Account -> Manage License Keys -> Generate New License Key. Copy this key somewhere because you'll need it again soon.
|
||||
|
||||
You'll need to download the GeoIP database file to your Graylog container. Access your Graylog container's shell from your Docker host like so
|
||||
|
||||
`sudo docker exec -it graylog /bin/bash`
|
||||
|
||||
Then download the database file, replace `YOUR_LICENSE_KEY` with the key you generated above.
|
||||
|
||||
```
|
||||
curl "https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-Country&license_key=YOUR_LICENSE_KEY&suffix=tar.gz" -o GeoLite2-Country.tar.gz \
|
||||
&& tar -xzvf GeoLite2-Country.tar.gz \
|
||||
&& mv GeoLite2-Country_*/GeoLite2-Country.mmdb /usr/share/graylog/data/data/
|
||||
```
|
||||
|
||||
### Configure Additional Settings
|
||||
|
||||
In a browser navigate to http://(ip of docker server):9000 and login.
|
||||
|
||||
For Graylog, it's recommended to create an index set. To do so, navigate to System -> Indices. Create an index set with the name "OPNsense / filterlog" and set the index prefix to `opnsense_filterlog`.
|
||||
|
||||
|
||||
|
||||
Once that's done, download the [content pack](https://raw.githubusercontent.com/bsmithio/OPNsense-Dashboard/master/config/OPNsense-pack.json) and install it on Graylog by navigating to System -> Content Packs -> Upload, choose the pack, Upload, then Install.
|
||||
|
||||
Now, add your index set from earlier to the "OPNsense / filterlog" stream. Navigate to Streams -> More Actions -> Edit Stream -> select your index set and save.
|
||||
|
||||
|
||||
|
||||
There's one more step we need to do here, navigate to System -> Configurations -> click on Update under Message Processors, and reorder like so:
|
||||
|
||||
|
||||
|
||||
Ensure that all of these are enabled, and click save.
|
||||
|
||||
### Add Graylog server as syslog target on OPNsense
|
||||
|
||||
Once that is all done, login to your OPNsense router and navigate to System -> Settings -> Logging / targets. Add a new target with the following options:
|
||||
|
||||
|
||||
|
||||
Add a description if you'd like, then click save.
|
||||
|
||||
## Configuring Grafana
|
||||
|
||||
### Add InfluxDB and ElasticSearch data sources
|
||||
|
||||
You will need to add the data sources on Grafana. Navigate to http://(ip of docker server):3000, login and click on the cog wheel and Add a Data Source.
|
||||
|
||||
For InfluxDB, make the following configurations
|
||||
|
||||
|
||||
|
||||
|
||||
For ElasticSearch, make the following configurations
|
||||
|
||||
|
||||
|
||||
### Import Dashboard
|
||||
|
||||
To import the dashboard, copy the JSON in [OPNsense-Grafana-Dashboard.json](OPNsense-Grafana-Dashboard.json) and navigate to Dashboards -> Browse -> Import and paste under Import via panel json.
|
||||
|
||||
Dashboard Settings -> Variables
|
||||
|
||||
WAN - $WAN is a static variable defined so that a separate dashboard panel can be created for WAN interfaces stats. Use a comma-separated list for multiple WAN interfaces.
|
||||
|
||||
LAN - $LAN uses a regex to remove any interfaces you don't want to be grouped as LAN. The filtering happens in the "Regex" field. I use a negative lookahead regex to match the interfaces I want excluded. It should be pretty easy to understand what you need to do here. I have excluded igb0 (WAN) and igb1,igb2,igb3 (only used to host vlans)..
|
||||
|
||||
Lastly, I don't recommend setting the time range beyond 24 hours, due to how many data points that will return in grafana.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Telegraf Plugins
|
||||
|
||||
- You can run most plugins from a shell/ssh session to verify the output. (the environment vars may be different when telegraf is executing the plugin)
|
||||
- If you're copying from a windows system, make sure the [CRLF is correct](https://www.cyberciti.biz/faq/howto-unix-linux-convert-dos-newlines-cr-lf-unix-text-format/)
|
||||
- The below command should display unix line endings (\n or LF) as $ and Windows line endings (\r\n or CRLF) as ^M$.
|
||||
|
||||
`# cat -e /usr/local/bin/telegraf_pfinterface.php`
|
||||
|
||||
### Telegraf Troubleshooting
|
||||
If you get no good output from running the plugin directly, try the following command before moving to the below step.
|
||||
|
||||
# telegraf --test --config /usr/local/etc/telegraf.conf
|
||||
|
||||
To troubleshoot plugins further, add the following lines to the agent block in /usr/local/etc/telegraf.conf and send a HUP to the telegraf pid. You're going to need to do this from a ssh shell. One you update the config you are going to need to tell telegraf to read the new configs. If you restart telegraf from pfSense, this will not work since it will overwrite your changes.
|
||||
|
||||
### Telegraf Config (Paste in to [agent] section)
|
||||
debug = true
|
||||
quiet = false
|
||||
logfile = "/var/log/telegraf/telegraf.log"
|
||||
|
||||
### Restarting Telegraf
|
||||
# ps aux | grep '[t]elegraf.conf'
|
||||
# kill -HUP <pid of telegraf proces>
|
||||
|
||||
Now go read /var/log/telegraf/telegraf.log
|
||||
|
||||
### InfluxDB
|
||||
When in doubt, run a few queries to see if the data you are looking for is being populated.
|
||||
I recommend doing this in Grafana's Explore tab.
|
||||
|
||||
### View measurements
|
||||
import "influxdata/influxdb/schema"
|
||||
|
||||
schema.measurements(bucket: "opnsense")
|
||||
|
||||
### View field values
|
||||
|
||||
from(bucket: "opnsense")
|
||||
|> range(start: -24h)
|
||||
|> filter(fn: (r) => r["_measurement"] == "system")
|
||||
|> limit(n:10)
|
||||
|
||||
### How to drop an InfluxDB v2 measurement
|
||||
|
||||
You must access your influx instance's shell to do this.
|
||||
To do so run
|
||||
`sudo docker exec -it influxdb /bin/bash`
|
||||
on your docker host.
|
||||
|
||||
Then use the following
|
||||
|
||||
bash-4.4# influx delete --bucket "$YourBucket" --predicate '_measurement="$Example"' -o $organization --start "1970-01-01T00:00:00Z" --stop "2050-12-31T23:59:00Z" --token "$YourAPIToken"
|
||||
|
||||
### Learn more about Flux queries
|
||||
|
||||
https://docs.influxdata.com/influxdb/cloud/query-data/flux/query-fields/
|
||||
|
||||
https://docs.influxdata.com/influxdb/cloud/query-data/flux/explore-schema/
|
||||
Loading…
Reference in a new issue