---
title: 'Paxeer Node Advanced Configuration & Monitoring'
sidebarTitle: 'Advanced Configuration & Monitoring'
description: "Optimize Paxeer node performance with advanced configuration, Prometheus and Grafana monitoring, and alerting."
keywords: ['paxeer node', 'node monitoring', 'prometheus', 'grafana', 'performance optimization', 'system tuning', 'infrastructure alerts']
---

> **For AI agents:** the complete documentation index is at [llms.txt](/llms.txt). Append `.md` to any page URL for its markdown version.

## Optimizing System Configuration

There are a virtually unlimited number of unique individual setups that cannot be covered in this document. As well, even similar builds and configurations can behave differently due to external factors, so your results may vary.

Here are some general guidelines to use as a starting point. Be cautious, make incremental changes, testing and observing before moving forward.
Always focus on only one specific area at a time - avoid making changes to memory, storage, and CPU configs all at once. Diagnosing potential problems becomes nearly impossible otherwise.

### Memory Management

The following settings in `/etc/sysctl.conf` can optimize memory usage and disk I/O patterns:

```bash
# Minimize swapping
vm.swappiness = 1

# Control disk write behavior
vm.dirty_background_ratio = 3
vm.dirty_ratio = 10
vm.dirty_expire_centisecs = 300
vm.dirty_writeback_centisecs = 100
```

Apply changes: `sudo sysctl -p`

### Network Stack

The following settings in `/etc/sysctl.conf` may improve network performance:

```bash
# Increase connection handling capacity
net.core.somaxconn = 32768
net.core.netdev_max_backlog = 32768
net.ipv4.tcp_max_syn_backlog = 16384

# Optimize buffer sizes
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 87380 16777216
```

### Storage Configuration

For NVMe drives, optimize I/O scheduling:

```bash
# Set IO scheduler
echo "none" > /sys/block/nvme0n1/queue/scheduler

# Set read-ahead buffer
blockdev --setra 4096 /dev/nvme0n1

# Set IO priority in systemd service
sudo tee -a /etc/systemd/system/paxd.service << EOF
[Service]
IOSchedulingClass=realtime
IOSchedulingPriority=2
EOF

# Configure disk mount options
sudo tee -a /etc/fstab << EOF
/dev/nvme0n1p1 /data ext4 defaults,noatime,nosuid,nodev,noexec,commit=60 0 0
EOF
```

## Infrastructure Monitoring

Monitoring is one of the most critical components of network infrastructure. performance tuning, and alerting configuration for Cosmos-SDK/Tendermint nodes.

### Prometheus Setup

First, install Prometheus:

```bash
wget https://github.com/prometheus/prometheus/releases/download/v2.42.0/prometheus-2.42.0.linux-amd64.tar.gz
tar xvf prometheus-2.42.0.linux-amd64.tar.gz
```

Example Prometheus configuration:

```yaml
global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: 'paxeer_node'
    static_configs:
      - targets: ['node1_ip:port']
    metrics_path: /metrics
  - job_name: 'node'
    static_configs:
      - targets: ['node2_ip:port']
```

### Grafana Integration

Install and configure Grafana:

```bash
sudo apt install -y apt-transport-https software-properties-common
sudo add-apt-repository "deb https://packages.grafana.com/oss/deb stable main"
sudo apt update && sudo apt-get install grafana
```

<Accordion title="Sample Grafana Dashboard JSON">

```json
{
	"annotations": {
		"list": [
			{
				"builtIn": 1,
				"datasource": "-- Grafana --",
				"enable": true,
				"hide": true,
				"iconColor": "rgba(0, 211, 255, 1)",
				"name": "Annotations & Alerts",
				"type": "dashboard"
			}
		]
	},
	"editable": true,
	"gnetId": null,
	"graphTooltip": 0,
	"id": 1,
	"links": [],
	"panels": [
		{
			"alerting": {},
			"aliasColors": {},
			"bars": false,
			"dashLength": 10,
			"dashes": false,
			"datasource": null,
			"fieldConfig": {
				"defaults": {
					"custom": {}
				},
				"overrides": []
			},
			"fill": 1,
			"fillGradient": 0,
			"gridPos": {
				"h": 8,
				"w": 12,
				"x": 0,
				"y": 0
			},
			"hiddenSeries": false,
			"id": 2,
			"legend": {
				"avg": false,
				"current": false,
				"max": false,
				"min": false,
				"show": true,
				"total": false,
				"values": false
			},
			"lines": true,
			"linewidth": 1,
			"nullPointMode": "null",
			"options": {
				"alertThreshold": true
			},
			"percentage": false,
			"pluginVersion": "7.2.0",
			"pointradius": 2,
			"points": false,
			"renderer": "flot",
			"seriesOverrides": [],
			"spaceLength": 10,
			"stack": false,
			"steppedLine": false,
			"targets": [
				{
					"expr": "tendermint_consensus_height",
					"interval": "",
					"legendFormat": "",
					"refId": "A"
				}
			],
			"thresholds": [],
			"timeRegions": [],
			"title": "Block Height",
			"tooltip": {
				"shared": true,
				"sort": 0,
				"value_type": "individual"
			},
			"type": "graph",
			"xaxis": {
				"buckets": null,
				"mode": "time",
				"name": null,
				"show": true,
				"values": []
			},
			"yaxes": [
				{
					"format": "short",
					"label": null,
					"logBase": 1,
					"max": null,
					"min": null,
					"show": true
				},
				{
					"format": "short",
					"label": null,
					"logBase": 1,
					"max": null,
					"min": null,
					"show": true
				}
			],
			"yaxis": {
				"align": false,
				"alignLevel": null
			}
		}
	],
	"schemaVersion": 26,
	"style": "dark",
	"tags": [],
	"templating": {
		"list": []
	},
	"time": {
		"from": "now-6h",
		"to": "now"
	},
	"timepicker": {},
	"timezone": "",
	"title": "Paxeer Node Metrics",
	"uid": "paxeer_metrics",
	"version": 1
}
```

</Accordion>

### Alert Management

Install Alertmanager:

```bash
wget https://github.com/prometheus/alertmanager/releases/download/v0.25.0/alertmanager-0.25.0.linux-amd64.tar.gz
tar xvf alertmanager-0.25.0.linux-amd64.tar.gz
```

<Accordion title="Create Alert Rules Configuration">

```yaml
groups:
  - name: validator_alerts
    rules:
      - alert: NodeDown
        expr: up == 0
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: 'Node {{ $labels.instance }} down'

      - alert: BlockProductionSlow
        expr: rate(tendermint_consensus_height[5m]) < 0.1
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: 'Block production is slow on {{ $labels.instance }}'
      - alert: ValidatorMissedBlocks
        expr: increase(tendermint_consensus_validator_missed_blocks[1h]) > 0
        labels:
          severity: critical
        annotations:
          summary: 'Validator missing blocks'

      - alert: ValidatorJailed
        expr: tendermint_consensus_validator_status == 0
        labels:
          severity: critical
        annotations:
          summary: 'Validator has been jailed'

      - alert: ConsensusStalled
        expr: tendermint_consensus_height_status == 0
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: 'Consensus has stalled'
```

</Accordion>

## Log Management

### Loki Setup

Using Loki for log aggregation:

```bash
wget https://github.com/grafana/loki/releases/download/v2.8.0/loki-linux-amd64.zip
unzip loki-linux-amd64.zip
```

<Accordion title="Promtail Configuration">

```yaml
server:
  http_listen_port: 9080

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://localhost:3100/loki/api/v1/push

scrape_configs:
  - job_name: paxeer_logs
    static_configs:
      - targets:
          - localhost
        labels:
          job: paxd_logs
          __path__: /var/log/paxd/*.log
```

</Accordion>

### Log Rotation

Configure logrotate to manage log files:

```bash
sudo tee /etc/logrotate.d/paxeer << EOF
/var/log/paxeer/*.log {
    daily
    rotate 14
    compress
    delaycompress
    notifempty
    create 0640 paxeer paxeer
    sharedscripts
    postrotate
        systemctl reload paxd
    endscript
}
EOF
```

## Security Configuration

### Network Security

UFW firewall configuration:

```bash
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 26656/tcp comment 'Paxeer P2P'
sudo ufw allow 26657/tcp comment 'Paxeer RPC'
sudo ufw allow 9090/tcp comment 'Paxeer gRPC'
sudo ufw enable
```

### Rate Limiting

<Accordion title="Example Nginx Configuration with Rate Limiting">

```nginx
http {
    limit_req_zone $binary_remote_addr zone=paxeer_rpc:10m rate=10r/s;

    server {
        listen 26657;
        location / {
            limit_req zone=paxeer_rpc burst=20 nodelay;
            proxy_pass http://localhost:26657;
        }
    }
}
```

</Accordion>

## Validator-Specific Monitoring

### Status Query

Query validator status through SDK:

```bash
paxd query staking validator $(paxd keys show --bech val -a <validator_keyfile_name>)
```

Query through REST API:

```sh
curl -s "http://localhost:1317/cosmos/staking/v1beta1/validators/<valoper_address>"
```

<Accordion title='Validator "Status" Query Script'>

```sh
#!/bin/bash

MONIKER="$1"
API_URL="http://localhost:1317/cosmos/staking/v1beta1/validators?pagination.limit=500"

echo "Querying validators from $API_URL..."

VALIDATOR_DATA=$(curl -s "$API_URL" | jq -c --arg MONIKER "$MONIKER" '.validators[] | select(.description.moniker == $MONIKER)')

if [[ -z "$VALIDATOR_DATA" ]]; then
    echo "No validator found with moniker: $MONIKER"
    exit 1
fi

echo "Validator details:"
echo "$VALIDATOR_DATA" | jq '.'
```

</Accordion>

### Critical Metrics

Monitor these validator-specific metrics:

```bash
# Check signing status
paxd query slashing signing-info $(paxd tendermint show-validator)

# Check current delegations
paxd query staking delegations-to $(paxd keys show -a $VALIDATOR_KEY)
```

## Backup Management

<Accordion title="Complete Automated Backup Script">

```bash
#!/bin/bash
BACKUP_DIR="/backup/paxeer"
DATE=$(date +%Y%m%d)
NODE_HOME="/root/.paxeer"

# Create backup directory
mkdir -p $BACKUP_DIR

# Stop service
systemctl stop paxd

# Backup configuration
tar czf $BACKUP_DIR/paxeer-config-$DATE.tar.gz $NODE_HOME/config

# Backup data directory
tar czf $BACKUP_DIR/paxeer-data-$DATE.tar.gz $NODE_HOME/data

# Backup key files
tar czf $BACKUP_DIR/paxeer-keys-$DATE.tar.gz $NODE_HOME/keyring-file

# Start service
systemctl start paxd

# Remove backups older than 7 days
find $BACKUP_DIR -type f -mtime +7 -name '*.tar.gz' -delete

# Log backup completion
echo "Backup completed successfully on $(date)" >> $BACKUP_DIR/backup.log
```

</Accordion>

## Host System Monitoring

### Resource Usage Tracking

Install and configure node_exporter:

```bash
wget https://github.com/prometheus/node_exporter/releases/download/v1.5.0/node_exporter-1.5.0.linux-amd64.tar.gz
tar xvf node_exporter-1.5.0.linux-amd64.tar.gz
```

Add to Prometheus configuration:

```yaml
scrape_configs:
  - job_name: 'node'
    static_configs:
      - targets: ['localhost:9100']
```

For specific customizations or additional metrics, consult the Paxeer
technical communities.