# Installing Prometheus - Alertmanager on Ubuntu

To send and manage alerts, we will set up Alertmanager. Alertmanager is responsible for deduplicating, grouping, and routing alerts to various receivers, such as email, PagerDuty, or Slack. Multiple instances of Alertmanager can be configured for high availability, but for this demonstration, we will install a single instance.

1. Start by creating a system user for Alertmanager:
    

```shell
sudo useradd \
    --system \
    --no-create-home \
    --shell /bin/false alertmanager
```

1. Download Alertmanager from the [Prometheus](https://prometheus.io/download/#alertmanager) downloads page:
    

```shell
wget https://github.com/prometheus/alertmanager/releases/download/v0.26.0/alertmanager-0.26.0.linux-amd64.tar.gz
```

1. Extract the Alertmanager binary:
    

```shell
tar -xvf alertmanager-0.26.0.linux-amd64.tar.gz
```

1. Alertmanager requires storage for storing notification states and silences. Create the necessary directories:
    

```shell
sudo mkdir -p /alertmanager-data /etc/alertmanager
```

1. Move the Alertmanager binary to the local bin and copy a sample configuration:
    

```shell
sudo mv alertmanager-0.26.0.linux-amd64/alertmanager /usr/local/bin/
sudo mv alertmanager-0.26.0.linux-amd64/alertmanager.yml /etc/alertmanager/
```

1. Remove the downloaded archive and folder:
    

```shell
rm -rf alertmanager*
```

1. Verify that Alertmanager is executable:
    

```shell
alertmanager --version
```

You can also get help and view all supported configuration options by running:

```shell
alertmanager --help
```

1. Create a systemd service definition for Alertmanager:
    

```shell
sudo vim /etc/systemd/system/alertmanager.service
```

Here's an example of the "alertmanager.service" file:

```shell
[Unit]
Description=Alertmanager
Wants=network-online.target
After=network-online.target

StartLimitIntervalSec=500
StartLimitBurst=5

[Service]
User=alertmanager
Group=alertmanager
Type=simple
Restart=on-failure
RestartSec=5s
ExecStart=/usr/local/bin/alertmanager \
  --storage.path=/alertmanager-data \
  --config.file=/etc/alertmanager/alertmanager.yml

[Install]
WantedBy=multi-user.target
```

1. Enable the Alertmanager service:
    

```shell
sudo systemctl enable alertmanager
```

1. Start Alertmanager:
    

```shell
sudo systemctl start alertmanager
```

1. Check the status of Alertmanager:
    

```shell
sudo systemctl status alertmanager
```

Alertmanager will be accessible on port 9093 at http://:9093.

1. Now, let's create a simple alert. In most Prometheus setups, there's always an active alert used to validate the monitoring system itself. It's a crucial alert to ensure the monitoring system is functioning correctly.
    

Create an alert rule file for this dead man's switch alert:

```shell
sudo vim /etc/prometheus/dead-mans-snitch-rule.yml
```

Here's an example of the "dead-mans-snitch-rule.yml" file:

```yaml
---
groups:
- name: dead-mans-snitch
  rules:
  - alert: DeadMansSnitch
    annotations:
      message: This alert is integrated with DeadMansSnitch.
    expr: vector(1)
```

You also need to update the Prometheus configuration to specify the location of Alertmanager and the path to the new alert rule:

```shell
sudo vim /etc/prometheus/prometheus.yml
```

Here's an example of the "prometheus.yml" file:

```yaml
...
alerting:
  alertmanagers:
    - static_configs:
        - targets:
          - localhost:9093
rule_files:
  - dead-mans-snitch-rule.yml
```

1. Before restarting Prometheus, validate the Prometheus configuration:
    

```shell
promtool check config /etc/prometheus/prometheus.yml
```

1. Restart Prometheus:
    

```shell
sudo systemctl restart prometheus
```

1. Check the status of Prometheus:
    

```shell
sudo systemctl status prometheus
```

Now, you have Alertmanager installed and configured to manage alerts, and a basic alert rule has been set up for monitoring the monitoring system itself.