Self Hosting Journey: The Server Talks Back

One of the most annoying things about running a home server is not knowing what's happening when you're not looking at it. Did it reboot? Did someone try to break in? Did the VPN go down? Is Nginx still running? For months, the only way to find out was to SSH in and check manually. Sometimes I'd discover the server had been offline for days and I had no idea.

So I built a notification system. Now my server sends me Telegram messages whenever something important happens. It's like having a tiny sysadmin in my pocket.

The Foundation: A Telegram Bot

Telegram bots are free, instant, and dead simple to set up. Created one through @BotFather on Telegram — gave it a name, got an API token, done. Then I grabbed my Chat ID by sending a message to the bot and checking the API:

curl -s https://api.telegram.org/bot[TOKEN]/getUpdates | python3 -m json.tool

The Chat ID is in the response under message.chat.id. With these two pieces, I can send myself messages from any bash script.

The Core Script

Created a simple notification script at /usr/local/bin/notify.sh that everything else calls:

#!/bin/bash

TOKEN="[BOT_TOKEN]"
CHAT_ID="[CHAT_ID]"
MESSAGE="$1"

curl -s -X POST "https://api.telegram.org/bot${TOKEN}/sendMessage" \
    -d chat_id="${CHAT_ID}" \
    -d text="${MESSAGE}" \
    -d parse_mode="Markdown" > /dev/null

Every notification on the system goes through this one script. Want to send a message? Just call /usr/local/bin/notify.sh "your message here". Simple.

What Gets Monitored

🔄 Server Reboots

Created a systemd service that runs once on boot and sends a notification:

[Unit]
Description=Notify on boot
After=network-online.target
Wants=network-online.target

[Service]
Type=oneshot
ExecStart=/usr/local/bin/notify.sh "🔄 Server just booted up"

[Install]
WantedBy=multi-user.target

Now every time the server restarts — whether planned or not — I immediately know about it. No more discovering a week later that it's been offline.

🔑 SSH Logins

This one uses PAM (Pluggable Authentication Modules) to trigger a notification on every SSH login. Created a script at /usr/local/bin/notify-ssh.sh:

#!/bin/bash
if [ "$PAM_TYPE" = "open_session" ]; then
    /usr/local/bin/notify.sh "🔑 SSH login: *${PAM_USER}* from *${PAM_RHOST}*"
fi

Added it to /etc/pam.d/sshd:

session optional pam_exec.so /usr/local/bin/notify-ssh.sh

Now I get a Telegram message every time someone logs in via SSH, with the username and the IP they connected from. Since SSH is only accessible through the VPN or local network, any login notification is expected — and if I ever get one I don't recognize, I know something is very wrong.

🌐 IP Changes

Added a single line to my Cloudflare DDNS script. Whenever it detects a new public IP and updates the DNS records, it also pings me:

/usr/local/bin/notify.sh "🌐 Public IP changed to *${IP}*"

This was especially useful after an incident where my ISP changed my IP and the server was unreachable for hours because the DDNS script only ran once a day. (I've since changed it to every 2 hours.)

🔴 Service Failures

A cron job checks critical services every 5 minutes and alerts me if any of them are down:

#!/bin/bash
SERVICE="$1"
STATUS=$(systemctl is-active "$SERVICE")
if [ "$STATUS" != "active" ]; then
    /usr/local/bin/notify.sh "🔴 Service *${SERVICE}* is *${STATUS}*"
fi

Monitored services: nginx, openvpn-server@server, webhook. If any of them crash, I know within 5 minutes. No news is good news — the script only sends a message when something is actually broken.

🔒 VPN Connections & Disconnections

This one was the most fun to set up — and also the one that caused the most problems ahahah.

OpenVPN supports client-connect and client-disconnect hooks — scripts that run when a client connects or disconnects. I created two small scripts:

# client-connect.sh
#!/bin/bash
/usr/local/bin/notify.sh "🔒 VPN connected: *${common_name}* from *${trusted_ip}*"

# client-disconnect.sh
#!/bin/bash
DURATION=$((time_duration / 60))
/usr/local/bin/notify.sh "🔓 VPN disconnected: *${common_name}* after *${DURATION} minutes*"

Added them to the OpenVPN server config:

script-security 2
client-connect /etc/openvpn/server/client-connect.sh
client-disconnect /etc/openvpn/server/client-disconnect.sh

The Permission Nightmare

And then the VPN completely broke. Every client got AUTH_FAILED when trying to connect. The TLS handshake worked fine — certificates verified, encryption established — but then authentication failed at the last step.

After some investigation, the server logs revealed the problem:

WARNING: Failed running command (--client-connect): could not execute external program

The issue? OpenVPN drops privileges to nobody:nogroup after initialization (for security), but the scripts were owned by root with 700 permissions. The nobody user simply couldn't execute them. And when a client-connect script fails, OpenVPN treats it as an authentication failure — which is actually correct behavior, just not what I expected.

The fix was two things:

# Make the scripts owned by nobody (the user OpenVPN runs as)
sudo chown nobody:nogroup /etc/openvpn/server/client-connect.sh
sudo chown nobody:nogroup /etc/openvpn/server/client-disconnect.sh

# Make notify.sh readable by everyone (it needs to be callable by nobody)
sudo chmod 755 /usr/local/bin/notify.sh

After that, VPN connections worked again AND I got a beautiful Telegram notification: "🔒 VPN connected: mose2 from 192.168.1.1". Worth the debugging.

🚫 fail2ban Bans

Created a custom fail2ban action at /etc/fail2ban/action.d/telegram.conf:

[Definition]
actionban = /usr/local/bin/notify.sh "🚫 fail2ban: Banned *<ip>* from *<name>* for <bantime> seconds"
actionunban = /usr/local/bin/notify.sh "✅ fail2ban: Unbanned *<ip>* from *<name>*"

Now whenever someone tries to brute-force my server and gets banned, I get a notification with their IP. And when they get unbanned after the timeout, another notification. It's oddly satisfying to see these come in — like watching a bouncer at work.

⚠️ TLS Certificate Expiry

Certbot auto-renews certificates, so this should never trigger. But "should never" and "will never" are different things in the world of servers. Created a daily check that warns me if the certificate expires within 14 days:

#!/bin/bash
EXPIRY=$(sudo certbot certificates 2>/dev/null | grep "Expiry Date" | head -1 | awk '{print $3}')
DAYS_LEFT=$(( ($(date -d "$EXPIRY" +%s) - $(date +%s)) / 86400 ))

if [ "$DAYS_LEFT" -lt 14 ]; then
    /usr/local/bin/notify.sh "⚠️ TLS certificate expires in *${DAYS_LEFT} days*"
fi

A safety net for the safety net. Paranoid? Maybe. But I'd rather get a warning I don't need than lose HTTPS on my site because a cron job silently failed.

---

Living With Notifications

After a few days of running this system, here's what my typical Telegram feed looks like:

• Morning: nothing (server ran fine overnight — no news is good news)
• When I connect to VPN: "🔒 VPN connected: mose from [IP]"
• When I SSH in: "🔑 SSH login: mose from 10.8.0.2"
• When I disconnect: "🔓 VPN disconnected: mose after 45 minutes"
• Occasionally: "🚫 fail2ban: Banned [some random IP]" (always satisfying)

The system is designed to be quiet when everything is fine and loud when something needs attention. That was the goal — not a flood of messages, but a reliable alert when it matters.

The little anxiety about "is my server still running?" is basically gone now. If something goes wrong, my phone buzzes. If my phone is quiet, the server is happy. Simple.

The Blind Spot: What If the Server Is Offline?

There's one problem with this entire notification system: if the server is offline, it can't send notifications. The Telegram bot lives on the server — if the server dies, the bot dies with it. I'd have no idea until I tried to visit my website or connect to the VPN.

The solution is external monitoring — something outside my server that checks if it's alive. I set up UptimeRobot (free tier), which pings https://mosearc.eu every 5 minutes from their servers around the world. If my site doesn't respond, UptimeRobot sends me a Telegram notification. When it comes back online, another notification.

It also tracks uptime percentage over time, which is nice for bragging rights (and for the blog ahahah).

---

What's Next

• Containerization with Docker
• UPS (power outages are still the #1 enemy)
• Self-hosted drive with NAS
• Mail server
• Local LLM (the dream lives on)