RetroFed Forum
Your home away from modern life.
How do you monitor your servers?
Post Reply
AuthorMessage
InFerNo lemmy.ml InFerNo@lemmy.ml
Joined: 2023-06-07
#52 · Quote Reply

I plan to perhaps, just maybe, I am not sure, to try self-hosting an email server. But I'll need a good uptime. Although I am really scared about security.

Been running one for many years, mostly issue-free. Had to request whitelists way in the beginning, but things turned out ok.

Check out this super great guide. https://workaround.org/ispmail-trixie

4
moldy_rice piefed.keyboardvagabond.com moldy_rice@piefed.keyboardvagabond.com
Joined: 2026-08-07
#53 · Quote Reply

I go the easy route with munin and wazuh. It comes with a ton of stuff preconfigured

3
CompactFlax discuss.tchncs.de CompactFlax@discuss.tchncs.de
Joined: 2025-02-14
#54 · (edited) Quote Reply

And the best time to notice that the backups weren't working is when you need access to the backup. šŸ˜…

Backups? In this economy?

But I need to get something working. I am about to move to VaultWarden and that makes me very nervous.

5
moldy_rice piefed.keyboardvagabond.com moldy_rice@piefed.keyboardvagabond.com
Joined: 2026-08-07
#55 · Quote Reply

I use FreeBSD so I don't need to check it to make sure it's working properly

Wasn't there an RCE recently discovered with AI?

1
Carrots quokk.au Carrots@quokk.au
Joined: 2026-09-30
#56 · Quote Reply

Kernel panic?

My kernel is chill.

3
Carrots quokk.au Carrots@quokk.au
Joined: 2026-09-30
#57 · Quote Reply

I recently found out that this is not always true.

I never said anything about it responding.

5
Rioting Pacifist lemmy.world RIotingPacifist@lemmy.world
Joined: 2023-06-22
#58 · Quote Reply

Monit, simple to setup, gets the job done, runs on my router too.

4
Dultas lemmy.world Dultas@lemmy.world
Joined: 2023-08-03
#59 · Quote Reply

Well, it’s a good learning experience. But….. don’t. Good luck getting around the major email providers’ smart screen shit. It’s probably not gonna happen. I tried a few years ago, and even microslop couldn’t figure out why their shit screen crap wouldn’t allow my emails. It’s not in their best interest to fix it. So they don’t. But it’s worth doing it in the short term to learn how to do it. T...

Yeah, email is one of the more annoying things to self host. Google rejects mine because my ISP has all their non static IPs on block lists (even though my IP hasn't changed once since I've been with them it's still not "static", can't change the reverse DNS) and I'm not paying 150% more a month for a business account to get a static IP. I ended up just using a free email relay. Still had to setup SPF, DKIM, DMARC, and all that jazz still.

4
šŸ‡° šŸŒ€ šŸ‡± šŸ‡¦ šŸ‡³ šŸ‡¦ šŸ‡° šŸ‡® pawb.social Kolanaki@pawb.social
Joined: 2024-10-06
#60 · Quote Reply

I just look at the console.

2
shadshack piefed.world shadshack@piefed.world
Joined: 2026-10-07
#61 · Quote Reply

Uptime Kuma for monitoring web service status (just like if the jellyfin, etc. website is up), and Zabbix for monitoring system metrics like CPU/Memory/Disk I/O, etc. Zabbix can do a whole lot, but can also get a little complicated. There are plenty of guides out there for configuring it though.

0
Lka1988 lemmy.dbzer0.com lka1988@lemmy.dbzer0.com
Joined: 2024-11-18
#62 · Quote Reply

My 3-node Proxmox cluster has a dashboard, my NAS has a dashboard, and Home Assistant has a dashboard. Hell, my gaming PC has a dashboard, too (Webmin and Cockpit).

1
lIlIlIlIlIlIl lemmy.world lIlIlIlIlIlIl@lemmy.world
Joined: 2025-03-31
#63 · Quote Reply

Can you teach me this wizardry?

Sure! In Uptime Kuma, add a monitor with type Push. It gives you a URL with a unique token, and the monitor goes into ā€œdownā€ state if nothing calls that URL within the heartbeat interval. Then have a cron (or a systemd timer or whatever) run this on the machine with the drive every 5 or 10 minutes:

use=$(df --output=pcent /mnt/yourdrive | tail -1 | tr -dc ā€˜0-9’)
curl -fsS -G ā€œhttp://your-kuma:3001/api/push/TOKEN --data-urlencode ā€œstatus=upā€ --data-urlencode ā€œmsg=${use}% usedā€ --data-urlencode ā€œping=${use}ā€

Ping is of course meant to be response time, but Kuma will graph any number you toss in, so you get a ā€œfilledā€ chart for the drive. I also send status=down once usage passes a threshold. And if the box itself should fall over for some reason, the missing check-ins trigger the alert too.

(I typed this quick on my phone, so please double-check and probably don’t blindly copy-paste)

5
Lka1988 lemmy.dbzer0.com lka1988@lemmy.dbzer0.com
Joined: 2024-11-18
#64 · Quote Reply

Does this monitoring system also happen to be a dependent on your taxes?

I've got a few of those services running.

4
Carol2852 discuss.tchncs.de Carol2852@discuss.tchncs.de
Joined: 2023-06-27
#65 · Quote Reply

Backups? In this economy? But I need to get something working. I am about to move to VaultWarden and that makes me very nervous.

I run a vaultwarden container since forever and it's really smooth. All updates went through just fine, never had an issue with it.

3
hsedr lemmy.zip hsedr@lemmy.zip
Joined: 2026-05-16
#66 · Quote Reply

I am using prometheus-nodeexporter which is scraped by Prometheus and then visualized in Grafana

1
blackshirt lemmy.world blackshirt@lemmy.world
Joined: 2026-06-05
#67 · (edited) Quote Reply

Ultime Kuma, Beszel and Ntfy

2
WeirdGoesPro lemmy.dbzer0.com WeirdGoesPro@lemmy.dbzer0.com
Joined: 2023-08-15
#68 · Quote Reply

Sure! In Uptime Kuma, add a monitor with type Push. It gives you a URL with a unique token, and the monitor goes into ā€œdownā€ state if nothing calls that URL within the heartbeat interval. Then have a cron (or a systemd timer or whatever) run this on the machine with the drive every 5 or 10 minutes: use=$(df --output=pcent /mnt/yourdrive | tail -1 | tr -dc ā€˜0-9’) curl -fsS -G ā€œhttp://your-kuma:3...

Thank you! This is dope!

2
Alfredolin lemmy.zip Alfredolin@lemmy.zip
Joined: 2026-07-28
#69 · Quote Reply

I run a vaultwarden container since forever and it's really smooth. All updates went through just fine, never had an issue with it.

Same. And I was also nervous in the beginning, but in the end if you are the only user, you'll have your vault replicated everywhere you have the app.

1
Alfredolin lemmy.zip Alfredolin@lemmy.zip
Joined: 2026-07-28
#70 · Quote Reply

My primary monitoring system is Beszel, but since I just migrated from VMs to LXCs it can’t properly see the memory/CPU usage, so it just checks for disk usage and liveliness. Home assistant with Proxmox & Dockhand integrations gives me the rest of my monitoring needs.

I've been checking several other solutions before and Beszel just hits the spot.

1
communism lemmy.ml communism@lemmy.ml
Joined: 2024-03-19
#71 · Quote Reply

I plan to perhaps, just maybe, I am not sure, to try self-hosting an email server. But I'll need a good uptime. Although I am really scared about security.

I self-host my own email, have done so for years. I've not gotten pwned. Use an SSH key and a firewall, only expose the ports that should be public, have reasonable access controls, etc, just practise general cybersecurity—it applies to servers too.

I know everyone complains about hosting email. But I've not run into major problems. My emails get through spam filters and I have good uptime. It's also a fun learning experience.

6
TrollTrollrolllol lemmy.world TrollTrollrolllol@lemmy.world
Joined: 2023-07-01
#72 · Quote Reply

"can't access this thing, server must be down"

I'll fix it when I get home

18
Tealk rollenspiel.forum Tealk@rollenspiel.forum
Joined: 2023-08-08
#73 · Quote Reply

Icinga and Phare Uptime

2
NotEasyBeingGreen slrpnk.net NotEasyBeingGreen@slrpnk.net
Joined: 2026-03-30
#74 · Quote Reply

I self-host my own email, have done so for years. I've not gotten pwned. Use an SSH key and a firewall, only expose the ports that should be public, have reasonable access controls, etc, just practise general cybersecurity—it applies to servers too. I know everyone complains about hosting email. But I've not run into major problems. My emails get through spam filters and I have good uptime. It'...

I think that you have to do quite a bit more setup on email than most other services. There's SPF and DKIM and DMARC and so on. Plus spam filtering, setting up accounts and authentication, ...

Plus email is arguably more important than anything else, since it's kind of the lowest common denominator of the Internet; your dentist will probably send you an email to remind you of appointments, job offers will arrive via email, password resets arrive via email, and so on.

So it is a bit scary.

Having said that, as long as you're not hosting from a residential IP addresses it is definitely achievable. You can start with one domain and use it for low-value purposes (or for accounts that allow multiple email addresses, like GitHub).

6
SayCyberOnceMore feddit.uk Cyber@feddit.uk
Joined: 2023-06-17
#75 · Quote Reply

Monit, simple to setup, gets the job done, runs on my router too.

I have this on my NAS, works well

1
bertlewirth lemmy.world bertlewirth@lemmy.world
Joined: 2026-09-15
#76 · (edited) Quote Reply

Yeah, email is one of the more annoying things to self host. Google rejects mine because my ISP has all their non static IPs on block lists (even though my IP hasn't changed once since I've been with them it's still not "static", can't change the reverse DNS) and I'm not paying 150% more a month for a business account to get a static IP. I ended up just using a free email relay. Still had to se...

Yup. It’s so damn annoying hosting email. Do it once to learn, but don’t do it as a permanent solution. It takes so much maintenance to keep it going, if you even do get it working.

1
electric_nan lemmy.ml electric_nan@lemmy.ml
Joined: 2022-01-15
#77 · Quote Reply

And the best time to notice that the backups weren't working is when you need access to the backup. šŸ˜…

Backups are pretty much the only thing I get notifications for. Otherwise, one app or another on my phone will complain if it loses connection to my server.

2
Foofighter discuss.tchncs.de Foofighter@discuss.tchncs.de
Joined: 2023-08-04
#78 · Quote Reply

Use service, wait for 404 or data loss...

9
dimjim sh.itjust.works dimjim@sh.itjust.works
Joined: 2023-10-15
#79 · Quote Reply

I've been checking several other solutions before and Beszel just hits the spot.

If you run Proxmox and Proxmox Backup Server, I actually JUST found a better product for my needs, Pulse

My only problem with Beszel is that it doesn’t know about LXCs within Proxmox, so I can’t see if there are CPU/memory issues happening within them. Pulse integrates directly and can see all of it!

If you don’t run LXCs within Proxmox, I’d say stick with Beszel.

1
Samsy lemmy.ml Samsy@lemmy.ml
Joined: 2021-02-05
#80 · Quote Reply

Beszel & Pangolin.

1
motruck lemmy.zip motruck@lemmy.zip
Joined: 2025-08-23
#81 · Quote Reply

Gatus. Works and easy incremental set up.

2
timochka lemmy.zip timochka@lemmy.zip
Joined: 2026-05-22
#82 · Quote Reply

I use Signoz and OTel collectors to bring in everything from Kubernetes (and also SNMP-to-OTel for the network and NAS stats.)

That covers dashboards and basic alerting. Then on top of that I have an n8n workflow that runs every morning - it pulls all the main stats out of Signoz, as well as issues and their status from my infrastructure repo in Gitlab, and feeds it all into a local LLM (Qwen-3.8-Flash-Next at the moment), which emails me a report on overall cluster health. It also automatically opens a ticket in Gitlab for anything that might need fixing.

It works really nicely, gotta say.

3
InFerNo lemmy.ml InFerNo@lemmy.ml
Joined: 2023-06-07
#83 · Quote Reply

Yup. It’s so damn annoying hosting email. Do it once to learn, but don’t do it as a permanent solution. It takes so much maintenance to keep it going, if you even do get it working.

What maintenance do you mean? run OS updates and pay the vps bill. that's true for any self hosted application. which specific maintenance are you referring to?

1
frongt lemmy.zip frongt@lemmy.zip
Joined: 2025-07-31
#84 · Quote Reply

What maintenance do you mean? run OS updates and pay the vps bill. that's true for any self hosted application. which specific maintenance are you referring to?

Setting up the newfangled security thing. TLS, DKIM, DMARC, SPF are the big ones over the years. Finding out you've been silently blackisted by a provider and hoping and praying they'll answer your request to unblock. Managing your own spam filter.

1
Vendetta9076 sh.itjust.works Vendetta9076@sh.itjust.works
Joined: 2023-06-08
#85 · Quote Reply

Backups? In this economy? But I need to get something working. I am about to move to VaultWarden and that makes me very nervous.

Don't take this as a reason to not do backups, but vaultwarden is essentially its own back up as it replicates the vault everywhere you use the app. I've recovered my vault several times from my phone.

3
DeathByDenim lemmy.world DeathByDenim@lemmy.world
Joined: 2023-08-24
#86 · Quote Reply

Backups are pretty much the only thing I get notifications for. Otherwise, one app or another on my phone will complain if it loses connection to my server.

That and degraded RAID!

1
ryannathans aussie.zone ryannathans@aussie.zone
Joined: 2023-09-20
#87 · Quote Reply

Wasn't there an RCE recently discovered with AI?

You'd need to be running kerberised NFS in the kernel, give the attacker access to your NFS server, and have a valid kerberos login

I don't know anyone sharing NFS access let alone opening it up on the open internet

2
communism lemmy.ml communism@lemmy.ml
Joined: 2024-03-19
#88 · Quote Reply

I think that you have to do quite a bit more setup on email than most other services. There's SPF and DKIM and DMARC and so on. Plus spam filtering, setting up accounts and authentication, ... Plus email is arguably more important than anything else, since it's kind of the lowest common denominator of the Internet; your dentist will probably send you an email to remind you of appointments, job ...

Yes, it can be a bit involved. I'd say I can spin up a mail server in 1–2 days though provided the OS is already installed and I have a domain name and port 25 isn't blocked (that's the main one that tends to be blocked). It may take longer for people who aren't familiar with mail servers but it's not some huge operation.

You can just try spinning up a mail server for fun. If it goes badly then just don't move over your dentist appointments to it. It's only critical infra if you use it for critical things, which you probably shouldn't till it's stable.

2
CompactFlax discuss.tchncs.de CompactFlax@discuss.tchncs.de
Joined: 2025-02-14
#89 · Quote Reply

Don't take this as a reason to not do backups, but vaultwarden is essentially its own back up as it replicates the vault everywhere you use the app. I've recovered my vault several times from my phone.

I’m just using Dropbox now and replicating my 1Password everywhere and for some reason it makes me nervous to self host.

I think I’m just going to back up everyone’s vault to a USB and bring it to a second location twice a year. If storage ever gets affordable, I’ll send replication snapshots instead.

1
modus lemmy.world modus@lemmy.world
Joined: 2023-06-12
#90 · Quote Reply

You guys are monitoring your servers?

Seriously. I just kinda check it out once in a while.

1
Joelk111 lemmy.world Joelk111@lemmy.world
Joined: 2023-07-01
#91 · Quote Reply

They're in my attached garage. In the living room when it's quiet I can hear them humming. Does that count?

6
kaotic lemmy.world kaotic@lemmy.world
Joined: 2023-06-12
#92 · Quote Reply

Prometheus & Grafana

1
teuto lemmy.teuto.icu teuto@lemmy.teuto.icu
Joined: 2023-07-23
#93 · Quote Reply

They're in my attached garage. In the living room when it's quiet I can hear them humming. Does that count?

"Oh the fans ramped up, something must be going down"

3
TheHive feddit.org luftruessel@feddit.org
Joined: 2025-03-09
#94 · Quote Reply

I have a basic dashboard set up in my home assistant that I never check. Don't even know how it works, I forgot. That's why I really want to just put my whole setup in code. That's why nothing is happening, the chore just seems so daunting.

So I listen for complains ringing through the house and react accordingly.

3
Magiilaro feddit.org DarkMetatron@feddit.org
Joined: 2024-06-24
#95 · Quote Reply

I have the best detector for issues with my self hosted services: my wife!

7
mr_jay lemmy.zip mr_jay@lemmy.zip
Joined: 2026-10-03
#96 · Quote Reply

I run UptimeKuma for monitoring a couple of things, but honestly, I’m pretty much my only user and my homelab is independent of the regular home network, so I don’t really care too much. I’ve been considering some simple log monitoring… when I get around to it :-)

2
2x4 piefed.social 2x4@piefed.social
Joined: 2026-02-01
#97 · Quote Reply

Beszel, uptime kuma, wazuh

1
trilobite lemmy.ml trilobite@lemmy.ml
Joined: 2024-02-17
#98 · Quote Reply

I have the best detector for issues with my self hosted services: my wife!

Thats cool! My wife thinks i'm crazy just because i homeserve ...

2
trilobite lemmy.ml trilobite@lemmy.ml
Joined: 2024-02-17
#99 · Quote Reply

Uptime Kuma, only really care if it's up or not. Got about 30 services monitored, sending notifications to Signal.

Ah cool! How do you get notifications sent to signal. I've always avoided because i didnt like emails of my home setup leaving home in clear

2
trilobite lemmy.ml trilobite@lemmy.ml
Joined: 2024-02-17
#100 · Quote Reply

Ah cool! How do you get notifications sent to signal. I've always avoided because i didnt like emails of my home setup leaving home in clear

Edit: ah, i see! It's kuma that allows you to i guess by adding new device on phone Signal

1
CameronDev programming.dev CameronDev@programming.dev
Joined: 2023-06-18
#101 · Quote Reply

Ah cool! How do you get notifications sent to signal. I've always avoided because i didnt like emails of my home setup leaving home in clear

https://github.com/bbernhard/signal-cli-rest-api

Its very finicky to setup, but once you get it working its pretty good. Sometimes needs to be reauthenticated though.

Kuma notification settings:

1
Post Reply Page 2 of 3   Goto page « First 1, 2, 3 Last »
Back to top