RetroFed Forums
Your home away from modern life.
For people using OpnSense who want some automatic snapshots.
Post Reply
AuthorMessage
MuttMutt lemmy.world MuttMutt@lemmy.world
Joined: 2023-06-18
(edited) Quote

So here is the deal. OpnSense is based on FreeBSD (as the users know but others may not understand) and utilizes ZFS Snapshots which are an amazing way to go back to a previous state when something breaks after an update. However for some reason the cron section has been changed so you can not add any random cron job and can only select from predefined options. This means we have to work from the CLI to add jobs, but even worse after an update the cron jobs are removed.... which becomes self defeating.

I think I have a decent solution that will work to not only persist the backup script that will automatically create and remove snapshots but also recreate the cron jobs automatically. Ideally there would be some sort of addon where we could adjust things but this is the best I can do for now and want to share with everyone.

Here is the script, place it at: /usr/local/etc/rc.syshook.d/start/99-snapshot-addon.sh
and preform a chmod +x so that it is executable. Then reboot and the script to create the snapshot automatically and the cron job should be created. Since this runs on boot it will do this check after every update that requires a reboot or reboot in general and if the script and cron jobs are removed due to an update it will recreate them.

#!/bin/sh

CAT_PATH="/usr/local/opnsense/scripts/auto_be_retention.sh"

# Ensure target directory exists
mkdir -p /usr/local/opnsense/scripts

# 1. Write/Update the auto_be_retention.sh script
cat << 'EOF' > "$CAT_PATH"
#!/bin/sh

WEEKLY_RETENTION=6
MONTHLY_RETENTION=4

TYPE=$1

if [ "$TYPE" != "weekly" ] && [ "$TYPE" != "monthly" ]; then
    echo "Usage: $0 [weekly|monthly]"
    exit 1
fi

DATE_TAG=$(date +%Y%m%d_%H%M%S)
BE_NAME="auto_${TYPE}_${DATE_TAG}"

/sbin/bectl create "${BE_NAME}"

if [ "$TYPE" = "weekly" ]; then
    KEEP=$WEEKLY_RETENTION
else
    KEEP=$MONTHLY_RETENTION
fi

/sbin/bectl list -H | awk '{print $1}' | grep "^auto_${TYPE}_" | sort | \
awk -v keep="$KEEP" '
    { lines[NR] = $1 }
    END {
        to_delete = NR - keep
        for (i = 1; i <= to_delete; i++) {
            print lines[i]
        }
    }
' | while read -r OLD_BE; do
    if [ -n "$OLD_BE" ]; then
        /sbin/bectl destroy -F -o "$OLD_BE"
    fi
done
EOF

chmod +x "$CAT_PATH"

# 2. Inject cron tasks into /etc/crontab if missing
CRON_FILE="/etc/crontab"

if ! grep -q "auto_be_retention.sh weekly" "$CRON_FILE"; then
    echo "0 3 * * 0 root /usr/local/opnsense/scripts/auto_be_retention.sh weekly" >> "$CRON_FILE"
fi

if ! grep -q "auto_be_retention.sh monthly" "$CRON_FILE"; then
    echo "30 3 1 * * root /usr/local/opnsense/scripts/auto_be_retention.sh monthly" >> "$CRON_FILE"
fi

# 3. Reload cron daemon
service cron restart

This will check for and if missing create a script at /usr/local/opnsense/scripts/auto_be_retention.sh and it will check for and if missing create two cron jobs.

The script located at /usr/local/opnsense/scripts/auto_be_retention.sh will create a snapshot with a timestamp named similar to: auto_monthly_20261001_033000

It will also look the count of previous snapshots (6 for weekly and 4 for monthly which can be adjusted if you choose) and delete the oldest one. Weekly snapshots will happen every Sunday at 3:00 AM and monthly snapshots will happen on the 1st of the month at 3:30 AM.


Why do we need this?

We don't "need" this but the advantage is that if you can have automatic updates (which are an option that is available in the cron section) it's probably better to have a snapshot that can easily be rolled back to in the case of something acting up automatically created as well. The other option is try and undo the issue or take a config backup and reinstall then use that to get everything up and running again.

Rolling back a snapshot takes a few minutes and gets everything back up and running until a solution for the issue/s are available.

I ran the script and I am not seeing anything in the Settings > cron section!!

You won't... that is part of the deal with how OpnSense has decided to handle things. You will need to look in: /etc/crontab This is because of how OpnSense is using menu drive cron selections, since the selections are not available it seems to not want to create or show them.

I have done some testing and removed the files and jobs and also had them recreated. At the next update we will see if everything sticks around or if it is removed.

Please remember that the script above is given without warranty or guarantee, use it at your own risk. I am not responsible if this script causes things not limited to but including data loss, testicular/breast shrinkage/growth, rude children, nagging wives/girlfriends/husbands/partners, alien visitation, global warming, or thermonuclear war. ID10T errors are also your own responsibility... you should not be running things you find on the internet without understanding how they work.

45   Original
zo0 programming.dev zo0@programming.dev
Joined: 2024-12-07
#2 · Quote Reply

Have you tried rrstoring to a previous snapshot after a borked update?

2
non_burglar lemmy.world non_burglar@lemmy.world
Joined: 2023-06-16
#3 · Quote Reply

  • I personally wouldn't be touching /etc/crontab, but adding or editing a tab in /var/cron/tabs.
  • I would use ~test~ or just a bash eval instead of mkdir to check for the existence of a directory. Mkdir has backfired on me spectacularly in bash and sh scripts.
  • always grep, then sort, then awk. Just as a habit, any other order can make run time long and cause globbing problems.
  • good job
7
MuttMutt lemmy.world MuttMutt@lemmy.world
Joined: 2023-06-18
#4 · Quote Reply

Have you tried rrstoring to a previous snapshot after a borked update?

On OpnSense no. I previously did not have snapshots created. I have on FreeNAS when I had a setting issue that came up. Fundamentally it should be no different.

I have had to reinstall OpnSense before and would rather not go through that again if at all possible.

1
ikidd lemmy.dbzer0.com ikidd@lemmy.dbzer0.com
Joined: 2026-04-23
#5 · Quote Reply

I run my OPNsense as a proxmox VM, so I get high availability, replication, live backup and restore via PBS and update-time snapshots.

I couldn't recommend this method more, it's saved my ass repeatedly.

2
tofu lemmy.nocturnal.garden tofu@lemmy.nocturnal.garden
Joined: 2025-03-04
#6 · Quote Reply

I run my OPNsense as a proxmox VM, so I get high availability, replication, live backup and restore via PBS and update-time snapshots. I couldn't recommend this method more, it's saved my ass repeatedly.

Depends on what you do with it I guess. It's my gateway between my family LAN and my homelab LANs, and also the DHCP server for the latter. It's supposed to be up when the proxmox cluster boots and all nodes would need 3 NICs for HA to work, and all be directly connected to the family LAN (in which they aren't supposed to be).

1
MuttMutt lemmy.world MuttMutt@lemmy.world
Joined: 2023-06-18
#7 · Quote Reply

I run my OPNsense as a proxmox VM, so I get high availability, replication, live backup and restore via PBS and update-time snapshots. I couldn't recommend this method more, it's saved my ass repeatedly.

I've never been comfortable virtualizing a router. Nearly anything else is not an issue.

5
ikidd lemmy.dbzer0.com ikidd@lemmy.dbzer0.com
Joined: 2026-04-23
#8 · Quote Reply

I've never been comfortable virtualizing a router. Nearly anything else is not an issue.

It's been amazing. The thing I like the most is being able to live-motion it to another node to upgrade the host then migrate back. Doesn't drop a ping.

I'm using some ancient Dell rackmount servers for nodes, and a couple times the prime server dropped off entirely because of power issues, and I didn't notice for a couple weeks that it had jumped hosts. I've also had to restore it from backup because of a borked upgrade, and it took maybe 5 minutes.

I thought that it would be a bad idea but I tried it with the original baremetal router that I lift and shifted to a VM then put the source on standby, and 5 years later, I would not go back.

2
ikidd lemmy.dbzer0.com ikidd@lemmy.dbzer0.com
Joined: 2026-04-23
#9 · (edited) Quote Reply

Depends on what you do with it I guess. It's my gateway between my family LAN and my homelab LANs, and also the DHCP server for the latter. It's supposed to be up when the proxmox cluster boots and all nodes would need 3 NICs for HA to work, and all be directly connected to the family LAN (in which they aren't supposed to be).

Set the boot order in PVE. You can also group HA targets to shift as a group. I have a physical nic on WAN and LAN, and the LAN nic is VLANned for DMZ, guest, IOT, etc. As long as you keep them consistent from node to node, I've never had an issue.

I have been thinking about a failover LTE modem, and that one has me a bit stumped about how to deal with HA moves. Maybe a GLiNet LTE router that I can VLAN on the WAN nic. Or maybe I just hook it into the LAN network and have failover scripts for my critical devices

2
CaptainShardmind lemmy.world CaptainShardmind@lemmy.world
Joined: 2026-10-21
#10 · Quote Reply

I'm not on OPNsense, but I lean on the same idea with my Ubuntu servers, and being able to roll back is what makes me willing to update at all. One thing I'd add: boot environments live on the same disk, so export config.xml somewhere off the box on the same schedule. And do one deliberate rollback before an update forces you to, so you know how long it really takes.

1
MuttMutt lemmy.world MuttMutt@lemmy.world
Joined: 2023-06-18
#11 · Quote Reply

I'm not on OPNsense, but I lean on the same idea with my Ubuntu servers, and being able to roll back is what makes me willing to update at all. One thing I'd add: boot environments live on the same disk, so export config.xml somewhere off the box on the same schedule. And do one deliberate rollback before an update forces you to, so you know how long it really takes.

I keep config backups of the config as well but my boot drive is a pair of mirrored Intel D3-S4510's that are underprovioned by 50GB.

Rollback time will technically depend on the size of the change since a rollback only replaces changed files so that will depend on how far back you are rolling back to. My largest snapshot references about 2GB and is a little over 4 months old, I would estimate about 5 minutes at most rolling back to that point.

1
Post Reply
Back to top