I personally run a ProxMox cluster. I run both Windows and Linux servers. I perform local full-VM backups using the hypervisor to a USB disk. That gives me a fast way to restore VMs if I need to. I also run Veeam, which handles the offsite copy and provides granular file restores. It’s nice because the community edition supports hardened disk immutability, which can help prevent ransomware attacks and Unfortunate Incidents. That just runs over SSH, and installs a Veeam agent/repo on the remote linux box.
If you are all linux, perhaps you’d like Borg. It’s pretty easy to set up, and perhaps it offers the de-duplication you crave. It’s also got some GUIs you can run, if you are into that kind of thing. If you don’t feel inclined to use any backup software, then using rsync over ssh (scp, yes) instead of SFTP is a solid way to go.