I use rsync for this purpose and the only notable bottleneck is my download speed, fwiw.

I use mailfence. They offer imap, caldav, and carddav. It’ll check all those boxes, but I don’t think those are unique offerings among the privacy respecting email services.

I use Proton Mail’s Proton Calendar app for my calendar though. I was using caldav + davx5 but I had issues with reminder settings getting lost on recurring events.

My suggestion just changes your threat model, so may not be a good one based on your wants.

Perhaps consolidate systems? Managing less devices = less points of failure. But adds the risk of any given failure being more severe.