My setup is smaller, but when my venerable old router died about a year ago, I acquired an Asus TUF-AX3000_V2 where I installed FreshTomato. One can login via SSH and dump all settings for backup. Likewise, individual or all settings can be done on the command line instead of the GUI. I have a script on my computer that reads CSV files with MAC addresses and more to apply changes in an automated way.
Yes, XMPP with proper TLS on the server side and Conversations or one of its forks (preferably fetched from F-Droid) using OMEMO encryption should be good enough. If you are brave or paranoid, give Tox a try: https://tox.chat/
Maybe the first question is what your budget is, both regarding money and time. For example, you could buy a pre-configured NAS from Synology or QNAP, which requires less technical skills but more money, or a home-made solution reusing used components (but fresh disks for reliability). Depending on your electricity costs, you may want to choose a low-power solution or something which you power off when not used. For storage, maybe a three-disk RAID5 is a good compromise. For backups, plain S3 cloud storage encrypted via restic is a good idea.
Well, you have Finland in the north-east, Ireland in the north-west, and every land border faces a Euro-zone country. Few other countries can claim the latter.
Many projects accept donations, for example for server costs or travel expenses (conferences, meetings). You can setup recurring monthly transfers to projects whose software you use most often. Examples are the Free Software Foundation for various GNU tools or the KDE project.
#Peertube got already mentioned, but just serving video files may already suffice. Modern webbrowsers are capable of playing videos. Some tweaking of parameters may be necessary when encoding them. Also, no frills such as dynamic adoption of bitrate/quality or high-level stuff like commenting, likes, or subtitles.
If you do not trust Tailscale as a company, here is an open source re-implementation of the server called headscale. Some/all clients are open source as well. So, you can review all components yourself or pay for a professional third-party review. Otherwise, if you take a binary blob from any origin, including Tailscale, and have it run with privileges on your server, there are few limits on what this blob can do. Yes, backdoors are technically possible, but probably bad for Tailscale’s business if that ever came to light.