You can have a hosted e-mail system with your own domain, many service providers will do this. They would have the storage, archive, etc. on their servers. Access would be a web client, e-mail client of your choice, or a bridge demon. Option 2 is a hosted vertual server, so your self hosting but renting a VM server on a providers data center, they take care of the hardware, power, backup (sometimes depending on service), etc. You would set up your OS and e-mail services.

I use a combo of this, I have a service with my own domain and point my MX records there, I use a bridge and a sctipt to pull backups and store them locally but use their email clients for mobile and Thunderbird on PCs.

Edit: FWIW I’ve been using Proton (paid version) for this but will be moving to hosted virtual server at some point, when I get a minute to migrate. I expect the cost to be about the same

Membrane keypad would have 2 issues that would prevent it, first the membrane is most likely not heat resistant like flexable PCBs and would melt if you tried to solder to it, second the black traces on the membrane are made of powdered carbon in a binder and would not take the solder or would break apart. You could use a pressure fit to hold the contact of the switch to the traces if you where somehow able to hold the switch in place.