The server my clients' sites run on
The Linux server behind my hosting, and the day it took my own shop offline for 14 hours.
What it is
My hosting runs on a Linux server in Australia that I set up and look after myself. It serves several client websites, some WordPress and some Node, plus Print Bench. When you buy hosting from me, this is the machine your site lives on, and I’m the one who patches it.
How it’s set up
- Locked doors. It accepts logins by key only, with no passwords and no direct admin login. A firewall leaves open only what the websites need.
- Updates. Security updates install themselves. I check the bigger upgrades by hand.
- PHP kept in its lane. A security profile limits which folders the PHP sites can write to: uploads and caches, not the site’s own code. A hacked plugin has a much harder time planting anything that sticks.
- Backups. The whole server is snapshotted by the hosting provider. Print Bench also gets its own nightly backup, copied to two machines at home, with an alert to my phone if it’s late.
The day it went wrong
I’d rather show you this than pretend it never happens.
On 12 November, a maintenance task installed a package to get one small security profile. The package quietly brought twenty others with it. One was a profile for PHP, and it switched on straight away. Nothing broke, because PHP was already running and didn’t pick it up.
Three days later, an automatic system update restarted PHP. It came back under the new profile, which wasn’t allowed to read the websites’ own files. From about 7am every PHP site on the server answered “Access denied”, including my own shop.
I found it that evening and had it fixed by 9:30pm. I wrote rules that let PHP read what it needs and write only where it should. I kept the profile switched on, because it’s genuinely useful once it’s configured, and switched off the eighteen others that had no reason to be there. I then checked every site and confirmed PHP wasn’t restarting any more.
What I changed because of it
- I check what a package brings with it before installing it, not just the one thing I wanted.
- The rules now sit in a file that package upgrades don’t touch, so the same thing can’t come back on the next update.
- Fourteen hours is too long to not know. Outside monitoring that alerts me when a site stops answering is on my list for exactly this reason.