Followup: our apologies for the extended downtime this morning. An interrupted host upgrade caused a problem and we had to restore from backups.
The database was unaffected and there was no data loss. However, site performance is still recovering. We have closed registrations in the meantime (though users can still invite others directly).
Alright! We're back at our former performance levels! So the fix worked but it reverted the fix for our memory problems.
That's okay! We're stable for now. We can re-try the memory fix later. I'll let you know when that's coming up.
Again, a sincere thank you to everyone for their patience today. It's been a stressful day and the best part of getting the site back up has been getting your messages of support. Thank you.
A postmortem of today's downtime can be found here: https://gist.github.com/ashfurrow/c0f3f13da10c6609c3e3cf20b319dd3e
@announcements great work, I missed the maintenance notice and was saddened to see the site down. Thanks for all you do!
This Mastodon instance is for people interested in technology. Discussions aren't limited to technology, because tech folks shouldn't be limited to technology either!