This morning, Salsa Labs experienced an unplanned service outage from 7:00 to 8:47 a.m. Eastern due to a virtual server bug which prevented access to an internally critical system.

Although we were able to begin troubleshooting the problem immediately, resolution required an interlocking sequence of systems fixes. Throughout the downtime period, Salsa pages (both public-facing pages and headquarters login pages for campaign managers) experienced a complete outage. Email blasts scheduled for that time frame were delayed, but were queued and automatically sent as soon as service was restored.

This outage has been fully resolved. Going forward, we're exploring several options for extending our existing layers of redundancy to minimize the prospects of any similar bug impacting service uptime.

We apologize for any disruption to your work. If you are still encountering issues, please contact support@salsalabs.com. Twitter users should be sure to follow @SalsaLabsOps for future real-time incident updates.