Root Cause Analysis:
Recent database optimization changes intended to improve performance inadvertently introduced adverse effects, resulting in degraded system stability and availability. Upon identifying this issue, we promptly reverted these changes, restoring the system to a stable state.
Steps to be taken:
Incremental Rollout of Configuration Changes:
Adopt a phased or incremental rollout strategy for significant configuration changes to limit potential impact and allow for early identification and mitigation of issues.
Enhanced Proactive Monitoring and Alerting:
Improve our database monitoring capabilities and adjust alerting thresholds to proactively detect and respond quickly to performance degradation or instability.
Posted Mar 25, 2025 - 09:05 CDT
Resolved
This incident has been resolved.
Posted Mar 21, 2025 - 11:26 CDT
Monitoring
System has been restored and we are closely monitoring the implemented solution
Posted Mar 21, 2025 - 09:43 CDT
Identified
Our engineers are working to roll back the changes related to system maintenance that was performed overnight to return the system to the previous state and resolve the current issues. This will take approximately 30 minutes. Updates will be posted as available
Posted Mar 21, 2025 - 09:07 CDT
Update
We continue to investigate the cause of this issue
Posted Mar 21, 2025 - 08:57 CDT
Update
We are now experiencing a complete outage. All engineers are working to restore as quickly as possible
Posted Mar 21, 2025 - 08:25 CDT
Investigating
Our engineering team is actively investigating reports of lag when using RevolutionEHR. We will post updates as soon as they are available
Posted Mar 21, 2025 - 08:10 CDT
This incident affected: RevolutionEHR United States.