Agents are no longer getting logged out or marked as away too early.
We will provide further details about today’s incident via an RCA, which we aim to compile as soon as possible.
IMPORTANT NOTE: Today's hotfix will impact Gladly’s ability to set Agents to auto-away or auto-logout in a timely manner. To ensure that calls and messages do not get routed incorrectly, please ensure that Agents log out at the end of their shifts and go away when they step away from their desks.
Please contact Gladly Support should you have any questions.
Posted Nov 28, 2022 - 16:55 PST
Update
We are continuing to monitor for any further issues.
Posted Nov 28, 2022 - 14:59 PST
Update
Gladly has fully deployed a fix to address the Agent idle timing (away / logout) issues. We will continue to monitor Gladly operations in the meantime.
Now this hotfix has deployed Agents should refresh their Gladly instance via the update banner or log out and log back in again for this hotfix to take effect and for them to remain online with Gladly.
Please Note: During the interim, this hotfix will impact Gladly’s ability to set Agents to auto-away or auto-logout in a timely manner. To ensure that calls and messages do not get routed incorrectly, please ensure that Agents log out at the end of their shifts and go away when they step away from their desks.
We will continue to work on reinstating auto-away and auto-logout capabilities.
Please contact Gladly Support should you have any questions.
Posted Nov 28, 2022 - 14:55 PST
Update
Gladly continues to deploy a fix to address the Agent idle timing (away / logout) issues. (This is taking longer than anticipated Vs. the previously communicated rollout timing)
After this hotfix is deployed it will require Agents to refresh their Gladly instance if/when they receive the update banner or log out and log back in again for this hotfix to take effect.
Please note: This hotfix will impact Gladly’s ability to set Agents to auto-away or auto-logout in a timely manner. To ensure that calls and messages do not get routed incorrectly, please ensure that Agents log out at the end of their shifts and go Away when they step away from their desks.
Posted Nov 28, 2022 - 14:28 PST
Update
Gladly is deploying a fix to address the Agent idle timing (away / logout) issues. We expect this to be deployed to production within the next 30 minutes.
No other functionality on Gladly (e.g.: routing, IVRs) is impacted or was previously impacted by this issue.
We will continue to monitor system performance in collaboration with our infrastructure partners.
Please contact Gladly Support should you have any questions.
Posted Nov 28, 2022 - 13:50 PST
Update
We are seeing marked improvement and recovery across Agent idle timing issues.
We recommend re-opening your Gladly Voice & Chat channels at this time.
We will continue to monitor activities while services resume normal Cyber Monday activities in conjunction with our infrastructure partners.
Please contact Gladly Support should you have any further questions.
Posted Nov 28, 2022 - 13:21 PST
Update
It is recommended to close your Gladly Voice & Chat channels, once again while this issue persists.
We will provide further updates on reopening these channels as it becomes available and the situation improves.
Please contact Gladly Support should you have any further questions.
Posted Nov 28, 2022 - 12:35 PST
Monitoring
We are continuing to work to alleviate the strain on the system as we investigate the root cause.
This includes ~ spinning up extra instances, disabling non-essential services, and pushing for a suggested / urgent resolution with AWS; our database provider.
It is recommended to reconvene your Gladly Voice & Chat channels at this time, while we are seeing improvements in Gladly operations.
Please contact Gladly Support should you have any further questions.
Posted Nov 28, 2022 - 12:23 PST
Update
We are working to alleviate strain on the system as we investigate the root cause. Spinning up extra instances, disabling non-essential services and pushing for urgent resolution with AWS; our database provider.
Please contact Gladly Support should you have any further questions.
Posted Nov 28, 2022 - 11:37 PST
Identified
We are going to apply further database capacity to handle the problematic traffic and Gladly idle / timeout issues.
It is recommended to close your Gladly Voice & Chat channels while this issue persists.
We will provide further updates on reopening channels as it becomes available and the situation improves.
Please contact Gladly Support should you have any questions.
Posted Nov 28, 2022 - 11:16 PST
Investigating
We are currently investigating an issue with Agents becoming logged out from the Gladly platform at an earlier time than expected.
Please contact Gladly Support should you have any questions.