Norce Storm had intermittent problems with payment processing between 11:20 2023-10-11 to 12:30 2023-10-12. The problems manifested as requests related to payment processing timed out in periods (see below for a summary of periods where payment processing is considered down).
The problem manifested as timeouts or very long response times to payments-related requests to StormApi. The problem was caused by problems in the underlying OS where the number of network sockets available was well below configured levels and thus services were competing for a reduced amount of network sockets than normal. It affected primarily the Payment Service since it's a hub for communicating with all Payment Service and Voucher Providers which requires a substantial amount of network socket usage.
The problem was worsened by a significant increase in traffic during the night, on a Payment Service Adapter which requires more than double the number of network connections.
The problem was resolved at ca 12:53 on 2023-10-12 by moving Payment Service to a secondary VM cluster where the service behaved as normal.
Due to the intermittent nature of the problem, downtime for Payment Services/StormApi is calculated using the percentage of failed requests, 14,33% which translates to 2h 9m downtime in total.
For a more detailed report, please contact your Customer Success Manager