ANALYSIS:It would appear a small software patch was applied without an adequate disaster plan. A huge backlog results
IT WAS supposed to be a routine “patch” to fix or upgrade its software systems, but one week on it seems that the IT issues at Ulster Bank, which stem from a technological problem at its parent Royal Bank of Scotland (RBS), are a lot more serious than they initially appeared.
So what might have gone wrong?
According to the bank, the problems began when a software upgrade to the payment processing system applied last Tuesday was corrupted. Typically a patch is used to apply a small update or to fix a small glitch with software. However, there has been nothing small about the problems that have ensued.
Stephen Edward Walsh, chief executive of Keeper Systems, says it is possible that one of the bank’s mainframes “fell over” or crashed, “a once in 20 years” event.
When the upgrade was being applied, it could have corrupted a core file detailing all customers’ payment information which put all the account balances out of sync.
The question then is why the bank couldn’t have rolled back to an earlier version, leading to suggestions that the corruption could have flowed on to the back-up files.
According to IT experts, it is possible that the bank did not perceive the upgrade to be a serious affair. If this had been the case, it is likely that it would have stopped operations temporarily to protect its data, and undertaken the process over a bank holiday weekend not a Tuesday night.
Now the longer it goes on, the more overnight processing will need to be done, which pushes out the prospect of it being resolved in the short term.
“Each day it’s not operating as normal, a backlog is being generated, which makes the problem more serious,” says Walsh.
Indeed Ulster Bank has indicated that the problem will not now be resolved until next Monday, and it may have to deal with a flood of monthly pay cheques kicking in this week, as it’s the end of the month
Another question is why the bank’s disaster-recovery process did not deal with the ensuing difficulties, given that it would have been expected that the bank would have a second mainframe operating at a different location.
“If the main system is damaged, it could then switch over to disaster planning,” notes Walsh, adding, “there should be questions for the Regulator as to why the bank’s business continuity system isn’t sufficiently robust”.
While the problems at Ulster Bank are more severe than usual, it is not an isolated incident.
In 2010, Bank of Ireland also suffered a problem when its system went offline.
At the time, it chose to impose offline limits which meant that while it couldn’t reconcile withdrawals, people were still allowed to withdraw money from ATMs, leading some to exploit the situation. According to Walsh, while this won’t be an issue for Ulster Bank, it will still incur considerable expense given costs that people will have incurred due to failing to meet a payment on a credit card for example.
In such cases, he would expect the bank to meet the cost of these “consequential damages”.
The problems will also cause the bank significant reputational damage.
“It’s a massive mistake,” says Christopher Skinner, chairman at the Financial Services Club and a markets commentator.
“I always say that banks are a bit like aircraft. If you land safely each time you’ll be back, but if they crash you may not be.”
Ulster Bank said it had no comment to make on the IT issues.
