I just stepped out of the train taking me to work when my phone rang. “Raj is looking for you”, told me Darren, my project administrator. “He asked me to urgently let you know that while his off-shore team is scheduled to commence integration testing later on today, over night security incidents in the city will prevent the majority of his team from getting to work. Given the internal political situation it is not clear when regular, normal, work will resume”.
The preparation for the integration testing phase has taken a considerable amount of time and required the collaborative effort of quite a few resources, representing the core technologies and support teams involved in the new infrastructure deployment. While planning was carried out both on-site and off-shore, testing itself was planned to execute using a predominantly off-shore resource pool across a number of geographical locations, thus allowing for testing to proceed almost around the clock.
This is not the first time one of my infrastructure projects is impacted by off-shore events. Heavy rain and flooding in one of our regional data centres in South East Asia has caused me major headaches in the past. The transition to the new data centre has just taken place few months earlier and the risk realisation of weather impacts and how to prepare for them has not yet sunk in. This lack of proper risk identification has caused my project major delays and cost overruns as I have had to divert some of the project operations to a different site.
This time I was much better prepared. Though not specifically targeting security risks on my risks register I did investigate the possibility of delays due to regional issues like weather conditions, terror threats and alike. I have involved the company’s risk department in the project’s risk reviews and assessments and have prepared a mitigation strategy I could invoke should such circumstances arise.
The risk documented in my risk register was documented as follows:
“Local circumstances (e.g. unusual weather conditions, political tension or terror activities) at offshore site1 could result in major delays to scheduled activities due to inability of local resources to reach the office and perform their allocated tasks“.
Offshore Site1 has not experienced devastating weather conditions since the one I mentioned above and having consulted weather forecasts indicated that the likelihood of such event re-occurring during the lifetime of my project was very low.
The political tension in the area however has been steadily rising over recent months and security experts consulted on this matter have estimated the likelihood of a disturbing incident taking place at being likely, or about 60%.
From a project perspective, the consequences of not being able to execute the testing on time was quite severe. The project outputs were directly contributing to a business outcome whose needs were based on strict regulatory requirements. Not meeting those requirements could result in harsh financial penalties and loss of public credibility.
It was clear from the outset that this risk cannot be accepted and that an operative mitigation strategy needs to be put in place in order to have it controlled.
The plan of attack for mitigating and addressing this risk was basically agreed as following:
A second, alternative, team will be put on stand-by on-site to be able to pick up the testing should the off-shore team become unapproachable.
The off-shore team will be provided with remote access capability (mobile phone and laptops) so they could execute their test scripts remotely if required and be able to communicate with the team as required.
Operational instructions and guidelines were distributed and reviewed by all respective teams, on site and off-shore, so they are aware of the communication plan and execution plan associated with invoking the alternative mitigation plan.
“Thanks Darren”, I said, Please convene the project risk committee so we can discuss and obtain an approval to embark on plan B, and execute the communication plan associated with it”.
In a bizarre way, our previous failure to deal with a similar event in the past has made us aware of the scenario and allowed us to better prepare for the event we’re just facing now. Lesson learned well executed.
Think about it!
*Disclaimer: This is a fictional story and is meant for educational purposes only. Any resemblance to real persons, living or dead is purely coincidental.
P.S. After publishing this post I discovered that Chris O’Halloran published a post with a similar message (see HERE).