November 2009 Vol. 236 No. 11
Features
SCADA Cyber Security Training
“Security requires a particular mindset. Security professionals—at least the good ones–see the world differently. They cannot walk into a store without noticing how they might shoplift. They cannot use a computer without wondering about the security vulnerabilities. They cannot vote without trying to figure out how to vote twice. They just cannot help it.” –Bruce Schneier, CRYPTO-GRAM, April-May 2008.
The authors had the unique opportunity of attending a very rare event the week of July 20, 2009 in the high desert plains of Idaho Falls, ID. The five-day event, entitled National SCADA Test Bed Advanced Training, was sponsored by Idaho National Laboratory (INL), one of the lead Department of Energy (DOE) laboratories responsible for the National SCADA Test Bed (NSTB) under the DOE NSTB program.
The Advanced Control System Cyber Security Training program focused on energy sector (oil, natural gas, and electric) participants to teach the difficult subject of cyber security, to raise awareness, to make better defenders and to take advantage of the true security professionals in the organization to encourage more security disciples.
The training was attended by 33 industry participants representing 20 energy companies with impressive educational, technical and industry backgrounds. Attendees had the benefit of several focused sessions with leaders in Network Design, Operating Systems, Critical Communication, Application Design, the application of contemporary security mitigation strategies and knowledge of the latest attack vectors.
Participants were provided defensive cyber security skills for their control systems. Through the instructions and their participation in a Red Team (attacker)/Blue Team (defender) exercise, participants gained an understanding of how cyber attacks against control systems could be launched, why and how they work and mitigation strategies that will increase a company’s cyber security posture to thwart potential attacks.
The attending individuals learned about tools and strategies they had only heard of or had tried in an effort to protect their company’s information technology and control systems without verification of its effectiveness. Their attempts to protect their company’s cyber exploits occurred without really having a clear understanding of the availability of very powerful and fairly easy to use open source cyber tools. These tools are readily available on the Internet and in some cases are free of charge, allowing a malevolent perpetrator to develop and implement exploits to cause information stack overflow and Reverse TCP. This was evident from a comment by an attendee representing the oil and gas sector.
“I would highly recommend this class for anyone involved in securing their energy industry IT assets. This class was very informative and definitely an eye opener. One of the items that I found most helpful was the plethora of software tools available for download via the Internet that can be utilized by hackers to cause severe disruption to an environment. However, it was demonstrated how these same tools can be utilized by a proactive IT group to protect their assets.” Allen Lykins – Manager, Network, SCADA and Control Systems Genesis Energy.
Another attendee said, “I wish I had learned (earlier) what power some of these exploits have and how readily available these tools can be. It would have made my job a lot easier to have this level of understanding.”
The training was organized by the INL team, and the program leaders such as Jeff Hahn and Gary Finco kept the training agenda to the timeline and accommodated the needs of the students. Participants had an opportunity to talk to experts on the impact of Beagle, Blaster, Stormworm, Spam, IRC bots, P2P bots, as well as other significant cyber events that have affected our information superhighway and information technology environments. This training helped frame the way participants need to prepare for any cyber security event.
A one-day 12-hour exercise tying all the concepts together was a revelation and an eye-opener for the industry participants to find out what they did not know and to see how easy it was for the attackers (Red Team) to exploit vulnerabilities by way of the World Wide Web Internet gateway through the corporate enterprise connectivity to the company control systems.
The defenders (Blue Team) made an interesting discovery. They found that, from their perspective, it was difficult to distinguish an “attack” from other normal business functions that were required to maintain the operation of the real-life simulated industry process.
The Blue Team quickly realized from their defensive perspective how easy it was to exploit their process control system vulnerabilities. There would often be a great deal of reconnaissance before the Red Team would launch an attack. After a grueling day filled with a lot of pressure and wall-to-wall activity, the Blue Team, “giddy” with excitement after the exercise, held their ground in the end.
From the start, the Red Team played their role with an attacker attitude – a mentality of nothing to lose and everything to gain. The Red Team showed up late on this day, very relaxed and ready for work. They were actually a leg up on the Blue Team as the activity started because one of their creative fellow attackers managed to score early through a social engineering act and took a key network diagram away from an unsuspecting Blue Team Member who was just a bit too trusting. The Red Team members were cool and collected after the exercise, definitely identifying with the “Nothing to Lose” philosophy of a good attacker.
Participants were fully engaged during the five days of training. They were provided with mini training sessions in preparation for the exercise in the use of tools for defense as well as those tools used by attackers.
The rules of engagement forced the Blue Team to conduct business as if they were running the real company. The time provided to the team for organizing their company, roles and responsibilities during the exercise may have been limited as some pointed out, but also provided a realistic approach similar to what companies in the real world face in such a crisis.
In their after-action comments, all the participants recommended building upon the relationships established during the week, and to continue the networking and exchange of information among their peer organizations. The participants felt that the network also provided an excellent resource to learn and collaborate in the future. All participants agreed that they would like to see the availability of more courses for the industry in SCADA and Cyber Security Training each year.
In summary, all the participants felt that this type of training needs to continue because it is what oil and natural gas and electric communities have been looking for. The training and the control systems architecture were geared toward the sector, allowing participants to achieve a higher level of training and understanding as they were familiar with the philosophy, equipment, and solutions. While we can argue the similarities and differences with the electric sector, the shared experiences are unique, and the participants appreciated the chance to talk to peers and other communities to learn that the issues are the same when it comes to cyber-security vulnerabilities.
After some retrospection, several participants sent comments saying they thought they were ahead of the game when they arrived for the training, doing what they thought needed to be done, only to discover during the five days that they were not as prepared as they thought. It was described as a realistic environment and highly effective in showing people where the gaps are in their security practices.
During the brief 12-hour period the Red Team was able to almost penetrate the control systems. Imagine what a highly trained team that had been together for a long time and with the proper financial backing and motivation could do – and we are not even talking about the resources available to a nation-state.
Al Rivero, PE, is Director of Professional Services at Telvent Energy. He can be reached at 713-346-0654 or al.rivero@telvent.com. www.telvent.com
Comments