In the last couple of weeks, there have been multiple reports of hackers exploiting a security vulnerability in Cisco Smart Install Client tool to attack hundreds of thousands of network switches. Cisco had previously identified the vulnerability, published an advisory and provided patches. As is often the case, it appears that only a very small portion of network devices were updated immediately, leaving many of them vulnerable. In most of the attacks in this event, the router or switch had their configuration wiped out and were left inaccessible and inoperable, causing the network to become unavailable.
Notable with this Cisco hack is the unprecedented severity and scale of an attack affecting networking devices. While most network devices today leverage a mainstream Operating System (such as Linux), network vendors normally remove components and disable services that are not needed in a purpose-built network appliance, reducing the possible vulnerability surface. Another factor is that the fragmentation caused by proprietary customizations by each vendor make it more difficult for a hacker to affect a very large number of devices by exploiting a single vulnerability.
That is why most broad attacks usually target mainstream versions of hardware, operating systems and application frameworks used in consumer or data center computing devices.
The current trend is the increasing use of commodity hardware with a mainline operating system and software in network nodes. This convergence enables the use of common tools to manage all systems in a large infrastructure, including networking, compute and storage systems. But it also gives hackers a large vulnerability surface to attack systems that were not attractive targets before.
With the adoption of these new technologies, a network can no longer count on the “security by obscurity” of vendor-proprietary architectures. As network devices use more mainstream hardware and software components, network engineers will have not only to pay more attention to security advisories and react quickly, but also make sure to have the tools and processes to prevent attacks and recover from a disruption.
We have all become used to the automatic update in our cell phones and laptops to fix vulnerabilities quickly, but the same is not true for networking devices – and for good reason. Without vendor-neutral automated configuration management and provisioning systems, network operations groups in enterprise networks are conservative when introducing changes. Human error and configuration change are the two primary causes of network downtime, and so Network Engineers minimize firmware updates because they are afraid of creating disruptions.
Automated configuration management and provisioning systems not only eliminate human errors (by far, the biggest cause of downtime), but also allow the roll-back of changes if they cause disruption. Monitoring of security threats is important, but without enabling patches and fixes to be quickly deployed, we will continue to see major attack events in networking from time to time.
A reliable network infrastructure can leave network engineers with the feeling that they can always manage the network in-band (i.e. relying on network connectivity to manage network devices). Events such as this attack, which left the devices inoperable and brought the network down, are a strong reminder: when there is a network disruption, engineers need an alternative path to access and manage these devices without reliance on the production network. With an Out-of-Band Management infrastructure in place to maintain uptime, including 4G-LTE cellular access, network engineers can reduce the time-to-repair and ensure maximum network availability. So, when a vulnerability is exploited, the network can be up and running again in as short a time as possible.