Providing Out-of-Band Connectivity to Mission-Critical IT Resources

Key Automation Infrastructure Components That Enable End-to-End Network Automation

A resilient network containing automation infrastructure components and concepts overlays a busy industrial plant that uses OT automation.

As inflation rises, new business declines, and another COVID-19 surge looms on the horizon, many organizations are bracing for a recession. CIOs and IT managers are having to do more with less—less staff, less budget for upgrades and repairs, and less access to on-site infrastructure. Despite these restrictions, they still need to ensure the 24/7 availability and optimal performance of enterprise network resources as any amount of downtime could severely impact business revenue.

The ability to continue providing digital services in less-than-ideal situations is known as network resiliency. Network automation is a key tool for ensuring resiliency during staffing shortages and lockdowns, and a network automation framework provides the tools and methodologies needed to create a fully-automated network infrastructure.

The four building blocks of a resilient network automation framework include:

  1. IT/OT production infrastructure
  2. Automation infrastructure
  3. Orchestration infrastructure
  4. AIOps

We’ve previously discussed the role of IT/OT production infrastructure in network automation and how an IT/OT convergence strategy accelerates network automation. In this post, we’ll describe the automation infrastructure components that enable end-to-end network automation. Future blogs will explain how the orchestration infrastructure layer and AIOps layer build upon these components to ensure business resiliency.

What is automation infrastructure?

Automation infrastructure is composed of all the hardware and software solutions that enable automation to occur. These solutions target the IT and OT production infrastructure and automate some or all of their workflows.

Key automation infrastructure components

There are a variety of hardware and software solutions that provide automation capabilities for specific workflows, use cases, and deployment models. As part of a resilient network automation framework, the most important automation infrastructure components include:

Gen 3 OOB serial consoles

Serial consoles are typically installed in data centers and used to manage other devices over a serial cable connection. They create an out-of-band management (OOBM) network that’s dedicated to troubleshooting, management, and orchestration traffic, and which is accessible via a secondary internet connection (often using cellular). This secondary connection ensures administrators always have remote management access to critical data center infrastructure even when the primary ISP, WAN link, or production LAN goes down. That means businesses can recover from outages faster and without dispatching expensive truck rolls.

The latest generation of serial consoles, Gen 3, gives administrators the ability to automate workflows on all data center infrastructure. Gen 3 serial consoles are vendor-neutral, which means they can extend their automated management capabilities to any vendor’s device. That vendor neutrality also means that Gen 3 serial consoles support custom scripts and third-party automation tools in addition to whatever automation capabilities are built-in.

For peak resiliency, data center deployments should follow a two-tier OOB architecture. That means each rack of IT/OT production infrastructure should connect to its own Gen 3 serial console, which provides OOB management access and automation. These top-of-rack serial consoles should then connect to an OOB appliance in the middle or end of the row. This ensures OOBM access for the top-of-rack appliances and creates an additional layer of redundancy and resiliency.

Screenshot 2022-12-05 202130
Another important aspect of Gen 3 serial consoles is security. Since serial consoles provide comprehensive management access to critical infrastructure, they’re a tempting target for cybercriminals. A secure Gen 3 OOBM solution includes:

  • → Integration support for third-party security solutions like next-generation firewalls (NGFWs), security service edge (SSE), and SAML 2.0
  • → An up-to-date operating system (OS) kernel that’s frequently patched by the vendor when vulnerabilities are identified
  • → Onboard firewall functionality to inspect traffic on both the OOB network and the production network
  • → Hardware security features like encrypted boot sequences and BIOS protection to prevent unauthorized access on stolen serial consoles

Gen 3 OOB serial consoles are the automation infrastructure components that enable automation and resiliency for data center deployments at the core of enterprise networks.

SD-WAN gateway routers

A gateway router is used to connect a LAN infrastructure to the internet and the enterprise WAN architecture. As part of a resilient network automation framework, all gateway routers should support SD-WAN (software-defined wide area networking).

SD-WAN separates the control and management processes from underlying WAN hardware and virtualizes them as software. SD-WAN uses features like application awareness and guaranteed minimum bandwidth to automatically optimize network performance. An SD-WAN solution can also use automatic load balancing and failover to ensure continuous availability in the event of a localized failure or data center outage.

SD-WAN is usually a cloud-based service that delivers centralized management and orchestration of automated workflows. This service runs on top of the gateway routers deployed at each site.

An SD-WAN gateway router is a key automation infrastructure component for the main office, data center, branch, and edge deployments because it enables automated WAN management and orchestration. An all-in-one cloud-managed gateway router is particularly useful for OT automation in remote facilities like warehouses and factories because it provides SD-WAN capabilities, OOBM, and routing in one multi-function device.

Monitoring, visibility, and analytics

Monitoring and visibility solutions give administrators virtual eyes and ears on remote network infrastructure. As part of a resilient network automation framework, a visibility solution should be vendor neutral so it can dig its probes into any device in a mixed vendor environment. It should also include environmental monitoring sensors that collect data on conditions in the rack.

Device monitoring and environmental sensors give administrators the ability to detect potential issues and respond quickly to prevent outages. Monitoring and visibility solutions also collect valuable data that can feed into the AIOps building block of the network automation framework.

Infrastructure as Code

Infrastructure as Code, or IaC, uses software abstraction to decouple infrastructure configurations from the underlying hardware. Configurations are written as scripts or definition files that automatically provision virtual machines (VMs), containers, or software-defined networking (SDN) devices. An IaC definition file can be deployed repeatedly, which means many identical resources can be spun up quickly while ensuring consistent configurations. An IaC config can also undergo automatic security testing before it’s deployed to any devices to prevent vulnerabilities from affecting production.

Another important aspect of IaC is automatic configuration management. Configuration management solutions like RedHat Ansible allow administrators to define the desired state of a system or network resource. The configuration management tool continuously monitors the resource to detect unauthorized changes, which might be made by a careless sysadmin or could be a sign of a malware infection. As soon as the change is detected, the configuration management solution uses a programmatic playbook to take whatever actions are needed to restore the system to its proper state.

IaC helps ensure network resiliency by reducing human error in device configurations and updates, as well as by enabling the use of pre-production automated security vulnerability scanning and configuration management. Infrastructure as Code also facilitates another key automation infrastructure component—immutable infrastructure.

Immutable infrastructure

In-place system and device updates are a common cause of hangs or failures which can be challenging to resolve remotely. Immutable infrastructure resolves this problem by eliminating updates and configuration changes altogether. Immutable infrastructure refers to virtual systems and network resources that are never changed in place. If an immutable resource has an issue or vulnerability, or if its OS is out of date, an entirely new resource is spun up and the old one is simply deleted.

IaC is an immutable infrastructure best practice because it gives administrators the ability to provision many devices very quickly and with identical configurations. Immutable infrastructure is secure, easy to deploy, and resilient to failure, making it an important part of the network automation framework.

Why Nodegrid is a key automation infrastructure component

The automation infrastructure building block of the network automation framework relies on vendor-neutral OOBM devices like gateway routers and Gen 3 serial consoles that extend automation to converged IT/OT production infrastructure. These devices must also support monitoring and visibility solutions, Infrastructure as Code with configuration management, and immutable infrastructure.

For example, the Nodegrid platform from ZPE Systems includes OOB management hardware for a variety of data centers, branch, and edge deployments. Nodegrid serial consoles, such as the NSCP, can dig their hooks into any device in your data center to enable end-to-end network automation. A Nodegrid Gen 3 OOB serial console can even extend IaC and immutable practices to legacy devices to ensure resiliency without expensive forklift upgrades.

Nodegrid services routers, such as the Mini SR, are compact edge gateways that deliver SD-WAN support, OOBM, and cloud management capabilities to IT/OT infrastructure in smaller branch office and edge data center deployments. Nodegrid SRs can help you consolidate an entire rack of branch infrastructure into a single device to reduce management complexity, CapEx, and OpEx.

Nodegrid out-of-band is delivered via WiFi, Ethernet, or 5G/4G LTE to ensure administrators have fast and reliable access to remote infrastructure. All Nodegrid OOB devices are protected by robust hardware security features like BIOS protection, UEFI Secure Boot, geofencing, disk encryption, and TPM 2.0. Plus, Nodegrid supports integrations with Zero Trust Security solutions like identity and access management (IAM) and SAML 2.0, as well as providing an on-ramp to SSE.

Nodegrid serial consoles and services routers also include interfaces for environmental monitoring sensors to collect crucial data about conditions in your rack. These sensors, as well as any other connected devices, can all be observed and managed from a single, centralized monitoring and reporting platform.

What makes Nodegrid a crucial element of automation infrastructure is its ability to directly host Infrastructure as Code and automated configuration solutions, including Ansible, Chef, Puppet, SaltStack, Monit, and Docker. Nodegrid appliances can then extend the capabilities of the IaC solution to any of the modern, legacy, and mixed-vendor devices it manages.

ZPE’s Network Automation Blueprint

Automation infrastructure works together with IT/OT production infrastructure, orchestration, and AIOps to ensure network resiliency during uncertain times. The Network Automation Blueprint from ZPE Systems provides a reference architecture for achieving Gartner’s definition of hyperautomation as well as meeting the Open Networking User Group (ONUG) Orchestration and Automation recommendations.

In future blog posts, we’ll discuss the remaining two building blocks of the Network Automation Blueprint in depth. In the meantime, you can read about IT/OT production infrastructure or click here to get a sneak peek of the blueprint, which includes a 10-step checklist to get started with automation now.

Want to learn more about key automation infrastructure?

To learn more about Nodegrid as a key automation infrastructure component, contact ZPE Systems today.

Contact Us

How an IT/OT Convergence Strategy Accelerates Network Automation

An ITOT convergence strategy visualized with many digital services organized together in a data center.
In the face of a looming recession, Covid-19 uncertainty, global political instability, and an increasing frequency of natural disasters, network resiliency should be on every organization’s mind. Network resiliency is the ability to continue providing services and connectivity even during disruptions, such as when buildings are locked down or layoffs reduce the number of staff available to maintain or operate the technology. Network automation is the key to ensuring continuous, consistent, and streamlined management during tumultuous times.

A network automation framework provides all the tools and processes needed to create an efficient, resilient, fully automated network infrastructure. The four building blocks of a resilient network automation framework include:

  1. IT/OT production infrastructure
  2. Automation infrastructure
  3. Orchestration infrastructure
  4. AIOps

In this blog, we’ll discuss why an IT/OT convergence strategy is critical for forming the foundation of a network automation framework. Future posts will discuss the other three building blocks and how they work together to ensure business resiliency.

What is IT/OT convergence?

IT/OT convergence is exactly what it sounds like—bringing your information technology (IT) and operational technology together under unified management.

Operational technology, or OT, controls equipment interacting with the physical world, such as industrial machinery or HVAC systems. OT automation runs on specialized industrial computers, such as programmable logic controllers (PLCs) and supervisory control and data acquisition systems (SCADAs). Those computers are usually completely isolated from IT networks, which means operators have no way to access them remotely. If operators can’t get onsite, whether due to a Covid-19 lockdown or natural disaster, they lose the ability to manage OT.

For example, Southern California is home to many high tech manufacturing plants, especially in the aerospace and defense industries. Due to the effects of climate change, there’s been an increase in the frequency and severity of wildfires in this region, leading to more frequent evacuation orders and plant closures. That means operators can’t access their computer systems to control and monitor OT devices, forcing these businesses to pause their operations.

In addition, OT control systems aren’t usually within the purview of IT management because they use specialized computers and automation software that needs to be operated and supported by OT experts. That means IT infrastructure automation and OT infrastructure automation are siloed, which can lead to cost and management inefficiencies. With recession anxieties running high, many organizations are looking for ways to reduce such inefficiencies by converging their IT and OT infrastructure.

IT/OT convergence involves bringing your operational technology under the same management and automation umbrella as your IT network infrastructure. In a converged IT/OT infrastructure, OT control systems like PLCs and SCADAs connect to the same management hardware (e.g., serial consoles or cloud-managed gateway routers) as IT servers and network devices. This gives administrators a single platform from which to orchestrate automation across both IT and OT infrastructure.

What does IT/OT convergence look like?

IT and OT equipment being managed
First, you have the IT and OT equipment being managed. On the IT side, this includes things like servers, storage, security appliances, and SD-WAN devices. On the OT side, you have devices like environmental sensors, cameras, and power distribution units, as well as industrial computers used to monitor and control physical equipment. Some examples of those industrial systems include:

  • Programmable logic controllers (PLCs), which control industrial machines, robotic devices, and other manufacturing processes.
  • Supervisory control and data acquisition (SCADA), which is a control system for high-level supervision of industrial processes, including PLCs.
  • Building management systems (BMSs) which manage building equipment such as HVAC, fire suppression, lighting, and automatic doors.

These IT devices and OT computers all connect to common management hardware. For large deployments, these might be high-density serial consoles; in smaller deployments, these might be network edge routers with integrated serial console management functionality. This management hardware then connects to an orchestration platform that’s used to monitor, deploy, and manage automation across the converged IT/OT infrastructure.

How an IT/OT convergence strategy accelerates network automation

Bringing operational technology onto IT networks makes it possible for operators to remotely access their OT systems when they’re unable to come onsite. That means that your business can continue to function even during pandemic lockdowns, extreme weather events, or wars that prevent your staff from entering the building.

IT/OT convergence also allows you to bring operational technology under the same management umbrella as IT, so you can use the automation tools you’re already familiar with on the IT side to automate your OT. This reduces the overall management complexity of the IT/OT infrastructure and facilitates holistic orchestration of a fully automated—or even hyperautomated—enterprise network. This level of automation can help organizations reduce wasteful processes, eliminate redundancies, and increase operational efficiency so they can weather recessions and other economic difficulties.

Building IT/OT convergence into a resilient network automation framework

Your IT and OT infrastructure represent the target devices that are automated as part of a network automation framework. For maximum resiliency, your IT/OT convergence strategy should include:

Out-of-band (OOB) connectivity

Out-of-band (OOB) connectivity provides an alternative path to remote IT and OT infrastructure when the primary ISP connection goes down. In addition, OOB management devices (like serial consoles) directly connect to IT/OT devices, so administrators can manage them without an IP address or LAN connectivity. While OOB is not itself a component of IT/OT infrastructure, it’s a crucial element of the management devices and orchestration solution you’ll use to converge your IT and OT infrastructure.

Wired and wireless connectivity

Your converged IT/OT management solution also needs to support a variety of wired and wireless connectivity options to ensure resilience and flexibility. For example, if the ISP’s wired network infrastructure is disrupted due to extreme weather or warfare, you should be able to fail over to a 5G or 4G cellular connection. Or you may have some devices that lack RJ-45 ports, which means you need a management solution that supports USB. The goal is for your management solution to be adaptable to any scenario so that sudden changes or unforeseen issues don’t cripple your network operations.

Power control with UPS backup

As a remote network infrastructure, one of the most frustrating issues to deal with is a device that locks up after a system crash or failed firmware update. Often, a power cycle is all that’s needed to fix the problem, but that requires an on-site technician, which means an expensive and time-consuming truck roll. To ensure network resiliency while reducing the incidence of truck rolls, you need an IT/OT management solution that includes rack PDUs and IPMI options to facilitate remote power control of all connected devices.

In addition, an uninterruptible power supply (UPS) improves resiliency by providing backup power in case of an outage. This gives network teams time to investigate the problem and (hopefully) implement a fix before losing power. As part of the network resilience framework, all UPS units should hook into the management solution to allow for automated monitoring, optimization, and troubleshooting.

Environmental Sensors

Environmental sensors are used to monitor conditions in the location where IT and OT infrastructure is deployed. Traditionally, these sensors monitor racks in remote data centers, but they’re especially critical for IT/OT infrastructure that resides in less-ideal locations. For example, environmental sensors can provide data on the temperature and humidity levels in remote warehouses, offshore oil rigs, outdoor “smart city” deployments, and other locations when environmental conditions can’t be controlled.

Environmental sensors alert administrators when conditions grow too extreme for IT/OT equipment to function optimally. That means that teams can respond quickly and prevent equipment failures from bringing down critical resources. In addition, your infrastructure orchestration solution can analyze the data from these sensors to predict future issues or recommend optimizations to improve efficiency and resiliency.

How Nodegrid accelerates IT/OT convergence

The most successful IT/OT convergence strategy relies on vendor-agnostic platforms that can connect to both IT and OT infrastructure. For example, the Nodegrid solution includes management hardware that can connect to modern and legacy devices in a mixed vendor IT/OT infrastructure, such as the Nodegrid Serial Console Plus (NSCP) for large and hyperscale data center deployments and the Nodegrid Net Services Router (NSR) for flexible edge and branch deployments. These devices allow you to use the ZPE Cloud management platform to extend automation and orchestration to all your IT and OT targets to create a unified, efficient, and resilient converged network infrastructure.

ZPE’s Network Automation Blueprint

IT/OT production infrastructure works together with automation infrastructure, orchestration, and AIOps to ensure network resiliency during uncertain times. The Network Automation Blueprint from ZPE Systems provides a reference architecture for achieving Gartner’s definition of hyperautomation as well as meeting the Open Networking User Group (ONUG) Orchestration and Automation recommendations.

In future blog posts, we’ll discuss the remaining three building blocks of the Network Automation Blueprint in depth. In the meantime, click here to get a sneak peek of the blueprint, which includes a 10-step checklist to get started with automation now.

Ready to learn more about implementing an IT/OT convergence strategy?

To learn more about implementing an IT/OT convergence strategy with Nodegrid, contact ZPE Systems today.

Contact Us

What Is Edge Computing for Machine Learning?

Edge computing for machine learning is visualized as an artificial brain on the monitor of a remote industrial machine
Edge computing and machine learning technologies are helping organizations use their data more efficiently and effectively. In this blog, we’ll explain what edge computing is and discuss how it’s used with machine learning to improve performance and keep data secure. We’ll also explore two different edge computing deployment models for machine learning and provide advice on how to manage them.

What is edge computing?

For many modern enterprises, most of their data is no longer generated in a centralized data center or office building. These days, that data comes from IoT devices, “smart” industrial systems, and other remote locations around the globe. Transferring all that data to and from a central data center for processing can introduce latency and negatively impact performance. Transmitting sensitive data over the internet also increases the risk of interception by hackers.

Edge computing moves computational power closer to the source of data so that data doesn’t need to be sent to a separate location for processing. The benefit of edge computing is that data doesn’t need to travel as far, which translates to less latency and improved application performance. Plus, the data stays behind the firewall on the local network, reducing security risks.

What is edge computing for machine learning?

Machine learning (ML) is powered by data, and with data moving to the edge of enterprise networks, machine learning needs to decentralize as well. Edge computing for machine learning places ML applications closer to remote sources of data. The benefits of edge computing for machine learning are the same for edge computing in general, just supercharged.

Machine learning requires data to make intelligent predictions and decisions. In many cases, that data originates from the edge of the network. For example, the healthcare industry uses ML algorithms to analyze health data from smart devices in hospitals and clinics around the world, sometimes in hard-to-reach and politically unstable regions.

Getting patient health data from these remote facilities back to a centralized data center for machine learning processing can be very challenging, especially if the internet infrastructure is outdated or inconsistent. In addition, this data is personal and sensitive, and healthcare organizations are obligated to ensure its protection, so transferring it over uncertain internet connections is too risky.

Instead, organizations can install the ML algorithm on servers in each remote facility, or even on the smart devices themselves. This drastically reduces their reliance on outside network infrastructure for running machine learning workloads, which improves performance and ensures patient health data stays private.

How to deploy edge computing for machine learning

There are two basic deployment models for machine learning at the edge.

A traditional edge machine learning deployment uses one or more racks of heavy-duty servers with high-performance machine learning processing units. This deployment model is best suited to large ML workloads that process massive amounts of edge data.

A “thin” or “nano” edge machine learning deployment runs on smaller servers or multi-purpose devices that share rack space with other edge infrastructure. This deployment model is more cost-effective and works best for smaller ML workloads in buildings where space is limited.

For either deployment model, you need a solution in place for remote management so administrators can maintain and troubleshoot edge infrastructure without traveling on-site. The best way to ensure reliable management access is through out-of-band (OOB) management. OOB management creates a separate network dedicated to remote management and troubleshooting, and that  provides an alternative path to remote infrastructure (typically via cellular LTE) in case the primary ISP or WAN link goes down.

Through that OOB management network, you can orchestrate workloads, push out security patches, and monitor the health and performance of edge infrastructure.

Deploy and manage edge computing machine learning infrastructure with Nodegrid

The Nodegrid Net Services Router (NSR) from ZPE Systems supports both traditional and nano edge computing machine learning deployment models. The NSR is a modular and customizable solution that delivers OOB management, cellular failover, edge routing and switching, and automation in a single device.

You can use the NSR’s serial console modules to monitor, manage, and orchestrate an entire rack of edge machine learning servers. For less intensive workloads, you can use the edge compute module to host ML applications, virtual machines, and Docker images.

Either way, you can take advantage of 5G/4G LTE to ensure fast and reliable OOB access and cellular failover. The NSR is also secured by Zero Trust features like SAML 2.0 integration and BIOS protection to keep edge machine learning data protected.

Ready to learn more about 5G/4G LTE to ensure fast and reliable OOB access?

To learn more about edge computing for machine learning with Nodegrid, contact ZPE Systems today.

Contact Us

Comparing Cellular Failover Router, Gateway & Bridge for Business Continuity

NSRSTACK2-1(1)
Cellular failover is critical for business continuity because it ensures uninterrupted internet access even if the primary ISP connection goes down. When looking for an enterprise cellular failover solution, you’re likely to see the terms “cellular failover router,” “cellular failover bridge,” and “cellular failover gateway” used somewhat interchangeably. These three types of devices offer similar (and often overlapping) capabilities, which can make it difficult to tell which is the best option for your particular use case. In this post, we’ll define and compare these different cellular failover devices before discussing the best option for business continuity.

Cellular failover router vs. cellular failover bridge vs. cellular failover gateway

Let’s define the network functions provided by these devices and describe how cellular failover fits in. We’ll start with bridges, which provide the least amount of functionality.

What is a cellular failover router?

This router connects multiple LANs together but can also forward traffic to and from locations outside the domain. It forwards packets on the network layer of the OSI model (layer 3) using IP addresses. A basic router does not provide access to the internet—it must route traffic through a modem to forward packets outside of the LAN.

A cellular failover router provides a secondary internet connection over which traffic can be routed if the primary ISP link goes down. It includes a cellular modem for internet access as well as IP routing capabilities, giving it more functionality than a cellular failover bridge. Since cellular failover routers combine a modem and router into a single device, they’re often referred to as cellular failover gateways.

What is a cellular failover bridge?

A network bridge connects multiple local area networks (LANs) into a single domain but is not capable of moving data outside of the domain. It forwards frames on the data link layer of the OSI model (layer 2) using MAC addresses (also known as physical or hardware addresses).

A cellular failover bridge is essentially a device that connects the primary network to the cellular failover network so LAN devices can access that network if the ISP connection goes down. Usually, these come in the form of cellular modems configured in bridge mode. A cellular modem in bridge mode provides internet access via the cellular LTE network and gives devices on the LAN a link (or “bridge”) to cross over to that cellular network. It does not provide routing functionality itself, however, so it needs the primary router for that.

A basic network topology using a cellular modem in bridge mode for failover.

Fig. 1: A basic network topology using a cellular modem in bridge mode for failover.

What is a cellular failover gateway?

A gateway is a device that connects multiple networks with different transmission protocols together. All traffic flowing into and out of an enterprise network must pass through a gateway. Network gateways combine the functionality of a modem and a router, so they provide both an internet connection and the ability to route packets to and from IP addresses. That’s why cellular failover routers—which combine a cellular internet connection and IP routing—are frequently called cellular failover gateways.

A basic network topology using a cellular failover gateway router

Fig. 2: A basic network topology using a cellular failover gateway router

Cellular failover routers/gateways also function as cellular failover bridges, but the reverse is not true. A cellular failover bridge must rely on an external router for IP-based packet forwarding.

Why choose an integrated cellular failover device?

Generally speaking, the terms cellular failover router, bridge, or gateway refer to standalone devices. Often, they’re designed to provide simple cellular connectivity and rely on the primary router/gateway in order to function, such as Cradlepoint cellular failover adapters. Another option is to get a standalone cellular gateway, such as a Meraki, that you deploy alongside your primary router and fail traffic over to when the primary connection goes down.

In both cases, you’re investing in a single-purpose cellular failover device that must be purchased, installed, and managed in addition to the primary gateway router, network switches, serial consoles, etc. While this may not seem like a big deal in a single-site, centralized enterprise LAN, it grows much more onerous in a large and distributed network with many remote sites and a complicated SD-WAN architecture.

A much better option is to buy an all-in-one device that combines many networking capabilities into one, such as a Nodegrid Services Router. Nodegrid devices include production gateway, routing, and switching functionality in addition to cellular failover, remote out-of-band (OOB) management over serial, and more.

A basic network topology using a Nodegrid Net Services Router with integrated networking, cellular failover, hosted firewall solution, and a serial console module.

Fig. 3: A basic network topology using a Nodegrid Net Services Router with integrated networking, cellular failover, hosted firewall solution, and a serial console module.

The Nodegrid solution is highly customizable, with six integrated routers to choose from depending on your deployment size and use case. The most flexible option is the Nodegrid Net Services Router (NSR) with a modular design that lets you swap out expansion modules to get the exact functionality you need without paying for extras that you don’t. A single Nodegrid device can replace an entire rack of networking equipment, simplifying deployments in branch offices, edge computing data centers, manufacturing plants, and other remote sites.

Plus, Nodegrid’s vendor-neutral hardware and software allow you to consolidate infrastructure management behind a single pane of glass. You can use Nodegrid to orchestrate cellular failover, SD-WAN, DCIM, and more over a dedicated, reliable, and blazing-fast OOB management network.

Ready to learn more about Nodegrid cellular failover router connectivity?

To learn more or see a demo of Nodegrid in action, contact ZPE Systems today.

Contact Us

The Importance of Out-of-Band Data Center Connectivity

The importance of data center connectivity is illustrated with overlapping digital globes superimposed over racks of data center equipment

Data center connectivity is more crucial than ever. Data, applications, and digital services power every aspect of business, which means your infrastructure needs to be available 24/7. However, according to the Uptime Institute’s 2022 Outage Analysis report, outages are still a frequent problem for enterprises and data centers, and the financial consequences of the resulting business interruptions are staggering.

One of the best tools for maintaining data center connectivity is remote out-of-band (OOB) management. OOB management creates an alternative path to remote infrastructure on a dedicated management network. An OOB management solution uses serial consoles and data center infrastructure management (DCIM) software to give administrators the ability to monitor and control remote data center infrastructure. With OOB, you can recover from outages faster and regain control over remote data center infrastructure even when the main network is down.

The importance of out-of-band data center connectivity

The first major takeaway from the Uptime Institute report is that outage rates have remained high over recent years. Twenty percent of responding organizations experienced a serious outage in the last three years, which is slightly higher than in the 2021 report. It was noted that 80% of data centers reported an outage of some kind (with varying severity), which hasn’t changed much since previous reports. The implication here is that businesses and data centers are both still struggling to maintain the 24/7 availability expected by their customers. Let’s dig deeper into the causes and effects of data center outages and discuss how out-of-band management can help.

  1. Network issues are the biggest cause of downtime
    According to the 2022 report, networking problems were the single largest cause of outages over the last three years. These issues are frequently due to the complexity of distributed and software-defined network architectures, especially in cloud or hybrid cloud deployments.Out-of-band data center connectivity solutions use serial consoles which directly connect to other data center devices using the serial port. That means administrators can access and manage those devices without needing to use their IP addresses. So, if a configuration mistake causes the production LAN to go down, administrators can still remotely fix the problem, shortening the duration of the outage. And, since OOB serial consoles provide a secondary network interface—often an LTE cellular modem—you’ll still have remote access even if human error brings down the WAN or SD-WAN architecture.
    .
  2. Power failures are another leading cause of outages
    Respondents reported that 43% of significant outages—ones that resulted in business interruption and financial loss—were caused by power issues. Many of those incidents were due to uninterruptible power supply (UPS) failures.As part of a data center infrastructure management (DCIM) solution, an OOB serial console gives administrators the ability to remotely monitor and manage UPS devices in the rack. Admins get alerts when devices aren’t performing efficiently or begin to show signs of imminent failure. That means organizations can proactively schedule repairs or deploy replacements before a power outage occurs.
    .
  3. Out-of-band data center connectivity shortens recovery time
    One of the most alarming statistics from the report is the percentage of public outages lasting more than 24 hours. In 2017, just 8% of outages lasted longer than a day, but that increased to nearly 30% in 2021.Out-of-band data center connectivity can significantly reduce the time to recovery by ensuring administrators always have remote access to data center infrastructure. That means your organization will waste less time waiting for on-site managed services to arrive or for in-house technicians to travel to the data center. As soon as DCIM monitoring alerts them to an issue, admins can begin diagnosing and fixing the problem from their remote desktop.
    .
  4. Outages are more expensive than ever
    Over 60% of reported outages resulted in at least $100,000 in losses, an increase of 21% since 2019. The number of outages costing more than $1 million also increased by 4%.OOB management gives teams the ability to remotely troubleshoot and recover from many issues, so you don’t need to pay for truck rolls or on-site managed services. If remote troubleshooting reveals that the problem requires an on-site fix, technicians can go in already knowing the source of the issue and with all the necessary tools to repair it. Either way, your organization saves time and money.

Out-of-band data center connectivity gives organizations reliable access to remote infrastructure even during a network outage. OOB serial consoles also provide visibility into the health and performance of critical data center devices like UPSs, so you can proactively address issues and prevent downtime from occurring. Through 24/7 remote access, monitoring, and management, you can reduce the incidence, duration, cost, and impact of data center downtime.

Gen 3 OOB data center connectivity with Nodegrid

The Nodegrid Serial Console Plus (NSCP) is a Gen 3 out-of-band data center connectivity solution that delivers reliable and blazing-fast remote access to up to 96 data center devices from a single 1U rack-mounted box. Nodegrid’s vendor-neutral OOB DCIM platform supports integrations with your choice of infrastructure solutions and automation tools, giving you total and efficient control over your data center infrastructure.

Ready to learn more about out-of-band data center connectivity?

To learn more about out-of-band data center connectivity with Nodegrid, contact ZPE Systems today.

Contact Us

Network Engineers: 5 Must-Have Tools During a Slow Economy

Network Engineers: 5 Must-Have Tools During a Slow Economy

Network engineers need powerful tools to keep digital services online and customers happy. This is especially true during economic downturn, when organizations must freeze hiring and put more strain on existing staff. Revenue relies on network availability, and with experts predicting a recession this winter, significant operational challenges are inevitable for most organizations.

The burden of overcoming these challenges falls on network engineers. Success means maintaining reliable services and reaping any professional benefits (salary increases, promotions, etc.). Failure, on the other hand, means the very realistic possibility of major business losses and job cuts, including yours.

In order to make sure you don’t fall into the latter scenario, here are five must-have tools and techniques to help network engineers overcome these challenges.

Tool 1. OOBI-LAN™

Out-of-band (OOB) management is an essential part of a network engineer’s toolkit. At the conceptual level, out-of-band is meant to provide management access to production equipment, even if the production equipment is offline.

One major problem is that many organizations invest a lot of time and money into their production infrastructure, but not into any dedicated OOB infrastructure. In other words, they deploy OOB solutions that rely in part on their production equipment, such as OOB VLANs connected to in-band switches. All it takes is a mistake, misconfiguration, or attack to bring down the production and management networks, leaving network engineers to rebuild the entire system from scratch while their services remain offline to customers. This is simply not acceptable in a slow economy, where the business’ resources and revenue are already too thin.

From the pandemic lockdowns, organizations have learned that they need a way to more quickly recover their network locations. According to the Uptime Institute’s 2022 Outage Analysis, outages lasting longer than 24 hours increased to nearly 30% in 2021. This has led many to build dedicated OOB infrastructure for the LAN (OOBI-LAN). They deploy a serial console locally to establish connectivity to the management ports of their sensitive equipment. Network engineers must use this serial console to access their production infrastructure. This serial console minimizes the attack surface since it’s the only device connected to the Internet, and allows network engineers to restore services even if production equipment is down.

Tool 2. OOBI-WAN™

A critical tool for network engineers is out-of-band that enables remote WAN management. But typically, organizations employ a WAN management strategy that also relies on their production infrastructure, such as for creating VPN tunnels for management traffic. If a VPN tunnel becomes broken or the production gear fails, network engineers are suddenly left without remote access to their equipment.

Aside from a lack of availability, traditional OOB access comes with real security risks. Exposing LTE modems to the Internet, leveraging untrusted third-party VPN services, using OOB hardware that’s old and unpatched, and worse — exposing the management port of devices to public Internet. All of these are attack surfaces, any of which can give access to your infrastructure and be used as the pivot point to get to the rest of the infrastructure.

traditional WAN management approach

Image: Management access depends on production equipment to establish VPN tunnels. 

On top of their OOBI-LAN, organizations have built dedicated OOB infrastructure for the WAN (OOBI-WAN – there’s a Star Wars reference somewhere in there) for added resilience against these scenarios.

OOBI-WAN is the WAN best practice

Image: OOBI-WAN and OOBI-LAN create a fully separate out-of-band infrastructure that can be used to completely rebuild production infrastructure. 

OOBI-WAN uses MPLS, IPsec, or SD-WAN links to create an overlay network dedicated specifically to management traffic. This gives network engineers private access to their infrastructure for management and troubleshooting, essentially creating a completely separate OOB network that does not rely on any part of the production network. OOBI-WAN lets network engineers use their WAN connection to remotely access their OOBI-LAN and fully rebuild their distributed networks, regardless of the state of their production infrastructure.  

A key part of OOBI-WAN is the inherent security that is built at all layers. To build secure OOBI-WAN, the best practice is to use OOBI-SDWAN™ which automates the building of VPN tunnels between all the nodes that need to be managed. OOBI-SDWAN provides the expected auto-VPN feature which means VPN encryption keys remain secure, as they don’t need to be copied/pasted/typed into multiple third-party devices. OOBI-SDWAN also ensures that an SLA is provided on the OOBI network along with observability dashboards of connectivity and the access state of the network. The combination of OOBI-SDWAN with a zero trust security framework is the best way to gain reliability in a way that reduces your risk.

OOBI-WAN hub and spoke

Tool 3. Fully independent automation infrastructure

Another tool that network engineers are becoming familiar with is automation. Network automation codifies repetitive tasks to reduce workloads for configuration management, compliance, and troubleshooting. During a slow economy, being able to scale an IT team’s efforts is especially valuable to business operations and end customers.

There is one major concern, however: having automation that runs loose and begins destroying the network, much like a bull in a China shop. Network engineers typically must learn new automation tools and programming languages, which requires trial and error. And because there is a lack of a best practice reference architecture, teams don’t know any better than to automate directly on the production network. This causes anxiety, as one mistake could bring down the network, cause catastrophic losses, and leave network engineers without an efficient way to recover.

Image: The orange section describes dedicated automation infrastructure used for safely implementing automation.

In recent years, teams have been deploying automation on dedicated infrastructure like their OOB network. This automation infrastructure sits between the production infrastructure and the orchestration infrastructure, and serves as a safe way to build an automation pipeline. Open, Linux-based appliances like the Nodegrid Net SR combine a variety of functions and can host automation tools, like those for observability and analytics, version control, and source of truth. This independent automation infrastructure allows network engineers to ensure the integrity of configuration changes, software updates, and remediation protocols in an out-of-band manner, rather than testing directly on the production network. They can scale their capabilities, and in case of errors, roll back to a golden configuration that keeps services online.

Tool 4. Remote access to local jump box

Network engineers have another tool at their disposal: the jump box (a.k.a. jump server, jump host). A jump box hosts tools for maintaining operations, and these include file servers, image storage, configuration management tools, and troubleshooting commands. The jump box is a valuable asset for normal operations and for restoring services, such as when a device fails and needs its image rebuilt.

The issue with jump boxes is that they are typically a separate device that requires power, cooling, rack space, and maintenance. Some jump boxes also require on-site technicians to physically connect to the equipment needing repair.

Many organizations have adapted by upgrading their OOB infrastructure with appliances that can run full virtual machines (VMs). These can run all the tools mentioned above as well as with Docker containers, while consolidating power consumption, cooling resources, and rack space. The OOB appliance can double as a jump box. Combined with OOBI-LAN and OOBI-WAN, network engineers get remote access to re-image a device, diagnose DNS/routing issues, and perform any other necessary tasks. Key point is that discrete jump boxes – Like the Intel NUC — to be converted to virtual jump boxes running on a secure OOB platform like the Nodegrid Service routers.

Tool 5. Smart hands

A final way that network engineers get help through a slow economy is by outsourcing to so-called ‘smart hands.’ Employing smart hands means involving a third-party expert who can take on some of the IT workload. It’s a viable strategy, especially for teams feeling crushed by corporate belt tightening and the resulting mountain of tasks.

Companies who take this approach must be aware that the skills of smart hands varies greatly, as does the cost. This means it’s essential to strike a balance between which tasks to outsource, and which tasks to keep in house. For example, many organizations use smart hands for simple jobs such as replacing hardware and installing equipment at new sites. For more specialized jobs that require deeper knowledge of the environment, such as fixing a misconfigured IP address or route, teams use in-house personnel. This balance helps organizations get the support they need to keep operations running.

Get a cheat sheet to implement these tools fast

Some companies thrive during economic downturn, because they’ve intelligently placed these tools within their network architecture. Over the past decade, we’ve worked with these companies — including the largest tech giants — to describe in painstaking detail how they set up their infrastructure. We just released all 40+ pages of this validated reference architecture, complete with implementation diagrams and examples.

It’s called the network automation blueprint and it combines all of these tools. Network engineers can confidently answer questions like:

  • How do we meet SLAs with a smaller workforce?
  • How can we keep sites operating without physical access to equipment?
  • How can we perform weekly updates/patching without breaking things?

The blueprint is your cheat sheet to implementing a more resilient network, and fast. Click the button below to download your copy now.