IT automation is driven by data and that data comes from an increasing variety of sources – and just like the data itself, those sources are rarely ever static. The ecosystem of IT applications, services, and data sources has changed over time and seems to be changing more often as time goes on. Different IT teams select the appropriate tools and applications for their domain and skillset to help get their work done, and typically focus less on how they interoperate with other systems and teams, or if they do interoperate, to what extent they can interoperate. The world of networking introduces yet another challenge to automation due to its complexity, management techniques, systems, and skillsets.
Because IT systems and processes across network domains have become increasingly dynamic, managing these integrations and the necessary data transformation techniques needed to automate between systems has posed a challenge for DevOps and NetDevOps personnel. Both teams need a way to simplify this process and utilize automation solutions that can make these different, complex, disparate systems work together in unison so their organizations can get the most out of their automations.
The Integration Tax of Automation for DevOps Teams
Automating even the simplest of IT processes requires accessing and manipulating data from more sources than you might realize, which means integrating all these sources is necessary for realizing the maximum benefits of automation.
Consider a “simple” network change for an application. There was probably a request from an ITSM system submitted by the application owner. That request is received by a network engineer, who has to determine the location of the change, the device to change, and the nature of the change which may need to be referenced in different applications, sources of truth, or even people. Before the change is made, a change request process may be needed with details of the change – gathered from all those sources and changed into the appropriate format by the network engineer. After review and approval, the changes can then be made and all of those data sources, and potentially other network services, like DNS, firewalls, or load balancers, must be updated with the most current network information.
As you can see and have probably experienced, there’s A LOT of different IT systems as part of this process. And for a more complex use case where multiple network domains are involved (SD-WAN, data center, multi-cloud), there would be even more integration required.
This puts a huge burden on the DevOps team to write the integration code. While they are well prepared and skilled for the task at hand, the burden comes by having to manage all this integration code over time. Remember, we live in an incredibly dynamic world. That even applies to the world of networking where change hasn’t historically been as rapid, as we have still seen more adoption of systems and tools to help automate. And even in the lifecycle of the same system, software is upgraded to a higher version, which can break the previous integration code and further adding to the integration tax that DevOps teams must overcome.
How Data Transformation Impacts Your Ability to Automate
The integration tax isn’t the only hurdle for DevOps teams. While integration gives your automation code the ability to communicate with your organization’s IT and network systems, it doesn’t mean the data from every system is in the same format.
Considering the data that is needed for automation tasks, it’s never returned from one system in the format necessary for another system. For that next system to act correctly on the data, it must be transformed and arranged into the appropriate format for that system. Once again, DevOps teams are well prepared and skilled with data manipulation techniques required in code to make these transformations but it’s managing them over time is so burdensome. Because data transformations often occur in different directions between multiple systems, the problem grows exponentially as changes occur in any of these systems.
To manage this, a programmer has to understand the nature of the changes needed and then back-track through lines and lines of code in multiple files to identify and fix the integration code, transformation code, or both. If a different system is upgraded next week that breaks the automation, they must repeat the entire process again, taking them away from building new automations so they can fix older ones. For many DevOps teams, this has become a full-time job and for others, this has become a considerable backlog and in either case it has severely impacted the ability to automate.
How Itential Simplifies Integration & Data Transformation for DevOps Teams
So how can DevOps teams go about alleviating the burden put on automation efforts by integration and data transformation? While the easier answer would be to reduce the number of systems, services, API changes, data formats, or even limit the adoption of new technology, that’s just not feasible. Every team wants the freedom to select the right tool for the job at hand.
The way to help DevOps teams overcome these challenges is to provide a better method to quickly integrate, at the API level, with any IT or network system at any time, and to manage those systems easily, providing a clear method to leverage API access for both old and new versions of applications.
Since data transformations follow the integration challenge, DevOps teams need a better way to create, update, and manage existing common data transformations between systems. For multiple systems that are frequently used together in automations, they need a way of building a standardized library of data transformations that can be reused by the DevOps team as well as the network team.
That’s precisely what we’ve engineered at Itential — an automation platform that provides a better way to manage the integration and data transformation necessary to build meaningful and scalable automations. With the Itential Automation Platform, you can rapidly integrate with any IT or infrastructure system, manage the data transformations between these systems within an automation, and allow any team to access the automation so that DevOps and network teams can work together to build automations that integrate and scale as IT continues to change and evolve.
Recently, I demoed exactly our how solution does this, making integration and data transformation a simple process for DevOps teams. You can see it in action here.
Rich Martin
Director of Technical Marketing ‐ Itential
Rich Martin is the Director of Technical Marketing at Itential. Previously, Rich has worked at several networking vendors as a both a Pre-Sales Systems Engineer and Systems Engineering Manager but started his career with a background in software development and Linux. He has a passion for automation in the networking domain, and at Itential he helps networking teams to get started quickly and move forward successfully on their network automation journey.
