Category: Technology

Transforming Manufacturing

Overview

Growing Up in Manufacturing

My father ran his own business when I was growing up.  His business had two components: first, he manufactured pinion wire (steel or brass rods of various diameters with teeth from which gears are cut) and, second, as a producer of specialty gears that were used in various applications (e.g., the timing mechanism of an oil pump).  It was something he used to raise and support a large Italian family and it was pretty much a one-man show, with help from his kids as needed, whether that was counting and quality testing gears with mating parts or cutting, packing, and shipping material to various customers across North America.  He acquired and learned how to operate screw machines to produce pinion wire but eventually shifted to a distribution business, where he would buy finished material in quantity and then distribute to middle market customers at lower volumes at a markup. 

His business was as low tech as you could get, with a little card file he maintained that had every order by customer written out on an index card, tracking the specific item/part, volume, and pricing so he had a way to understand history as new requests for quotes and orders came in and also understand purchase patterns over time.  It was largely a relationship business, and he took his customer commitments to heart at a level that dinner conversation could easily deflect into a worry about a disruption in his supply chain (e.g., something not making it to the electroplater on time) and whether he might miss his promised delivery date.  Integrity and accountability were things that mattered and it was very clear his customers knew it.  He had a note pad on which he’d jot things down to keep some subset of information on customer / prospect follow ups, active orders, pending quotes, and so on, but to say there was a system outside what he kept in his head would be unfair to his mental capacity, which was substantial.

It was a highly manual business and, as a person who taught myself how to write software in the third grade, I was always curious what he could do to make things a little easier, more structured, and less manual, even though that was an inescapable part of running his business on the whole.  That isn’t to say he had that issue or concern, given he’d developed a system and way of operating over many years, knew exactly how it worked, and was entirely comfortable with it.  There was also a small point of my father being relatively stubborn, but that didn’t necessarily deter me from suggesting things could be improved.  I do like a challenge, after all… 

As it happened, when I was in high school, we got our first home computer (somewhere in the mid-1980s) and, despite its relatively limited capacity, I thought it would be a good idea to figure out a way to make something about running his business a little easier with technology.  To that end, I wrote a piece of software that would take all of his order management off of the paper index cards and put it into an application.  The ability to look up customer history, enter new orders, look at pricing differences across customers, etc. were all things I figured would make life a lot easier, not mention reducing the need to maintain this overstuffed card file that seemed highly inefficient to me.

By this point, I suspect it’s clear to the reader what happened… which is that, while my father appreciated the good intentions and concept, there was no interest in changing the way he’d been doing business for decades in favor of using technology he found a lot more confusing and intimidating than what he already knew (and that worked to his level of satisfaction).  I ended up relatively disappointed, but learned a valuable lesson, which is that, the first challenge in transformation is changing mindsets… without that, the best vision in the world will fail, no matter what value it may create.

It Starts with Mindset

I wanted to start this article with the above story because, despite nearly forty years having passed since I tried to introduce a little bit of automation to my father’s business, manufacturing in today’s environment can be just as antiquated and resistant to change as it was then, seeing technology as an afterthought, a bolt on, or cost of doing business rather than the means to unlock the potential that exists to transform a digital business in even the most “low tech” of operating environments.

While there is an inevitable and essential dependence on people, equipment, and processes, my belief is that we have a long way to go on understanding the critical role technology plays in unlocking the potential of all of those things to optimize capacity, improve quality, ensure safety, and increase performance in a production setting.

The Criticality of Discipline

Having spent a number of years understanding various approaches to digital manufacturing, one point that I wanted to raise prior to going into more of the particulars is the importance of operating with a holistic vision and striking the balance between agility and long-term value creation.  As I addressed in my article Fast and Cheap Isn’t Good, too much speed without quality can lead to complexity, uncontrolled and inflated TCO, and an inability to integrate and scale digital capabilities over time.  Wanting something “right now” isn’t an excuse not to do things the right way and eventually there is a price to pay for tactical thinking when solutions don’t scale or produce more than incremental gains. 

This is also related to “Framework-Driven Design” that I talk about in my article on Excellence by Design.  It is rarely the case that there is an opportunity to start from scratch in modernizing a manufacturing facility, but I do believe there is substantial value in making sure that investments are guided by an overall operating concept, technology strategy, and evolving standards that will, over time, transform the manufacturing environment as a whole and unlock a level of value that isn’t possible where incremental gains are always the goal.  Sustainable change takes time.

The remainder of this article will focus on a set of areas that I believe form the core of the future digital manufacturing environment.  Given this is a substantial topic, I will focus on the breadth of the subject versus going too deep into any one area.  Those can be follow-up articles as appropriate over time.

 

Leveraging Data Effectively

The Criticality of Standards

It is a foregone conclusion that you can’t optimize what you can’t track, measure, and analyze in real-time.  To that end, starting with data and standards is critical in transforming to a digital manufacturing environment.  Without standards, the ability to benchmark, correlate, and analyze performance will be severely compromised.  This can be a basic as how a camera system, autonomous vehicle, drone, conveyor, or digital sensor is integrated within a facility, to the representation of equipment hierarchies, or how operator roles and processes are tracked across a set of similar facilities.  Where standards for these things don’t exist, value will be constrained to a set of individual point solutions, use cases, and one-off successes.  Where standards are, however, implemented and scaled over time, the value opportunity will eventually cross over into exponential gains that aren’t otherwise possible, because the technical debt associated with retrofitting and mapping across various standards in place will create a significant maintenance effort that limits focus on true innovation and optimization.  This isn’t to suggest that there is a one-size-fits-all way to thinking about standards and that every solution needs to conform for the sake of an ivory tower ideal.  The point is that it’s worth slowing down the pace of “progress” at times to understand the value in designing solutions for longer-term value creation.

The Role of Data Governance

It’s impossible to discuss the criticality of standards without also highlighting the need for active, ongoing data governance, both to ensure standards are followed, that data quality at the local and enterprise level is given priority (especially to the degree that analytical insights and AI become core to informed decision making), and also to help identify and surface additional areas of opportunity where standards may be needed to create further insights and value across the operating environment.  The upshot of this is that there need to be established roles and accountability for data stewards at the facility and enterprise level if there is an aspiration to drive excellence in manufacturing, no matter what the present level of automation is across facilities.

 

Modeling Distributed Operations

Applying Distributed Computing

There is a power in distributed computing that enables you to scale execution at a rate that is beyond the capacity you can achieve with a single machine (or processor).  The model requires an overall coordinator of activity to distribute work and monitor execution and then the individual processors to churn out calculations as rapidly as they are able.  As you increase processors, you increase capacity, so long as the orchestrator can continue to manage and coordinate the parallel activity effectively.

From a manufacturing standpoint, the concept applies well across a set of distributed facilities, where the overall goal is to optimize the performance and utilization of available capacity given varying demand signals, individual operating characteristics of each facility, cost considerations, preventative maintenance windows, etc.  It’s a system that can be measured, analyzed, and optimized, with data gathered and measured locally, a subset of which is used to inform and guide the macro-level process.

Striking the Balance

While I will dive into this a little further towards the tail end of this article, the overall premise from an operating standpoint is to have a model that optimizes the coordination of activity between individual operating units (facilities) that are running as autonomously as possible at peak efficiency, while distributing work across them in a way that maximizes production, availability, cost, or whatever other business parameters are most critical. 

The key point being that the technology infrastructure for distributing and enabling production across and within facilities should ideally be a matter of business parameters that can be input and adjusted at the macro-level and the entire system of facilities be adjusted in real-time in a seamless, integrated way.  Conversely, the system should be a closed loop where a disruption at the facility level can inform a change across the overall ecosystem such that workloads are redistributed (if possible) to minimize the impact on overall production.  This could be manifest in one or more micro-level events (e.g., a higher than expected occurrence of unplanned outages) that informs production scheduling and distribution of orders to a major event (e.g., a fire or substantial facility outage) that redirects work across other facilities to minimize end customer impact.  Arguably there are elements that exist within ERP systems that can account for some of this today, but the level and degree of customization required to make it a robust and inclusive process would be substantial, given much of the data required to inform the model exists outside the ERP ecosystem itself, in equipment, devices, processes, and execution within individual facilities themselves.

Thinking about Mergers, Acquisitions, and Divestitures

As I mentioned in the previous section on data, establishing standards is critical to enabling a distributed paradigm for operations, the benefit of which is also the speed at which an acquisition could be leveraged effectively in concert with an existing set of facilities.  This assumes there is an ability to translate and integrate systems rapidly to make the new facility function as a logical extension of what is already in place, but ultimately a number of those technology-related challenges would have to be worked through in the interest of optimizing individual facility performance regardless.  The alternative to having this macro-level dynamic ecosystem functioning would likely be excess cost, inefficiency, and wasted production capacity.

 

Advancing the Digital Facility

The Role of the Digital Facility

At a time when data and analytics can inform meaningful action in real-time, the starting point for optimizing performance is the individual “processor”, which is a digital facility.  While the historical mental model would focus on IT and OT systems and integrating them in a secure way, the emergence of digital equipment, sensors, devices, and connected workers has led to more complex infrastructure and an exponential amount of available data that needs to be thoughtfully integrated to maximize the value it can contribute over time.  With this increased reliance on technology, likely some of which runs locally and some in the cloud, the reliability of wired and wireless connectivity has also become a critical imperative of operating and competing as a digital manufacturer.

Thinking About Auto Maintenance

Drawing on a consumer example, I brought my car in for maintenance recently.  The first thing the dealer did was plug in and download a set of diagnostic information that was gathered over the course of my road trips over the last year and a half.  The data was collected passively, provided the technicians with input on how various engine components were performing, and also some insight on settings that I could adjust given my driving habits that would enable the car to perform better (e.g., be more fuel efficient).  These diagnostics and safety systems are part of having a modern car and we take them for granted.

Turning back to a manufacturing facility, a similar mental model should apply for managing data at a local and enterprise level, which is that there should be a passive flow of data to a central repository that is mapped to processes, equipment, and operators in a way that enables ongoing analytics to help troubleshoot problems, identify optimization and maintenance opportunities, and look across facilities for efficiencies that could be leveraged at broader scale.

Building Smarter Equipment

Taking things a step further… what if I were to attach a sensor under the hood of my car, take the data, build a model, and try to make driving decisions using that model and my existing dashboard as input?  The concept seems a little ridiculous given the systems already in place within a car to help make the driving experience safe and efficient.  That being said, in a manufacturing facility with legacy equipment, that intelligence isn’t always built in, and the role of analytics can become an informed guessing game of how a piece of equipment is functioning without the benefit of the knowledge of the people who built the equipment to begin with. 

Ultimately, the goal should be for the intelligence to be embedded within the equipment itself, to enable a level of self-healing or alerting, and then within control systems to look at operating conditions across a connected ecosystem to determine appropriate interventions as they occur, whether that be a minor adjustment to operating parameters or a level of preventative maintenance.

The Role of Edge Computing and Facility Data

The desire to optimize performance and safety at the individual facility level means that decisions need to be informed and actions taken in near real-time as much as possible.  This premise then suggests that facility data management and edge computing will continue to increase in criticality as more advanced uses of AI become part of everyday integrated work processes and facility operations.

 

Enabling Operators with Intelligence

The Knowledge Challenge

With the general labor shortage in the market and retirement of experienced, skilled laborers, managing knowledge and accelerating productivity is a major issue to be addressed in manufacturing facilities.  There are a number of challenges associated with this situation, not the least of which can be safety related depending on the nature of the manufacturing environment itself.  Beyond that, the longer it takes to make an operator productive in relation to their average tenure (something that statistics would suggest is continually shrinking over time), the effectiveness of the average worker can become a limiting factor in the operating performance of a facility overall.

Understanding Operator Overload

One way that things have gotten worse is the proliferation of systems that comes with “modernizing” the manufacturing environment itself.  When confronted with the ever-expanding set of control, IT, ERP, and analytical systems, all of which can be sending alerts and requesting action (to varying degrees of criticality) on a relatively continuous basis, the pressure being created on individual operators and supervisors in a facility has increased substantially (with the availability of exponential amounts of data itself).  This is further complicated in situations where an individual “wears multiple hats” in terms of fulfilling multiple roles/personas within a given facility and arbitrating which actions to take against that increased number of demands can be considerably more complex.

Why Digital Experience Matters

While the number of applications that are part of an operating environment may not be something that is easy to reduce or simplify without significant investment (and time to make change happen), it is possible to look at things like digital experience platforms (DXPs) as a means to manage multiple applications into a single, integrated experience, inclusive of AR/VR/XR technologies as appropriate.  Organizing around an individual operator’s responsibilities can help reduce confusion, eliminate duplicated data entry, improve data quality, and ultimately improve productivity, safety, and effectiveness by extension.

The Role of the Intelligent Agent

With a foundation in place to organize and present relevant information and actions to an operator on a real-time basis, the next level of opportunity comes with the integration of intelligent agents (AI-enabled tools) into a digital worker platform to inform meaningful and guided actions that will ultimate create the most production and safety impact on an ongoing basis.  Again, there is a significant dependency on edge computing, wireless infrastructure, facility data, mobile devices, a delivery mechanism (the DXP mentioned above), and a sound underlying technology strategy to enable this at scale, but it is ultimately where AI tools can have a major impact in manufacturing moving forward.

 

Optimizing Performance through Orchestration

Why Orchestration Matters

Orchestration itself isn’t a new concept in manufacturing from my perspective, as the legacy versions of it are likely inherent in control and MES systems themselves.  The challenge occurs when you want to scale that concept out to include digital equipment, digital workers, digital devices, control systems, and connected applications into one, seamless, integrated end-to-end process.  Orchestration provides the means to establish configurable and dynamic workflow and associated rules into how you operate and optimize performance within and across facilities in a digital enterprise.

While this is definitely a capability that would need to be developed and extended over time, the concept is to think of the manufacturing ecosystem as a seamless collaboration of operators and equipment to ultimately drive efficient and safe production of finished goods. Having established the infrastructure to coordinate and track activity, the process performance can be automatically recorded and analyzed to inform continuous improvement on an ongoing basis.

Orchestrating within the Facility

The number of uses of orchestration within a facility can be as simple as coordinating and optimizing material movement between autonomous vehicles and fork lifts within a facility, to computer vision applications for safety and quality management.  With the increasing number of connected solutions within a facility, having the means to integrate and coordinate activity between and across them offers a significant opportunity in digital manufacturing moving forward.

Orchestrating across the Enterprise

Scaling back out to the enterprise level, looking across facilities, there are opportunities to look at things like procurement of MRO supplies and optimizing inventory levels, managing and optimizing production planning across similar facilities, benchmarking and analyzing process performance and looking for improvements that can be applied across facilities in a way that will create substantially greater impact than is possible if the focus is limited to the individual facility alone.  Given that certain enterprise systems like ERPs tend to operate at largely a global versus local level, having infrastructure in place to coordinate activity across both can create visibility to improvement opportunities and thereby substantial value over time.

Coordinated Execution

Finally, to coordinate between the local and global levels of execution, a thoughtful approach to managing data and the associated analytics needs to be taken.  As was mentioned in the opening, the overall operating model is meant to leverage a configurable, distributed paradigm, so the data that is shared and analyzed within and across layers is important to calibrate as part of the evolving operating and technology strategy.

 

Wrapping Up

There is a considerable amount of complexity associated with moving between a legacy, process and equipment-oriented mindset to one that is digitally enabled and based on insight-driven, orchestrated action.  That being said, the good news is that the value that can be unlocked with a thoughtful digital strategy is substantial given we’re still on the front-end of the evolution overall.

I hope the ideas were worth considering.  Thanks for spending the time to read them.  Feedback is welcome as always.

-CJG 02/12/2024

Fast and Cheap, Isn’t Good…

Overview

Having touched on the importance of quality in accelerating value in my latest article Creating Value Through Strategy, I wanted to dive a little deeper into the topic of “speed versus quality”.

For those who may be unfamiliar, there is a general concept in project delivery that the three primary dimensions against which you operate are good (the level of effort you put into ensuring a product is well architected and meets functional and non-functional requirements at the time of delivery), fast (how quickly or often you produce results), and cheap (your ability to deliver the product/solution at a reasonable cost). 

The general assumption is that the realities of delivery lead you to having to prioritize two of the three (e.g., you can deliver a really good product fast, but it won’t be cheap; or you can deliver a really good product at a low cost, but it will take a lot of time [therefore not be “fast”]).  What this translates to, in my experience, has nearly always been that speed and cost are prioritized highest, with quality being the item compromised.

Where this becomes an issue is in the nature of the tradeoff that was made and the longer-term implications of those decisions.  Quality matters.  My assertion is that, where quality is compromised, “cheap” is only true in the short-term and definitely not the case overall.

The remainder of this article will explore several dimensions to consider when making these decisions.  This isn’t to say that there aren’t cases where there is a “good enough” level of quality to deliver a meaningful or value-added product or service.  My experience, however, has historically been that the concepts like “we didn’t have the time” or “we want to launch and learn” are often used as a substitute for discipline in delivery and ultimately undermine business value creation.

Putting Things in Perspective

Dimensions That Matter

I included the diagram above to put how I think of product delivery into perspective.  In the prioritization of good, fast, and cheap, what often occurs is that too much focus and energy goes into the time spent getting a new capability or solution to market, but not enough on what happens once it is there and the implications of that.  The remainder of this section will explore aspects of that worth considering in the overall context of product/solution development.

Some areas to consider in how a product is designed and delivered:

  • Architecture
    • Is the design of the solution modular and component- or service-based? This is important to the degree that capabilities may emerge over time that surpass what was originally delivered and, in a best-of-breed environment, you would ideally like to be able to replace part of a solution without having to fundamentally rearchitect or materially refactor the overall solution
    • Does the solution conform to enterprise standards and guidelines? I’ve seen multiple situations where concurrent, large-scale efforts were designed and developed without consideration for their interoperability and adherence to “enterprise” standards.  By comparison, developing on a “program-“ or “project-level”, or in working with a monolithic technology/solution (e.g., with a relatively closed ERP system), creates technology silos that lead to a massive amount of technical debt as it is almost never the case that there is leadership appetite for refactoring or rewriting core aspects of those solutions over time
    • Is the solution cloud-native and does it support containerization to enable deployment of workloads across public and private clouds as well as the edge? In the highly complex computing environments of today, especially in industries like Manufacturing, the ability to operate and distribute solutions to optimize availability, performance, and security (at a minimum) is critical.  Where these dimensions aren’t taken into account, there would likely be almost an immediate need for modernization to offset the risk of technology obsolescence at some point in the next year or two  
  • Security
    • Does the product or service leverage enterprise technologies and security standards? Managing vulnerabilities and migrating towards “zero trust” is a critical aspect of today’s technology environment, especially to the degree that workloads are deployed on the public cloud.  Where CI/CD pipelines are developed as part of a standard cloud platform strategy with integrated security tooling, the enterprise level ability to manage, monitor, and mitigate security risk will be significant improved
  • Integration
    • Does the product or service leverage enterprise technologies and integration standards? Interoperability with other internal and external systems, as well as your ability to introduce and leverage new capabilities and rationalize redundant solutions over time is fundamentally dependent on the manner in which applications are architected, designed, and integrated with the rest of a technology footprint.  Having worked in environments with well-defined standards and strictly enforced governance versus ones where neither were in place, the level of associated complexity and costs in the ultimate operating environments was materially different
  • Data Standards
    • Does the product or service align to overall master data requirements for the organization? Master data management can be a significant challenge from a data governance standpoint, which is why giving this consideration up front in a product development lifecycle is extremely important.  Where it isn’t considered in design, the end result could be master data that doesn’t map or align to other hierarchies in place, complicating integration and analytics intended to work across solutions and the “cleanup” required of data stewards (to the degree that they are in place) could be expensive and difficult post-deployment
    • Are advanced analytics aspirations taken into account in the design process itself? This is an area becoming increasingly important given AI-enabled (“intelligent”) applications as discussed in my article on The Intelligent Enterprise.  Designing with data standards in mind and an eye towards how it will be used to enable and drive analytics, likely in concert with data in other adjacent or downstream systems is a step that can save considerable effort and cost downstream when properly addressed early in the product development cycle
  • “Good Enough”/Responsive Architecture
    • All the above points noted, I believe architecture needs to be appropriate to the nature of the solution being delivered. Having worked in environments where architecture standards were very “ivory tower”/theoretical in nature and made delivery extremely complex and costly versus ones where architecture was ignored and the delivery environment was essentially run with an “ask for forgiveness” or cowboy/superhero mentality, the ideal state in my mind should be somewhere in between, where architecture is appropriate to the delivery circumstances, but also mindful of longer-term implications of the solution being delivered so as to minimize technical debt and further interoperability in a connected enterprise ecosystem environment.

Thinking Total Cost of Ownership

What makes product/software development challenging is the level of unknowns that exist.  At any given time, when estimating a new endeavor, you have the known, the known unknown, and the complete unknown (because what you’re doing is outside your team’s collective experience).  The first two components can be incorporated into an estimation model that can be used for planning and the third component can be covered through some form of “contingency” load that is added to an estimate to account for those blind spots to a degree.

Where things get complicated is, once execution begins, the desire to meet delivery commitments (and the associated pressure thereof) can influence decisions being made on an ongoing basis.  This is complicated by the normal number of surprises that occur during any delivery effort of reasonable scale and complexity (things don’t work as expected, decisions or deliverables are delayed, requirements become increasingly clear over time, etc.).  The question is whether a project has both disciplined, courageous leadership in place and the appropriate level of governance to make sure that, as decisions need to made in the interest of arbitrating quality, cost, time, and scope, that they are done with total cost of ownership in mind.

As an example, there was a point in the past where I encountered a large implementation program ($100MM+ in scale) with a timeline of over a year to deploy an initial release.  During the project, the team announced that all the pivotal architecture decisions needed to be made within a one-week window of time, suggesting that the “dates wouldn’t be met” if that wasn’t done.  That logic was then used at a later point to decide that standards shouldn’t be followed for other key aspects of the implementation in the interest of “meeting delivery commitments”.  What was unfortunate in this situation was that, not only were good architecture and standards not implemented, the project encountered technical challenges (likely due to one or two of those root causes, among other things) that caused it to be delivered over a year late regardless.  The resulting solution was more difficult to maintain, integrate, scale, or leverage for future business needs.  In retrospect, was any “speed” obtained through that decision making process and the lack of quality in the solution?  Certainly not, and this situation unfortunately isn’t unique to larger scale implementations in my experience.  In these cases, the ongoing run rate of the program itself can become an excuse to make tactical decisions that ultimately create a very costly and complex solution to manage and maintain in the production environment, none of which anyone typically wants to remediate or rewrite post-deployment.

So, given the above example, the argument could be made that the decisions were a result of inexperience or pure unknowns that existed when the work was estimated and planned to begin with, which is a fair point.  Two questions come to mind in terms of addressing this situation:

  • Are ongoing changes being reviewed through a change control process in relation to project cost, scope, and deadline, or are the longer-term implications in terms of technical debt and operating cost of ownership also considered? Compromise is a reality of software delivery and there isn’t a “perfect world” situation pretty much ever in my experience.  That being said, these choices should be conscious ones, made with full transparency and in a thoughtful manner, which is often not the case, especially when the pressures surrounding a project are high to begin with.
  • Are the “learnings” obtained on an ongoing basis factored into the estimation and planning process so as to mitigate future needs to compromise quality when issues arise? Having been part of and worked closely with large programs over many years, there isn’t a roadmap that ever plays out in practice how it is drawn up on paper at the outset.  That being said, every time the roadmap is revised, as pivot points in the implementation are reached and plans adjusted, are learnings being incorporated such that mistakes or sacrifices to quality aren’t being repeated over and over again.  This is a tangible thing that can be monitored and governed over time.  In the case of Agile-driven efforts, it would be as simple as looking for patterns in the retrospectives (post-sprint) to see whether the process is improving or repeating the same mistakes (a very correctible situation with disciplined delivery leadership)

Speed on the Micro- Versus Macro-Scale

I touched on this somewhat in the previous point, but the point to call out here is that tactical decisions made in the interest of compromising quality for the “upcoming release” can and often do create technical issues that will ultimately make downstream delivery more difficult (i.e., slower and more costly).

As an example, there was a situation in the past where a team integrated technology from multiple vendors that provided the same underlying capability (i.e., the sourcing strategy didn’t have a “preferred provider”, so multiple buys were done over time using different partners, sometimes in parallel).  In each case, the desire from the team was to deliver solutions as rapidly as possible in the interest of “meeting customer demand” and they were recognized and rewarded for doing so at speed.  The problem with this situation was that the team perceived standards as an impediment to the delivery process and, therefore, either didn’t leverage any or did so on a transactional or project-level basis.  Where this became problematic was where there became a need to:

  • Replace a given vendor – other partners couldn’t be leveraged because they weren’t integrated in a common way
  • Integrate across partners – the technology stack was different and defined unique to each use case
  • Run analytics across solutions – data standards weren’t in place so that underlying data structures were in a common format

The point of sharing the example is that, at a micro-level, the team’s approach seems fast, cheap, and appropriate.  The accumulation of the technical debt, however, is substantial when you scale and operate under that mindset for an extended period of time, and it does both limit your ability to leverage those investments, migrate to new solutions, introduce new capabilities quickly and effectively, and integrate across individual point solutions where needed.  Some form of balance should be in place to optimize the value created and cost of ownership over time.  Without it, the technical debt will undermine the business value in time.

Consulting Versus Corporate Environments

Having worked in both corporate and consulting environments, it’s interesting to me that there can be a different perspective on quality depending on where you sit (and the level of governance in place) in relation to the overall delivery.

Generally speaking, it’s somewhat common on the corporate side of the equation to believe that consultants lack the knowledge of your systems and business to deliver solutions you could yourself “if you had the time”.  By contrast, on the consulting side, ideally, you believe that clients are thinking of you as a “hired gun” when it comes to implementations, because you’re bringing in necessary skills and capacity to deliver on something they may not have the experience or bench strength to deliver on their own.

So, with both sides thinking they know more than the other and believing they are capable of doing a quality job (no one does a poor job on purpose), why is quality so often left unattended on larger scale efforts?  

On this point:

  • The delivery pressures and unknowns I mentioned above apply regardless of who is executing a project.
  • A successful delivery in many cases requires a blend of internal and external resources (to the extent they are being leveraged) so there is a balance of internal knowledge and outside expertise to deliver the best possible solution from an objective standpoint.
  • Finally, you can’t deliver to standards of excellence that aren’t set. I’ve seen and worked in environments (both as a “client” and as a consultant) where there were very exacting standards and expectations of quality and ones where quality wasn’t governed at the level it should be

I didn’t want to belabor this aspect of delivery, but it is interesting how the perspective and influence over quality decisions can be different depending on one’s role in the delivery process (client, consultant, or otherwise).

Wrapping Up

Bringing things back to the overall level, the point of writing this article was to provide some food for thought on the good, fast, cheap concept and the reality that, in larger and more complex delivery situations, the cost of speed isn’t always evaluated effectively.  There is no “perfect world”, for certain, but having discipline, thinking through some of the dimensions above, and making sure the tradeoffs made are thoughtful and transparent in nature could help improve value/cost delivered over time.

I hope the ideas were worth considering.  Thanks for spending the time to read them.  Feedback is welcome as always.

-CJG 12/10/2023

Creating Value Through Strategy

Context

One of the things that I’ve come to appreciate over the course of time is the value of what I call “actionable strategy”.  By this, I mean a blend of the conceptual and practical, a framework that can be used to set direction and organize execution without being too prescriptive, while still providing a vision and mental model for leadership and teams to understand and align on the things that matter.

Without a strategy, you can have an organization largely focused on execution, but that tends to create significant operating or technical debt and complexity over time, ultimately having an adverse impact on competitive advantage, slowing delivery, and driving significant operating cost.  Similarly, a conceptual strategy that doesn’t provide enough structure to organize and facilitate execution tends to create little impact over time as teams don’t know how to apply it in a practical sense, or it can add significant overhead and cost in the administration required to map its strategic objectives to the actual work being done across the organization (given they aren’t aligned up front or at all).  The root causes of these situations can vary, but the important point is to recognize the criticality of an actionable business-aligned technology strategy and its role in guiding execution (and thereby the value technology can create for an organization).

In reality, there are so many internal and external factors that can influence priorities in an organization over time, that one’s ability to provide continuity of direction with clear conceptual outcomes (while not being too hung up on specific “tasks”) can be important in both creating the conditions for transformation and sustainable change without having to “reset” that direction very often.  This is the essence of why framework-centric thinking is so important in my mind.  Sustainable change takes time, because it’s a mindset, a culture, and way of operating.  If a strategy is well-conceived and directionally correct, the activities and priorities within that model may change, but the ability to continue to advance the organization’s goals and create value should still exist.  Said differently: Strategies are difficult to establish and operationalize.  The less you have to do a larger-scale reset of them, the better.  It’s also far easier to adjust priorities and activities than higher-level strategies, given the time it takes (particularly in larger organizations) to establish awareness of a vision and strategy.  This is especially true if the new direction represents a departure from what has been in place for some time.

To be clear, while there is a relationship between this topic and what I covered in my article on Excellence By Design, the focus there is more on the operation and execution of IT within an organization, not so much the vision and direction of what you’d ideally like to accomplish overall.

The rest of this article will focus on the various dimensions that I believe compromise a good strategy, how I think about them, and ways that they could create measurable impact.  There is nothing particularly “IT-specific” about these categories (i.e., this is conceptually akin to ‘better, faster, cheaper’) and I would argue they could apply equally well to other areas of a business, but differ in how they translate on an operating level.

In relation to the Measures outlined in each of the sections below, a few notes for awareness:

  • I listed several potential areas to consider and explore in each section, along with some questions that come to mind with each.
  • The goal wasn’t to be exhaustive or suggest that I’d recommend tracking any or all of them on an “IT Scorecard”, rather to provide some food for thought
  • My general point of view is that it’s better to track as little as possible from an “IT reporting” standpoint, unless there is intention to leverage those metrics to drive action and decisions. My experience with IT metrics historically is that they are overreported and underleveraged (and therefore not a good use of company time and resources).  I touch on some of these concepts in the article On Project Health and Transparency

Innovate

What It Is and Why It Matters

Stealing from my article on Excellence By Design: “Relentless innovation is the notion that anything we are doing today may be irrelevant tomorrow, and therefore we should continuously improve and reinvent our capabilities to ones that create the most long-term value.”

Technology is evolving at a rate faster than most organizations’ ability to adopt or integrate those capabilities effectively.  As a result, a company’s ability to leverage these advances becomes increasingly challenging over time, especially to the degree that the underlying environment isn’t architected in a manner to facilitate their integration and adoption. 

The upshot of this is that the benefits to be achieved could be marginalized as any attempts to capitalize on these innovations will likely become point solutions or one-off efforts that don’t scale or create a different form of technical debt over time.  This is very evident in areas like analytics where capabilities like GenAI and other artificial intelligence-oriented solutions are only as effective as the underlying architecture of the environment into which they are integrated.  Are wins possible that could be material from a business standpoint?  Absolutely yes.  Will it be easy to scale them if you don’t invest in foundational things to enable that?  Very likely not.

The positive side of this is that technology is in a much different place than it was ten or twenty years ago, where it can significantly improve or enhance a company’s capabilities or competitive position.  Even in the most arcane of circumstances, there likely is an opportunity for technology to fuel change and growth in a digital business environment, whether that is internal to the operations of a company, or through its interactions with customers, suppliers, or partners (or some combination thereof).

Key Dimensions to Consider

Thinking about this area, a number of dimensions came to mind:

  • Promoting Courageous Leadership
    • This begins by acknowledging that leadership is critical to setting the stage for innovation over time
    • There are countless examples of organizations that were market leaders who ultimately lost their competitive advantage due to complacency or an inability to see or respond to changing market conditions effectively
  • Fueling Competitive Advantage
    • This is about understanding how technology helps create competitive advantage for a company and focusing in on those areas rather than trying to do everything in an unstructured or broad-based way, which would likely diffuse focus, spread critical resources, and marginalize realized benefits over time
  • Investing in Disciplined Experimentation
    • This is about having a well-defined process to enable testing out new business and technology capabilities in a way that is purposeful and that creates longer-term benefits
    • The process aspect of this important as it is relatively easy to spin up a lot of “innovation and improvement” efforts without taking the time to understand and evaluate the value and implications of those activities in advance. The problem of this being that you can either end up wasting money where the return on investment isn’t significant or that you can develop concepts that can’t easily be scaled to production-level solutions, which will limit their value in practice
  • Enabling Rapid Technology Adoption
    • This dimension is about understanding the role of architecture, standards, and governance in integrating and adopting new technical capabilities over time
    • As an example, an organization with an established component (or micro-service) architecture and integration strategy should be able to test and adopt new technologies much faster than one without them. That isn’t to suggest it can’t be done, but rather that the cost and time to execute those objectives will increase as delivery becomes more of a brute force situation than one enabled by a well-architected environment
  • Establishing a Culture of Sustainability
    • Following onto the prior point, as new solutions are considered, tested, and adopted, product lifecycle considerations should come into play.
    • Specifically, as part of the introduction of something new, is it possible to replace or retire something that currently exists?
    • At some point, when new technologies and solutions are introduced in a relatively ungoverned manner, it will only be a matter of time before the cost and complexity of the technology footprint will choke an organization’s ability to continue to both leverage those investments and to introduce new capabilities rapidly.

Measuring Impact

Several ways to think about impact:

  • Competitive Advantage
    • What is a company’s absolute position relative to its competition in markets where they compete and on metrics relative to those markets?
  • Market Differentiation
    • Is innovation fueling new capabilities not offered by competitors?
    • Is the capability gap widening or narrowing over time?
    • I separated these first two points, though they are arguably flavors of the same thing, to emphasize the importance of looking at both capabilities and outcomes from a competitive standpoint. One can be doing very well from a competitive standpoint relative to a given market, but have competitors developing or extending their capabilities faster, in which case, there could be risk of the overall competitive position changing in time
  • Reduced Time to Adopt New Solutions
    • What is the average length of time between a major technology advancement (e.g., cloud computing, artificial intelligence) becoming available and an organization’s ability to perform meaningful experiments and/or deploy it in a production setting?
    • What is the ratio of investment on infrastructure in relation to new technologies meant to leverage it over time?
  • Reduced Technical Debt
    • What percentage of experiments turn into production solutions?
    • How easy is to scale those production solutions (vertically or horizontally) across an enterprise?
    • Are new innovations enabling the elimination of other legacy solutions? Are they additive and complementary or redundant at some level?

Accelerate

What It Is and Why It Matters

“Take as much as time as you need, let’s make sure we do it right, no matter what.”  This is a declaration that I don’t think I’ve ever heard in nearly thirty-two years in technology.  Speed matters, “first mover advantage”, or any other label one could place upon the desire to produce value at a pace that is at or beyond an organization’s ability to integrate and assimilate all the changes.

That being said, the means to speed is not just a rush to iterative methodology.  The number of times I’ve heard or seen “Agile Transformation” (normally followed by months of training people on concepts like “Scrum meetings”, “Sprints”, and “User Stories”) posed as a silver bullet to providing disproportionate delivery results goes beyond my ability to count and it’s unfortunate.  Similarly, I’ve heard glorified versions of perpetual hackathons championed, where the delivery process involves cobbling together solutions in a “launch and learn” mindset that ultimately are poorly architected, can’t scale, aren’t repeatable, create massive amounts of technical debt, and never are remediated in production.  These are cases where things done in the interest of “speed” actually destroy value over time.

That being said, moving from monolithic to iterative (or product-centric) approaches and DevSecOps is generally a good thing to do.  Does this remedy issues in a business/IT relationship, solve for a lack of architecture, standards and governance, address an overall lack of portfolio-level prioritization, or a host of other issues that also affect operating performance and value creation over time?  Absolutely not.

The dimensions discussed in this section are meant to highlight a few areas beyond methodology that I believe contribute to delivering value at speed, and ones that are often overlooked in the interest of a “quick fix” (which changing methodology generally isn’t).

Key Dimensions to Consider

Dimensions that are top of mind in relation to this area:

  • Optimizing Portfolio Investments
    • Accelerating delivery begins by first taking a look at the overall portfolio makeup and ensuring the level of ongoing delivery is appropriate to the capabilities of the organization. This includes utilization of critical knowledge resources (e.g., planning on a named resource versus an FTE-basis), leverage of an overall release strategy, alignment of variable capacity to the right efforts, etc.
    • Said differently, when an organization tries to do too much, it tends to do a lot of things ineffectively, even under the best of circumstances. This does not help enhance speed to value at the overall level
  • Promoting Reuse, Standards, and Governance
    • This dimension is about recognizing the value that frameworks, standards and governance (along with architecture strategy) play in accelerating delivery over time, because they become assets and artifacts that can be leveraged on projects to reduce risk as well as effort
    • Where these things don’t exist, there almost certainly will be an increase in project effort (and duration) and technical debt that ultimately will slow progress on developing and integrating new solutions into the landscape
  • Facilitating Continuous Improvement
    • This dimension is about establishing an environment where learning from mistakes is encouraged and leveraged proactively on an ongoing basis to improve the efficacy of estimation, planning, execution, and deployment of solutions
    • It’s worth noting that this is as much an issue of culture as of process, because teams need to know that it is safe, expected, and appreciated to share learnings on delivery efforts if there is to be sustainable improvement over time
  • Promoting Speed to Value
    • This is about understanding the delivery process, exploring iterative approaches, ensuring scope is managed and prioritized to maximize impact, and so on
    • I’ve written separately that methodology only provides a process, not necessarily a solution to underlying cultural or delivery issues that may exist. As such, it is part of what should be examined and understood in the interest of breaking down monolithic approaches and delivering value at a reasonable pace and frequency, but it is definitely not a silver bullet.  They don’t, nor will they ever exist.
  • Establishing a Culture of Quality
    • In the proverbial “Good, Fast, or Cheap” triangle, the general assumption is that you can only choose two of the three as priorities and accept that the third will be compromised. Given that most organizations want results to be delivered quickly and don’t have unlimited financial resources, the implication is that quality will be the dimension that suffers.
    • The irony of this premise is that, where quality is compromised repeatedly on projects, the general outcome is that technical debt will be increased, maintenance effort along with it, and future delivery efforts will be hampered as a consequence of those choices
    • As a result, in any environment where speed is important, quality needs to be a significant focus so ongoing delivery can be focused as much as possible on developing new capabilities and not fixing things that were not delivered properly to begin with

Measuring Impact

Several ways to think about impact:

  • Reduced Time to Market
    • What is the average time from approval to delivery?
    • What is the percentage of user stories/use cases delivered per sprint (in an iterative model)? What level of spillover/deferral is occurring on an ongoing basis (this can be an indicator of estimation, planning, or execution-related issues)?
    • Are retrospectives part of the delivery process and valuable in terms of their learnings?
  • Increase in Leverage of Standards
    • Is there an architecture review process in place? Are standards documented, accessible, and in use?  Are findings from reviews being implemented as an outcome of the governance process?
    • What percentage of projects are establishing or leveraging reusable common components, services/APIs, etc.?
  • Increased Quality
    • Are defect injection rates trending in a positive direction?
    • What level of severity 1/2 issues are uncovered post-production in relation to those discovered in testing pre-deployment (efficacy of testing)?
    • Are criteria in place and leveraged for production deployment (whether leveraging CI/CD processes or otherwise)?
    • Is production support effort for critical solutions decreasing over time (non-maintenance related)?
  • Lower Average Project Cost
    • Is the average labor cost/effort per delivery reducing on an ongoing basis?

Optimize

What It Is and Why It Matters

Along with the pursuit of speed, it is equally important to pursue “simplicity” in today’s complex technology environment.  With so many layers now being present, from hosted to cloud-based solutions, package and custom software, internal and externally integrated SaaS and PaaS solutions, digital equipment and devices, cyber security requirements, analytics solutions, and monitoring tools… complexity is everywhere.  In large organizations, the complexity tends to be magnified for many reasons, which can create additional complexities in and across the technology footprint and organizations required to design, deliver, and support integrated solutions at scale.

My experience with optimization historically is that it tends to be too reactive of a process, and generally falls by the wayside when business conditions are favorable.  The problem with this is the bloat and inefficiency that tends to be bred in a growth environment, that ultimately reduces the value created by IT with increasing levels of spend.  That is why a purposeful approach that is part of a larger portfolio allocation strategy is important.  Things like workforce and sourcing strategy, modernization, ongoing rationalization and simplification, standardization and continuous improvement are important to offset what otherwise could lead to a massive “correction” the minute conditions change.  I would argue that, similar to performance improvement in software development, an organization should never be so cost inefficient that a massive correction is even possible.  For that to be the case, something extremely disruptive should have occurred, otherwise the discipline in delivery and operations likely wasn’t where it needed to be leading up to that adjustment.

I’ve highlighted a few dimensions that are top of mind in regard to ongoing optimization, but have written an entire article on optimizing value over cost that is a more thorough exploration of this topic if this is of interest (Optimizing the Value of IT).

Key Dimensions to Consider

Dimensions that are top of mind in relation to this area:

  • Reducing Complexity
    • There is some very simple math related to complexity in an IT environment, which is that increasing complexity drives a (sometimes disproportionate) increase in cost and time to deliver solutions, especially where there is a lack of architecture standards and governance
    • In areas like Integration and Analytics, this is particularly important, given they are both foundational and enable a significant amount of business capabilities when done well
    • It is also important to clarify that reducing complexity doesn’t necessarily equate to reducing assets (applications, data solutions, technologies, devices, integration endpoints, etc.), because it could be the case that the number of desired capabilities in an organization requires an increasing number of solutions over time. That being said, with the right integration architecture and associated standards, as an example, the ability to integrate and rationalize solutions will be significantly easier and faster than without them (which is complexity of a different kind)
  • Optimizing Ongoing Costs
    • I recently wrote an article on Optimizing the Value of IT, so I won’t cover all that material again here
    • The overall point is that there are many levers available to increase value while managing or reducing technology costs in an enterprise
    • That being said, aggregate IT spend can and may increase over time, and be entirely appropriate depending on the circumstances, as long as the value delivered increases proportionately (or in excess of that amount)
  • Continually Modernizing
    • The mental model that I’ve had for support for a number of years is to liken it to city planning and urban renewal. Modernizing a footprint is never a one-time event, it needs to be a continuous process
    • Where this tends to break down in many organizations is the “Keep the Lights On” concept, which suggests that maintenance spend should be minimized on an ongoing basis to allow the maximum amount of funding for discretionary efforts that advance new capabilities
    • The problem with this logic is that it can tend to lead to neglect of core infrastructure and solutions that then become obsolete, unsupportable, pose security risks, and that approach end of life with only very expensive and disruptive paths to upgrade or modernize them
    • It would be far easier to carve out a portion of the annual spend allocation for a thoughtful and continuous modernization where these become ongoing efforts, are less disruptive, and longer-term costs are managed more effectively at lower overall risk
  • Establishing and Maintaining a Workforce Strategy
    • I have an article in my backlog for this blog around workforce and sourcing strategy, having spent time developing both in the past, so I won’t elaborate too much on this right now other than to say it’s an important component in an organizational strategy for multiple reasons, the largest being that it enables you to flex delivery capability (up and down) to match demand while maintaining quality and a reasonable cost structure
  • Proactively Managing Performance
    • Unpopular though it is, my experience in many of the organizations in which I’ve worked over the years has been that performance management is handled on a reactive basis
    • Particularly when an organization is in a period of growth, notwithstanding extreme situations, the tendency can be to add people and neglect the performance management process with an “all hands, on deck” mentality that ultimately has a negative impact on quality, productivity, morale, and other measures that matter
    • This isn’t an argument for formula-driven processes, as I’ve worked in organizations that have forced performance curves against an employee population, and sometimes to significant, detrimental effect. My primary argument is that I’d rather have an environment with 2% involuntary annual attrition (conceptually), than one where it isn’t managed at all, market conditions change, and suddenly there is a push for a 10% reduction every three years, where competent “average” talent is caught in the crossfire.  These over-corrections cause significant disruption, have material impact on employee loyalty, productivity, and morale, and generally (in my opinion) are the result of neglecting performance management on an ongoing basis

Measuring Impact

Several ways to think about impact:

  • Increased Value/Cost Ratio
    • Is the value delivered for IT-related effort increasing in relation to cost (whether the latter is increasing, decreasing, or remaining flat)?
  • Reduced Overall Assets
    • Have the number of duplicated/functionally equivalent/redundant assets (applications, technologies, data solutions, devices, etc.) reduced over time?
  • Lower Complexity
    • Is the percentage of effort on the average delivery project spent on addressing issues related to a lack of standards, unique technologies, redundant systems, etc. reducing over time?
  • Lower Technical Debt
    • What percentage of overall IT spend is committed to addressing quality, technology, end-of-life, or non-conformant solutions (to standards) in production on an ongoing basis?

Inspire

What It Is and Why It Matters

Having written my last article on culture, I’m not going to dive deeply into the topic, but I believe the subject of employee engagement and retention (“People are our greatest asset…”) is often spoken about, but not proportionately acted on in deliberate ways.  It is far different, as an example, to tell employees their learning and development is important, but then either not provide the means for them to receive training and education or put “delivery” needs above that growth on an ongoing basis.  It’s expedient on a short-term level, but the cost to an organization in loyalty, morale, and ultimately productivity (and results) is significant.

Inspiration matters.  I fundamentally believe you achieve excellence as an organization by enrolling everyone possible in creating a differentiated and special workplace.  Having worked in environments where there was a contagious enthusiasm in what we were doing and also in ones I’d consider relatively toxic and unhealthy, there’s no doubt on the impact it has on the investment people make in doing their best work.

Following onto this, I believe there is also a distinction to be drawn in engaging the “average” employees across the organization versus targeting the “top performers”.  I have written about this previously, but top performers, while important to recognize and leverage effectively, don’t generally struggle with motivation (it’s part of what makes them top performers to begin with).  The problem is that placing a disproportionate amount of management focus on this subset of the employee population can have a significant adverse impact, because the majority of an organization is not “top performers” and that’s completely fine.  If the engagement, output, and productivity of the average employee is elevated even marginally, the net impact to organizational results should be fairly significant in most environments.

The dimensions below represent a few ways that I think about employee engagement and creating an inspired workplace.

Key Dimensions to Consider

Dimensions that are top of mind in relation to this area:

  • Becoming an Employer of Choice
    • Reputation matters. Very simple, but relevant point
    • This becomes real in how employees are treated on a cultural and day-to-day level, compensated, and managed even in the situation where they exit the company (willingly or otherwise)
    • Having worked for and with organizations that have had a “reputation” that is unflattering in certain ways, the thing I’ve come to be aware of over time is how important that quality is, not only when you work for a company, but the perception of it that then becomes attached to you afterwards
    • Two very simple questions to employees that could serve as a litmus test in this regard:
      • If you were looking for a job today, knowing what you know now, would you come work here again?
      • How likely would you be to recommend this as a place to work to a friend?
    • Promoting a Healthy Culture
      • Following onto the previous point, I recently wrote about The Criticality of Culture, so I won’t delve into the mechanics of this beyond the fact that dedicated, talented employees are critical to every organization, of any size, and the way in which they are treated and the environment in which they work is crucial to optimizing the experience for them and the results that will be obtained for the organization as a whole
    • Investing in Employee Development
      • Having worked in organizations where there was both an explicit, dedicated commitment to ongoing education and development and others where there was “never time” to invest in or “delivery commitments” that interfered with people’s learning and growth, the consequent impact on productivity and organizational performance has always been fairly obvious and very negative from my perspective
      • A healthy culture should create space for people to learn and grow their skills, particularly in technology, where the landscape is constantly changing and there is a substantial risk of skills becoming atrophied if not reinforced and evolved as things change.
      • This isn’t an argument for random training, of course, as there should be applicability for the skills into which an organization invests on behalf of its employees, but it should be an ongoing priority as much as any delivery effort so you maintain your ability to integrate new technology capabilities as and when they become available over time
    • Facilitating Collaboration
      • This and the next dimension are both discussed in the above article on culture, but the overall point is that creating a productive workplace goes beyond the individual employee to encouraging collaboration and seeking the kind of results discussed in my article on The Power of N
      • The secondary benefit from a collaborative environment is the sense of “connectedness” it creates across teams when it’s present, which would certainly help productivity and creativity/solutioning when part of a healthy, positive culture
    • Creating an Environment of Transparency
      • Understanding there are always certain things that require confidentiality or limited distribution (or both), the level of transparency in an environment helps create connection between the individual and the organization as well as helping to foster and engender trust
      • Reinforcing the criticality of communication in creating an inspiring workplace is extremely obvious, but having seen situations where the opposite is in place, it’s worth noting regardless

Measuring Impact

Several ways to think about impact:

  • Improved Productivity
    • Is more output being produced on a per FTE basis over time?
    • Are technologies like Copilot being leveraged effectively where appropriate?
  • Improved Average Utilization
    • Are utilization statistics reflecting healthy levels (i.e., not significantly over or under allocated) on an ongoing basis (assuming plan/actuals are reasonably reflected)?
  • Improved Employee Satisfaction
    • Are employee surveys trending in a positive direction in terms of job satisfaction?
  • Lower Voluntary Attrition
    • Are metrics declining in relation to voluntary attrition?

Perform

What It Is and Why It Matters

Very simply said: all the aspirations to innovate, grow, and develop capabilities don’t mean a lot if your production environment doesn’t support business and customer needs exceptionally well on a day-to-day basis.

As a former account executive and engagement manager in consulting at various organizations, any account strategy for me always began with one statement: “Deliver with quality”. If you don’t block and tackle well in your execution, the best vision and set of strategic goals will quickly be set aside until you do.  This is fundamentally about managing infrastructure, availability, performance of critical solutions, and security.  In all cases, it can be easy to operate in a reactive capacity and be very complacent about it, rather than looking for ways to improve, simplify, and drive greater stability, security, and performance over time. 

As an example, I experienced a situation where an organization spent tens of millions of dollars annually on production support, planning for things that essentially hadn’t broken yet, but had no explicit plan or spend targeted at addressing the root cause of the issues themselves.  Thankfully, we were able to reverse that situation, plan for some proactive efforts that ultimately took millions out of that spend by simply executing a couple projects.  In that case, the issue was the mindset, assuming that we had to operate in a reactive rather than proactive way, while the effort and dollars being consumed could have been better applied developing new business capabilities rather than continuing to band-aid issues we’d never addressed.

Another situation that is fairly prevalent today is the role of FinOps in managing cloud costs.  Without governance, the convenience of spinning up cloud assets and services can add considerable complexity, cost, and security exposure, all under the promise of shifting from a CapEx to OpEx environment.  The reality is that the maturity and discipline required to manage it effectively requires focus so it doesn’t become problematic over time.

There are many ways to think about managing and optimizing production, but the dimensions that come to mind as worthy of some attention are expressed below.

Key Dimensions to Consider

Dimensions that are top of mind in relation to this area:

  • Providing Reliability of Critical Solutions
    • Having worked with a client where the health of critical production solutions was in a state where that became the top IT priority, this can’t be overlooked as a critical priority in any strategy
    • It’s great to advance capabilities through ongoing delivery work, but if you can’t operate and support critical business needs on a daily level, it doesn’t matter
  • Effectively Managing Vulnerabilities
    • With the increase in complexity in managing technology environments today, internal and external to an organization, cyber exposure is growing at a rate faster than anyone can manage it fully
    • To that end, having a comprehensive security strategy, from managing external to internal threats, ransomware, etc. (from the “outside-in”) is critical to ensuring ongoing operations with minimal risk
  • Evolving Towards a “Zero Trust” Environment
    • Similar to the previous point, while the definition of “zero trust” continues to evolve, managing a conceptual “least privilege” environment (from the “inside-out”) that protects critical assets, applications, and data is an imperative in today’s complex operating environment
  • Improving Integrated Solution Performance
    • Again, with the increasing complexity and distribution of solutions in a connected enterprise (including third party suppliers, partners, and customers), the end user experience of these solutions is an important consideration that will only increase in importance
    • While there are various solutions for application performance monitoring (APM) on the market today, the need for integrated monitoring, analytics, and optimization tools will likely increase over time to help govern and manage critical solutions where performance characteristics matter
  • Developing a Culture Surrounding Security
    • Finally, in relation to managing an effective (physical and cyber) security posture, while a deliberate strategy for managing vulnerability and zero trust are the methods by which risk is managed and mitigated, equally there is a mindset that needs to be established and integrated into an organization for risk to be effectively managed
    • This dimension is meant to recognize the need to provide adequate training, review key delivery processes (along with associated roles and responsibilities), and evaluate tools and safeguards to create an environment conducive to managing security overall

Measuring Impact

Several ways to think about impact:

  • Increased Availability
    • Is the reliability of critical production solutions improving over time and within SLAs?
  • Lower Cybersecurity Exposure
    • Is a thoughtful plan for managing cyber security in place, being executed, monitored, and managed on a continuous basis?
    • Do disaster recovery and business continuity plans exist and are they being tested?
  • Improved Systems Performance
    • Are end user SLAs met for critical solutions on an ongoing basis?
  • Lower Unplanned Outages
    • Are unplanned outages or events declining over time?

Wrapping Up

Overall, the goal of this article was to share some concepts surrounding where I see the value of strategy for IT in enabling a business at an overall level.  I didn’t delve into what the makeup of the underlying technology landscape is or should be (things I discuss in articles like The Intelligent Enterprise and Perspective on Impact Driven Analytics), because the point is to think about how to create momentum at an overall level in areas that matter… innovation, speed, value/cost, productivity, and performance/reliability.

Feedback is certainly welcome… I hope this was worth the time to read it.

-CJG 12/05/2023

Perspective on Impact-Driven Analytics

Overview

I’ve spent a reasonable amount of time in recent years considering data strategy and how to architect an enterprise environment responsive and resilient to change.  What’s complicated matters is the many dimensions to establishing a comprehensive data strategy and the pace with which technologies and solutions have and continue to be introduced, none of which appears to be slowing down… quite the opposite.  At the same time, the focus on “data centricity” and organizations’ desire to make the most of the insights embedded within and across their enterprise systems has created a substantial pull to drive experimentation and create new solutions aimed at monetizing those insights for competitive advantage.  With the recent advent of Generative AI and large language models, the fervor surrounding analytics has only garnered more attention as to the potential it may create, not all to a favorable end.

The problem with the situation is that, not unlike many other technology “gold rush” situations that have occurred over the last thirty-one years I’ve been working in the industry, the lack of structure and discipline (or an overall framework) to guide execution can lead to a different form of technical debt, suboptimized outcomes, and complexity that ultimately doesn’t scale to the enterprise.  Hopefully this article will unpack the analytics environment and provide a way to think about the various capabilities that can be brought to bear in a more structured approach, along with the value in doing so.

Ultimately, analytics value is created in insight-driven, orchestrated actions taken, not on presentment or publication of data itself.

 

Drawing a “Real World” Comparison

The anecdotal hallmark of “traditional business intelligence” is the dashboard, which in many cases, reflects a visual representation of data contained in one or more underlying system, meant to increase end user awareness of the state of affairs, whatever the particular business need may be (this is a topic I’ve peripherally addressed in my On Project Health and Transparency article).

Having leased a new car last summer, I was both impressed and overwhelmed by the level of sophistication available to me through the various displays in the vehicle.  The capabilities have come a long way from a bunch of dials on the dashboard with a couple lights to indicate warnings.  That being said, there was a simplicity and accessibility to that design.  You knew the operating condition of the vehicle (speed, fuel, engine temp, etc.), were warned about conditions you could address (add oil, washer fluid), and situations where expert assistance might be needed (the proverbial “check engine” light).

What impressed me about the current experience design was the level of configurability involved, what I want to see on each of the displays, from advanced operating information, to warnings (exceeding the speed limit, not that this ever happens…), to suggestions related to optimizing engine performance and fuel efficiency based on analytics run over the course of a road trip.

This isn’t very different than the analytics environment available to the average enterprise, the choices are seemingly endless, and they can be quite overwhelming if not managed in some way.  The question of modeling the right experience comes down to this: starting with the questions/desired outcome, then working backwards in terms of capabilities and data that need to be brought to bear to address those needs.  Historical analytics can feel like it becomes a “data- or source-forward” mental model, when the ideal environment should be defined from the “outcome-backwards”, where the ultimate solution is rooted in a problem (or use case) meant to be solved.

 

Where Things Break Down

As I stated in the opening, the analytics landscape has gotten extremely complex in recent years and seemingly at an increasing pace.  What this can do, as is somewhat the case with large language models and Generative AI right now, is create a lot of excitement over the latest technology or solution without a sense of how something can be used or scaled within and across an enterprise.  I liken this to a rush to the “cool” versus the “useful”, and it becomes a challenge the minute it becomes a distraction from underlying realities of the analytics environment.

Those realities are:

  • Business ownership and data stewardship are critical to identifying the right opportunities and unlocking the value to be derived from analytics. Technology is normally NOT the underlying issue in having an effective data strategy, though disciplined delivery can obviously be a challenge depending on the capabilities of the organization
  • Not all data is created equal, and it’s important to discriminate in what data is accessed, moved, stored, curated, governed… because there is a business and technology cost for doing so
  • Technologies and enabling capabilities WILL change, so the way they are integrated and orchestrated is critically important to leveraging them effectively over time
  • It is easy to develop solutions that solve a specific need or use case but not to scale and integrate them as enterprise-level solutions. In an Intelligent Enterprise, this is where orders of magnitude in value and longer-term competitive advantage is created, across digitally connected ecosystems (including those with partners), starting with effective master data management and extending to newer capabilities that will be discussed below

At an overall level, while it’s relatively easy to create high-level conceptual diagrams or point solutions in relation to data and analytics, it takes discipline and thought to architect an environment that will produce value and agility at scale… that is part of what this article is intended to address.

 

Thoughts on “Data Centricity”

Given there is value to be unlocked through an effective data strategy, “data centricity” has become fairly common language as an anchor point in discussion.  While I feel that calling attention to opportunity areas can be healthy and productive, there is also a risk that concepts without substance (the antithesis of what I refer to as “actionable strategies”) can become more of a distraction than a facilitator of progress and evolution.  A similar situation arguably exists with “zero trust” right now, but that’s a topic worthy of its own article at a future date.

In the case of being “data centric”, the number of ways the language can be translated has seemed problematic to me, largely because I fundamentally believe data is only valuable to the extent it drives a meaningful action or business outcome. To that end, I would much rather be “insight-centric” or “value-focused”, “action-oriented”, or some other phrase that leans towards what we are doing with the data we acquire and analyze, not the fact that we have it, can access, store, or display it.  Those things may be part of the underlying means to an end, but they aren’t the goal in itself, and place emphasis on the road versus the destination of a journey.

To the extent that “data centricity” drives a conversation on what data a business has that may, if accessed and understood, create value, fuel innovation, and provide competitive advantage, I believe there is value in pursuing it, but a robust and thoughtful data strategy requires end-to-end thinking at a deeper level than a catch phrase or tagline on its own.

 

What “Good” Looks Like

I would submit that there are two fundamental aspects of having a robust data strategy once you address business ownership and stewardship as a foundational requirement: asking the right questions, and architecting a resilient environment.

 

Asking the Right Questions

Arguably the heading is a bit misleading here, because inference-based models can suggest improvements to move from an existing to a desired state, but the point is to begin with the problem statement, opportunity, or desired outcome, and work back to the data, insights, and actions required to achieve that result.  This is a business-focused activity and is, therefore, why establishing ownership and stewardship is so critical.

“We can accomplish X if we optimize inventory across Y locations while maintaining a fulfillment window of Z”

The statement above is different than something more “traditional” in the sense of producing a dashboard that shows “inventory levels across locations”, “fulfillment times by location”, etc. that then is intended to inform someone who ultimately may make a decision independent of secondary impacts or, better yet, recommends or enables actions to keep the inventory ecosystem calibrated in a dynamic way that continuously recalibrates to changing conditions, within defined business constraints.

While the example itself may not be perfect, the point is whether we think about analytics as presentment-focused or outcome-focused.  To the degree we focus on enabling outcomes, the requirements of the environment we establish will likely be different, more dynamic, and more biased towards execution.

 

Architecting a Resilient Environment

With the goals identified, the technology challenge becomes about enabling those outcomes, but architecting an environment that can and will evolve as those needs change and as the underlying capabilities continue to advance in what we are able to do in analytics as a whole.

What that means, and the next section will explore, is having a structured and layered approach so that capabilities can be applied, removed, and evolved with minimal disruption to other aspects of the overall environment.  This is, at its essence, a modular and composable architecture that enables interoperability through standards-based interaction across the layers of the solution in a way that will accelerate delivery and innovation over time.

The benefit to designing an interoperable environment is simple: speed, cost, and value.  As I mentioned in where things tend to break down, in technology, there should always a bias towards rapid delivery.  That being said, focusing solely on speed can tend to create a substantial amount of technical debt and monolithic solutions that don’t create cumulative or enterprise-level value.  Short-term, they may produce impact, but medium- to longer-term, they make things considerably worse once they have to be maintained and supported and the costs for doing so escalate.  Where a well-designed environment can help is in creating a flywheel effect over time to accelerate delivery using common infrastructure, integration standards, and frameworks so that the distance between idea and implementation is significantly reduced.

 

Breaking Down the Environment

The following diagram represents the logical layers of an analytics environment and some of the solutions or capabilities that can exist at each tier.  While the diagram could arguably be drawn in various ways, the reason I’ve drawn it like this is to show the separation of concerns between where content and data originates and ultimately where it’s consumed, along with the layers of processing that can occur in between.

Having the separation of concerns defined and standards (and reference architecture) established, the ability to scale, integrate new solutions and capabilities over time, and retire or modernize those that don’t create the right level of value, becomes considerably easier than when analytics solutions are purpose built in an end-to-end manner.

The next section will elaborate on each the layers to provide more insight on why they are organized in this manner.

 

Consume and Engage

The “outermost” tier of the environment is the consumption layer, where all of the underlying analytics capabilities of an organization should be brought to bear.

In the interest of transforming analytics, as was previously mentioned in the context of “data centricity”, the dialogue needs to move from “What do you want to see?” in business terms to “What do you want to accomplish and how do you want that to work from an end user standpoint?”, then employing capabilities at the lower level tiers to enable that outcome and experience (both).

The latter dimension is important, because it is possible to deliver both data and insights and not enable effective action, and the goal of a modern analytics environment is to enable outcomes, not a better presentment of a traditional dashboard.  This is why I’ve explicitly called out the role of a Digital Experience Platform (DXP) or minimally an awareness of how end users are meant to consume, engage, and interact with the outcome of analytics, ideally as part of an integrated experience that enables or automates action based on the underlying goals.

As analytics continue to move from passive and static to more dynamic and near real-time solutions, the role of data apps as an integrated part of applications or a digital experience for end users (internal or external) will become critical to delivering on the value of analytics investments.

Again, the requirements at this level are defined by the business goals or outcomes to be accomplished, questions to be answered, user workflows to be enabled, etc. and NOT the technologies to be leveraged in doing so.  Leading with technologies is almost certainly a way to head down a path that will fail over time and create technical debt in the process.

At an overall level, the reason for separating consumption and thinking of it independent of anything that “feeds” it, is that, regardless of how good the data or insights produced in the analytics environment are, if the end user can’t take effective action upon what’s delivered, there will be little value created in the solution.

 

Understand and Analyze

Once the goal is established, the capabilities to be brought to bear becomes the next level of inquiry:

  • If there is a set of activities associated with this outcome that requires workflow, rules, and process automation, orchestration should be integrated into the solution
  • If defined inputs are meant to be processed against the underlying data and a result dynamically produced, this may be a case where a Generative AI engine could be leveraged
  • If natural language input is desired, a natural language processing engine should be integrated
  • If the goal is to analyze the desired state or outcome against the current environment or operating conditions and infer the appropriate actions to be taken, causal models and inference-based analytics could be integrated. This is where causal models take a step past Generative AI in their potential to create value at an enterprise level, though the “describe-ability” of the underlying operating environment would likely play a key role in the efficacy of these technologies over time
  • Finally, if the goal is simply to run data sets through “traditional” statistical models for predictive analytics purposes (as an example), AI/ML models may be leveraged in the eventual solution

Having referenced the various capabilities above there are three important points to understand in why this layer is critical and separated from the rest:

  • Any or all of these capabilities may be brought to bear, regardless of how they are consumed by an end user, and regardless of how the underlying data is sourced, managed, and exposed.
  • Integrating them in ways are standards-based will allow them to be applied as and when needed into various solutions to create considerable cumulative analytical capability at an enterprise level
  • These capabilities definitely WILL continue to evolve and advance rapidly, so thinking about them in a plug-and-play based approach will create considerable organizational agility to respond and integrate innovations as and when they emerge over time, which translates into long-term value and competitive advantage.

 

Organize and Expose

There are three main concepts I outlined in this tier of the environment:

  • Virtualization – how data is exposed and accessed from underlying internal and external solutions
  • Semantic Layer – how data is modeled for the purpose of allowing capabilities at higher tiers to analyze, process, and present information at lower levels of the model
  • Data Products – how data is packaged for the purposes of analysis and consumption

These three concepts can be implemented with one or more technologies, but the important distinction being that they offer a representation of underlying data in a logical format that enables analysis and consumption, not necessarily that they are a direct representation of the source data or content itself.

With regard to data products in particular, while there is a significant amount of attention paid to their identification and development, they represent marginal value in an overall data strategy, especially when analytical capabilities and consumption models have evolved to such a great degree.  Where data products should be a focus (as a foundational step) is where the underlying organization and management of data is in such disarray that an examination of how to restructure and clean up the environment is important to reducing the chaos that exists in the current state.  What that implies, however, is less distractions and potential technical debt by extension, but not the kind of competitive advantage that comes from advanced capabilities and enabled consumption.  The other scenario where data products create value in themselves is when they are packaged and marketed for external consumption (e.g., credit scores, financial market data).  It’s worth noting in this case, however, that the end customer is assuming the responsibility of analyzing, integrating, and consuming those products as they are not an “end” in themselves in an overall analytics value chain.

 

Manage, Structure, and Enrich

While I listed a number of different types of solutions that can comprise a “storage” layer in the analytics environment, the best-case scenario would be that it doesn’t exist at all.  Where the storage layer creates value in analytics is providing a means to map, associate, enrich, and transform data in ways that would be too time consuming or expensive to do “on the fly” for the purposes of feeding the analytics and consumption tiers of the model.  There is certainly value, for instance, in graph databases for modeling complex many-to-many relationships across data sets, marts and warehouses for dealing with structured data, and data lakes for archival, managing unstructured data, and training of analytical models, but where source data can be exposed and streamed directly to the downstream models and solutions, there will be lower complexity, cost, and latency in the overall solution.

 

Acquire and Transmit

As capabilities continue to advance and consumption models mature, the desire for near real-time analytics will almost certainly dominate the analytics environment.  To that end, leveraging event-based processing, whether through an enterprise service or event bus, will be critical.  To the degree that enterprise integration standards can be leveraged (and canonical objects, where defined), further simplification and acceleration of analytics efforts will be possible.

Given the varied capabilities across cloud platforms (AWS, Azure, and GCP), not to mention the probability that data will be distributed between enterprise systems that could be hosted in a different cloud platform than its documents (as those in Office 365), the ability to think critically about how to integrate and synthesize across platforms is also important.  Without a defined strategy for managing multi-cloud in this domain in particular, costs for egress/ingress of data could be substantial depending on the scale of the analytics environment itself, not to mention the additional complexities that would be introduced into governance and compliance efforts surrounding duplicated content across cloud providers.

 

Generate and Provide

The lowest tier of the model is the simplest to describe, given it’s where data and content originate, which can be a combination of applications, databases, digital devices and equipment, and so forth, internal and external to an organization.  Back to the original point on business ownership and stewardship of data, if the quality of data emanating from these sources isn’t managed and governed, everything downstream will bear the fruit of the poisoned tree depending on the degree of issues involved.

Given the amount of attention given to large language models and GenAI right now, I thought it was worth noting that I consider these as another form of content generation more logically associated with the other types of solutions at this tier of the analytics model.  It could be the case that generated content makes its way through all the layers as a “data set” delivered directly to a consumer in the model, but by orienting and associating it with the rest of the sources of data, we create the potential to apply other capabilities at the next tiers of processing to that generated content, and thereby could enrich, analyze, and do more interesting things with it over time.

 

Wrapping Up

As I indicated at the opening, the modern analytics environment is complex and highly adaptive, which presents a significant challenge to capturing the value and competitive advantage that is believed to be resident in an organization’s data.

That being said, through establishing the right level of business ownership, understanding the desired outcomes, and applying disciplined thinking in how an enterprise environment is designed and constructed, there can be significant and sustainable value created for an enterprise.

I hope the ideas were thought provoking.  I appreciate those taking the time to read them.

 

-CJG 07/27/2023

Optimizing the Value of IT

Overview

Given the challenging economic environment, I thought it would be a good time to revisit something that was an active part of my work for several years, namely IT cost optimization.

In the spirit of Excellence by Design, I don’t consider cost optimization to be a moment in time activity that becomes a priority on a periodic (“once every X years”) or reactive basis.  Optimizing the value/cost ratio is something that should always be a priority in the interest of having disciplined operations, maintaining organizational agility, technical relevance, and competitive advantage.

In the consulting business, this is somewhat of a given, as most clients want more value for the money they spend on an annualized basis, especially if the service is something provided over a period of time.  Complacency is the fastest path to lose a client and, consequently, there is a direct incentive to look for ways to get better at what you do or provide equivalent service at a lower cost to the degree the capability itself is already relatively optimized.

On the corporate side, however, where the longer-term ramifications of technology decisions bear out in accumulated technical debt and complexity, the choices become more complex as they are less about a project, program, or portfolio and become more focused on the technology footprint, operating model, and organizational structure as a whole.

To that end, I’ll explore various dimensions of how to think about the complexity and makeup of IT from a cost perspective along with the various levers to explore in how to optimize value/cost.  I’m being deliberate in mentioning both because it is very easy to reduce costs and have an adverse impact on service quality or agility, and that’s why thoughtful analysis is important in making informed choices on improving cost-efficiency.

Framing the Problem

Before looking at the individual dimensions, I first wanted to cover the simple mental model I’ve used for many years in terms of driving operating performance:

 

The model above is based on three connected components that feed each other in a continuous cycle:

  • Transparency
    • We can’t govern what we can’t see. The first step in driving any level of thoughtful optimization is having a fact-based understanding of what is going on
    • This isn’t about seeing or monitoring “everything”. It is about understanding the critical, minimum information that is needed to make informed decisions and then obtaining as accurate a set of data surrounding those points as possible.
  • Governance
    • With the above foundation in place, the next step is to have leadership engagement to review and understand the situation, and identify opportunities to improve.
    • This governance is a critical step in any optimization effort because, if there are not sustainable organizational or cultural changes made in the course of transforming, the likelihood of things returning to a similar condition will be relatively high.
  • Improvement
    • Once opportunities are identified, executing effectively on the various strategies becomes the focus, with the goal of achieving the outcomes defined through the governance process
    • The outcomes of this work should then be reflected in the next cycle of operating metrics and the cycle can be repeated on a continuing basis.

The process for optimizing IT costs is no different than what is expressed here: understand the situation first, then target areas of improvement, make adjustments, continue.  It’s a process, not a destination.  From here, we’ll explore the various dimensions of complexity and cost within IT, and the levers to consider in adjusting them.

 

At an Operating-Level

Before delving into the footprint itself, a couple areas to consider at an overall level are portfolio management and release strategy.

 

Portfolio management

Given that I am mid-way through writing an article on portfolio management and am also planning a separate one on workforce and sourcing strategy, I won’t explore this topic much beyond saying that having a mature portfolio management process can help influence cost-efficiency. 

That being said, I don’t consider ineffective portfolio management to be a root cause of IT value/cost being imbalanced.  An effective workforce and sourcing strategy that aligns variable capacity to sources of demand fluctuation (within reasonable cost constraints) should enable IT to deliver significant value even during periods of increased business demand.  That being said, a lack of effective prioritization, disciplined estimation and planning, resource planning, and sourcing strategy in combination with each other can have significant and harmful effects on cost-efficiency and, therefore, generally provide opportunities for improvement.

Some questions to consider in this area:

  • Is prioritization effective in your organization? When “priority” effort arise, are other ongoing efforts stopped or delayed to account for them or is the general trend to take on more work without recalibrating existing commitments?
  • Are estimation and planning efforts benchmarked, reviewed, analyzed and improved, so the integrity of ongoing prioritization and slotting of projects can be done effectively?
  • Is there a defined workforce and sourcing strategy to align variable capacity to fluctuating demand so that internal capacity can be reallocated effectively and sourcing scaled in a way that doesn’t disproportionately have an adverse impact on cost? Conversely, can demand decline without significant need for recalibration of internal, fixed capacity?  There is a situation I experienced where we and another part of the organization took the same level of financial adjustment, but they had to make 3x the level of staffing adjustment given we were operating under a defined sourcing strategy and the other organization wasn’t.  This is an important reason to have a workforce and sourcing strategy.
  • Is resource planning handled on an FTE (e.g., role-based) or resource-basis (e.g., named resource), or some combination thereof? What is the average utilization of “critical” resources across the organization on an ongoing basis?

Release strategy

This is an area that often seems overlooked in my experience (outside product delivery environments) as a means to both improve delivery effectiveness, manage cost, and improve overall quality.

Having a structured release strategy that accounts for major and minor releases, with defined criteria and established deployment windows, versus an arbitrary or ad-hoc approach can be a significant benefit both from an IT delivery and business continuity perspective.  Generally speaking, delivery cycles (in a non-CI/CD, DevSecOps-oriented environment) tend to consume time and energy that slows delivery progress.  The more windows that exist, the more disruption that occurs over a calendar year.  When those windows are allowed to occur on an ad-hoc basis, the complexities of integration testing, configuration management, and coordination from a project, program, and change management perspective tends to increase proportional to the number of release windows involved.  Similarly, the risk of quality issues occurring within and across a connected ecosystem increases as the process for stabilizing and testing individual solutions, integrating across solutions, and managing post-deployment production issues is spread across multiple teams in overlapping efforts.  Where standard integration patterns and reference architecture is in place to govern interactions across connected components, there are means to manage and mitigate risk, but generally speaking, it’s better and more cost-effective to manage a smaller set of larger, scheduled release windows than allow a more random or ad-hoc environment to exist at scale.

 

Applications

In the application footprint, larger organizations or those built through acquisition tend to have a fairly diverse and potentially redundant application landscape, which can lead to significant cost and complexity, both in maintaining and integrating the various systems in place.  This is also true when there is a combination of significant internally (custom) developed solutions working in concert with external SaaS solutions or software packages.

Three main levers can have a significant influence along the lines of what I discuss in The Intelligent Enterprise:

  • Ecosystem Design
    • Whether one chooses to refer to this as business architecture, domain-driven design, component architecture, or something else, the goal is to identify and govern a set of well-defined connected ecosystems that are composable, made up of modular components that provide a clear business (or technical) capability or set of services
    • This is critical enabler to both optimizing the application footprint as well as promoting interoperability and innovation over time, as new capabilities can be more rapidly integrated into a standards-based environment
    • Where complexity comes about is where custom or SaaS/package solutions are integrated in a way that blurs these component boundaries and creates functional overlaps that create technical debt, redundancy, data integrity issues, etc.

 

  • Integration strategy
    • With a set of well-defined components, the secondary goal is to leverage standard integration patterns with canonical objects to promote interoperability, simplification, and ongoing evolution of the technology footprint over time.
    • Without standards for integration, an organization’s ability to adopt new, innovative technologies will be significantly hindered over time and the leverage of those investments marginalized, because of the complexity involved in bringing those capabilities into the existing environment rapidly without having refactor or rewrite a portion of what exists to leverage them.
    • At an overall level, it is hard to argue that technologies are advancing at a rate faster than any organization’s ability to adopt and integrate them, so having a well-defined and heavily leveraged enterprise integration strategy is critical to long-term value creation and competitive advantage.

 

  • Application Rationalization
    • Finally, with defined ecosystems and standards for integration, having the courage and organizational leadership to consolidate like solutions to a smaller set of standard solutions for various connected components can be a significant way to both reduce cost and increase speed-to-value over time.
    • I deliberately focused on the organizational aspects of rationalization, because one of the most significant obstacles in technology simplification is the courageous leadership needed to “pick a direction” and handle the objections that invariably result in those tradeoff decisions being made.
    • Technology proliferation can be caused by a number of things, but organizational behaviors can certainly contribute when two largely comparable solutions exist without one of them being retired solely based on resistance to change or perceived control or ownership associated with a given solution.
    • At a capability-level, evaluating similar solutions, understanding functional differences and associating the value with those dimensions is a good starting point for simplifying what is in place. That being said, the largest challenge in application rationalization doesn’t tend to be identifying the best solution, it’s having the courage to make the decision, commit the investment, and execute on the plan given “new projects” tend to get more organizational focus and priority in many companies than cleaning up what they already have in place.  In a budget-constrained environment, the new, shiny thing tends to win in a prioritization process, which is something I’ll write about in a future article.

Overall, the larger the organization, the more opportunity may exist in the application domain, and the good news is that there are many things that can be done to simplify, standardize, rationalize, and ultimately optimize what’s in place in ways that both reduce cost and increase the agility, speed, and value that IT can deliver.

 

Data

The data landscape and associated technologies, especially when considering advanced analytics, has significantly added complexity (and likely associated cost) in the last five to ten years in particular.  With the growing demand for AI/ML, NLP, and now Generative AI-enabled solutions, the ability to integrate, manage, and expose data, from producer to ultimate consumer has taken on significant criticality.

Some concepts that are directionally important in my opinion in relation to optimizing value/cost in data and analytics enablement:

  • Managing separation of concerns
    • Similar to the application environment, thinking of the data and analytics environment (OLTP included) as a set of connected components with defined responsibilities, connected through standard integration patterns is important to reducing complexity, enabling innovation, and accelerating speed-to-value over time
    • Significant technical debt can be created where the relationship of operational data stores (ODS), analytics technologies, purpose-built solutions (e.g., graph or time series databases) master data management tools, data lakes, lake houses, virtualization tools, visualization tools, data quality tools, and so on are not integrated in clear, purposeful ways.
    • Where I see value in “data centricity” is in the way it serves as a reminder to understand the value that can be created for organizations in leveraging the knowledge embedded within their workforce and solutions
    • I also, however, believe that value will be unlocked over time through intelligent applications that leverage knowledge and insights to accelerate business decisions, drive purposeful collaboration, and enable innovation and competitive advantage. Data isn’t the outcome, it’s an enabler of those outcomes when managed effectively.

 

  • Minimizing data movement
    • The larger the landscape and number of solutions involved in moving source data from the original producer (whether it’s a connected application, device, or piece of equipment) to the end consumer (however that consumption is enabled) has a significant impact on innovation and business agility.
    • As such, concepts like data mesh / data fabric, enabling distributed sourcing of data in near-real time with minimized data movement to feed analytical solutions and/or deliver end user insights is critical in thinking through a longer-term data strategy.
    • In a perfect world, where data enrichment is not a critical requirement, the ability to virtualize, integrate, and expose data across various sources to conceptually “flatten” the layers of the analytics environment is an area where end consumer value can be increased while reducing cost typically associated with ETL, storage, and compute spread across various components of the data ecosystem
    • Concepts like zero ETL, data sharing, and virtualization are also key enablers that have promise in this regard

 

  • Limiting enabling technologies
    • As in the application domain, the more diverse and complex a data ecosystem is, the likelihood that a diverse set of overlapping technologies is in place, with overlapping or redundant capabilities.
    • At a minimum, a thoughtful process for reviewing and governing any new technology introductions, to evaluate how they complement, replace, or are potentially redundant or duplicative with solutions already in place is an important capability to have in place
    • Similarly, it is not uncommon to introduce new technologies with somewhat of a “silver bullet” mindset, without considering the implications for supporting or operating those solutions, which can increase cost and complexity, or having a deliberate plan to replace or retire other solutions that provide a similar capability in the process.
    • Simply said, technical debt accumulates over time, through a set of individually rationalized and justified, but overall suboptimized short-term decisions.
  • Rationalize, simplify, standardize
    • Finally, where defined components exist, data sourcing and movement is managed, and technologies introductions are governed, there should be an ongoing effort to modernize, simplify, and standardize what is already in place.
    • Data solutions can tend to be very “purpose-built” in their orientation to the degree that the enable a specific use case or outcome. The problem that occurs in this situation is if the desired business architecture becomes the de facto technical architecture and significant complexity is created in the process.
    • Using a parallel, smaller scale analogy, there is a reason that logical and physical data modeling are separate activities in application development (the former in traditional “business design” versus the latter being part of “technical design” in waterfall-based approaches). What makes sense from a business or logical standpoint likely won’t be optimized if architected as defined in that context (e.g., most business users don’t think intuitively in third normal form, nor should they have to).
    • Modern technologies allow for relatively cheap storage and giving thought to how the underlying physical landscape should be designed from producer to consumer is critical in both enabling insight delivery at speed, but also doing so within a managed, optimized technology environment.

Overall, similar to the application domain, there are significant opportunities to enable innovation and speed-to-value in the data and analytics domain, but a purposeful and thoughtful data strategy is the foundation for being cost-effective and creating long-term value.

 

Technologies

I’ve touched on technologies through the process of discussing optimization opportunities in both the application and data domains, but it’s important to understand the difference between technology rationalization (the tools and technologies you use to enable your IT environment) and application or data rationalization (the solutions that leverage those underlying technologies to solve business problems).

The process for technology simplification is the same as described in the other two domains, so I won’t repeat the concepts here beyond reiterating that a strong package or technology evaluation process (that considers the relationship to existing solutions in place) and governance of new technology introductions with explicit plans to replace or retire legacy equivalents and ensure organizational readiness to support the new technologies in production is critical to optimizing value/cost in this dimension.

 

Infrastructure

At an overall level, unless there is a significant compliance, competitive, privacy, or legal reason to do so, I would argue that no one should be in the infrastructure business unless it IS their business.  That may be a somewhat controversial point-of-view, but at a time when cloud and hosting providers are both established and mature, arguing the differentiated value of providing (versus managing) these capabilities within a typical IT department is a significant leap of faith in my opinion.  Internal and external customer value and innovation is created in the capabilities delivered through applications, not the infrastructure, networking, and storage underlying those solutions.  This isn’t to say these capabilities aren’t a critical enabler.  They definitely are, though, the overall organizational goal in infrastructure from my perspective should be to ensure quality of service at the right cost (through third party providers to the maximum extent possible), and then manage and govern the reliability and performance of that set of environments, focusing on continuous improvement and enabling innovation as required by consuming solutions over time.

There are a significant number of cost elements associated with infrastructure, a lot of financial allocations involved, and establishing TCO through these indirect expenses can be highly complex in most organizations.  As a result, I’ll focus on three overall categories that I consider significant and acknowledge there is normally opportunity to optimize value/cost in this domain beyond these three alone (cloud, hosted solutions, and licensing).  This is partially why working with a defined set of providers and managing and governing the process can be a way to focus on quality of service and desired service levels within established cost parameters versus taking on the challenge of operationalizing a substantial set of these capabilities internally.

Certainly, a level of core network and cyber security infrastructure is necessary and critical to an organization under any circumstances, something I will touch on in a future article on the minimum requirements to run an innovation-centric IT organization, but even in those cases, that does not imply or require that those capabilities be developed or managed internally.

 

Cloud

With the ever-expanding set of cloud-enabled capabilities, there are three critical watch items that I believe have significant impact on cost optimization over time:

  • Innovation
    • Cloud platform providers are making significant advancements in their capabilities on an annual basis, some of which can help enable innovation
    • To the extent that some of the architecture and integration principles above are leveraged, and a thoughtful, disciplined process is used to evaluate and manage introduction of new technologies over time, organizations can benefit from their leverage of cloud as a part of their infrastructure strategy

 

  • Multi-cloud Integration
    • The reality of cloud providers today is also that no one is good at everything and there is differentiated value in various services provided from each of them (GCP, Azure, AWS)
    • The challenge is how to integrate and synthesize these differentiated capabilities in a secure way without either creating significant complexity or cost in the process
    • Again, having a modular, composable architecture mindset with API- or service-based integration is critical in finding the right balance for leveraging these capabilities over time
    • Where significant complexity and cost can be created is where data egress comes into play from one cloud platform to another and, consequently, the need for such data movement should be minimized in my opinion to situations where the value of doing so (ideally without persisting the data in the target platform) greatly outweighs the cost to operate in that overall environment

 

  • FinOps Discipline
    • The promise of having managed platforms that convert traditional capex to opex is certainly an attractive argument for moving away from insourced and hosted solutions to the cloud (or a managed hosting provider for that matter). The challenge is in having a disciplined process for leveraging cloud services, understanding how they are being consumed across an organization, and optimizing their use on an ongoing basis.
    • Understandably, there is not a direct incentive for platform providers to optimize this on their own and tools largely provide transparency into spend related to consumption of various services over time.
    • Hopefully, as these providers mature, we’ll see more of an integrated platform within and across cloud providers to help continuously optimize a footprint so that it provides reliability and scalability, but also without promoting over provisioning or other costs that don’t provide end customer value in the process.

Given the focus of this article is cost optimization and not cloud strategy, I’m not getting into cloud modernization, automation and platform services, containerization of workloads, or serverless computing, though arguably some of those also can provide opportunities to enable innovation, improve reliability, enable edge-based computing, and optimize value/cost as well.

 

Internally Managed / Hosted

Given how far we are into the age of cloud computing, I’m assuming that legacy environments have largely been moved into converged infrastructure.  In some organizations, this may not be the case and should be evaluated along with the potential for outsourcing the hosting and management of these environments where possible (and competitive) at a reasonable value/cost level.

One interesting anecdote is how organizations don’t tend to want to make significant investments in modernizing legacy environments, particularly those in financial services resting on mainframe or midrange computing solutions.  That being said, given that they are normally shared resources, as the burden of those costs shift (where teams selectively modernize and move off those environments) and allocations of the remaining MIPS and other hosting charges are adjusted, the priority in revisiting those strategies tends to change.  Being proactive on modernization should be a continuous, proactive process rather than a reactive one, because the resulting technology decisions can otherwise be suboptimized and turned into lift-and-shift based approaches versus true modernization or innovation opportunities (I’d consider this under the broader excellence topic of relentless innovation).

 

Licensing

The last infrastructure dimension that I’d call out in relation to licensing.  While I’ve already addressed the opportunity to promote innovation and optimize expense through rationalizing applications, data solutions, or underlying technologies individually, there are three other dimensions that are worth consideration:

  • Partner Optimization
    • Between leverage of multi-year agreements on core, strategic platforms and consolidation of tools (even in a best-of-breed environment) to a smaller set of strategic, third-party providers, there are normally opportunities to reduce the number of technology partners and optimize costs in large organizations
    • The watch item would be to ensure such consolidation efforts consider volatility in the underlying technology environment (e.g., the commitment might be too long for situations where the pace of innovation is very high) while also ensuring conformance to the component and integration architecture strategies of the organization so as not to create dependencies that would make transition of those technologies more complex in the future

 

  • Governance and Utilization
    • Where licensing costs are either consumption-based or up for renewal, having established practices for revisiting the value and usage of core technologies over time can help in optimization. This can also be important in ensuring compliance to critical contract terms where appropriate (e.g., named user scenarios, concurrent versus per-seat agreements)
    • In one example a number of years ago, we decided to investigate indirect expense coming through software licenses and uncovered nearly a million dollars of software that had been renewed on an annual basis that wasn’t being utilized by anyone. The reality is that we treated these as bespoke, fixed charges and no one was looking at them at any interval.  All we needed to do in that case was pay attention and do the homework.

 

  • Transition Planning
    • The most important of these three areas is akin to having a governance process in place.
    • With regard to transition, establishing a companion process to the software renewal cycle for critical, core technologies (i.e., those providing a critical capability or having significant associated expense). This process would involve a health check (similar to package selection, but including incumbent technologies/solutions) at a point commensurate with the window of time it would take to evaluate and replace the solution if it was no longer the best option to provide a given capability.
    • Unfortunately, depending on the level of dependency that exists for third-party solutions, it is not uncommon for organizations to lack a disciplined process to review technologies in advance of their contractual renewal period and be forced to extend their licenses because of a lack of time to do anything else.
    • The result can be that organizations deploy new technologies in parallel with ones that are no longer competitive purely because they didn’t plan in advance for those transitions to occur in an organic way

Similar to the other categories, where licensing is a substantial cost component of IT expense, the general point is to be proactive and disciplined about managing and governing it.  This is a source of overhead that is easy to overlook and that can create undue burden on the overall value/cost equation.

 

Services

I’m going to write on workforce and sourcing strategy separately, so I won’t go deeply into this topic or direct labor in this article beyond a few points in each.

In optimizing cost of third-party provided services, a few dimensions come to mind:

  • Sourcing Strategy
    • Understanding and having a deliberate mapping of primary, secondary and augmentation partners (as appropriate) for key capabilities or portfolios/solutions is the starting point for optimizing value/cost
    • Where a deliberate strategy doesn’t exist, the ability to monitor, benchmark, govern, manage, and optimize will be both complex and effective only on a limited basis
    • Effective sourcing and certain approaches to how partners are engaged can also be a key lever in both enabling rapid execution of key strategies, managing migration across legacy and modernized environments, establishing new capabilities where a talent base doesn’t currently exist internal to an organization, and in optimizing expense that may be either fragmented across multiple partners or enabled through contingency labor in ad-hoc ways, all of which can help optimize the value/cost ratio on an ongoing basis

 

  • Vendor Management
    • Worth noting that I’m using the word “vendor” here because the term is fairly well understood and standard when it comes to this process. In practice, I never use the word “vendor” in deference to “partner” as I believe the latter signals a healthy approach and mindset when it comes to working with third-parties.
    • Having worked in several consulting organizations over a number of years, it was very easy to tell which clients operated in a vendor versus a partnership mindset and the former of the two can be a disincentive to making the most of these relationships
    • That being said, organizations should have an ongoing, formalized process for reviewing key partner relationships, performance against contractual obligations, on-time delivery commitments, quality expectations, management of change, and achievement of strategic partner objectives.
    • There should also be a process in place to solicit ongoing feedback both on how to improve effectiveness and the relationship but also to understand and leverage knowledge and insights a partner has on industry and technology trends and innovation opportunities that can further increase value/cost performance over time.

 

  • Contract Management
    • Finally, having a defined, transparent, and effective process for managing contractual commitments and the associated incentives where appropriate can also be important to optimizing overall value/cost
    • It is generally true that partners don’t deliver to standards that aren’t established and governed
    • Defining service levels, quality expectations, utilizing fixed price or risk sharing models and so on and then reviewing and holding both partners and the internal organization working with those partners accountable to those standards is important in having both a disciplined operating and a disciplined delivery environment
    • There’s nothing wrong with assuming everyone will do their part when it comes to living into the terms of agreements, but there also isn’t harm in keeping an eye on those commitments and making sure that partner relationships are held to evolving standards that promote maturity, quality, and cost effectiveness over time

Similar to other categories, the level of investment in sourcing, whether through professional service firms or contingent labor, should drive the level of effort involved in understanding, governing, and optimizing it, but some level of process and discipline should be in place almost under any scenario.

 

Labor

The final dimension to optimizing value and cost is direct labor.  I’m guessing, in writing this, that it’s fairly obvious I put this category last and I did so intentionally.  It is often said that “employees are the greatest source of expense” in an organization.  Interestingly enough “people are our greatest asset” has also been said many times as well.

In the section on portfolio management, I mentioned the importance of having a workforce and sourcing strategy and understanding the relationship between the alignment of people to demand on an ongoing basis.  That is a given and should be understood and evaluated with a critical eye towards how things flex and adjust as demand fluctuates.  It is also a given and assumed that an organization focused on excellence should be managing performance on a continuing basis (including times of favorable market conditions) so as not to create organizational bloat or ineffectiveness.  Said differently, poor performance that is unmanaged in an organization drags down average productivity, has an adverse impact on quality, and ultimately a negative impact on value cost because the working capacity of an organization isn’t being applied to ongoing demand and delivery needs effectively.  Where this is allowed to continue unchecked over too long a duration, the result may be an over-correction that also can have adverse impacts on performance, which is why it should be an ongoing area of focus by comparison with an episodic one.

Beyond performance management, I believe it’s important to think of all of the expense categories before this one to be variable, which is sometimes not the case in the way they are evaluated and managed.  If non-direct labor expense is substantial, a different question to consider is the relative value of “working capacity” (i.e., “knowledge workers”) by comparison with expense consumed in other things.  Said differently, a mental model that I used with a team in the past was that “every million dollars we save in X (insert dimension or cost element here… licensing, sourcing, infrastructure, applications) is Y people we can retain to do meaningful work.”

Wrapping Up

Understanding that this has been a relatively long article, but still only a high-level treatment of a number of these topics, hopefully it has been useful in calling out many of the opportunities that are available to promote excellence in operations and optimize value/cost over time.

In my experience, having been in multiple organizations that have realigned costs, it takes engaged and courageous leadership to make thoughtful changes versus expedient ones… it matters… and it’s worth the time invested to find the right balance overall.  In a perfect world, disciplined operations should be a part of the makeup of an effectively led organization on an ongoing basis, not the result of a market correction or fluctuation in demand or business priorities.

 

Excellence always matters, quality and value always matter.  The discipline it takes to create and manage that environment is worth the time it takes to do it effectively.

 

Thank you for taking the time to read the thoughts.  As with everything I write, feedback and reactions are welcome.  I hope this was worth the investment in time.

-CJG 04/09/2023

The Intelligent Enterprise

Setting the Stage

I’ve mentioned the concept of “framework-centric design” in a couple articles now, and the goal here is to provide some clarity with regard to where I believe technology and enterprise architecture strategy is heading.  A challenge with trying to predict the future of this industry is the number of variables involved, including how rapidly various capabilities evolve, new technologies are introduced, skills of the “average developer” advance along with the tools enabling them to perform their work, methods of integrating and analyzing data arise, etc.  I will make various assertions here.  Undoubtedly some things will come to pass and others will not (and that’s ok), but the goal is to look forward and consider the possibilities in the interest of stirring discussion and exploration.  Innovation is borne out of such ideas, and hopefully it will be worth the read.

That being said, I believe certain things will drive evolution in how digital business will operate:

  • A more “open systems” approach will be driven by market demand that reduces proprietary approaches (e.g., on various SaaS and ERP platforms) that constrain or inhibit API-based interactions and open data exchange between applications
  • Technologies will be introduced so rapidly that organizations will be forced to revisit integration in a way that enables accelerated adoption of new capabilities to remain competitive
  • Application architecture will evolve to where connected ecosystems and strategic outcomes become the focus of design, not the individual components themselves.
  • As an extension of the previous point, secure digital integration between an organization and its customers, partners, and suppliers will become more of a collaborative, fluid process than the transaction-centric model that is largely in place today (e.g., product design through fulfillment versus simple order placement).
  • “Data-centric” thinking will be replaced by intelligent ecosystems and applications as the next logical step in its evolution. Data quality processes and infrastructure will be largely supplanted by AI/ML technology that interprets and resolves discrepancies and can autocorrect source data in upstream systems without user intervention, overcoming the behavioral obstacles that inhibit significant progress in this domain today.
  • Cloud-enabled technology will reach edge computing environments to the degree that cloud native architecture becomes agnostic to the deployment environment and enables more homogenous design across ecosystems (said differently, we can design towards an “ideal state” without artificially constraining solutions to a legacy hosted/on-prem environment). Bandwidth will decide where workloads execute, not the physical environment.

Again, some of the above assertions may seem very obvious or likely where others are more speculative.

The point is to look at the cumulative effect of these changes on digital business and ask ourselves whether the investments we’re making today are taking us where we need to be, because the future will arrive before we expect or will be prepared for it.

The Emerging Ecosystem

In the world of application architecture, the shift towards microservices, SaaS platforms, and cloud native technologies (to name a few) seem to have pulled us away from a more strategic conceptualization of the role ecosystems play in the future environment.

Said simply: It’s not about the technology, it’s about the solution.  Once you define the solution architecture, the technology part is solvable.

Too many discussions in my experience start with a package, or a technology, or an infrastructure concept without a fundamental understanding of the problem being solved in the first place, let alone a concept of what “good” would look like in addressing that problem at a logical (i.e., solution architecture) level.

Looking back in time, we started with a simple premise of automating activities performed by “knowledge workers” to promote efficiency, consistency, collaboration, and so on, where the application was the logical center of the universe.  In some respects, this can still be the case where SaaS platforms, ERP systems, and even some custom built (and much beloved) home grown solutions tend to become dominant players in the IT landscape.  Those solutions then force a degree of adaptation of the systems around them to conform to their integration and data models, with constraints generally defined by the builders of that core system.  In a relatively unstructured or legacy environment, that could lead to a significant amount of point-to-point integration, custom solutions, and ultimately technical debt.  What seems like a good solution in these cases can eventually hinder evolution as those primary elements in the technology footprint effectively become a limiting constraint on advancing the overall capabilities of the ecosystems they inhabit.

Moving one step in the right direction, we decouple systems and move towards a level of standard integration, whether through canonical objects or other means, that provides a means to replace systems more easily over time.  In the case of a publish-and-subscribe based approach, there is also a way to support distributed transactions (e.g., address change in a system of record being promoted to multiple, dependent downstream applications) in a relatively linear sequential manner (synchronous or otherwise).

In a way, the “enterprise middleware” discussion has felt like a white whale of its own to me, having seen many attempts at it fail over years of delivery work, from the CORBA/DCOM days through many iterations of “EAI” technologies since.  The two largest problems I’ve seen in this regard: scope and priorities.  Defining the transactions/objects that matter, modeling them properly from a business standpoint, and enforcing standardization to a potentially significant number of new and legacy systems is a very difficult thing to do.  I’ve seen “enterprise integration” over-scoped to the point it doesn’t create value (e.g., standards for standards sake), which then undermines the credibility of the work that actually could reduce complexity, promote resiliency and interoperability, and ultimately create speed to value in delivery.  The other somewhat prevalent obstacle is the lack of leadership engagement on the value of standard integration itself, to the point that it is de-prioritized in the course of ongoing delivery work (e.g. date versus quality issue).  In these situations, teams commit to “come back” and retrofit integrations to a strategic model, but ultimately never do, and the integrity of the overall integration architecture strategy falls apart for a lack of effective governance and adoption during execution.

Even with the best defined and executed integration strategy, my contention is that they fall short, because the endgame is not about modeling transactions, it’s about effective digital orchestration.

The shift in mental model is a pivot from application and transaction-centric thinking to an ecosystem-centric approach, where we look at the set of components in place as a whole and the ways in which they interoperate to bring about meaningful business outcomes.  Effective digital orchestration is at the center of how the future ecosystems need to operate.

The goal in the future state is to think of the overall objective (e.g., demand generation for a sales ecosystem, production for manufacturing, distribution for transportation and logistics), then examine how each of the individual ecosystem components enable or support those overall goals, how they interoperate, and what the coordination of those elements needs to be across various use cases (standard and exception based) in an “ideal state” to optimize business outcomes.  The applications move from the center of the model to components as endpoints and the process by which they interact is managed in the center through defined but highly configurable process orchestration.  The configurability is a critical dimension because, as I will address in the data section below, the continuous optimization of that process through AI/ML is part of how the model will eventually change the way digital enterprises perform.

Said slightly differently, in an integration-centric model, transactions are initiated, either in a publish-and-subscribe or request-response manner, they execute, and complete.  In an orchestration-centric model, the ecosystem itself is perpetually “running” and optimizing in the interest of generating its desired business outcomes.  In the case of a sales ecosystem, as an example, monitoring associated with a lag in demand could automatically initiate lead generation processes that request actions executed through a CRM solution.  In manufacturing, delays in a production line could automatically trigger actions in downstream processes or equipment to limit the impact of those disruptions on overall efficiency and output.  The ecosystem becomes “self-aware” and therefore more effective in delivering on overall value than its individual components can on their own.

With a digitally connected ecosystem in place, the logical question becomes, “ok, what next?”  Well, once you have discrete components that are integrated and orchestrated in a thoughtful manner that drives business outcomes, there are two ways to extend that value further:

  • Optimize both the performance of individual components and the overall ecosystem through embedded AI/ML (something I’ll address in the next section)
  • Connect that ecosystem internally to other intelligent ecosystems within the enterprise and use it as a foundation to drive digital integration with customers, partners, and suppliers

In the latter case, given the orchestration model itself is meant to be dynamic and configurable, an organization’s ability to drive digital collaboration with third parties (starting with customers) should be substantially improved beyond the transaction-centric models largely used today.

Moving Beyond “Data Centricity”

There is a lot of discussion related to data-centricity right now, largely under the premise that quality data can provide a foundation for meaningful insights that create business opportunity.  The challenge that I see is in relation to how many organizations are modeling and executing strategies in relation to leveraging data more effectively.

The diagram above is meant to provide a simplistic representation of the logical model that is largely in place today to provide insights leveraging enterprise data.  The overall assumption is that core systems should first publish their data to a centralized enterprise environment (data lake(s) in many instances).  That data is then moved, transformed, enriched, etc. for the purpose of feeding downstream data solutions or analytical models that provide insights as to how to address whatever questions were originally the source of inquiry (e.g., customer churn, predictive maintenance).

The problem with the current model is that it is all reactive processing when the goal of our future state environment should be intelligent applications.

In the intelligent enterprise, AI/ML serves two purposes:

  • Making the orchestration across a set of connected components more effective. This would be accomplished presumably by collecting and analyzing process performance data gathered as part of the ecosystem’s ongoing operation.
  • Making applications more intelligent to the point that user intervention is greatly simplified, data quality is automatically enforced and improved, execution and performance of the component itself is continuously improved in line with its role in the larger ecosystem. Given there are not a significant number of “intelligent applications” today, it would seem as if the logical progression would be to first have a “configurable AI” capability that could integrate with existing applications, analyze their data and user interactions, and then make recommendations or simulate user actions in the interest of improving application performance.  That capability would eventually be integrated within individual applications as a means to make it more attuned to individual application-oriented needs, user task flows, and data sets.

The implication on enterprise data strategy stemming from this shift in application architecture could be significant, as the goal of centralizing data for a larger “post-process” analytical effort would be moving upstream (at least in part) to the source applications, eliminating the need for a larger enterprise lake environment in deference to smaller data pools that facilitate change at a component/application level.

Wrapping Up

Bringing it all together, the purpose of this article was to share a concept of a connected future state technology environment where orchestration within and across ecosystems with embedded optimization could drive a significant amount of disruption in digital business.

As a conceptual model, there are undoubtedly gaps and unknowns to be considered, not the least of which is how to manage complexity in a distributed data strategy model and refactor existing transactional execution when it moves from being initiated by individual applications to a central orchestrator.  Much to contemplate and consider, but fundamentally I believe this is where technology is going, and a substantial amount of digital capability will be unlocked in the process.

Feedback is welcome.  The goal was to initiate discussion, not to suggest a well defined “answer”.  Hopefully the concepts will stir questions and reaction.

-CJG 03/27/2022

Excellence By Design

Background

As I began this journey and subsequently to assemble topics about which to write, I noticed that there were both an overwhelming set of ideas coming (a good problem to have) and a very unclear relationship in the concepts that were running quite rapidly through my mind (not a good thing).

Upon further reflection, it occurred to me that the ideas all centered around the various dimensions of leading a technology organization at different levels of specificity.  To that end, I thought I should set the stage a bit, in the interest of making things more cohesive in what I may write from here.

 

On the Pursuit of Excellence

At an overall level, what better place to start than a simple premise: Excellence is a choice.

Shooting for excellence is a commitment that requires a lot on a practical level, starting with courageous leadership, because it is a perpetually moving target, requires adaptability, tenacity, and a willingness to accept change as a way of life.  Excellence isn’t accidental, it is a matter of organizational will and the passion to pursue aspirations beyond what, at times, may feel “realistic” or “practical”.  It requires a belief in what is possible and is defined along multiple dimensions, which we’ll explore briefly here, namely:

  • Relentless Innovation
  • Operating with Agility
  • Framework-Driven Design -and-
  • Delivering at Speed

Relentless Innovation

Starting with vision, some questions to consider in the context of an overall strategy:

  • Is it clear and understood across the organization, along with its intended outcome (e.g., what success looks like)?
  • Is it one that connects to individuals in the organization, their roles and ongoing contributions, or are those disconnected concepts (i.e., is it something that individuals take to heart)?
  • Can it evolve as circumstances change while maintaining a degree of fundamental integrity (e.g., will it stand the test of time or need to be continually redefined)?
  • Is it actionable? Can tangible steps be taken to drive progress towards its ultimate goals?
  • Is it “deliberate”/intended/proactive or was it defined in a reactive context (e.g., in response to a competitor’s actions)?
  • Are day-to-day decisions made with the strategy in mind?

Overall, the point is to have a thoughtful, proactive strategy, that is actionable, connected to ongoing decisions, and embraced by the broader organization.

Where this becomes more interesting is in how we think of strategy in relation to change, which is where the next concept comes into play.  Relentless innovation is the notion that anything we are doing today may be irrelevant tomorrow, and therefore we should continuously improve and reinvent our capabilities to ones that create the most long-term value. This is much easier said than done, because it requires a lot of organizational humility and a willingness to tear down existing structures and rebuild new ones in their place.  That forces a degree of risk tolerance, because there is safety in the established practices and solutions of today, especially if they’ve created value.  On the other hand, success can be very detrimental insofar as complacency can become part of the organizational mindset and change slows down to an environment that is essentially an iteration of the present.

 

Operating with Agility

Looking at IT Operations, a number of questions come to mind that may be the subject of future articles:

  • Is there a mindset of being cost-efficient (driving the highest value/cost ratio)?
  • Is there a culture of continuous improvement and innovation in place?
  • Is there a strategy for incorporating and optimizing the relationship of project and product teams (to the extent that a full product orientation isn’t feasible)?
  • Is there a sourcing strategy in place that is deliberate, governed, optimized (whether insourced, outsourced, or some combination thereof)?
  • Are portfolio management processes effective and aligned to business strategy?
  • Is there a highly transparent, but extremely lightweight operating infrastructure in place to facilitate engagement and value creation?
  • To what degree is talent rotation and development part of the culture? Are people stuck in the same organization or silo for long periods of time, or are high potential leaders moved between teams to facilitate a higher degree of knowledge sharing, development, and improvement?

Having worked in IT Ops, the largest issue I’ve seen in a number of companies is an overly significant focus on process and infrastructure by comparison with transparency and enablement.  This is a tricky balance to strike, but arguably, I’d much rather have a less “mature” operating environment (IT for IT) that produces directionally correct information and drives engagement than a heavy, cumbersome process that becomes a distraction from producing business outcomes.  A simple litmus test on the latter type of environment being in place is whether, in discussion, teams talk about the process and tools versus the outcomes, decisions, and impact.

 

Framework-Driven Design

Shifting focus to technology, I believe the opportunity is to think differently about the overall solution architecture of future ecosystems.  Much has been written and discussed relative to modern or cloud native applications, data-centric design, DevSecOps, domain-driven design, and so on.

What fundamentally bothers me about solution design approaches is that, when focusing on one dimension (e.g., data centricity), other dimensions of the more holistic view of modern application design is left out, and then it becomes a challenge to delivery teams to integrate one or more of these concepts in practice without a way to synthesize them into one cohesive approach.  This is where framework-centric design can be an interesting approach to consider.

In my definition, framework-centric design is focused on architecting a connected ecosystem and operating environment intended to promote resiliency, interoperability, and application-agnostic integration such that individual solution components can be upgraded or replaced over time at a rapid pace without disrupting the capability of the ecosystem as a whole.

I will explore this topic further in a future article, but the base premise is to design an overall solution that performs complex tasks given multiple components integrated in standardized ways, leveraging modern, cloud native technologies, with integrated data that feeds embedded analytics capabilities as part of the operation of the ecosystem.

The framework itself, therefore, becomes a platform and the individual components are treated as replaceable parts that enable a best-of-breed mentality as new capabilities emerge that become advantageous to integrate with the framework over time.

 

Delivering at Speed

From a delivery standpoint, as tempting as it is to write about iterative development (or Agile in particular) as a cure all, the reality is that more organizations suffer from a lack of discipline than a lack of methodology. 

The unfortunate myth that needs to be explored and unwound is that executing with discipline means value will be delayed when, in fact, the exact opposite is true.  It is a generalization, but the faster a build team moves (to the extent that process or rigor is abandoned), the immediate impact is usually a level of technical debt that will create drag, either in the initial or subsequent delivery efforts.

Quality doesn’t happen by accident.  It is something that needs to be planned and built into a work product from the kickoff of a delivery effort, regardless of the methodology or operating model employed.

I will likely write more on this topic given the number of opportunities that exist, but it’s sufficient to say that you can’t achieve excellence when you don’t execute as flawlessly as possible… and discipline is needed to accomplish that.

 

Wrapping Up

Overall, the goal was to provide a quick summary of the various dimensions that I believe are important to consider in leading an organization.  No doubt, there may be questions or omissions (intentional or unintended) as this was a first blush at how I think about it. 

What about people and culture?  Well… that’s part of operating effectively… as an example.

Hopefully this was a good starting point and provided some food for thought.  Feedback, questions, and reactions are always welcome.

Looking forward to continuing this journey.

-CJG 10/28/2021