Data Mesh Principles and Logical Architecture The great divide of data What do we really mean by data? The answer depends on whom you ask. Today’s landscape is divided into operational data and analytical data . Operational data sits in databases behind business capabilities served with microservices, has a transactional nature, keeps the current state and serves the needs of the applications running the business. Analytical data is a temporal and aggregated view of the facts of the business over time, often modeled to provide retrospective or future-perspective insights; it trains the ML models or feeds the analytical reports. The current state of technology, architecture and organization design is reflective of the divergence of these two data planes - two levels of existence, integrated yet separate. This divergence has led to a fragile architecture. Continuously failing ETL (Extract, Transform, Load) jobs and ever growing complexity of labyrinth of data pipel...
Alignment maps are organizational information radiators that help visualize the alignment of ongoing work with business outcomes. The work may be regular functionality addition or technical work such as re-architecting or repaying technical debt or improving the build and deployment pipeline. Team members use alignment maps to understand what business outcomes their day-to-day work is meant to improve. Business and IT sponsors use them to understand how ongoing work relates to the business outcomes they care about.
Here’s an example scenario (inspired by real life) that illustrates how these maps may be useful. A team of developers had inefficiently implemented a catalog search function as N+1 calls. The first call to the catalog index returned a set of SKU IDs. For each ID returned, a query was then made to retrieve product detail. The implementation came to the attention of an architect when it failed performance tests. He advised the team to get rid of the N+1 implementation.
“Search-in-one” was the mantra he offered the team as a way to remember their objective. Given the organizational boundary between architects and developers and the low frequency of communication between them, the mantra was taken literally. The team moved heaven and earth to implement a combined index query and detail query in a single call. They lost sight of the real objective of improving search performance and slogged away in an attempt to achieve acceptable performance in exactly one call. Funding ran out in a few months and after some heated discussions, the project was cancelled and the team disbanded.
The above example may seem absurd but sadly, enterprise IT is no stranger to architecture and business projects that are cancelled after a while because they lost sight of why they were funded in the first place. In the terminology of organizational design, these are problems of alignment.
Visualizing Alignment
Broadly, IT strategy has to align with business strategy and IT outcomes with desired business outcomes. A business outcome may be supported (in part) by one or more IT outcomes. Each IT outcome may be realized by one or more initiatives (program of work—architectural or business). At this point, it may also be useful to identify an owner for each initiative who then sponsors work (action items) across multiple teams as part of executing the initiative. Depending on the initiative the owner may be a product owner, architect, tech lead or manager. Here's an alignment map for the “search-in-one” case. Had it been in public display in the team’s work area, it might have prompted someone to take a step back and ask what their work was really meant to achieve.
Global Map
A global alignment map for the IT (appdev+ops) organization may look more like this (although real maps tend to be much larger).
As with all information radiators, such a map is a snapshot in time and needs to be updated regularly (say once a month). Each team displays a big printout of the global map in its work area.
Big organizations are likely to realize value early in this exercise by collaborating to come up with a version 1.0 of such a map that everyone agrees to. The discussions around who owns what initiatives and what outcomes an initiative contributes to leads to a fair bit of organizational clarity of what everyone is up to. Usually, the absence of well-articulated and commonly understood business and IT strategies come in the way of converging on a set of business and IT outcomes. Well-facilitated workshops with deep and wide participation across the relevant parts of the organization can help address this.
Tracing alignment paths
Once a global alignment map is in place, it allows us to trace alignment from either end. IT and business sponsors can trace what action items are in play under a given initiative. Development team members can trace through the map to understand the real purpose of items they are working on. In addition to in-progress items, we could also include action items that are planned, done or blocked.
As illustrated in the map above, each team highlights their section of the map on their copy of the global map.
Qualitative benefits validation
Once a month (or quarter), IT and business people get together to validate if all the IT activity has made any difference to business outcomes. Business people come to the meeting with red-amber-green (RAG) statues for business outcomes and IT people may come with RAG statues for their side of the map. Both parties need to be able to back up their RAG assessments with data and/or real stories from the trenches (narrative evidence).
These maps can be combined
Yorumlar
Yorum Gönder