Ana içeriğe atla

Data Mesh Principles and Logical Architecture

 Data Mesh Principles and Logical Architecture The great divide of data What do we really mean by data? The answer depends on whom you ask. Today’s landscape is divided into  operational data  and  analytical data . Operational data sits in databases behind business capabilities served with microservices, has a transactional nature, keeps the current state and serves the needs of the applications running the business. Analytical data is a temporal and aggregated view of the facts of the business over time, often modeled to provide retrospective or future-perspective insights; it trains the ML models or feeds the analytical reports. The current state of technology, architecture and organization design is reflective of the divergence of these two data planes - two levels of existence, integrated yet separate. This divergence has led to a fragile architecture. Continuously failing ETL (Extract, Transform, Load) jobs and ever growing complexity of labyrinth of data pipel...
Yorum Gönder

Monolith First

Monolith First 

As I hear stories about teams using a microservices architecture, I've noticed a common pattern.

1.      Almost all the successful microservice stories have started with a monolith that got too big and was broken up

2.      Almost all the cases where I've heard of a system that was built as a microservice system from scratch, it has ended up in serious trouble.

This pattern has led many of my colleagues to argue that you shouldn't start a new project with microservices, even if you're sure your application will be big enough to make it worthwhile. .

 


Microservices are a useful architecture, but even their advocates say that using them incurs a significant MicroservicePremium, which means they are only useful with more complex systems. This premium, essentially the cost of managing a suite of services, will slow down a team, favoring a monolith for simpler applications. This leads to a powerful argument for a monolith-first strategy, where you should build a new application as a monolith initially, even if you think it's likely that it will benefit from a microservices architecture later on.

The first reason for this is classic Yagni. When you begin a new application, how sure are you that it will be useful to your users? It may be hard to scale a poorly designed but successful software system, but that's still a better place to be than its inverse. As we're now recognizing, often the best way to find out if a software idea is useful is to build a simplistic version of it and see how well it works out. During this first phase you need to prioritize speed (and thus cycle time for feedback), so the premium of microservices is a drag you should do without.

The second issue with starting with microservices is that they only work well if you come up with good, stable boundaries between the services - which is essentially the task of drawing up the right set of BoundedContexts. Any refactoring of functionality between services is much harder than it is in a monolith. But even experienced architects working in familiar domains have great difficulty getting boundaries right at the beginning. By building a monolith first, you can figure out what the right boundaries are, before a microservices design brushes a layer of treacle over them. It also gives you time to develop the MicroservicePrerequisites you need for finer-grained services.

I've heard different ways to execute a monolith-first strategy. The logical way is to design a monolith carefully, paying attention to modularity within the software, both at the API boundaries and how the data is stored. Do this well, and it's a relatively simple matter to make the shift to microservices. However I'd feel much more comfortable with this approach if I'd heard a decent number of stories where it worked out that way. [1]

A more common approach is to start with a monolith and gradually peel off microservices at the edges. Such an approach can leave a substantial monolith at the heart of the microservices architecture, but with most new development occurring in the microservices while the monolith is relatively quiescent.

Another common approach is to just replace the monolith entirely. Few people look at this as an approach to be proud of, yet there are advantages to building a monolith as a SacrificialArchitecture. Don't be afraid of building a monolith that you will discard, particularly if a monolith can get you to market quickly.

Another route I've run into is to start with just a couple of coarse-grained services, larger than those you expect to end up with. Use these coarse-grained services to get used to working with multiple services, while enjoying the fact that such coarse granularity reduces the amount of inter-service refactoring you have to do. Then as boundaries stabilize, break down into finer-grained services. [2]

While the bulk of my contacts lean toward the monolith-first approach, it is by no means unanimous. The counter argument says that starting with microservices allows you to get used to the rhythm of developing in a microservice environment. It takes a lot, perhaps too much, discipline to build a monolith in a sufficiently modular way that it can be broken down into microservices easily. By starting with microservices you get everyone used to developing in separate small teams from the beginning, and having teams separated by service boundaries makes it much easier to scale up the development effort when you need to. This is especially viable for system replacements where you have a better chance of coming up with stable-enough boundaries early. Although the evidence is sparse, I feel that you shouldn't start with microservices unless you have reasonable experience of building a microservices system in the team.

I don't feel I have enough anecdotes yet to get a firm handle on how to decide whether to use a monolith-first strategy. These are early days in microservices, and there are relatively few anecdotes to learn from. So anybody's advice on these topics must be seen as tentative, however confidently they argue.

 

Yorumlar

Yorum Gönder

Bu blogdaki popüler yayınlar

Continuous Integration with Visual C++ and COM

  William E. Caputo ThoughtWorks Oren Miller ThoughtWorks July 2002 The Continuous Integration principles are applicable in many different development environments. While the principles remain largely unchanged from project to project, the implementation of this practice can vary considerably. Variables such as language, platform, team size & team location provide unique implementation challenges. Here we will outline how we've implemented CI in a COM/Windows environment for a project developing primarily in Visual C++. The More Often the Better What Is a Successful Build? Single Source Point Building the Code Self-Testing Code Automated Build Dependency Management What We Could Have Done Better Summing up The More Often the Better As noted in the main article, one of the least intuitive notions about integration is that less often does not result in less difficulty, it results in more difficulty. This is especially true when developing with C++. The build time on a development...
Yorum Gönder
Devamı

Rotation

  Rotation I've spent a lot of time of the last year wandering around ThoughtWorks, talking to lots of people on lots of projects. One message that's come home really firmly to me is the value of rotation. We practice rotation in lots of ways. One of the most notable is rotating around countries. We've put in a deliberate program to encourage people to spend 6-18 months in a different country. Living a good length of time in a different country does a huge amount to widen people's perspective of the world. I've benefitted personally from living both in the UK and USA, even though they are very similar cultures. This mental expansion is even greater for those that spend time in somewhere like India, where the cultural differences are greater. Geographic rotation presents lots of challanges, particular for older people with familes. One of the things we need to figure out is how to make geographic rotation easier for people, so more people do it. Already there's a...
Yorum Gönder
Devamı

Business Capability Centric

 Business Capability Centric A business-capability centric team is one whose work is aligned long-term to a certain area of the business. The team lives as long as the said business-capability is relevant to the business. This is in contrast to project teams that only last as long as it takes to deliver project scope. For example, an e-commerce business has capabilities such as buying and merchandising, catalog, marketing, order management, fulfilment and customer service. An insurance business has capabilities such as policy administration, claims administration, and new business. A telecom business has capabilities such as network management, service provisioning and assurance, billing, and revenue management. They may be further divided into fine-grained capabilities so that they can be owned by teams of manageable size. Business-capability centric teams are “think-it, build-it and run-it” teams. They do not hand over to other teams for testing, deploying or supporting what they...
Yorum Gönder
Devamı