Data Mesh Principles and Logical Architecture The great divide of data What do we really mean by data? The answer depends on whom you ask. Today’s landscape is divided into operational data and analytical data . Operational data sits in databases behind business capabilities served with microservices, has a transactional nature, keeps the current state and serves the needs of the applications running the business. Analytical data is a temporal and aggregated view of the facts of the business over time, often modeled to provide retrospective or future-perspective insights; it trains the ML models or feeds the analytical reports. The current state of technology, architecture and organization design is reflective of the divergence of these two data planes - two levels of existence, integrated yet separate. This divergence has led to a fragile architecture. Continuously failing ETL (Extract, Transform, Load) jobs and ever growing complexity of labyrinth of data pipel...
Lay Programmer
I use the term lay programmer to mean someone who is programming without thinking themselves as a programmer. Someone who spends a large part of her day working on spreadsheets is doing programming, often very intense programming. Usually however she won't call herself a programmer, nor think of spending much time learning how to program better.
It's easy for professional programmers to get sniffy at lay programmers, but lay programmers usually are domain experts who know a great deal about what a program should do. Our challenge is to think of ways to engage them more effectively in software development, and provide tools that are easy for them to use, but also capable of being well structured programs that can evolve efficiently and integrate well into the wider software ecosystem.
Yorumlar
Yorum Gönder