Data mesh is the killer app for supercloud, says Zhamak Dehghani

[ad_1]

The data mesh concept continues to take the world by storm, because it incorporates a domain-oriented and self-serve design, enabling end users to easily access vital data without transferring it to a data warehouse or data lake.

Since data mesh entails sharing data across boundaries responsibly, it is a prerequisite for supercloud, according to Zhamak Dehghani (pictured), founder and author of “Data Mesh: Delivering Data-Driven Value at Scale.”

“The terminology is supercloud, open cloud, but I think in the spirit of it it’s this embracing of diversity and giving autonomy for people to make decisions for what’s right for them and not yet lock them in,” Dehghani said. “I think just embracing that is baked into how data mesh assumed the world would work.”

Dehghani spoke with theCUBE industry analysts John Furrier and Dave Vellante at the Supercloud2 event, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed why data mesh is a perfect fit for supercloud and the paradigms happening in the data space. (* Disclosure below.)

The evolution of data APIs

To reduce the primitives needed to train machine-learning models, data APIs ought to be revamped. This is because they will play a pivotal role in streamlining data owners’ operations, according to Dehghani.

“Perhaps one of the trends that we might see is this evolution of data APIs to become more and more computational aware to bring the compute to the data to reduce that surface area so you can really leave the control of the data to the sovereign owners of that data,” she noted. “So you describe what you want, and the data owner decides how to manage them.”

One way of connecting the dots entails using data to reason, and this necessitates an intelligence layer, Dehghani pointed out.

“By just defining that API and standardizing it, now we have this amazing extension point that we can say, ‘Well, the next revision of this API, you not just tell me who you are, but you actually tell me what intelligence you’re after,’” she stated. “What’s a logic that I need to go and now compute on your API? And you can kind of evolve that.”

An open policy framework comes in handy when making data addressable for coding and machines. Operating systems are also instrumental when abstracting complexities, according to Dehghani.

“We try to solve the problem of policies live with the data,” she said. “An enforcement of the policies happens at the most granular level, which is, in this concept, the data product. And that would happen whether you read, write or access a data product.”

Providing a cohesive end-to-end developer experience

Since businesses have to endure gymnastics to harness the value of decentralized data, cohesion should come in handy, according to Dehghani. Therefore, significant decoupling and coupling are needed to create that cohesive architecture.

“When you think about data as a decentralized problem, the cost of integration of these technologies in a cohesive developer experience is what’s missing,” she pointed out. “And we want to focus on that cohesive end-to-end developer experience to share data responsibly in these autonomous units, we call them data products, I guess, in data mesh.”

A role that mimics that of a site reliability engineer is emerging in the data space. This is based on the chaos and complexities endured, according to Dehghani.

“I think if we look at that evolution, perhaps something like that is happening by bringing data and apps closer and make them these domain-oriented data product teams or domain oriented cross-functional teams,” she explained. “And still have a very advanced, maybe at the platform infrastructure level, kind of operational team … they’re building infrastructure that is embracing that complexity, interconnectivity of this data process.”

Since many organizations have shown their inclination toward decentralized ownership of data, Nextdata Technologies, of which Dehghani is founder and chief executive officer, was recently created to make this a reality. This is attained through data governance, interconnectivity through standard APIs, and data discoverability.

“We do that with a developer experience that is easy and native to a generalist developer,” Dehghani stated. “So we try to find the common denominator that solves those problems and enables that developer experience for data sharing.”

Visit theCUBE’s Supercloud2 event page to watch full episodes on demand.

(* Disclosure: This is an editorial segment. TheCUBE is a paid media partner for Supercloud2, but sponsors for theCUBE’s event coverage do not have editorial control over content on theCUBE or SiliconANGLE.)