Data fabrics represent the next evolution of data architecture as they provide flexibility, scalability, and efficiency, yet they remain an enigma to many.
Author: Michael Nardacci
Interest in data fabrics, as evidenced by Google Trends, has increased nearly 5x since 2018, culminating in early June 2023 with the release of Microsoft Fabric. The concept of the data fabric picked up momentum in 2019 when Gartner identified it as a major technological trend for 2022, while RevGen’s own Vice President of Analytics and Insights, Ian Foley, identified it as a trend to watch for 2023.
However, despite years of analyst coverage and a Microsoft SaaS release, data fabrics remain a complex topic that often flies under the radar. As data size, complexity, and spread continues to grow, the data fabric architecture will be an important differentiator for those seeking to remain on the leading edge of analytics and insights.
A data fabric is an architectural design which can involve the integration of multiple interoperable products. A Data Fabric consists of multiple architecture components, also referred to as layers, which read, capture, integrate, and deliver data based on the understanding of who is using the data, the types of usage, and the changes in data use patterns. The key capabilities these components bring to a data fabric are:
Source Integration: Data fabrics can integrate any type of source data including third party, flat files, data warehouses, data lakes, or document repositories. They have the flexibility to securely aggregate these sources across cloud and on-premises locations. Once implemented, additional sources can be easily incorporated into the existing data fabric.
Data Orchestration: Data assets are inventoried and represented in visual information supply chains. The flow from source to data consumer is defined and includes documentation of the data cleansing, transformation, enrichment, and validation steps. Data can be delivered to the target via any method including APIs, ETL, CDC, or virtualization.
Insight Generation: Data fabrics continuously supply high-quality data to Business Intelligence groups as well as Machine Learning models. Both offline analytics, with these features running in the background and surfacing insights to the user through the fabric interface, as well as online operational intelligence are possible. Models can return real-time results which are persisted in the fabric for future analysis.
Enhanced Governance and Security: With all data in the virtualized data model, the organization has a holistic view of their data infrastructure and how it is being used. Flexible access and security protocols can be applied to different areas of the data model to lock down or promote different elements.
These key capabilities in addition to the ability to scale performance as needed can drive significant value for an organization.
Data fabrics leave source data in its original location (data warehouse, data lake, etc.) and create a virtualized data layer in a centralized platform using existing data assets as nodes within the fabric, thereby eliminating costly maintenance and integration processes. Gartner estimates that by 2024, data fabrics will quadruple the efficiency of data exploitation and halve the human data management tasks while reducing data discovery, analysis, and integration tasks by 70%. Additional benefits include:
Reduced Development Costs: The time and cost to develop new applications is reduced since there is little or no data integration required. Applications which need to represent data from multiple sources can simply reference the fabric without worrying about the disparate sources’ locations.
Enables Modular Development: Applications built on the data fabric can be modularized and quickly applied to other areas on the fabric. This enables the organization to standardize applications and tools for all users, which helps bring new data users into the fold.
Complete Visibility: All data sources across the organization can be integrated and streamlined to provide real-time insights driving better outcomes and break down existing data silos.
Simplified Use: Elimination of system-to-system APIs to communicate between sources within an organization. Data in various locations can be used without the user knowing the exact source or how to connect to it outside the fabric.
A data fabric is appropriate for any organization seeking to modernize their data architecture and analytics capabilities with a highly flexible, efficient, and scalable solution. Businesses with highly fragmented data sources, perhaps after a merger or acquisition, will see immediate benefits from the aggregation of all data on the virtualized data layer. Moreover, organizations seeking to expand organically will enjoy the benefits which come from advanced insight generation capabilities to ensure as their operational data grows it can be immediately leveraged for insights.
Ultimately, a data fabric is an effective tool for any organization seeking the agility to adapt within the evolving enterprise data world. ICD research into enterprise data culture reported that half of respondents were overwhelmed by the amount of organizational data while 44% also said they don’t have enough data to support decision making. If your organization feels similar, then a data fabric will help you put the correct data in the hands (or dashboards) of proper stakeholders leading to a positive impact on business performance.
RevGen has successfully implemented data fabric solutions for clients, including a major telecom company where we leveraged a data fabric to improve and deepen their marketing division’s data insights to better reach their customers. If your organization is considering modernizing your data architecture and analytic capabilities, contact RevGen for an initial consultation on how a data fabric may fit your needs or visit our Analytics & Insights site to learn more about our services.
Michael Nardacci is a Sr. Consultant at RevGen Partners where he works on projects related to business current state assessments and data transformation and migration.
As companies grapple with the opportunities and challenges of AI, the role of data governance is more important than ever.
Read More About Data Governance in the Age of AI
Data quality is key to operational efficiency, speed to insight, AI enablement, and other high-value analytics.
Read More About Data Quality: The Foundation for AI Enablement
Implementing an AI Data Analytics solution can bring additional efficiency and insight if you start by building these foundational components
Read More About AI Data Series Part 2: AI Data AnalyticsGet the latest updates and Insights from RevGen delivered straight to your inbox.
