Data Documentation Neglect: The Hidden AI Killer Lurking in Your Organization
Skip the painful guesswork in data documentation—Metaphor’s AI-powered platform keeps you compliant, collaborative, and innovative.
Learn why traditional data governance needs a significant overhaul, focusing on new strategies for fostering collaboration, enhancing data literacy, and more!
Join our Head of Product Kirit Basu as he explains how his team is revolutionizing traditional data governance methods, advocating for inclusivity in governance roles. Discover how modern tools elevate productivity, offering invaluable insights for data experts and curious minds alike.
Welcome, everyone. My talk today is titled "The End of Governance as We Know It." Yes, it's a bit clickbaity, but by the end of this presentation, I hope to convey why the traditional concept of data governance needs a significant overhaul.
Who Am I?
As Sean mentioned, I'm the Head of Product at Metaphor. Metaphor is essentially a social platform for data. While we do touch on governance, our primary focus is on creating a catalog that encourages company-wide participation.
Quick Survey
How many of you are currently in a data governance role? Quite a few, I see. And how many of you are data engineers or data scientists who find yourselves forced into governance tasks? A good number as well. Hopefully, what I share today will resonate with you.
The Reality of Collaboration
One of the promises of governance tools is that they foster collaboration. The word "collaboration" is plastered on every governance tool out there. However, the reality is that silos exist within companies. People tend to focus on their tasks, making genuine collaboration a challenge. Despite having governance tools designed for collaboration, they often fail to achieve this goal due to the inherent siloed nature of organizations.
Ease of Use and Implementation
Governance tools should be easy to use and implement, but the reality is often quite different. Many tools take 6-8 months just to get up and running, and they remain complex and difficult for end-users to navigate. Users looking for simple answers, like how to calculate revenue, are often faced with overly complicated systems.
The Cloud Promise
Many companies claim that moving their software to the cloud makes it easier to use. However, this is often just a revised version of the same complex system. While intricate software is necessary for complex tasks like running a nuclear power plant, most users need simplicity for everyday data queries.
Data Literacy and Support
Another critical aspect of governance is data literacy. Unfortunately, governance teams often focus on coverage analytics—what percentage of data is documented—instead of addressing the actual questions users have. This results in data teams spending over 30% of their time on repetitive support tasks, answering the same questions repeatedly.
Engagement and Usage
Initial engagement with new governance tools is high, but over time, usage dwindles to just the core data team. These tools are expensive, often costing hundreds of thousands of dollars, yet only a handful of people use them regularly.
The Next Generation of Governance
The future of governance needs to be outcome-oriented, focusing on delivering quick answers to users and facilitating organic discovery. Governance tools should support not just the technical producers of data but also the broader organization. They should make it easy to find and understand data, fostering genuine collaboration and breaking down silos.
Practical Example
Let me give you a practical example. One of our customers focused on the 25 most frequently asked questions in their organization. Instead of documenting every single data element, they answered these key questions in our tool. This approach provided immediate value and reduced repetitive support tasks.
Customer Focus
Data teams often don't interact directly with end-users, leading to a disconnect. Our platform brings together all roles, providing simple interfaces for business users and detailed views for technical users, facilitating better collaboration.
Organic Discovery
We aim to ensure organic discovery of information, connecting people working on similar problems across the organization. This social aspect of our platform is crucial for breaking down silos and enhancing collaboration.
Conclusion
In conclusion, our product helps solve these challenges by offering advanced search and discovery capabilities, improving data literacy, and providing a new way of doing governance. It's a powerful tool for data teams, designed to make their lives easier and their work more impactful. I'm happy to continue the conversation outside. Thank you all for listening.
The Metaphor Metadata Platform represents the next evolution of the Data Catalog - it combines best in class Technical Metadata (learnt from building DataHub at LinkedIn) with Behavioral and Social Metadata. It supercharges an organization’s ability to democratize data with state of the art capabilities for Data Governance, Data Literacy and Data Enablement, and provides an extremely intuitive user interface that turns even the most non-technical user into a fan of the catalog. See Metaphor in action today!