For the last decade, data lineage has been identified as a vital element of data-related regulatory compliance. Despite this, our experience has been that businesses, especially financial institutions, are not making the progress that might seem necessary.
Our VP of Strategic Technologies for Data Intelligence, Sue Habas, was recently a panellist on a webinar titled “Tracking Data Lineage for Regulatory Compliance and Change.” The poll questions presented during the webinar provided a range of great statistics on hot topics in the data management industry, including some surprising revelations about the progress of regulatory compliance automation.
The first webinar question appears to confirm our aforementioned suspicions – at least for the 300 financial services representatives responded to the question. We asked, “How much progress has your organisation made in implementing data lineage for regulatory compliance?”. The results showed that 47% of organisations had some implementation process, followed by 17% who were in a planning stage, 14% who had just started a program, 12% who had made as much implementation progress as possible, and 10% who had yet to make any progress.
Given that some key compliance deadlines have already passed, and that others are not far off, these results should be cause for concern. GDPR is less than a year away, MIFID II not much more than six months away and Basel III will be here in less than two years. Those are just a few of the key dates on the horizon.
When we talk to our clients about what’s going on, we have heard that some of them are getting some serious misdirection. While some may believe that data lineage is too hard and not necessary, and that all you need is Data Governance and Business Lineage, to me, this is like saying, “if your car’s broken, you can see what’s going on by checking the maintenance log and the blueprints.”
We encourage the implementation of data lineage because we feel that knowing your data’s origin can enable data stewards within your business to quickly pinpoint critical data elements. To support this claim, you needn’t look further than the results of the second poll question.
There’s no doubt that there are some real challenges in getting a data lineage program implemented. That was an issue that the second polling question tested. We asked, “What data management challenges does your organisation face when implementing data lineage for compliance?”. For this question, we allowed multiple responses and found that 79% of organisations found managing lineage across multiple systems as an issue, 54% felt they had a shortfall in understanding their data, 42% found the new regulatory requirements a challenge, closely followed by 40% who felt the change in regulatory requirements a challenge. A lucky 2% of those asked had no data management challenges.
Those first two are plainly related to the need to dig in and investigate the detail of data lineage. The issue with managing lineage across multiple systems and that of new regulatory problems point to another significant challenge. There are some excellent solutions available that address compliance with one regulation or another. However, how can multiple regulations be supported simultaneously when multiple systems are implicated? A Data Intelligence solution, such as ASG’s, is essential to meet the challenge quickly.
There are many benefits to data lineage, but perhaps the biggest is confident compliance. The attendees at the webinar expected much more. 77% saw secure regulatory compliance as an important benefit with similar numbers (74%) highlighting secure regulatory compliance was a top benefit. Over half of respondents (53%) felt that better operations were also positive, followed by 41% enjoying the reduction in redundant systems and costs, and 24% seeing new business opportunities and key.
The first two benefits are what we would expect. It’s encouraging to see the anticipated value deriving from improved operations and reduction of redundancy. Data lineage reveals inefficiencies in business and technical processes, and duplication of information assets. Tracing the data lineage, eliminating redundancies and smoothing operations saves costs and reduces the compliance workload.
To wrap all of this up; yes, data lineage is challenging, but the rewards are great. The need is urgent, and those who would divert you need to be firmly ignored!
