Separation Anxiety: Bridging the Digital Transformation Divide
A few months ago, I became a certified practitioner of Data Vault 2.0. I had been working with Data Vault for a few years, but I was finally able to take a class with Dan Linstedt himself, and then successfully passed the certification test. I must say I had a bit of swagger walking around the office, with my chest puffed out in light of my new accomplishment.
I was even excited when I looked at the news feed on my phone and noticed an article about Data Vault! Excited, that is, until I noticed the title: “Is Data Vault Becoming Obsolete?” (http://roelantvos.com/blog/is-data-vault-becoming-obsolete/) Say what? Surely my recent achievement was not already worthless? Once I opened the article, my anxiety increased – the article was originally published TWO YEARS ago! Was I obsolete before I even began?
Reading the article, the author seemed to be suggesting that Data Vault was not completely obsolete, but with current, modern technologies it would best be used simply as a factor in a data approach, as part of a more advanced Data Warehouse Virtualization approach. It was an excellently written and well thought out article, and for a moment, I was wondering if I had wasted my efforts. Did I really spend my time and money on something no organization will need? Had technology eliminated the need for the skills I had attained? Can modern tools replace the need to understand these concepts fully?
But after a few minutes of sweating, I stopped to consider the clients that I have worked with over the last several years. None of them were familiar with Data Vault. In fact, they were either unfamiliar with most modern data warehousing approaches or were very nascent in their understanding. Each organization I have encountered is only beginning to address their data needs – they are in no way the mature environments described in the article. For these data teams, implementing a new technology built on the implicit understanding of Data Vault and Data Warehousing approaches as a means to solve their data needs, without first establishing those approaches and the accompanying governance, would be a costly mistake.
It occurred to me that this gap, between the institutional knowledge, culture, and data maturity of an organization and the level of advancement of the current technologies creates a very real level of anxiety and can waste very real time and money. Technology advances at a rapid pace, with improvements and new options often available in months, but organizational change is slow – significant changes are usually measured over years.
Of course, this revelation is not uniquely mine – in fact, it is summed up best in the form of Martec’s Law: “Technology changes exponentially, organizations change logarithmically”. And the divide between the level of technology and the level of the organization is costly. I have worked with companies who decided to improve their data landscape by buying new software that was intended to “fix” their issues. Tens of thousands of dollars were spent on licenses, buying what in many instances was excellent software. But because the organization was not changed sufficiently, the software went unused, or under-utilized. And the Enterprise Data Landscape was more complicated, not less, because a few people were using the new system, others using the old systems, and the rest looking for the next new technological solution.
Ultimately, it is in that divide that true Digital Transformation must take place. There is no software that is a silver bullet – no technology that can bring an organization forward on its own. It is only by moving forward with enterprise wide organizational change, including policies, guidelines, training, governance, and participation from all areas (including the business, IT, and management areas) that a company can finally build a bridge across the chasm of Martec’s Law. Once that bridge has been built, technology can be leveraged to support the organization’s goals, instead of being a boulder that must be pushed up the mountain.
The fact of the matter is that if you can develop and mature your organization, narrowing the many worries about whether your technology is obsolete will become secondary, and you can walk around your office with a puffed-out chest all day long.