DataOps Needs a Mom
DataOps, you know I love you. But you act like a teenager trying to find yourself. Calm down. It’s not that hard. You just need a bit of context and a backstory as you set off to make your mark. Choosing a direction might help, instead of running in all directions at once.
Yeah, sure, you might take it upon yourself to immediately become your father. But, c’mon. Huge bureaucratic data governance is a well-paying job, especially at banks and insurance companies, but this is not really you. You can be something new, and you don’t have to try to pretend that every fast-moving digital enterprise will slow down to glacial speed and spend a year to define what the word “customer” means.
And, sure, your grandfather still works as janitor on the night shift, cleaning up the messes and making sure that all of the production equipment is ready to go for the morning shift. But simply noticing when the data machinery breaks down and fixing it, every night, over and over again, is not a career to aspire to.
I know your sister is an amazing artist. When she put her skills to use in drawing entity relationship diagrams and business process workflows, she found that she could coast along using only 10% of her skills, and helping everyone visualize what yesterday’s enterprise looks like. But just between you and me, her private studio where she envisions what the world could be is far more valuable, in spite of the fact that no one pays her for it. I don’t think you can follow her path, anyway. You need to find your true self.
So, DataOps, my darling boy, don’t settle for the past. Don’t limit yourself to defining the roles of data steward and data owner. Don’t slave away to cleanup data production messes that will only reappear on the next production run. And don’t spend your time drawing pictures of the way that enterprises used to function. Fancy data model diagrams make good museum pieces but they don’t lead to anything new.
What do you want to be? What do you want to do?
I suggest that you find out what makes an effective digital company. Study how capital is used differently. Use open-source to avoid the temptation for your engineering team to build code that represents a castle in the sky that they can go and live in and forget about the real world. Learn what your customers actually use your data & services to do. Don’t listen too closely to what your sales teams tell you about how to get prospects and turn them into contracts. The reality is a bit different after your customers start using your stuff.
If you take the time to study, here’s what DataOps (that’s you!) needs to learn.
- Lean Manufacturing Principles
- The DevOps and Test-First Culture
- Data Quality Automation
Yes, you’ll find yourself asking your father about metadata. And you’ll find that the best metadata is “just enough” metadata. Just enough to support continuous improvement. Just enough to automate continuous integration and delivery. Just enough to measure the root causes of data quality. Just enough to support the methods to produce it consistently. Just enough to stop creating messes that require daily janitorial staff to get the equipment running again every day. If you’re any good, then when your grandfather retires, he won’t need to be replaced.
And if you get a chance to work with your sister, ask her to bring her amazing imagination to work with her. She can help you design new data pipelines, new user interactions, and new metadata that your father never collected, which speaks to the significance of the data and how valuable it is to your customers.
Build a new world, my son. Don’t get stuck in the past. Build the new “DataOps” world that we’d all rather live in.
