Author: Rishi Sapra, Senior BI Consultant at Altius
By now we’ve all caught wind of the many benefits of having a data-driven culture within an organisation. Making decisions based on data can drive better decision-making, decrease expenditures and increase profits. But how do you get your organisation to become more data-driven, and eventually to take on serious projects within data science? It starts with understanding and implementing the fundamentals.
Begin with the fundamentals
In order to drive the change to a data-driven culture within your organisation, begin by educating yourself in the fundamental skill sets of data analysis and reporting. Over time, as data becomes increasingly embedded, high quality and more trusted by your teams, you’ll be able to perform more complex analysis.
Challenges in embracing data science
In companies with a strong data-driven culture, we see things like machine learning and decision automation. Big players like Microsoft, Amazon and Google – as well as small but fast-growing technology start-ups – base their entire business models around harnessing cognitive analytics, which refers to combining artificial intelligence and machine learning to mimic the way the human brain works, in order to enhance human decision making.
While the media may portray that every company in the digital age is embracing data science and artificial intelligence, the reality is that most companies – including the vast majority of that make up the FTSE 100 today – aren’t doing this on any real scale just yet. Data science initiatives are very likely on their agenda, but for now it’s mainly isolated projects (ones the thriving data science team at Altius can indeed tackle!).
There are three main reasons why companies have faced challenges in becoming data-driven:
- The data within these organisations isn’t yet centralised. It sits in pockets across the company, usually in a combination of corporate databases, file systems and spreadsheets on peoples’ computers.
- The data isn’t in the format or quality required for the type of analytics data science requires. The sources which capture this data aren’t typically sophisticated enough to capture the rich set of attributes needed to put it through a machine learning algorithm. Cognitive analytics is only as good as the data feeding into it – no matter how clever the logic is, it’s not exempt from the universal rule of ‘Garbage in, Garbage out’.
- It’s still ‘dark data’. This term – coined by Mark Whitehorn, Professor of Analytics at the University of Dundee – describes data that is invisible in an organisation. It may exist in transactional systems, communication systems like email, or file shares, but because it hasn’t been used by the business for basic analytics and reporting yet, it hasn’t yet earnt the trust of the stakeholders. The value of this data hasn’t yet been proven for simple use cases, so to try and use it in more complex ones – such as machine learning and decision automation – is too risky.
Cleaning data to create insights
If the above scenarios exist in your organisation, and you want to mature your data analytics approach, where do you start?
Get a sample of dark data. Put it into a reporting and analytics tool with a data visualisation element, like Power BI or Tableau. Then proceed to clean, transform and model the data using the visualisation tool itself or a third-party data wrangling tool like Alteryx.
Once you have clean data ripe for insights, you can quickly create business insights. You can do historical analysis, or even highlight the distributions of data by various attributes – like time, region, or product category – whatever type of descriptive attributes you have in your data.
While doing this, try to embed a better understanding of the data amongst business stakeholders. Establish them as the owners of the data, provide them with data dictionaries, and give them access to the data models you’ve built in tools they’re familiar with using, like Excel. Most importantly, ensure you have a single version of the truth and proper Governance in place to restrict who can access what data.
Seeing the data-driven shift happen
Once you’ve started to embed self-serve analytics in your organisation, the shift to a data-driven culture has begun. You’ll notice a difference: teams starting to understand and trust data, and even get excited about it. At this stage, company data is maintained in a data platform which should be owned by your IT department.
Now you’re ready to look at things like benchmarking and advanced reporting. You’ll be able to set up functions like automatic metric alerts, using tools like Power BI. You can then start to perform more advanced statistical analysis. With Power BI you can start to embed bits of R code into the data prep or the visualisation level to get predictive analytics, and you can supplement your report with a tool like Power Apps to do scenario planning.
Getting down to data science
Now you’re equipped to get serious with data science (the most exciting and fun side of data!). You can use the open source R and Python libraries and your favourite interactive developer interface (IDE) to develop complex models and deploy them on your analytical platform, one comprehensive deployment option being the Azure Machine Learning tools.
The output results of your models, or the models themselves, can be called through exposing the logic as an API, which can even be called from Excel. Now you’re firmly in the realm of changing the fundamental business model of the organisation using cognitive analytics.
It’s not easy, but it’s worth it!
If you’re lucky enough to work in a data-driven culture with exciting projects in data science, that’s fantastic. Get stuck in! But if the company or clients you work with aren’t at this stage, make sure to get the fundamentals right first, and embark on this thrilling journey together. Laying the groundwork from the start will make the job of data science smoother and more enjoyable in the future because you’ll have good quality, trusted data to work with.
Don’t worry if you don’t have the skills to do all this cool, complex stuff straight away. The skills you’ll hone when embarking on a data-driven journey are valuable in their own right for your career as an analyst or data scientist. And don’t forget to study up on business analysis, data governance, data preparation and data modelling, as this will be a huge help along the way.