Through education and work I’ve been in the data and analytics game for 20 years now! (More than half my actual life).
I’ve lived through significant advancements in the data and analytics profession. Not least in the last year. Whilst there are perhaps people that have been in the profession for double that, I would still consider myself somewhat of a data veteran…
If there are people in a saloon casually discussing “the problem with using algorithms to decide school grades is…”, I’ll slowly emerge from a dark corner like a wise data cowboy and ominously say (in a gruff voice):
“You think that’s bad, you weren’t there during the dark days. I remember it taking four hours to save an Access database. I remember people making maps in ‘paint’ (ok that might just have been me). I remember when them there folk (decision makers) were scared of data, and you’d be accused of witchcraft for doing any sort of analytics…”
*at this point most people have snuck off, and I’ll slope back to my corner on high alert for the next mention of ‘machine learning algorithm’.
Ironically, it’s difficult to quantify the relative growth of the profession and industry over these 20 years. Most people are aware of the increase in the volume of data that exists today (2.5 quintillion bytes created each day; Forbes 2018) and that data is easier than ever to access, process, store, etc. If you were to look at any of the biggest growing industries in this time (such as engineering, gambling, sales, etc.) they have all depended on data and analytics to grow. The amplified prominence and importance of data is clear to see.
Making physical changes is simple, changing minds is not
The global pandemic has acted as a catalyst for the continued progression of the profession. Over my 20 years in analytics I’d experienced some huge strides forward in analytics ‘enablers’ (accessible software, mainstreaming of data science methods, fit-for-purpose storage). However, this wasn’t enough for the data doubters. For people not in the data world it didn’t matter how quicker and easier it had become to produce and deploy more accurate and powerful predictive algorithms. And hence I still found myself repeating the oft quoted disheartening statistic that ‘50% of data science models never get implemented’.
In summary at the beginning of 2020, data was more fruitful, analysis more sophisticated, but the impact of analytics hadn’t made comparable headway.
And then March 2020 hit…
Like our own internal ‘life-cycle of a data science project’ the pandemic very quickly created a mass of problems.
Outside of the impact of the virus itself, a huge number of questions were being asked about the impact of COVID-19 and the lockdown on all our services and people.
Mental health and wellbeing, the economy, domestic abuse, education, homelessness, financial hardship, physical health, social care, etc – EVERYONE had questions they simply could not answer as the situation was unprecedented.
ecda was asked what it could do to help understand how and why some of these services were impacted – and, what happens next. Which moves us to the next stage of the life cycle…DATA
Before talking about ecda’s work, I’ll return to the national situation. Even in those halcyon days of January/February 2020 – it was evident that data and analytics would become increasingly important in the battle against the virus. Almost immediately everyone needed data. This was the case nationally and locally. It was all a bit understandably chaotic. If time pressures weren’t quite so severe, people may have thought about exactly what data is needed, what they want to do with it, and what it will tell them – but that luxury simply didn’t exist. So, people asked for ALL THE DATA.
Be careful what you wish for
‘Getting’ all the data is a problem itself. And, if people succeeded (and I’m not sure anyone did), ‘Having’ all the data is an even bigger problem!
Fundamentally (2019 and before) data was something for the data and analysis teams. But all of a sudden people that are not used to engaging with data needed to and were not fully prepared or trained for how to get the most from it.
ecda’s Societal Impact work began once there was some modicum of calm in July 2020. We were asked to understand the impact of COVID-19 on child abuse - which had been identified as an area of concern due to high levels of harm and vulnerability.
Like a lot of things in 2020, this ecda project was very different to what preceded it. The project was a priority because we feared significant latent demand building in child abuse due to increased pressures and the severance of referral mechanisms to childrens’ services (such as school closures). The need for this insight was immediate to allow us to understand impact on vulnerable children during the first wave, and to prepare for the re-opening of schools, any additional waves, and to determine what the longer-term landscape may look like. There was no manoeuvrability on these demanding time scales. Effectively we had about 5-6 weeks to generate and circulate an informed and robust view of current and future demand.
ecda needed to accelerate through all initial project phases to begin analysis as soon as possible. ‘Having’ all the data would have been a hindrance in this instance. We needed a clear and concise focus.
This focus was:
- to understand the impact of COVID-19 and the lockdown on childrens’ services (in particular regarding abuse) and;
- to make recommendations on future demand to prepare resources accordingly.
The second reason that allowed us to start work immediately was that, following a quick discussion weighing up the pros and cons of aggregated data vs granular data, we recognised that we could answer our business problem with Essex-level data alone. This was a first for ecda. (For details about the project read our previous blog).
We devised an approach to trend analysis and forecasting using a methodology called time series – within a couple of weeks we had an understanding of potential latent demand pools across all childrens’ services, the reasons behind this, and evidence driven forecast scenarios. If you're interested in seeing my video about forecasting, you'll find it here.
We saw such great value in the outputs from aggregated data sources that we are keen to explore more projects of this nature. And the approach to demand forecasting was also repeated for mental health services; adult social care demand; projected homelessness; and many other areas.
We were able to quickly deliver insight and recommendations stemming from analytics. And it was around about the middle of 2020 where people across the country were realising that ‘having’ data alone is not enough.
Data on its own is interesting. Applying analytics to the data makes it insightful. (And when this analytics is connected to a business problem it can become impactful).
Nationwide analytical models were starting to take centre stage, not always for the right reasons! (I’m looking at you Education algorithm). The importance of analytics and data science was being exposed to the masses. We began hearing phrases like ‘follow the science’ and ‘data not dates’ as the country looked towards analysts and data scientists to lead the resistance and recovery.
Analytics was becoming influential!
As far as the Societal Impact work goes, this helped inform recovery and budget planning, full evaluation is available here. And when expanded to include Mental Health services (in phase 2 of Societal Impact work) and combined with other service demand insight, we had this incredibly rich view of the impact across the entire system. It was only at this stage that I appreciated the sheer magnitude of this impact. It was seismic. And no service had gone unaffected.
A lot of the focus throughout the spring and summer of 2020 was understandably towards the virus (case rates, hospitalisations, deaths), but as 2020 was drawing to a conclusion the emphasis started to shift from the virus itself to the devastating legacy effects.
At the very beginning of 2021 we had ready made intelligence and demand forecasts for all our services for the year, enabling us to try and get one step ahead. This was presented to our chief executive, service directors, team leaders, commissioners, and pretty much every forum across Essex to initiate immediate planning and action.
Acting swiftly off the back of robust analytical intelligence is what will enable the country to overcome the continued challenges faced and to rebuild it stronger. It feels like a real watershed moment for analytics nationally and locally – through ecda we hope to continue to be the organisation driving the data and analytics agenda for social good.
Leave a comment