The 5 technological barriers to data-driven local government

Most local authorities have an ambition to be data-driven, but many technical obstacles stand in their way. We review the key barriers and propose a solution.

In the first two blogs in this series, I looked at how data, analytics and AI can help local government become more effective  and efficient .

But as anyone who’s ever worked on a digital transformation project in local government knows, there are many technical barriers standing in the way of local authorities as they seek to make better use of their data.

This blog outlines some of those barriers and proposes a solution: a Modern Data Warehouse architecture using the Microsoft Azure stack. It’s based on our new white paper: Data: The Key to Efficient and Effective Local Government , where you can read more about how councils are putting their data to work today, and how Adatis can help.

Five technical barriers to data-driven local government

Broadly, we come across five key challenges when working with public sector organisations:

1. Unlocking data from individual systems

Councils are sprawling organisations with many different services and departments, each with their own systems and databases. These are frequently legacy systems, with no reporting function and no easy way of extracting data from them.

As NESTA identified in its 2016 report Wise Council: Insights from the Cutting Edge of Local Government: “Silos of data are widespread in local government and, whilst they support their primary functions successfully, opportunities have been missed because data between councils (and even within councils) are often incompatible for connection, sharing and wider re-use for other purposes.”

The first challenge for many initiatives is to identify the systems that hold the relevant data, and extract it from the many different source systems it resides in. That requires a flexible architecture that not only takes data from the initial systems identified for the project, but also makes it easy to add and remove source systems over time .

2. Standardising and pooling data for analysis and sharing

Unlocking data from individual systems is just the start. It must then be cleaned and transformed into a standard format that allows it to be combined with other data.

There is a huge amount of duplication across council and local agency databases, so accurate master records must be created, and the data brought into a central repository where it can be made available to the right people at the right time.

This has long been an ambition for local authorities, but the size of their IT estates and lack of budget have often been a barrier. However, many realise that without a central repository of data, many other digital initiatives will fail.

In October 2019, for example, Lancashire County Council set out five digital priorities that will underpin the delivery of sustainable digital services, the first being “to ensure that all of the council’s data is captured accurately, kept up to date and accessible.”

For most authorities, this has often been a case of easier said than done. But now, Azure technologies like the Azure Data Lake make it both possible and affordable to bring data from multiple systems together into one place.

3. Applying the right tools to get the right insights

Once the data is available centrally, the right analytical tools must be applied to it to allow different kinds of users to get the insights they need.

Sometimes the aim will be to analyse historical performance. For example, West Midlands Police are examining historical policing data to identify crime hotspots and focus resources on them.

Other use cases require newly-generated data to be analysed in real-time, such as new tracking initiatives aimed at preventing ‘waste crime’ like mislabelling or mishandling rubbish in a bid to get around landfill and fly-tipping laws.

The most advanced cases require predictive models to be applied to data gathered from multiple datasets, like the project currently being undertaken by Suffolk, Cambridgeshire and Norfolk county councils to predict future demand for children’s social care.

New machine learning, data analytics and visualisation tools on the Azure platform support initiatives like these, and are becoming more accessible by the day.

4. Ensuring compliance, ethics and governance

Local authorities have a duty to respect and protect data gathered for analytical purposes. Data collected into a single place must be kept secure and well governed in line with GDPR and government commitments to data custodianship.

Initiatives like the use of algorithms to predict child abuse are already raising fears about profiling and stereotyping. As local authorities increase their use of AI and predictive models, they will need to ensure algorithms are free from bias, and be able to explain how the algorithms came to deliver the answers they did.

5. Acquiring the right skills

Using data effectively requires up-to-date skills and knowledge, especially in emerging areas like cloud, AI and predictive analytics. Local government is in competition with the private sector to attract highly sought-after skills, which means hiring new inhouse teams may be unachievable from a budget point of view.

But that doesn’t mean there’s a lack of talent in local government. In its March 2019 report, Data Science for Local Government, the Oxford Internet Institute, says of its work with local authorities:

“We found lots of examples of skilled analysts and BI specialists working on remarkable projects with shoestring budgets. Hence, we would encourage local governments to invest more in the people they currently have, by providing them with training and space to innovate.”

One of the most effective ways to do that is to partner with an external specialist which can transfer its knowledge to the in-house council team over the course of a project. This is something we at Adatis always aim to do with every data infrastructure modernisation project we work on.

The solution: A modern data warehouse delivered by an expert partner

At Adatis, we work with organisations across the public sector to tackle the challenges outlined in this blog, combining our analytics and data science expertise with a Modern Data Warehouse architecture built on the Microsoft Azure platform.

You can find out more in our new white paper, Data: The Key to Efficient and Effective Local Government . Get your copy and see how you can put the latest technologies and skills to work to support data-driven initiatives.

Get the white paper