A new Azure-based data lake and analytics platform gives King’s College London (King’s) analysts and university staff access to cross-functional data for strategic planning and decision-making.
At a glance
• Pan-university data lake, data warehouse and analytics platform
• Microsoft Azure, SQL Server and Power BI
• Overnight feeds from multiple source systems
• Ability to track complex KPIs and model scenarios
• Supports predictive analytics with machine learning
• Built by Adatis with full knowledge transfer to King’s
From data silos to data lake
Siloed data meant King’s was unable to get a clear picture of its operations for strategic planning. As part of a wider digital transformation, it has worked with Adatis to implement a pan-university data lake and analytics platform in Azure – creating a sustainable foundation for predictive analytics and evidence-based decision-making.
The Challenge:
Break down data silos to improve strategic planning
Like many universities, King’s had a problem with data silos. It divides its activities into five data domains: Students, Research, People, Finance and Space. Each domain has its own management systems, which meant getting a coherent view of operations was very difficult. Critical questions like ‘which teaching spaces should we use for which courses next academic year?’ were next to impossible to answer, often resulting in inefficient use of city-centre space in a global capital where every square foot comes at a hefty premium. Senior stakeholders were also unable to get the real-time insights they needed to track their progress against key performance indicators, for example the ratio of offers to acceptance, volumes of overseas students, and research income by faculty. The result was that many decisions were being made without evidence to support them. “The decisions being taken were not necessarily the ones people would have made if they’d had the data available,” says Matt Gordon, associate director for data and analytics at King’s.
A frustrated analytics team, unable to deliver true insight
Behind the scenes, the university’s strategy, planning and analytics team were struggling to keep up with demand for management information. Trying to get the evidence that stakeholders needed meant spending huge amounts of time attempting to access and collate data from multiple systems across different departments. “It was a real challenge for us to get the data. My boss often says it required at least three programming languages and a degree in diplomacy,” says Matt. “A huge amount of time was spent hand-cranking reports in Excel and trying to access data that was almost out of date by the time we got hold of it.” That left little time to analyse the data and extract meaningful insight – let alone to run scenarios or predict the outcomes of different strategic decisions. Something had to change.
The proposed solution:
A single, pan-university analytics platform in the cloud.
The first goal was to break down the data silos so that people could get the information they needed. “We wanted to provide a common view,” says Matt. “An Executive Dean of a Faculty should be able to see the number of students they have, the amount of space they have, the amount of research income they’re able to access – all in a single place.” Another key objective was to reduce the amount of time King’s analysts spent getting hold of data, so they could spend more time analysing it and producing meaningful reports and insights that could support strategic decision-making. The King’s strategy, planning and analytics team set about looking for a specialist analytics technology partner who could help them deliver on that vision. Their criteria: the solution had to be built on Azure, because King’s has a cloud-first philosophy and a strategic commitment to Microsoft. The partner also had to be able to help King’s make the most of the evolving analytics capabilities available in Azure, which presented some exciting opportunities. Importantly, they would need to show some progress early on, to keep people engaged with the transformation. Previous attempts to overhaul data and analytics had been abandoned after taking too long to deliver anything of value.
Adatis: A specialist Microsoft partner with an Agile approach
After evaluating a number of potential partners, King’s invited a shortlist of two to attend a five-day discovery session and present a proposal. From that exercise, one consultancy stood out. “Adatis came out very strongly as the preferred supplier,” Matt says. “They didn’t have a background in higher education, but their proposal fitted us better than some organisations that had experience in the sector.” As a Microsoft Gold Partner, Adatis had the Azure and SQL Server expertise that King’s was looking for, as well as detailed, up-to-date knowledge of Azure’s analytics services and expertise in Power BI, the university’s data visualisation tool of choice. Adatis are also Agile practitioners, with a focus on delivering value early. “Their approach of building something within a couple of weeks and being able to deploy at least a thin slice of development for people to start using was very attractive,” says Matt. But what really stood out for King’s was Adatis’s approach to data modelling, using the BEAM (Business Events Analysis and Modelling) methodology to focus on what King’s and its stakeholders wanted to get from the platform. “Other consultancies just asked us what fields we wanted,” says Matt. “Adatis focused on how they could model the data to get us the information we wanted. That gave us confidence that we’d
have a model that would deliver what we wanted.”
A phased approach to the platform build
With Adatis on board, the platform build got underway. The project rolled out in phases, starting with Admissions and Enrolment before moving on to Space & Timetabling, Research, Marketing, HR and Finance. Each stage has been completed progressively faster, with Adatis transferring skills and expertise to the King’s team and both teams applying lessons learned from the previous stage. The new platform ingests data from multiple source systems (see panel) into a single data lake. From there, some data is structured into a SQL Server-based data warehouses where it can be accessed by business users and stakeholders via a selection of pre-built reports, and King’s analysts can also pull relevant data from the lake to create bespoke Power BI reports and dashboards for senior stakeholders.
The Results:
Better decision support today, and a path to predictive analytics in future
Adatis approached the project in an Agile way, making sure to deliver usable functionality early. One such deliverable was the ability to correlate student numbers with classroom space and course timetables. The university can use these combined datasets to track how many students are taking which classes, in which spaces, throughout the year. This insight can be used to plan utilisation of the university’s teaching spaces more effectively. “We can now start to model out questions like ‘if we increase numbers on a specific course, can we accommodate those number with the space we have?’” Matt explains. That enables the university to extract maximum value from its current real estate by accurately allocating its existing space, ensuring the most efficient use.
As more source systems are added, the data available to drive complex planning and decision-making processes is growing, ensuring that more decisions can now be made based on solid evidence. “The improved transparency has been highly beneficial,” says Richard Salter, Director of Analytics at King’s. ‘It has led to a much deeper shared understanding of our organisation and greater consensus around what we need to prioritise moving forward. Evidence-based decision making has enabled the organisation to achieve great levels of equity and innovation.” The platform has already delivered huge benefits for King’s analysts, too. Time spent trying to access data has been minimised, enabling the team to spend more time analysing the data and creating reports and visualisations. “They can refocus their valuable time on finding new insights, building new reports and engaging with users across the university to explain their principal work and related conclusions,” Matt says. “Power BI reports allow users to access and explore data for themselves, so our analysts don’t need to email staff to request information from them.”
The Future:
Predictive analytics, machine learning, and a self-sufficient team
The end-goal for the platform is for it to support a predictive analytics approach, using data models to run different scenarios to understand how different strategic decisions might play out. To that end, King’s is now upskilling its analysts in building predictive models, and plans to make use of the cognitive and machine learning capabilities of Azure once those skills are in place. Another goal for King’s was to become self-sufficient at managing the platform. Adatis is now working on a full knowledge transfer – partially through a pair development approach, with Adatis developers sitting alongside King’s developers to transfer the skills over.
A successful implementation and a positive experience
In all, the project has been a very positive experience for the King’s strategy, planning and analytics team, with the benefits already beginning to be felt across the university. “I’ve got nothing but good things to say about working with Adatis,” says Matt Gordon. “They’ve worked collaboratively with us – if we’ve hit a problem, we’ve met, and everyone’s worked it through. Their culture is all about helping us meet our end goals. It’s been really good.”