Adatis provides tools to help online fashion retailer get datasets into Azure SQL Data Warehouse in days rather than months – significantly boosting the retailers ability to respond to trends.
New tools enable online fashion retailer to get data ready for analysis faster
This global fashion retail brand has an ambitious internal growth plan and has chosen Microsoft Azure as the strategic platform to support its strategy. Adatis worked with the retailer to provide tools for Azure SQL Data Warehouse that enable the retailer’s developers to get datasets ready for analysis in a matter of days.
About this Iconic online fashion retailer
This London based online fashion retailer is on a mission to be the world’s number-one fashion destination for twentysomethings. In 2016, they chose to migrate their core technology operations from an on-premises data centre to Microsoft Azure, to ensure seamless scalability as it rolls out its ambitious international growth plan.
The challenge: Getting datasets into the data warehouse was taking months
This retailer prides itself on understanding its target market of fashion-loving teens and twentysomethings, and on its ability to swiftly reflect consumer trends. However, the brand was hampered buy the slow process of getting datasets into the data warehouse for its business teams to analyse.
Before moving to Azure, the retailer had been operating an on-premises data warehouse, with a team of developers working in parallel to clean and format raw data, create datasets, and load them into the warehouse for analysis
But as the company has grown, so has the amount of data it generates. By 2017, the on-premises data warehouse was reaching its limits for data storage and for speed of executing queries. It was also taking the data development team a long time to get datasets into the warehouse – both because of the manual coding effort, and because multiple developers were often working in parallel on the same dataset, frequently causing them to overwrite each other’s changes.
As a result, if a business analyst requested a dataset that wasn’t already in the warehouse, it could take months for the data development team to make it available – seriously impeding the brand’s ability to discern trends and respond quickly.
The solution: A new data warehouse, and new data development tools to speed things up
A strategic move to Azure was already underway, including a new data warehouse using Azure SQL Data Warehouse. But to take full advantage of the new platform, the data development team needed to address the issues that were hampering its ability to get data into the warehouse fast.
Microsoft recommended that the retailer work with Adatis, a Microsoft Gold Partner specialising in building and maintaining modern data warehouse platforms. A team from Adatis worked onsite with the retailers’ team from June to November 2018, with a remit to speed up delivery of datasets into the ADS.
During that time, the Adatis team provided the retailer with a full enablement platform comprising:
- A SQL based metadata store used for handling and coordinating the many schedules, business rules and policies implemented throughout the company’s platform
- Code generation tools using a templated pattern approach, to simplify the generation process of objects within the warehouse, using the metadata store to ensure consistency across the platform
- Data contract-based process definitions to allow for granular modification of metadata and to prevent conflicts and overwrites that can causing further delays
- A benchmarking and performance testing framework to allow for experimentation with pattern and template configurations while maintaining performance against peak load benchmarks
All of these tools were developed in a rapid, iterative manner, with the retailer’s team testing them out at each stage and providing feedback to the Adatis team.
The results: Data availability timescales reduced from months to days
Using the framework and tools provided by Adatis, the retailer’s data development team can now get new datasets into the Azure data warehouse in a matter of 10-14 days, rather than the months it typically took previously. The impact on the brand’s ability to identify and respond to trends is likely to be significant.
What’s more, the retailer’s team can use and maintain the tools themselves, as they were created in a full knowledge transfer environment, with the Adatis team working closely with the brands team to build and test the tools and to train the brands team in their use.