Meet the guy trying to drag HM Treasury's data strategy into the 21st century

Interview The UK government is to spend £1.2 trillion in the current financial year overseen mainly by HM Treasury, the department also responsible for keeping the economy on track and overseeing financial markets. To do its job, the Treasury aims to create a "production model" of a new data platform by the end of the 2024-2025 financial year.

Appointed around 10 months ago, HM Treasury's chief data officer is looking to get the department's data ducks in a row.

"When I joined, there were around 300 analysts in Treasury," John Kelly tells The Register. "They are using different data science techniques to automate parts or all of their analysis, but much of it is quite traditional. Analysts take data from, say, Office for National Statistics... shape it, and it would end up in PowerPoint in a briefing.

"Those groups tend to work quite independently; analysts tend not to share common ways of performing that data analysis."

The result is duplicated effort and inconsistency, which leads to inefficiencies. It also creates inconsistency in the tools analysts use.

While the department has a Microsoft premium instance to run the Synapse data warehouse and Power BI analytics and visualization tools, it also uses data science software from Alteryx and Tableau, owned by Salesforce.

Although the Treasury feels it makes commercial sense to center the platform technology decisions on Microsoft, that does not always dictate the choice.

For example, in data catologs and metadata management, Microsoft has a product called Purview, but HM Treasury is looking elsewhere. Kelly says the department has already started exploring Informatica's tools and is considering going to market to find a solution.

"We've got a framework deal for Microsoft already," he says. "But what I wouldn't want to do is make a technology decision that's just completely bound by existing commercial arrangements and we end up using a suboptimal tool or platform for a particular purpose. We need to explore the best in class."

That said, the Treasury has been talking to Microsoft about changing its premium instance to a Fabric instance to get a peek at the Redmond giant's latest data platform and analytics offering.

As well as developing a new data platform, the Treasury is basing its strategy on people and practice. The people side is a "broad capability improvement program" which has so far resulted in the appointment of a new head of data management.

The data strategy resulted from an existing contract with consultancy and IT services firm NTT Data, which began working with HM Treasury under a £250 million deal in 2014. It reviewed data policy, finding the Treasury's approach was "not particularly mature."

From there, NTT helped create a so-called "thin slice," a narrow project that could show users and the leadership what was achievable with a more modern data stack based on Microsoft Synapse.

• UK government faces calls to end IR35 double tax anomaly
• 'There has never been a realistic plan' for UK's £11B Emergency Services Network
• Number of pensioners hurt by DWP legacy system error actually 165,000
• UK government's newest department to lead mega ERP procurement

"The thin slice element meant that we're able to stand up some architecture, that was not too costly or heavyweight, but just enough to prove that we can deliver what we said we would," Kelly tells us.

"This financial year, we'll stand up a small proof of concepts data catalogue so we can effectively test which platform we want to use, and then to have a production model delivered in the next financial year, 2024-2025."

With the data science tool set, the aim is to offer analysts some choice while keeping a lid on support costs.

The department provides tools to support development in data science languages R and Python as well as a choice of statistical libraries. But for sharing data with ministers, analysts are told to use the Treasury's Plotly Dash environments. Repeatable analysis should use Power BI and the department is ruling out some new languages and continuing with Tableau, at least in the short term.

"We need to get a skill set and community of analysts around Power BI and be proficient at that as a common platform before we start exploring other areas," Kelly adds.

But the restrictions are not intended to prevent the department from exploiting data science techniques in AI to improve understanding of government performance.

The team is developing a new forecasting process using a neural net which it hopes will lead to more accurate forecasting of how much cash the Treasury needs on a day-to-day basis, Kelly says. "AI is a big part of the strategy and identifying how we can use data science and AI to improve the way that we work."

The ultimate goal of the strategy is to help shape and evaluate policy, rather than it only being the result of a great deal of "stakeholder engagement," he tell us.

"There's room for bringing data much more to the center of that both in how we shape policy, and how we evaluate the impact of it. There's some work to be done in the process of scaling up and enabling our analysts to show policy professionals where data can really add value to the fast-paced work that they have to do."

With the UK government spending the equivalent of £42,000 per household per year, the electorate will hope it can bring in some data that helps make sure it is spent wisely. ®