Subscribe now for just £20 per year


Pressing the delete button

The service has a major issue dealing with poor quality legacy information on its systems. But how does it navigate those difficult backwaters safely? Gary Mason reports.

Pressing the delete button

Date - 16th February 2021
By - Gary Mason

The routine ‘data weed’ of the PNC that recently resulted in the accidental wiping of more than 15,000 offenders’ criminal records was a massive embarrassment for the Home Office and continues to give the department a severe headache.

Months of work lie ahead to try and retrieve the lost information which relates to more than 200,000 offences.

But while the data wipe is an epic cock-up in anybody’s language the importance of regularly weeding information held on police systems is well known to all forces, many of who have their own data issues.

Weeding data to comply with legislation, because it is out of date, factually inaccurate or simply no longer of any use is an essential task. But it is clear that many forces do not know what data they hold on all their systems, how much of it is relevant, duplicated or simply useless.

This covers a myriad of local systems but there is a spill over of the problem onto shared regional and national systems  - particularly the PND.

That database which holds intelligence relating to arrests was one of the back up systems that policing minister Kit Malthouse said should be used by ‘law enforcement partners’ while the lengthy task of retrieving the missing PNC data was ongoing.

But how reliable is the data on the PND? What we do know is the system is at the centre of a big Home Office-led project called NLEDS aiming to combine the PNC and PND into one huge database that can be accessed by all law enforcement agencies.

We also know that the project is significantly overdue and over budget. Paddy Tipping, the PCC for Nottinghamshire told the Police Digital Summit last week that police management and the Home Office “often fall out” about it.

Both systems hold a lot of legacy data but there are significant differences between the two.

For example there clearly will be a more complete data set for an offender than for a suspect but defining what that looks like and being really clear about it will be key to making the systems work hand in hand when they are combined and relaunched.

Teresa Ashforth is PND data quality and standards lead for the Home Office. Before that she spent 12 years at Durham Constabulary, whose current chief constable Jo Farrell is NPCC lead for data.

Defining what “good looks like” is clearly an issue given that 43 forces will have their own approaches to intelligence reports.

“We’ve got massive gaps and massive inconsistencies,” she told the Police Digital Summit. “Data no longer sits within force boundaries – it goes up to national systems whether that is the PNC or PND. It will go to other law enforcement partners and it will go to government. It will also go to our partners such as the NHS and Social Services.

“The next big step for me is defining what good looks like from a policing perspective. I’m not sure we know that yet. There are pockets of good work around location data but we want to see that across the board.

“This will mean that when an officer goes on to the PND they will know that the data they see from Avon and Somerset, Derbyshire, Leicestershire, Lancashire is the same quality and meets the same standard.”

Part of the issue is data weeding and nervousness around dealing with legacy information with the conflicting pressures of long-running historic investigations - such as IICSA (child sex abuse) and UCPI (undercover policing)  -  and differing interpretations of what is required under the Data Protection legislation.

Blind alley policing

But weeding data even on local systems potentially cuts down on workload and avoids blind alley policing. For example Durham weeded its information on missing persons and found out it had far fewer long term MISPERS than it thought.

“When we were looking at long term MISPERS we thought we had 32,” says Ashworth. “But when the data was tidied up to remove duplication and improve accuracy the real figure was 12.

“Front line officers and those who work in safeguarding were then able to trust the data.”

CC Jo Farrell is setting up a national data office 

CC Farrell said nationally she had secured central funding to set up a ‘data office’ with expert staff  within the next 12 months to tackle some of the issues facing the service including “the vast volumes of legacy data that exist across policing and put us in a better position.”

But how much do forces need to invest in dealing with their legacy data given the sheer volume of it that is rattling around local and national systems? At what point in its history does someone press the purge button?

The PND was created following the 2004 Bichard report into the murders of Holly Wells and Jessica Chapman in Soham, Cambridgeshire. The nub of the inquiry was that Ian Huntley was able to get a job as a caretaker at his victims’ school despite there being police intelligence in other forces indicating that he posed a serious risk to young girls.

Humberside Police said it deleted intelligence records on serious allegations of sexual assault made against Huntley because it thought this was required under Data Protection legislation.   

Understandably the case has made forces extremely “risk averse” in weeding legacy data that may no longer be relevant or useful even that not relating to intelligence on suspects who have no criminal convictions and therefore are not on the PNC

Teresa Ashworth told the summit: “Some forces like to hold onto everything because of ongoing enquires they are involved in and other forces are willing to dispose of data. Budgets and resources are key. If you are in a force that is going through a big ICT transformation  - and there a few who will be in that position over the next 12 months  - that will have a big impact.”

The Met is the largest force in the country so will hold more legacy data on its systems than anyone else.

Aimee Reed heads up the force’s data transformation programme having joined the MPS in 2001 as an Intelligence Analyst. She has also worked in intelligence and covert policing in 20 years with the organisation.

Data as an asset

Three years ago according to the Information Commissioner’s Office (ICO) the Met was the worst organisation in the UK at delivering quick and effective responses to requests from the public about information it held. There were a range of ICO enforcement actions in place and a landscape of “considerable legacy data,” Reed told the Police Digital Summit.

But she says the force also had “an ambitious and impressive” digital policing department that was ready to drive changes with technology.

This involved setting up a data office which would be an “objective guardian” over all the data the Met held in its systems. This involved changing the mindset of the force so that it started to “think about data as a valuable asset.”

The first thing needed to achieve this was leadership buy in. “The arrival of Cressida Dick into the organisation was a real game-changer for us,” she said.

Although a lot of progress has been made she said the force is “still not on the same page on some issues including the balance between addressing compliance issues with legacy data and the opportunity to do new things with data which is where management want to be in terms of investment. Otherwise you can end up just throwing money at people’s incumbent mistakes.”

What do forces need from the centre?

Aimee says forces need investment in data not just technology and an acceptance of the fact that it’s going to take time to see real improvements. “Data quality doesn’t become good overnight. It requires culture change and buy in.”

She also said forces need to understand that the police operate under a different part of data legislation to other organisations.

One of the key challenges and it is also a benefit is the Law Enforcement Directive. “Not all of our data is held under that; we are subject to Part 2 as well. There are exchanges between the two but we need to make that a lot easier than it is”.

She also says that police forces have a tendency to view data in terms of restrictions and legal requirements instead of its potential. 

“What you don’t want to become is ‘Mrs Compliance’, she said. "Data is an opportunity and an enabler; it is not the place that stops you doing things but it is often seen like that.”

Another issue highlighted by Teresa Ashworth is improving ‘data literacy’ particularly among senior police management.

She says they need to become more intelligent as clients when they are going out to market to buy technology and shouldn’t be looking for the “shiniest new thing” that’s out there.

“When we buy technology the data that sits on those systems needs to be well managed and well looked after. When I speak to policing colleagues who work in records management they are literally on their own. They don’t have the links to chief officers needed to drive change when they are key staff.”

She says forces need to “professionalise” that area of their business.

Front line buy in

The workforce generating a lot of the data that ends up in intelligence reports will be front line officers. How do forces ensure buy in from them about data quality?

Aimee Reed says: “We need to be really clear when we want people to fill in data at the front line and minimise the amount of times they have to do that.

“We also need to focus on what’s really critical to fill in well. Then give them the tools to access that data quickly and effectively so once they start using it and exploiting it themselves they will understand how important data quality issues are.”

But there is no point “analysts in an Ivory Tower” saying that to the front line.

Teresa Ashforth says more automation is also key: “We are still buying systems in policing that don’t have basic validation built in. We are seeing data inputted where a post code is in the date of birth field for example. There are opportunities for technology to manage those errors on our behalf."

Do you have an interesting news story? Contact the newsdesk on 0203 119 3303
or alternatively get in touch via the contact form.

Categories and Tags

Police Oracle welcomes readers’ comments but please keep them concise if possible. Personally abusive comments directed at named individuals and posted anonymously are not welcome. The editor reserves the right to block and delete any comments that fall into this category.


Ordered by: