Businesses today have the capacity to harness data in new ways, yet must take precautions against legal liability by carefully choosing what data to keep and use. With AI becoming a greater player in business operations, organizations need to secure both personal and sensitive company data while simultaneously managing it accordingly and purging older files that pose more of a risk than business value.
Forrester predicts an exponential increase in unstructured data by 2024, driven largely by AI. But due to an ever-evolving landscape and rising costs associated with breaches and privacy violations, creating an effective data retention and deletion plan must remain at the top of our minds.
Data Explosion and Rising Breaches With an ever-increasing increase in expected data volumes comes an exponential rise in breach costs and privacy violations. Ransomware criminals have taken over highly sensitive medical and government databases including Australia’s courts, Kentucky healthcare company 23andMe and large enterprises like Infosys, Boeing and Okta security provider Okta. IBM found that breaches average total costs were $4.45 Million per breach in 2023 — up 15% year over year from 2020!
Organizations need a policy for efficiently handling data. With the advent of artificial intelligence (AI), executives may question whether any obsolete information should ever be deleted; but as data storage increases over time, the more risk a company takes in terms of breaches or fines related to privacy laws. One way to reduce these risks is taking an in-depth look at how your company uses its data as well as exploring any relevant considerations or tangible benefits associated with any data retention strategies.
Organizations often face legal obligations to delete obsolete data due to data protection laws. Regulations require companies to retain personal information only as long as necessary; companies therefore need retention policies with variable retention periods depending on business areas and company strategies. Deleting outdated data not only reduces legal liabilities but can also cut storage costs significantly.
Finding Obsolete Data
A data map can be an effective way to identify which information should be considered obsolete and which will continue adding ongoing business value. A comprehensive map should outline all sources and types of incoming data, which fields they include and which systems or servers store it. A thorough map ensures a company knows where personal data resides as well as which types of protected or special category data are processed; its intended processing purposes, geographic locations of processing operations and applicable systems.
An accurate data inventory and classification system form the cornerstone of an effective privacy program and provide insight into how information flows throughout a company’s systems.
Once a company has created a map of their corpus of data, legal and technical teams can work closely with business stakeholders to ascertain its value, assess any regulatory restrictions for storage of it and consider potential ramifications should any data leak, breach or remain longer than necessary.
Most business stakeholders may be reluctant to delete anything, particularly as technology quickly changes. When discussing deletion and retention issues, discussions must center around what will provide maximum business benefit – for instance a data analytics team at a financial institution who want their lending eligibility models trained with as much data as possible is actually going against data protection and privacy laws.
With changes in interest rates, lending practices and consumers’ personal circumstances over the last 20 years, data collected 20 years ago may no longer provide an accurate assessment of today’s consumers. Companies would likely benefit more by turning to more up-to-date sources of data like credit information to formulate an accurate risk score.
Current commercial real estate market challenges bring this point home. Risk prediction models trained on pre-pandemic data – before online shopping and remote working were standard practices – often result in inaccurate predictions. To reduce inaccurate predictions, discuss with business stakeholders how data becomes obsolete over time and which data best reflects modern life.
Handling Obsolete Data: Determine, Delete or De-Identify
To help decide how long to retain data, start by considering any legal obligations surrounding financial records maintenance and sector-specific regulations related to transactions that involve personal data. Consider legal statute of limitation periods as a guide on whether data needs to be retained for defense against possible litigation – only keep personal information required as evidence in litigation cases such as transaction logs or evidence of user consent rather than every piece of data on individual users.
When the time comes to delete less valuable data, data can be deleted manually according to its retention period as specified by a retention schedule or automatically through a purge policy for improved reliability. Deidentification processes or anonymization technologies may also be employed; these bring with them their own set of challenges however.
Truly deidentified data generally falls within exemptions in data protection laws; however, doing it correctly requires stripping away much of its value so there’s not much left for use. Deidentifying means removing direct identifiers such as SSN or name from customer IP addresses as well as indirect ones like IP addresses of individual customers. For instance, to meet HIPAA safe harbor protection standard an organization must remove 18 unique identifiers. However, such an approach could help maintain performance for analytics or AI models; but before taking this path it’s essential that all stakeholders involved discuss its pros and cons with stakeholders before considering this route.
Avoiding Common Pitfalls
One of the key mistakes enterprises make in addressing obsolete data is rushing through it without first engaging in in-depth conversations and gathering feedback from multiple groups. Project owners must resist the urge to rush things along too quickly and recognize that feedback from various departments, legal, privacy, security teams as well as business leaders is essential in making sound decisions about which data needs to remain. Companies should collaborate across legal, privacy, security as well as business leaders to obtain feedback about which information needs to remain and avoid inadvertent deletion by policies & schedules which inadvertently deletes something crucially needed by business leaders & stakeholders – shorten retention periods over time but measure twice, before cutting once!
As we’ve noted above, when it comes to dealing with outdated data there are several considerations which must be made, including mapping data lineage and retention period criteria as well as devising effective policies. Successful data deletion requires taking an informed and strategic approach. By understanding legal, cybersecurity, and financial implications companies can craft an effective data retention plan which not only meets regulations but also effectively safeguards digital assets.
