Sorting 23,000 files without breaking anything

The problem

A community behavioral health center in New Hampshire had nearly three decades of files on a single shared network drive. The oldest dated to 1996. Over that time, twelve departments had each built their own corner of the drive with their own habits: different capitalization, different abbreviations, different ideas about what “archive” meant. Nobody owned the structure as a whole, and there was no retention standard anyone could point to.

That’s a nuisance in most organizations. In behavioral health, it’s a compliance exposure. The drive sat under several overlapping rule sets at once: HIPAA’s Privacy and Security Rules, 42 CFR Part 2 for substance-use-disorder records, New Hampshire’s He-M 400 licensing rules for community mental health centers, and state mental-health confidentiality law. Clinical-adjacent material, HR paperwork, billing exports, and everyday working documents were mixed together, so there was no quick way to tell what needed protecting, what had to be kept for a set number of years, and what could safely go.

The organization was also preparing to move off the on-premise drive. Migrating 24 GB of unexamined files into a new system would have carried every one of those problems forward, with a fresh coat of paint.

Why nothing moved first

The engagement started from a rule: nothing gets created, renamed, moved, or deleted until the full picture is understood and the right people have signed off in writing.

That can look slow from the outside. It isn’t caution for its own sake. Deleting a file that turns out to be under a retention requirement can’t be undone, and moving files around before they’re classified destroys the context — the path, the folder neighbors, the timestamps — that tells you what they are. A read-only first pass protects the evidence, keeps the audit record defensible, and means no regulatory obligation gets triggered by accident.

Anyone who has worked inside a clinical program has seen what happens when a well-meaning cleanup outruns the policy behind it. The audit was built so that couldn’t happen here.

How the inventory was built

A custom Python script walked the entire drive over an authenticated VPN connection in read-only mode. For every file and folder it captured the entry type, full path, name, extension, size, and the created, last-modified, and last-accessed dates, and wrote the results to a structured CSV.

The result was a complete inventory: 23,220 entries — 21,744 files and 1,476 folders — across roughly 24 GB. The deepest folder path ran twelve levels down.

That CSV fed a purpose-built Excel workbook that became the working audit log: one tab for the classification work itself, a summary dashboard that recalculated counts live as entries were classified, a reference library of the regulations and retention periods in play, and space for analysis and visuals. Every entry got a unique ID so any decision could be traced back to a specific file later.

The audit worked from metadata only. No file was opened and no content was read. That was deliberate — it kept the audit itself from becoming a new access to protected information — and it’s also why some items could only be flagged, not decided.

The classification schema

Every entry received one of seven disposition codes: retain as-is, retain and rename, retain and relocate, archive, recommend for deletion, escalate for compliance review, or clarification needed from staff.

On top of the disposition code, entries could carry one or more compliance flags: potential protected health information, potential substance-use-disorder records under Part 2, state licensing or regulatory records, a likely mandatory retention period, and access-control concerns — sensitive material stored somewhere too many people could reach it.

Separating “what should happen to this” from “what rules might apply to it” mattered. A file can be a perfectly good archive candidate and still need a compliance officer to see it first. Keeping those as two fields meant nothing sensitive could slip through on the strength of a routine-looking disposition.

What the audit found

Of the 23,220 entries, roughly 16,150 were marked for archive, about 4,130 were escalated for compliance review before any action, around 1,570 were retained as-is, about 800 were recommended for deletion pending approval, and roughly 540 needed direct input from staff before a call could be made.

The numbers told part of the story. The patterns behind them told more:

No consistent naming standard existed across the top-level folders. Capitalization, acronyms, and delimiters changed from department to department, and working files carried their own workflow cues — “FINAL,” “copy,” dates in several formats — that made it hard to tell the current version from an old one.

Several folders did the same job in parallel. Four different archive-style folders coexisted with no clear difference in retention status, which makes it hard to say whether something was archived or simply abandoned — and hard to defend that decision later. Resources, templates, forms, and agendas folders recurred across departments with no shared rules.

Depth was excessive in places. Past about five levels, folder depth starts to hurt navigation, runs into path-length limits on older systems, and adds cognitive load for staff who aren’t technical. The worst of it was in training and marketing content.

Access control needed attention. HR records that could contain personal or health information were reachable from the general shared drive, and a user-specific software installer sat at the top level of the drive where everyone could see it.

Where compliance risk concentrated

Most of the 4,130 escalations weren’t scattered randomly. They clustered in a handful of recognizable zones: HR forms, electronic billing and claims files, the folder tied to the organization’s EHR vendor, and quality-improvement data that tends to reference clinical populations. File-level signals added more — names suggesting clinical scheduling or demographic surveys, for example.

That clustering is useful in practice. It means compliance review can be organized by zone rather than file by file, and it points directly at where access restrictions should tighten first.

A smaller group of items genuinely couldn’t be decided from metadata: shortcuts, license and certificate files, registry files, media, and ambiguous legacy folders labeled simply “OLD.” Those went to the clarification queue for the people who actually know what they are.

What was delivered

The engagement produced a full Phase 1 audit report covering methodology, inventory results, structural findings, compliance considerations, open decisions, and next steps; the audit log workbook with all 23,220 entries, metadata, and classifications; a preliminary retention schedule mapping HIPAA, Part 2, New Hampshire rules, and grant requirements to record types; an audit framework and methodology document; an intake checklist for stakeholders; and a staff workflow guide covering preparation, read-only assessment, full audit, analysis and approval, implementation, and handoff.

Every disposition in the audit log was left in a pending-approval state on purpose. The retention schedule was marked for legal counsel review, and the report stated plainly that no deletion could be authorized until that review was complete and written sign-off was in hand. The audit’s job was to make those decisions possible and well-informed, not to make them for the organization.

Where it stands

The engagement is ongoing. The organization is now working through implementation alongside a specialized records-migration vendor, using the audit log, classification schema, and retention framework as the foundation for that plan. Because every entry is already classified and traceable, the migration can be planned against a known inventory instead of a guess.

What transfers to other organizations

Most small and mid-sized nonprofits have a drive that looks something like this one. The specific counts will differ, but the lessons hold: take inventory before you touch anything, keep “what happens to this file” separate from “what rules apply to it,” expect sensitive material to cluster, and get the retention schedule reviewed before anyone starts deleting. A cleanup that follows that order holds up after the consultant leaves. One that doesn’t tends to need doing again.