Classify years of old files as you migrate or archive them

StayCharted sits in the middle of the migration, so the source and destination can be different storage platforms. For documents it needs only a list of names and paths, and the workflow starts with an inventory or catalog export. Your migration tool reads the answers and applies them at the destination.

Document archives can be classified without restoring their contents. Use the names and paths in an available inventory or catalog, including for S3 Glacier Deep Archive, Azure Archive or tape. This avoids retrieving the archived file bodies just to classify their names; inventory generation, listing and classification can still have costs. Restore the categories you need to move, and apply your approved retention policy to the rest.

Convert Parquet inventories to CSV before uploading. Check inventory coverage and freshness: Google Cloud Storage notes that reports can be delayed or omit objects. For Azure, check that your account supports the intended blob index tag and lifecycle filters. An AzCopy command applies the same supplied tags to every blob it copies, so split jobs by category.

Azure allows blob index tags to be read and written in the Archive tier. Preserve existing tags when updating them. The storage actions are configured in your migration tools; they are not direct StayCharted connectors to every platform in the table.

1. List the files

A short script lists every file's folder, name and last-modified date, without opening any file. On a Windows file server:

# List file paths and dates for StayCharted. Reads names only; opens no files.
$Root    = '\\fs01\Archive'              # the share or folder to list
$OutFile = 'C:\Temp\file-list.csv'

Get-ChildItem -LiteralPath $Root -File -Recurse -ErrorAction Stop |
  ForEach-Object {
    [pscustomobject]@{
      Path     = $_.FullName.Substring($Root.Length).TrimStart('\')
      Modified = $_.LastWriteTime.ToString('yyyy')
    }
  } | Export-Csv -LiteralPath $OutFile -NoTypeInformation -Encoding UTF8

On Linux or an NFS mount, with Python 3 installed:

# Python 3 writes valid CSV, including names with commas and quotes.
python3 - <<'PYTHON'
import csv
import os
from datetime import datetime

root = '/mnt/archive'
def stop_on_error(error):
    raise error

with open('/tmp/file-list.csv', 'w', newline='', encoding='utf-8') as output:
    writer = csv.writer(output)
    writer.writerow(['Path', 'Modified'])
    for folder, dirs, files in os.walk(root, followlinks=False, onerror=stop_on_error):
        for name in files:
            path = os.path.join(folder, name)
            if os.path.islink(path):
                continue
            modified = datetime.fromtimestamp(os.stat(path).st_mtime).year
            writer.writerow([os.path.relpath(path, root), modified])
PYTHON

Check the listing finishes successfully before uploading it; fix any access errors rather than treating a partial inventory as complete. These examples list metadata and do not copy or delete the source files.

For data already in Amazon S3 or S3-compatible storage, use the bucket's inventory report or your provider's file listing.

2. Decide the categories and label a sample

Pick the categories the migration needs: the destination folders, the retention classes, or both. Then label a sample of files by adding a Category column to a few hundred rows of the list. Choose rows from across the old structure, not only from the folders you already understand.

If you don't know the categories yet, use Start from unlabelled rows on Essentials or Business. StayCharted groups the files by what their names and paths have in common, suggests a name for each group, and suggests a category for each row. A person accepts or corrects them before anything is trained.

3. Train and check the report

Train on the labelled sample with Path (and Modified, if age matters to your categories) as the input and Category as the answer. The report shows how often the model is right for each category on files it never saw, and which categories it confuses. Add examples where it's weak and train again.

4. Classify the rest

Fill the full list. Every row comes back with its category, a confidence and whether it needs review. Large lists can be split into files of your plan's rows per file and filled in turn. On Essentials and Business, answers below your review cutoff go into the Review Queue for a person.

5. Move each file by its category

Use the filled list with the tool you already migrate with:

• AWS DataSync: use the reviewed list to create a DataSync manifest for each category and configure its destination. Each manifest must follow DataSync’s CSV format, with paths relative to the task’s source location, and be stored in S3.
• Copy tools and scripts: robocopy, rclone or your own script can copy each category's files to their destination.
• Object tags and lifecycle rules: after the copy, apply category tags through your storage workflow. S3 Batch Operations’ Replace all object tags operation replaces the existing tag set; include any tags you need to retain. Configure lifecycle rules from your approved retention policy, and review expiry actions before enabling them.
• Unsure files: move them to a holding location and decide them after review.

Before generating a transfer manifest, inspect one filled file and identify its exact answer, confidence and review columns. Filter out rows that need review, have a missing answer or have an unrecognized review status. Map each remaining category to an approved destination. Export using the format your migration tool requires; DataSync and S3 Batch Operations use different manifest formats.

Test one reviewed batch before running the full migration. This workflow has not been verified end to end against a live DataSync task.

For images, the model reads the picture itself, so photos named IMG_0042.jpg are classified the same as well-named ones. How the pictures reach StayCharted depends on where they are:

• Amazon S3 and S3-compatible storage (Cloudflare R2, Backblaze B2, Wasabi, DigitalOcean Spaces, Google Cloud Storage): connect it directly. Train on a folder someone has sorted and fill the rest. StayCharted writes a copy of each photo into a folder for its category, can optionally tag each copy with its category, confidence and review status, and can write the category into JPEG and PNG keywords. Originals are never changed or deleted.
• Any other storage, such as Azure Blob Storage or a file server: upload ZIPs of photos, or a spreadsheet of links to them. During a migration, a simple way is to classify the photos where they land, if the destination is S3 or S3-compatible, or in a staging bucket on the way.
• Archive tiers (S3 Glacier Flexible Retrieval and Deep Archive, the Azure Archive tier): pictures have to be restored before anything can read them. Restore the pictures you intend to classify, then classify them as they become readable.

A migration is also the moment to fix access. StayCharted's file permissions model trains on your current NTFS, SMB or Linux permissions and suggests who should have access to each folder in its new location, for an admin to approve. See File and SharePoint permissions.

• File names carry the signal. Old archives are full of names like scan0001.pdf. The folder path often tells the model enough; when it doesn't, the model can be wrong, including at high confidence. Use the test report and review a sample of answers before applying them.
• Names leave your network; contents don't. StayCharted is a cloud service, so the file list you upload is processed there. On Business, the model itself can be trained in your own Amazon Bedrock or Microsoft Foundry account, but the file list is still uploaded to StayCharted. If file names can't leave your network at all, discuss self-hosted options before uploading that data.
• Volume. Plans include a number of rows filled a month (see current plan allowances). A full file server is often millions of files, so talk to us about Enterprise limits for the months of the migration.
• StayCharted doesn't move documents. It decides; your migration tool moves.
• Retention and deletion. Categories support your approved policy; a model prediction does not establish how long a record must be kept. Check retention requirements and legal holds before applying deletion rules.