Meet git filter-repo

You committed a .env file three months ago. Or hardcoded an API key that made it into a dozen commits before anyone noticed. Or a teammate’s email is wrong across the entire log because their laptop was misconfigured on day one. git rm or a new commit doesn’t fix any of this, the bad content is still sitting in every commit that ever touched it, reachable by anyone with the repo.

git filter-repo rewrites history itself: it walks every commit, applies the transformation you ask for, and produces a new history with the old content gone (or changed) everywhere it appeared. It’s the tool the Git project itself recommends over the old git filter-branch (deprecated, painfully slow) and over BFG Repo-Cleaner (great for a couple of specific jobs, but far less flexible).

It doesn’t ship with Git, install it first:

brew install git-filter-repo
# or: pip3 install git-filter-repo

How it works

  • git filter-repo only runs against a fresh clone, it refuses to touch a repo that looks like your regular working copy, this is a safety rail so you don’t nuke uncommitted work or the only copy you have
  • it rewrites every commit reachable from any ref, not just the current branch, then repacks the repo and expires the reflog, so the old objects actually disappear (once nothing else references them)
  • common flags:
    • --path <p> keeps only the given path; add --invert-paths to remove it instead
    • --replace-text <file> replaces matching strings/regexes in every blob, ever
    • --strip-blobs-bigger-than <size> drops any blob over the given size
    • --mailmap <file> rewrites author/committer identities
    • --message-callback / --blob-callback / --commit-callback run arbitrary Python against commit messages, file contents, or whole commits
    • --replace-refs (default delete-no-add) and the automatic reflog expiry are why the old data is actually gone afterward, not just hidden
  • after rewriting, every commit downstream of a touched one gets a new sha. Anyone with a clone needs to re-clone or hard-reset onto the new history, and you’ll need to force-push. This is the trade-off for actually removing something, not just hiding it going forward.

Setting the stage

One small repo, several problems piled into it on purpose:

mkdir filter-repo-demo && cd filter-repo-demo
git init -b main

echo "app source" > app.py
git add app.py
git commit -m "Initial commit"

echo "DB_PASSWORD=supersecret123" > .env
git add .env
git commit -m "Add local env config"

echo 'API_KEY = "sk_live_NOT_REAL_EXAMPLE_KEY_0000"' > config.py
git add config.py
git commit -m "Add API config"

echo "print('v2')" >> app.py
git add app.py
git commit --author="vlad flore <vlad@wrongdomain.com>" -m "Update app logic"

head -c 15000000 /dev/urandom > data.bin
git add data.bin
git commit -m "Add data file"

mkdir build && echo "compiled output" > build/app.out
git add build
git commit -m "Add build artifact"

git commit --allow-empty -m "wip"
git commit --allow-empty -m "fix typo asdasd"

git log --oneline

2fcc522 fix typo asdasd
345d1c1 wip
f1d0ae4 Add build artifact
76777fd Add data file
88d596a Update app logic
b7fd71f Add API config
9b01e83 Add local env config
33046d9 Initial commit

A committed secret file, a hardcoded key, a wrong author email, a stray 14MB blob, a build artifact, and a couple of junk commit messages. Every example below starts from a fresh clone of this repo, because filter-repo insists on it.

Removing a sensitive file from all of history

.env should never have been committed. Deleting it now only stops it appearing in future commits, it’s still readable in commit 9b01e83 and every commit after it.

cd .. && git clone filter-repo-demo demo-env && cd demo-env
git filter-repo --path .env --invert-paths

Parsed 8 commits
New history written in 0.20 seconds; now repacking/cleaning...
Repacking your repo and cleaning out old unneeded objects
Completely finished after 0.78 seconds.

git log --all --pretty=format: --name-only --diff-filter=A | sort -u | grep -v '^$'

app.py
build/app.out
config.py
data.bin

.env never shows up, in any commit, past or present. Every commit after it got a new sha, that’s expected: the shas encode the tree, and the tree changed.

Removing exposed secrets (keys, passwords, tokens)

Sometimes the whole file has to stay, the secret inside it doesn’t. --replace-text scans every blob in history and swaps matching strings, wherever they occur:

cd .. && git clone filter-repo-demo demo-secret && cd demo-secret
echo 'sk_live_NOT_REAL_EXAMPLE_KEY_0000==>REDACTED' > ../replacements.txt
git filter-repo --replace-text ../replacements.txt

git log --all -p -- config.py | grep -A1 API_KEY

+API_KEY = "REDACTED"

Note the ==> separator, that’s filter-repo’s literal-string syntax. It also accepts regex:<pattern>==><replacement> for anything more dynamic. Either way, the original key is gone from the commit that introduced it, not just from later ones.

Fixing author/committer information

One commit has the wrong email attached, vlad@wrongdomain.com instead of the real one. A .mailmap file maps the wrong identity to the right one, filter-repo applies it and bakes the correction into the commits themselves:

cd .. && git clone filter-repo-demo demo-mailmap && cd demo-mailmap
echo 'Vlad Flore <vlad@example.com> <vlad@wrongdomain.com>' > ../mailmap.txt
git filter-repo --mailmap ../mailmap.txt

git log --format='%an <%ae>' --all

Vlad Flore <vlad@example.com>
Vlad Flore <vlad@example.com>
Vlad Flore <vlad@example.com>
Vlad Flore <vlad@example.com>
Vlad Flore <vlad@example.com>
Vlad Flore <vlad@example.com>
Vlad Flore <vlad@example.com>
Vlad Flore <vlad@example.com>

All eight commits, same address now. (git commit --amend --author only fixes the last commit, this is the version of that for the whole history.)

Removing large files committed by accident

That 15MB data.bin is dead weight in every clone, forever, unless someone rewrites it out. --strip-blobs-bigger-than removes any blob over a size threshold, wherever it sits in history:

du -sh filter-repo-demo/.git

14M	filter-repo-demo/.git

cd .. && git clone filter-repo-demo demo-large && cd demo-large
git filter-repo --strip-blobs-bigger-than 1M

Processed 6 blob sizes
Parsed 8 commits
New history written in 0.08 seconds; now repacking/cleaning...
Completely finished after 0.19 seconds.

du -sh .git
git log --all --oneline -- data.bin

140K	.git

.git drops from 14M to 140K, and data.bin no longer shows up anywhere in the log, filter-repo even leaves a data.bin marker file behind by default noting a blob was stripped there, useful context for anyone wondering why a file disappeared.

Removing whole directories (build artifacts, generated files, dependencies)

Same idea as the .env case, but for a directory. build/ should never have been tracked:

cd .. && git clone filter-repo-demo demo-build && cd demo-build
git filter-repo --path build --invert-paths

git log --all --oneline

76777fd Add data file
88d596a Update app logic
b7fd71f Add API config
9b01e83 Add local env config
33046d9 Initial commit

Worth noticing: Add build artifact, wip, and fix typo asdasd are all gone from the log, not just build/. Removing the directory left those commits with no changes at all, and filter-repo prunes empty commits by default (--prune-empty=auto). Pass --prune-empty=never if you’d rather keep empty commits around.

Rewriting commit messages

A typo in a commit message, or a message that references something it shouldn’t, --message-callback runs arbitrary Python against every commit message in history:

cd .. && git clone filter-repo-demo demo-msg && cd demo-msg
git filter-repo --message-callback '
return re.sub(b"typo asdasd", b"typo in help text", message)
'

git log --all --format='%s'

fix typo in help text
wip
Add build artifact
Add data file
Update app logic
Add API config
Add local env config
Initial commit

message arrives as bytes, re is already imported for you inside the callback. For a single fixed replacement across every message, --replace-message <file> works the same way --replace-text does for file content.

Scripted transformations across all of history

The callbacks aren’t limited to messages. --blob-callback hands you every file’s raw content, --commit-callback hands you the whole commit object (author, message, file changes) for anything the built-in flags don’t cover:

cd .. && git clone filter-repo-demo demo-blob && cd demo-blob
git filter-repo --blob-callback '
blob.data = blob.data.replace(b"print(", b"logging.info(")
'

git log --all -p -- app.py | grep -E '^[+-][^+-]'

+logging.info('v2')
+app source

Every print( call in app.py, across every historical revision of the file, became logging.info(. Same mechanism works for renaming variables project-wide, normalizing line endings, stripping trailing whitespace, whatever a script can express.

After the rewrite

  • Force-push is required, the rewritten history shares no shas with the old one past the first changed commit: git push --force --all and git push --force --tags
  • Every clone is now stale. Collaborators can’t git pull, they need a fresh clone or git fetch + git reset --hard origin/<branch> on top of it
  • The old blobs aren’t gone until GC runs and nothing else references them. If the secret was ever pushed anywhere, treat it as burned regardless, rotate it. filter-repo cleans your repo, it can’t reach into forks, CI caches, or a leaked commit someone already has sitting on their disk
  • filter-repo writes a filter-repo/ directory with logs of everything it did, handy for double-checking a rewrite before you force-push it

Conclusion

git filter-repo is what to reach for when history itself is the problem, a secret, a huge blob, a wrong identity, a directory that never belonged, and a new commit on top won’t fix it. It’s more surgical than filter-branch, more flexible than BFG, and because it insists on a fresh clone and expires the reflog, when it says something is gone from history, it actually is.