I ran into this issue in the past week. My .PACK file was enormous (many many gigabytes) because I committed a lot of files I shouldn't have in the past. Large binary files, backups and other things that don't belong in a Git repo.
I deleted the files and committed those, but Git is really good at making sure you never lose data. So despite having a small repo, I had a .PACK file that was 10 times the size of the repo.
I googled around a lot and the best post I could find on StackOverflow was completely remove files from git repo and github on remote. It didn't quite work for me though.
git filter-branch -f --index-filter 'git rm -r --cached --ignore-unmatch "filename"' --prune-empty HEAD rm -rf .git/refs/original/ && git reflog expire --expire=now --all && git gc --aggressive --prune=now
I just had to add a few flags to get this working.
in the git filter-branch.
in the second line of commands.
Now you can run those two commands (replacing filename with a file name or directory) and it should permanently remove the file from the repo including the PACK file.
Be very careful when you run this command, I am not sure there is a way to undo this change.