How to Find and Remove Duplicate Files in Linux

The home folder (and sometimes the entire server) quickly becomes cluttered if you often download files and scatter them across different directories. After some time, it turns out that the same documents are in several places — and it is no longer possible to find all this by hand. 

Below is a clear overview of the tools that will help you get organized.

1. Rdfind — search for duplicates by content

rdfind (redundant data find) is a free console utility that recursively scans directories and finds files with identical content. It can identify the “original” and mark other copies as duplicates. The ranking logic is simple and predictable: the higher file that was encountered earlier when scanning input paths is located at a shallower depth and/or was found earlier among files in the same directory.

Installation

sudo apt install rdfind          # Debian/Ubuntu/Mint

sudo yum install rdfind          # RHEL/CentOS/Fedora/Rocky/AlmaLinux

sudo emerge -a sys-apps/rdfind   # Gentoo

sudo apk add rdfind              # Alpine

sudo pacman -S rdfind            # Arch

sudo zypper install rdfind       # openSUSE

Quick start

rdfind /home/user

The results are saved in results.txt (in the current directory). 

Remove duplicates:

rdfind -deleteduplicates true /home/user

2. Fdupes — classic duplicate scanner

fdupes also works from the terminal and compares files recursively, checking size and content. 

Installation

sudo apt install fdupes          # Debian/Ubuntu/Mint

sudo yum install fdupes          # RHEL/CentOS/Fedora/Rocky/AlmaLinux

sudo emerge -a sys-apps/fdupes   # Gentoo

sudo apk add fdupes              # Alpine

sudo pacman -S fdupes            # Arch

sudo zypper install fdupes       # openSUSE

Command examples

fdupes           # directory scan

fdupes -r         # recursively

fdupes -r # several paths, the second one has recursion

fdupes -S         # show total size of duplicates

fdupes -m         # summary

Interactive removal:

fdupes -d <dir>

The utility will show groups of identical files and ask which ones to delete — you need to enter the numbers. The -dN mode deletes everything, leaving the first file it comes across, but it is safe to do this only if you are completely sure:

fdupes -dN <dir>

3. Jdupes — fast fork of fdupes

jdupes is a modern and noticeably faster fork of fdupes, actively maintained and better suited for large directories. It compares the contents of files, but uses optimized algorithms and gives more control.

Installation

sudo apt install jdupes   # Debian/Ubuntu/Mint

sudo yum install jdupes   # RHEL/CentOS/Fedora/Rocky/AlmaLinux

sudo pacman -S jdupes     # Arch

sudo zypper install jdupes# openSUSE

Example commands:

jdupes     # scan

jdupes -r # recursion

jdupes -d # interactive deletion

jdupes -s # replace with symlinks

Total

All of the listed tools do a good job of finding duplicates, but the approach to deleting must be balanced. If you are not sure whether the file is needed, make a backup copy and remember the path — this makes it easier to roll back changes.

CONTENT:

Similar

All news