fdupes

fdupes is a program to scan directories for duplicate files with options to list, delete or replace the files with hard links pointing to the duplicate.fdupes is a Linux utility written in C language. The application is able to find duplicate files in a set of directories. It first compares file sizes, partial and full MD5 signatures, and then performs a byte-by-byte comparison for verification. Then it deletes the duplicate files from the directory.