When recovering files, or in particular doing a forensic investigation, it is very common for there to be duplicate entries of the same file, or photograph. Typically, there is little point in keeping multiple instances of the same file.

CnW has a very simple feature that will allow duplicates to be removed. The feature is found in the log, and any log showing recovered files can be deduplicated. As always with CnW software, the duplicate files are removed from the recovered drive, and never from the original drive. Once the DeDup function has been processed, the file entry for a file that has been removed will be marked as DeDup in the status column.

Deduplication works by using the MD5 hash value for each file. No two files ever have the same hash value, and so this represents a true digital signature. The name and date of the file are not relevant, so it is just the contents.

A very useful time to use this function is when doing a raw recovery. For instance, a disk may often contain multiple instances of photos, where only one is required, and with this function all instances are then deleted.

Another use of the hash MD5 value is made use of in the file filter by eliminating system files. One can download tables of MD5 values for known operating system files, and general applications. By using this with the file filter it does the equivalent of deduplicating any known good files