What is Deduplication?
- Deduplication
(occasionally called Single-Instance Storage, Capacity Optimization or Factoring) represents a data diminution technology designed to do away with redundant (duplicate) data on a storage system by saving just one instance of each piece of data for the purpose of reducing disk space and network bandwidth.There are various types of deduplication technologies:
- File deduplication – Only one copy of each identical file is stored. This technology is also called Single File Instance technology.
- Data compression – Some of the times data compression is included as a deduplication technology, yet data deduplication differs from compression in that compression searches only for recurring patterns of information and reduces it. On the other hand, data deduplication results in reducing the unique data no matter its internal format. The normal reduction ratio of compression is x2 and the reduction ratio of byte level deduplication is x50.
- Block level deduplication – Carves up the data in blocks and only one copy of each identical block is stored.
- Byte level deduplication – Examines the content of the data to be deduplicated at byte level and only stores the genuinely unique information. This is the lone technology that warrants full elimination of redundant information.
This implies that different deduplication technologies could also supply different granular control, trimming redundant portions of files, potentially down to the block level or even to the byte level.
When appraising a deduplication product, it is crucial to understand the granularity provided by their platform.
In a future article I will discuss and compare several deduplication products. Please comment on products you use or would like me to include in my analysis.
Data Ladder gives you an excellent Deduplication Software. You can have a look at that.