by Sreekanth Vadagiri (@sreeix) on Saturday, 22 March 2014
- Technical level
To learn about this wonderful probablilistic data structure called Bloom filters and things that redis provides to support this really fast and low memory way of finding data.
Bloom filters are probabilistic data structures that use very low memory to allow you to find if data does not existing. It gives false positives, but cannot give false negatives. This means that it can be used to query for existance of some stuff for sure. The structure also allows that you can tradeoff space to probability of false positives.
I also go over how we used it to store infomration about a large amount of video data( about 100 million) in a small 1G redis box.