To give an idea of the space savings that can be achieved, the table below shows the size in bytes for compressed Bitmaps that have varying amounts of individual bit set to 1 (for comparision uncompressed Bitmaps are 1,000,000 bytes or 0.95MB) # Bits SetĪs you can see it’s not until we get over 64,000 bits (62,016 to be precise) that we match the size of the regular Bitmaps. Then it is just a case of doing the relevant bitwise operations against the bits (a byte at a time), for example if you want to get the questions that have the C# AND Java Tags, you do the following:
To use these you have to pre-calculate an index in which each bit is set to 1 to indicate a match and 0 otherwise. So can we do any better, well yes, there is a fairly established mechanism for doing these types of queries, known as Bitmap indexes.
“ Ignored Tags” see Part 2 for more infomation). for larger value of skip or if there are a large amount of tagsToExclude (i.e. So this method takes longer as we skip more and more questions, i.e. In addition we have to initially load up the HashSet with all the ids for tag2, so that we can check matches. The main problem is that we have to scan through all the ids for tag1 until we have enough matches, i.e.
#EVOPDF OUT OF MEMORY CODE#
One of the most powerful features of the Stack Overflow Tag Engine is that it allows you to do complex boolean queries against multiple Tag, for instance:Ī simple way of implementing this is to write code like below, which makes use of a HashSet to let us efficiently do lookups to see if a particular questions should be included or excluded. This is the part 3 of a mini-series looking at what it might take to build the Stack Overflow Tag Engine, if you haven’t read part 1 or part 2, I recommend reading them first.
0 kommentar(er)
