The 3 variables that need to be considered when effectively scaling a search engine.

I am sure you have heard of the old adage where there are 3 concentric circles indicating, price, speed, and quality. You can have any one of these or possibly 2 out of 3 of them but never all 3.

The same is similar with text search engines at scale. If your infrastructure is responding to millions of searches each day you run into a similar problem. The variables are similar but different.

Speed: In the internet world, even a 500ms response from the time a request hits your VPC to the time it leaves is a bit on the slow side. Your SEO plummets if it gets much higher than that, not to mention the user’s experience.

Cost: A common work around for poorly designed systems is to just throw more hardware at its expensive hardware. You can just keep booting up more bigger DB instances to throw at the problem but eventually you are going to destroy your margins if you keep at it.

Quality: With search, the quality is complex and can be broken down into sub-categories.

Part 1 is the “coverage” which is the amount of fields and records indexed. Is every letter for every product description searchable? If I type in the word “the” into the search box, do we need to return every product that has that word in its description?

Part 2 of this equation is how complex is the sorting. How do you rank various search results against each other? Do you just return the results alphabetically? In the e-commerce world you would want to return the products in the order the user is most likely.

If you only sell a handful of widgets you can just show the top selling products but if you have a diverse collection of products ranging from apples to 19th century antiques the algorithms to sort those products next to each other can be computationally expensive.

Oftentimes these algorithms are derivative of other tables and fields beyond just the base product record to which, if joined at the time the actual request for the search happens can cause catastrophic latency.

Part 3 of the equation is how up to date the results need to be. Think of it like Google searches vs Google News. Google News has results that are updated every hour if not more frequently.

If you are in e-commerce, what good is showing a product that is out of stock to the user? Let's say you sell 100 blue widgets every hour on average but today you sold out of them for some reason. Would you want a user on mobile to have to scroll down past your sold out products to get to the products you have in stock? Likely no, so you need to update your search results virtually after every purchase and every time a new product arrives at your warehouse.

Bringing it all together. You can have any 2 but you can’t have all three…. Unless you are willing to get creative.

In my upcoming posts I will dig into how to have fast cost effective searches that make you money.