Some time ago, Bing’s search algorithm was based on deep neural network (DNN) architectures that individually analyze each word in a query, leading to rather vague results.
In April 2019, DNN was displaced by BERT (Bidirectional Encoder Representations from Transformers).

Transformers are able to grasp the relationship between each word in a search query and understand their context. BERT is a natural language processing technology that analyzes all words in a query as a single whole, providing more specific search results.

Change

This new innovation has greatly improved the user experience of Bing customers. You can now find information more quickly, bypassing semi-related content that does not accurately answer your questions.

For example, if you entered the query “how to repair a bike” in a search engine using DNN, you would be inundated with information on how to choose, buy, paint and ride a bike, all delivered on a par with repair tips, since the search engine would primarily offer content resonating with the word “bike.” You would still have to sort through a lot of fluff to find the repair guide your were looking for.

BERT saves you time by analyzing all words in your query as a unified concept, giving the same high value to each element. You do not need to dig pages deep in SERPs to find the content that actually addresses your needs.

In essence, BERT has made Bing more human, with a better understanding of a user’s search intent. Bing now provides results that users actually want to see. BERT is currently applied across the globe as an essential component of the Bing algorithm.

Limitations

In a recent interview with Bing administrators, they openly admit that BERT is a costly technology, especially considering the tremendous resources required to apply it worldwide.

BERT requires more time to process queries and find relevant information than DNN, since it analyzes a greater number of parameters and uses parallel computing, while DNN is based on simpler sequential computing.

In today’s dynamic world where people do not like to wait and expect valuable data to appear instantaneously at their fingertips, the slightest delay can alienate users from a search engine and send them looking for faster results elsewhere.

The initial version of BERT served twenty CPU cores at 77 milliseconds per request, which was not enough to satisfy the needs of Bing’s global audience. Tens of thousands of servers were necessary to improve the situation, which was prohibitively expensive.

BERT requires more time to find relevant information

Solutions

To reduce costs and computing time, Bing developers utilized Azure, a cloud computing platform by Microsoft. Azure graphics processing units (GPUs) are designed to serve a broad range of parameters in parallel, which makes them an ideal solution for accelerating BERT without additional expenses.

In the first stage of optimization, developers leveraged NV6 Virtual Machine, since it is inexpensive and available in all regions served by Bing. The first tests with a new GPU showed a 20 milliseconds latency, a great improvement over the initial 70 milliseconds.

The next step was to take advantage of the NVIDIA TensorRT platform and CUDA or CUBLAS libraries, as well as useful CUDA plugins. Developers deeply rebuilt BERT to bring out even more benefits of deep learning technology.

Nine-millisecond latency has now been achieved, eight times faster than the initial result. Also, the system’s throughput multiplied by 43 times.

In the final optimization stage, developers shifted to an NC6s_v3 Virtual Machine and Tensor Cores. BERT gained a latency of 6 milliseconds and the ability to process 64 requests in a single batch. As a result, throughput increased by 800 times. Impressive!

Importance of Azure GPUs to Bing

Without GPU Virtual Machines from Azure, BERT implementation would be improbable. Its original version was too expensive and too slow to serve Bing’s huge global audience.

To enable deep learning and natural language processing, Bing has launched 2000+ GPUs in four locations. These units process 1 million requests per second from different corners of the world.

They are able to cope perfectly with high loads, providing superior user experience thanks to Bing’s fast and responsive upgrade.

Another advantage of Azure that we cannot overlook is its immediate deployment on various GPU types. Without this property, Bing would have to invest tons of time adapting the technology to each GPU.

Natural Language Processing is the Future

The significance of this upgrade cannot be overestimated. The rapid development of artificial intelligence, and natural language processing in particular, puts Bing at the forefront of technological innovation.

Leading search engines like Google and Bing are constantly seeking ways to better understand long-tail keywords and provide more specific search results. Bing’s revelation that it had been using BERT for at least half a year before Google explains how it was able to outpace its competition and provide superior service to users. Other industry giants will no doubt adopt BERT technology in the near future, since the quality of search results and their overall success will depend on it.

For some time now, Intelligent Search has helped Bing remain among the most powerful and tech-savvy search engines in the world. Its Intelligent Answers collect information from multiple sources and tailor questions that accurately reflect search intent. Intelligent Image Search unmistakably recognizes items in photos and helps users to define unfamiliar words.

If Bing maintains its pace moving forward, the search engine should be able to retain its current position and even find itself on par with Google. Bing’s upgrade to BERT signals an important benchmark for search optimization.

Final Thoughts

As search engines become more human and intuitive, commercial websites will increasingly need to target their content to people and not robots. Times when keyword stuffing could help brands rank well in SERPs are long gone. We now live in an era of high-quality and original content tailored to the needs of web users. Keep this in mind when developing your company’s SEO campaign.