Google released a major algorithmic update on October 25, this year: BERT. It’s not the Muppet with unibrows of our collective youth. BERT stands for Bidirectional Encoder Representatives from Transformers. It’s a neural network-based open-source technique that focuses on natural language processing.
Neural Network Now What?
Google has just revolutionized how its internal algorithm interprets search queries and language through automated machine learning.
Come back… again, Okay. Imagine a simple scenario: How many times have you searched Google for “food near me” when you were hungry? Google must identify, for example, that you meant ‘food’ when you typed ‘near me.’ You misspelled it. That’s not it.
Google must also NOT misidentify your question. You do not mean what food items are near the word “me,” like’ li at’ or ‘li m. ‘ You ALSO do not refer to food in the vicinity of Maine (state code ME).
It is important to understand language and search queries. BERT has a massive impact on SEO. Google claims that the shift will affect 10% of all searches.
Decoding Human Language
Text messages are often the cause of miscommunication between people. Without exceptional writing skills, it’s hard to match the intention of a text with its vocabulary. We can lose ourselves in words, and the tone of others’ readings of our message can change the meaning. It’s debatable whether the writer or reader is to blame.
It’s a fact that Google’s algorithm for searching is very good at understanding what we are looking for.
- Google returns WebMD articles and home remedies when we search for “I have a cold.”
- Google returns articles about how to survive on an unbreathable planet.
Google receives billions of searches every day. About 15 percent of these queries were written and searched the first time.
It begs the question: How does Google produce quality results when it is presented with words it has never seen before?
How machines read and understand search queries
It is important to understand how humans read and comprehend sentences and then create a machine-learning algorithm that mimics these processes. Most people will be able to read and understand the following correction that has been misspelled.
- This is a sentence you can read even though the words are misspelled.
Machine algorithms can easily imitate what our brains do. It corrects the words in the sentence one by one and then runs it through the search query. This is just a spelling mistake. What happens if you use a sentence which is not grammatically correct?
- The sentence is difficult to read, but I can still understand it perfectly.
What is your brain doing as you read this sentence? It connects words to create a sense of plausibility. The transitional word “but” connects “Sentence difficult to read” with “still understand perfectly fine.” This tells the brain that two distinct ideas are related. “Hard to Read” refers to the sentence. “Perfectly Fine” refers to your ability to understand it.
How would an algorithmic machine know this? Why couldn’t the algorithm just as easily swap the subject and descriptor? There was no guarantee that the current Google algorithm would correctly address the sentence before BERT.
The BERT Difference
Now that the basics have been laid out let’s discuss what BERT is doing and what it’s fixing. Let’s talk about what BERT does and what it sets. As per Google
BERT models can, therefore, consider the full context of a word by looking at the words that come before and after it–particularly useful for understanding the intent behind search queries. BERT models are able to consider the context of a given word by looking at words before and after the talk. This is particularly useful when trying to understand the intention behind search queries.
Instead of running the search query backward, the mapping element compares and groups words with others in the sentence. The relationship between words and phrases in the sentence can now be used to deduce the intention.
What does BERT look like in practice?
Google gave examples of the positive impact BERT could have on search queries. These searches are a good example.
“Brazilian travelers to the USA in 2019 need a Visa.”
Prior to BERT, Google would struggle to understand the significance of the travel direction. In other words, “traveler,” “USA,” and “Brazil” all refer to travel plans between the two countries. It’s easy to overlook the importance we (humans) would place on the word “to” when describing going from Brazil to the USA. The articles that were created to satisfy the SEO intention had articles about travel to Brazil and the USA.
After the BERT, each word has been grouped and decoded. The directional word “to” is contextualized. This phrase will now bring up visa plans for travelers who are going from Brazil to the US. A tiny but important thing.
“Parking in a hill without curb.”
Before BETER, what does this leave you with in terms of your search query? So, What about the other “with no” filler words? Before BERT, they would have been ignored, thus giving the wrong information.
After the BERT, “parking on the hill” and “no curb” can be combined to give a better representation of the search and its intention.
Google’s public launch of BERT is part of its mission to acknowledge the shortcomings in its search engine algorithm and work on addressing them. The fact that it is open-source means there are more resources available to improve its functionality.
Open Source Algorithms: Benefits
Sorry to bore you with unnecessary information. We’re at a crossroads, and we need a sign. Open Source means anyone can access the source code for BERT, whether it’s your grandmother, your cat, or you. Anyone can then develop, upgrade, or reimagine code to create better functioning algorithms.
One team cannot handle the complexity, nuance, and layers of language. It is too complex to incorporate all it has to offer beyond “proper” language’s structural integrity. BERT was created to be open-source to allow users around the world to develop their systems. Google’s first development of BERT was:
“We have open-sourced a new method for NLP pretraining, called Bidirectional Representations From Transformers (BERT). This release allows anyone to train a state-of-the-art question-answering system (or other models) on a Cloud TPU in 30 minutes or a GPU in a few short hours. The release includes TensorFlow source code and pre-trained models for language representation.
The hundreds of thousands of individual search examples BERT ran helped it become the efficient program that it is today.
