This is the twenty-sixth article in a series dedicated to the various aspects of machine learning (ML). Today’s article will cover backpropagation algorithms, which is the most important type of learning method employed for neural networks in machine learning agents.
Think back to the last time you went to Taco Bell to get some heavenly tortilla-wrapped treats. Remember how much of a frenzy you were in to get to that drive thru before the 1 AM closing time? In that frenzy, you probably made some poor driving decisions, like popping a wheelie on your Harley Davidson because you thought it would make you go faster, but these decisions that in the moment probably seemed correct turned out to actually set you back and make your trip slower.
Lucky for you, you got there right at 12:59, with 1 second to spare, and had the workers make you an octuple-cheese Beefzilla burrito with refried beans on the side. As you sit on your bike in the parking lot, eating that burrito and washing it down with a Baja freeze, you look back on your trip over. Looking back, you realize that maybe popping that wheelie wasn’t too smart a move after all, and actually cost you precious time. So, from now on, when you burn rubber for a burrito, you’ll place “popping a wheelie” low on your list of things to do when you’re feeling that time crunch.
As it turns out, an AI agent has a similar method to learning which actions deserve what “weight,” or value, after their own (possibly suboptimal) accomplishment of a goal. This method of self-evaluation is called backpropagation.
What is Backpropagation?
Backpropagation is one of the most popular learning algorithms in machine learning, widely used where neural networks are found.
What this algorithm accomplishes is a search through the many possible performance-improving hypotheses available after a completed action. The main work done here is to assign new weights to hypotheses in light of new information. In training, the object of the algorithm is to reduce error in the neural networks, and after training it helps curb overfitting, which is the occurrence of an AI agent becoming just too used to the data it was trained on, which creates an inability to deal with new, unseen data. Additionally, backprogation eliminates biases that could limit performance.
An example of backpropagation in effect would be a window-washing robot, Scrubby, who in an effort to scrub a particularly hard stain ups the force it uses in applying a soaped-up sponge to the skyscraper window it is washing. As a result, it pushes too hard and breaks through the window, interrupting the meeting between cookie company Crispy Crumb and their potential business partner, milk company Cool Calcium. Not only is the window broken now, but Scrubby effectively ruins the meeting and upends any potential partnership between the two.
Now Scrubby is a machine learning agent, so it will use a backpropagation algorithm to assign different weights to the neurons that informed its recent problematic decisions.
For one, the decision to scrub the windows in front of a room full of people may be retooled, so when it comes to deciding which window to scrub at which time, it may choose ones that look into empty or lightly populated rooms so that it is not a potential distraction.
The second thing that it will reconsider is the impulses towards increasing force. It may lend less weight to nodes that advocate for increasing force in such situations, and/or focus on lending more weight to nodes that advocate for increasing force in lighter increments when scrubbing hard-to-wash stains.
This article introduced the most common algorithm in all of machine learning, which is backpropagation. The aim of the backpropagation algorithm is to readjust the weights and biases assigned to certain nodes in a neural network, which helps retool the agent’s assumptions about the environment and preferable decisions when creating a task. It has its human analogue in how we criticize our own reasoning and actions when making a decision or accomplishing a goal, and how we prioritize some reasons and actions and dismiss others after the fact, all for the purpose of improving our own proficiency at whatever it is we aim to do.