Machine learning is becoming more critical to cybersecurity every day. As I’ve written before, it’s a powerful weapon against the large-scale automation favored by today’s threat actors, but the dynamic between attackers and defenders is evolving.
Nowadays, machine learning is mostly used by security software to ingest massive quantities of data and identify threats, but that will all soon change as increasingly sophisticated cybercriminals tap into their own machine learning tools to counter this. The early stages of this malicious machine learning will likely take the form of bad guys directly targeting the good guys’ algorithms directly to sabotage, mislead, and reverse-engineer them.
We’re on the precipice of the age of adversarial machine learning, where dueling algorithms will determine an organization’s security, as well as the safety of its employees and customers. Here’s what that time will look like.
Machine learning models will be in the crosshairs
An organization’s threat detection ability is often only as good as its machine learning models, which would make these models a logical target for attackers. In terms of adversarial machine learning, this could mean cybersecurity vendors get hacked themselves by threat actors looking to gain access to the algorithms and data that trains their models. With this information, the bad guys can build their campaigns to evade detection, or they can build identical models they can test their attacks against.
Threat actors can also target these algorithms externally. Based on a model’s outputs, or how often they detect specific threats, threat actors can extrapolate a model’s signatures via trial and error and learn how to game them to avoid detection. Public models are the most susceptible to this type of manipulation because threat actors would have access to the same models that defenders use. This knowledge of how their campaigns and attacks are detected would let them completely disguise themselves, causing a host of problems for cybersecurity teams.
Because machine learning models don’t process and understand things on a human level, this could mean blurring a logo on a phishing page just enough so that it’s still recognizable to humans but utterly confusing to signature-based security models. It may also mean threat obfuscating the part of their code they know cybersecurity models are looking for to detect their threats.
It’s time to look ahead
The security community largely has ignored adversarial machine learning, but it’s almost certainly something we’ll be contending with sooner than later. I’ve covered just a couple examples here—there are many ways that machine learning models will be maliciously targeted.
The adversarial machine learning war isn’t raging yet, but my team is already taking measures to counter it. The two most effective ways to combat adversarial machine learning:
Blending: It’s far easier to game one model over several. Every data scientist has a “go-to” algorithm for training their models, but it’s essential to not only use other algorithms but try other algorithms together. We use blended models (also known as stacked models) to detect threats where the base models marry two or more different perspectives.
Co-training: Co-training is a semi-supervised machine learning method where two or more supervised models work together to classify unlabeled examples — examples that haven’t been classified by humans. By learning from several real examples, different Magecart scripts, for instance, they can learn to find threats in the wild, even as they evolve. If these models disagree on how to classify these examples, the disagreements escalate to our active learning system, which includes a review by human analysts. This way, the analyst can recognize when the model is no longer detecting threats at an acceptable rate, which may indicate that a threat actor is privy to its algorithms.
Placing an expert in the loop also helps when the model is unsure how to categorize a particular instance. Having this analyst or data scientist available when a model “asks for help,” is crucial as threats change, especially when they change with the intent to fool the model. Left alone, the model may make an incorrect assumption about if it’s a threat or if it’s benign. Developing a feedback mechanism that provides your model with the ability to identify and surface questionable items is critical to the success of your model.
It goes both ways
As data scientists in the age of adversarial machine learning, it’s our duty to make sure the bad guys won’t be the only ones with these kinds of tricks up their sleeve. Threats change all the time, so detection models must change accordingly. To keep up with adversaries, especially as they employ machine learning, it’s critical to use models that can learn incrementally.
When adversaries change their threats to beat your models, your models must transform to counter the new threats. It sounds like a chess match, but the stakes are much higher — it’s going to determine the future of security.
The post The Future of Machine Learning is Adversarial appeared first on RiskIQ.
Article Link: https://www.riskiq.com/blog/external-threat-management/adversarial-machine-learning/