Why SMS Fraud Detection Still Fails: The RoBERTa Solution

Scam SMS is all around. You’re checking your mobile phone for a delivery message, and then you get an immediate message from your bank, government agency, or even a family member saying that it’s an emergency and asking for urgent assistance. Many people believe that these attacks are being thwarted by spam filters, but as it is, many SMS fraud filters are still on legacy platforms that are easily circumvented by attackers.

This failure most likely impacts low-income communities and older users. Mobile phones play a significant role for these groups in communicating, banking notifications, healthcare reminders and government notifications. If fraud filters don’t work, the financial and emotional loss can be devastating. In 2024 alone, older Americans suffered $4.8 billion in scam losses, with SMS scams accounting for a large share of these losses, according to recent reports from the FBI’s IC3 Elder Fraud data. Americans received 78 billion scam text messages, an 18% increase over the same period in 2022.

The issue isn’t so much that scammers are growing in numbers. This is because traditional fraud detection systems haven’t kept up with the way fraud messages are sent these days. Most filters are still targeting the keywords that are considered suspicious such as “urgent,” ‘winner,” and “click now.” This, which may have worked years ago, is not the case with scammers these days. They are continually changing their wording, impersonating legitimate institutions and speaking a conversational style that sounds genuine.

The performance gap is significant, particularly when it comes to new scams that have yet to be discovered by legacy systems.

The Real Problem with Traditional SMS Fraud Filters

The majority of SMS fraud filters are similar to security guards looking for certain terms that are prohibited. If any of the terms in the message are already flagged as suspicious, then the message is flagged. Otherwise, it goes around. Sounds easy and effective, right?…until scammers adapt themselves. The FTC estimates the actual cost of fraud to older adults in 2024 at between $10.1 billion and $81.5 billion, depending on methodology, once underreporting is accounted for. 

Now, suppose that this message comes from a scammer:

“You’re receiving an email alert from your Social Security account that it has detected unusual activity, and that you should reach out with the SS team today to prevent temporary limitations.”

No recognizable spam words are found on this page. Avoid the “free prize”, “limited offer” and strong “clickbait” wording. This message may appear innocuous to a standard filter built on a keyword basis. But to an older person who is vulnerable, it may make them feel like they are in danger and need to do something quickly.

That’s why keyword detection is faltering. Fraudsters constantly change the content of the messages to evade detection. They use alternative language to replace blocked phrases, or they alter the domain, shorten the URL or act as a fake institution. The scam is evolving faster than the SMS fraud filters are able to catch it.

This problem is referred to as concept drift in machine learning. Simply put, the patterns are constantly in flux over the years. One fraud system that has been trained on last year’s scams could well be totally ineffective against today’s scams due to the different terminology used.

This problem has grown to be very serious, as evidenced by research comparing the detection systems. This meant that keyword-based systems only performed at a rate of 38.6% accuracy when facing new and evolving fraud messages. This translates to over half of the scam texts going undetected. Elderly Americans are disproportionately targeted because they are perceived as more financially stable, more trusting, and less digitally literate. Black and Hispanic consumers face higher victimization rates within this group.

No one would trust a smoke detector that only worked one-third of the time. Yet millions of users rely on SMS fraud filters operating at roughly that detection rate against modern scams.

Why Vulnerable Users Are Hit the Hardest

Scammers do not target randomly. They deliberately focus on people who are more likely to act on urgent messages or who have less digital literacy.

When a keyword gets flagged, scammers rephrase. When a domain gets blocked, they switch. The evaluation confirmed what prior research has shown: keyword filters drop below 40% accuracy when applied to novel fraud patterns not present in their training data. Critically, the categories with the worst keyword-filter performance are those causing the greatest financial harm. Government imposter fraud caused $789 million in losses in 2024. Crypto investment fraud caused $2 billion in losses to elderly victims alone. Neither category triggers keyword rules reliably.

If someone isn’t aware of the latest phishing methods, a scam text message claiming to be from Medicare, Social Security or a bank seems like a real possibility. The fear and urgency created by fraudsters makes them less critical thinkers. It’s a psychological trickery in disguise of customer support.

There are also more risks for low-income users. Many people will be unable to afford to have a high-cost protection service in the event of fraud or even a smartphone with the latest and greatest security built-in. They use their phones as the main channel for their banking, benefits and communication needs. If the scams are not detected, the consequences are immediate and severe.

There is another concealed issue which is underreporting. Numerous scam victims do not report the crime due to embarrassment or lack of confidence that it can be resolved. The actual monetary loss might be far greater than the reported losses, according to estimates from the FTC. When factoring in the unreported losses, the actual losses may be between $10 billion and more than $80 billion dollars each year.

This makes SMS fraud more than just a tech problem. It turns into a societal issue of trust, access and digital inequity.

How RoBERTa Understands Fraud Differently

The RoBERTa-based model is searching for meaning.

The distinction is all-important.

The Robustly Optimized BERT Approach (RoBERTa) is one of the transformer-based NLP models designed to better understand the context in language. It is able to evaluate the relation of words in a sentence rather than look for a particular word.

For example, the sentence:

You can’t get your account status verified, or else it will be suspended.

At first glance, it may seem like a normal sound. However, RoBERTa is aware of the fraud scheme: impersonation of an authority figure, along with urgency and an account pressure. It’s a brain that reads meaning behind the words, not word for word.

This contextual knowledge enables the model to detect scams even if the scammers completely change the message.

Using 12,400 labeled SMS messages across a number of fraud categories, the researchers were able to train a fine tuned version of RoBERTa:

• Government impersonation scams
• Crypto investment fraud
• Tech support scams
• Lotteries and prizes scams
• Emerging and novel scam formats

The model was then evaluated with an independent set of third-party messages which consisted of 3,200 unseen messages. The outcome was spectacular.

Fraud Type	Keyword Filter Accuracy	TF-IDF + SVM Accuracy	RoBERTa Accuracy
Government Imposter	71.2%	83.5%	94.8%
Tech Support Scam	68.4%	80.2%	93.1%
Crypto Investment Fraud	52.3%	71.4%	91.7%
Novel Scam Patterns	38.6%	62.1%	88.3%

The accuracy comparison across all fraud categories is shown below.

The most critical category is changing patterns of scams. It is here that these frauds are continuously changing in the real world. As seen with RoBERTa, contextual AI is vastly superior to static keyword systems because it can achieve high detection rates even when it encounters new types of scams.

False Positives in SMS Fraud Filters Are a Bigger Problem Than Most People Realize

A lot of people think that it won’t do them any harm to block additional messages. It is not.

If legitimate text messages get wrongly blocked by fraud systems, it can erode user trust at the moment. Now think about a senior who also forgets to take medication, an appointment with a doctor or an alert from the bank due to misidentification as spam.

In testing, traditional keyword filters resulted in false positives ranging from 12% to 18%. This results in potential incorrect rejection of up to 1 in 7 legit messages.

By using RoBERTa, that false-positive rate was dropped to just 1.8%.

In the real world that difference is quite significant. Users should be protected by a fraud filter without interfering with the flow of necessary communication. Users will discontinue monitoring alerts when they lose their confidence in the system.

This combination of protection and usability is where modern AI systems excel most:

Why Context Matters More Than Keywords

Language is complicated. The human understanding of meaning through tone, relationships and sentence structure is natural. Traditional spam systems don’t.

Let’s take these two as examples:

• Get your complimentary prize today!
• Some strange activity has been noticed on your account and it requires some urgent confirmation.

The first one is clearly phishing since it has a typical spam language. The second one is professional and serious sounding. But the second message could be even more harmful since it masquerades as a trusted institution.

RoBERTa has a strength for comprehending emotional and contextual cues within language. Identifies urgency, impersonation, persuasion techniques, and manipulation strategies.

This is particularly true of scams involving:

• Government agencies
• Banking notifications
• Crypto investment opportunities
• Healthcare impersonation
• Technical support requests

Scams today are written to sound calm, believable and human. AI detection systems need to be able to comprehend language just like humans.

The Importance of Continuous Learning

One of the biggest concerns with older fraud systems is they don’t change very often. Scammers evolve daily. Fraud detection needs to change as well.

They found that a static RoBERTa model tended to become less effective over time as the scam patterns evolve. If the model was not retrained, the accuracy on evolving scams decreased from 88.3% to 55.1% in 6 months.

That appears to be a scary thing until you watch this in action with continuous updates.

When monthly retraining was done with newly flagged scam messages the model performed well and remained over 85% accurate all the time.

This is a telling sign; fraud protection is no longer a “set it and forget it” process. It must continually be adapted.

Just like antivirus software. No one ever thinks that a five-year-old security program will protect against the latest malware. It’s time for SMS anti-fraud to grow like a tree.

Why Explainability Is Critical for User Trust

AI systems can sometimes be enigmatic as they are a black box. That does establish a trust issue in regard to fraud protection.

If there is just one statement, then it is:

      “Message blocked.”

It is frustrating or confusing for many users.

However, if the system says:

“This message appears to impersonate a government agency and requests personal information urgently.”

The Challenges of Deploying RoBERTa in the Real World

Explainability is not just a nice-to-have feature. For older users who may already be uneasy about digital safety, clear explanations build confidence and awareness over time. Proper use of AI makes this a must-have requirement.

Explainability is not just nice to have. Proper use of AI is a must-have requirement.

Despite its impressive performance, deploying transformer-based AI models is not simple.

Cost and lightweight SMS fraud filters are traditional keyword filters. They don’t need much computing power and are very easy to maintain. However, RoBERTa models need:

• GPU infrastructure
• Regular retraining
• Larger datasets
• Monitoring systems
• Ongoing engineering support

There is a higher level of inference latency too. The average latency for keyword systems is almost instantaneous, but RoBERTa pipelines are on average around 18 milliseconds per message. It’s still swift enough for SMS delivery, but still adds a layer of complexity to the operations.

Some smaller telecom and community service providers may not have the resources to implement such infrastructure. There are also regulatory issues to consider. The telecoms and financial industry sectors frequently demand systems of decision making that can be audited and are transparent.

This implies the increasing significance of explainable AI frameworks and simplified model variants for real-world use.

What the Future of SMS Fraud Detection Looks Like

But the future of detecting fraud isn’t just about blocking random keywords. It’s more about the human behavior, intent and pattern of deception.

The use of technologies such as AI models like RoBERTa is a departure from a rule-based filtering approach, and instead this allows for more intelligent contextual protection. Features the ability to adapt quickly, fewer erroneous alarms, and the ability to detect scams that previously were undetected.

What is the most interesting part is that this technology is already a reality today. The difference in performance between traditional SMS fraud filters and new NLP models is measurable, significant, and ready for deployment.

However, the technology is not a sufficient solution. The organizations that put these systems into place should consider:

• Vulnerable populations

• Continuous retraining
• Transparency
• Ethical AI practices
• Real-world accessibility

Fraudsters are constantly evolving but defensive AI is as well. The issue now is whether these cutting-edge safeguards will come to those who need them by the time that financial loss keeps mounting.

Explainability and regulatory acceptance are also open questions. Black-box transformer models may face resistance in regulated telecom or banking environments where model decisions must be auditable. Practitioners operating in those contexts should evaluate SHAP-based explanation layers or distilled interpretable variants before full deployment.

Conclusion

SMS spam has now become much more than spam. Modern scams are complex, situational, manipulative and carefully targeted to vulnerable users. Sadly, a lot of the traditional detection logic being used is hugely inefficient and is so easily evaded by scammers.

The technology is not experimental any longer. It is practical, effective and can provide protection to users at a higher level than legacy systems. The hard part now is to make sure they’re deployed responsibly, they’re constantly adapting, and that they’ll reach the communities that are most vulnerable to digital fraud.

FAQs

1. What is SMS fraud detection and why does it matter?

SMS fraud detection refers to the systems and technologies designed to identify and block scam or phishing text messages before they reach the user. It matters because SMS remains one of the most widely used communication channels, especially among elderly and low-income populations who are disproportionately targeted.

2. Why are SMS fraud filters that use keywords not effective?

They fail due to the fact that scammers always change the words around and don’t use any keywords that are well known, so one-time keyword rules won’t work.

3. What does RoBERTa mean in AI?

RoBERTa is a transformer-based natural language processing (NLP) model that has been developed to better capture the context and meaning of the language, as compared to traditional machine learning techniques.

4. What makes it particularly dangerous for them to become victims of SMS scams?

The elderly are frequently a specific target, since scammers believe they can be more trusting and less aware of scams than younger people are.

5. Is there a way to totally block SMS scams?

While no system is entirely flawless, sophisticated AI models can play a significant role in enhancing the accuracy of fraud detection and mitigate the negative impact of fraudulent activity.