Data Privacy Cybersecurity Paralegal

February 12, 2025

By: Myriam Sollberger

The Rise of AI-Generated Social Media Scams: How Scammers Use Artificial Intelligence to Target Individuals with Personalized Messages

Social media platforms have become fertile ground for scammers employing artificial intelligence (AI) to create convincing fake profiles. These AI-generated personas are used to target individuals with personalized messages, leading to various forms of fraud. This article examines the methods scammers use, the implications for victims, and strategies to mitigate these threats.

AI-Generated Profiles and Personalized Scams

Advancements in AI, particularly in generative adversarial networks (GANs), have enabled the creation of highly realistic images and personas. Scammers utilize these technologies to fabricate profiles that appear authentic, often complete with AI-generated photos and backgrounds. These profiles are then used to send personalized messages to potential victims, enhancing the credibility of their fraudulent schemes.

A study analyzing Twitter profiles found that approximately 0.052% of profile pictures were AI-generated, indicating a notable presence of such accounts on the platform (Ricker et al., 2024). These fake profiles are employed in various malicious activities, including spreading scams, disseminating spam, and amplifying coordinated messages.

Case Studies and Real-World Impacts

The use of AI-generated profiles has led to significant financial losses for individuals. For instance, an elderly woman was defrauded of £20,000 by a scammer posing as a U.S. Army colonel. The perpetrator used AI-generated videos and personalized messages to build trust before convincing the victim to transfer funds (The Scottish Sun, 2024).

In another instance, corporate executives have been targeted by sophisticated phishing scams crafted by AI bots. These fraudulent emails are hyper-personalized, exploiting personal details likely sourced through AI, making them more convincing and harder to detect (Financial Times, 2025).

Mitigation Strategies

To combat the rise of AI-generated scams, individuals and organizations should adopt several strategies:

1. Awareness and Education: Staying informed about the latest scamming techniques can help individuals recognize suspicious activities.
2. Verification Processes: Implementing robust verification methods, such as reverse image searches and cross-referencing information, can help identify fake profiles.
3. Advanced Detection Tools: Utilizing AI-powered tools designed to detect synthetic media can aid in identifying and mitigating threats.
4. Reporting Mechanisms: Encouraging users to report suspicious profiles and messages can help platforms take swift action against scammers.

Conclusion

The integration of AI into scamming tactics presents a significant challenge in the digital age. By understanding these methods and implementing proactive measures, individuals and organizations can better protect themselves from falling victim to AI-generated social media scams.

References

Ricker, J., Assenmacher, D., Holz, T., Fischer, A., & Quiring, E. (2024). AI-Generated Faces in the Real World: A Large-Scale Case Study of Twitter Profile Images. arXiv preprint arXiv:2404.14244.

The Scottish Sun. (2024, November 19). I was scammed out of £20,000 by AI-generated US Army colonel who promised me a briefcase full of £607k CASH. Retrieved from https://www.thescottishsun.co.uk/news/13879586/scammed-ai-generated-us-army-colonel-promised-cash/

Financial Times. (2025, January 15). AI-generated phishing scams target corporate executives. Retrieved from https://www.ft.com/content/d60fb4fb-cb85-4df7-b246-ec3d08260e6f