Artificial Intelligence (AI) is quickly becoming a game-changer not just in cybersecurity, but across all industries. As businesses leverage AI to improve operations, boost productivity, and enhance decision-making, cybercriminals are adapting these powerful tools to increase the sophistication of their attacks, creating an ongoing ‘arms race’ between defenders and attackers.
A recent phishing attack targeting YouTube CEO Neal Mohan exemplifies the growing threat AI poses in cybersecurity. Cybercriminals used AI-generated deepfake videos of Mohan, which were sent to YouTube content creators as private videos. These videos falsely claimed that the CEO was announcing changes in monetization policies, tricking users into clicking links that led to phishing websites. Once victims entered their credentials, they unknowingly gave attackers access to their accounts. This attack highlights how convincingly AI can now simulate voices, faces, and video footage, allowing cybercriminals to bypass traditional detection methods.
While AI is not necessarily creating entirely new forms of attack, it is significantly enhancing the speed, volume, and sophistication of traditional attack methods.
This article explores how cybercriminals are using AI today, the risks it poses, and how organizations can respond proactively to mitigate these new threats.
How Cybercriminals Are Using AI
AI is amplifying the threat landscape, making known attack techniques faster, more scalable, and harder to detect. Threat actors are applying AI to existing cybercriminal tactics — from malware development to phishing and scams. The difference lies in the scale, efficiency, and stealth with which these techniques can now be carried out.
1. Generating and Modifying Malware
While AI is not yet capable of generating entirely new malware from scratch, it has shown significant potential in modifying, rephrasing, and obfuscating existing malware to evade detection by traditional security tools.
Traditional malware obfuscation methods—such as code encryption, packing, and polymorphism—have long been used to hide malicious payloads. However, these techniques typically follow predictable patterns, making them increasingly detectable by advanced malware analysis tools.
AI takes these techniques to the next level by enabling malware to dynamically adapt its behavior and structure in real time, responding to the environment it operates within. Instead of simply altering code patterns in a fixed way, AI-driven malware can modify its behavior, execution strategy, and payload in response to the specific security measures it encounters. The ability of AI to not only modify existing malware but also create entirely new, unpredictable variants is forcing cybersecurity professionals to rethink traditional defense mechanisms. This makes AI-assisted malware far more difficult for traditional detection systems, which often rely on signature-based or heuristic analysis, to identify.
A notable example of AI’s impact on malware evasion comes from a study conducted by Unit 42 (Palo Alto Networks), where researchers demonstrated how LLMs could be used to rewrite 10,000 unique samples based on real-world, phishing-related JavaScript malware from 2021 and earlier. When these samples were analyzed by a deep learning-based detection model, the verdict flipped from malicious to benign in 88% of cases. Furthermore, when these samples were uploaded to platforms like VirusTotal, they were not detected as malicious, further demonstrating the effectiveness of AI in bypassing conventional security tools.
In addition to altering existing malware, AI is optimizing the speed and efficiency with which malware can be generated. For example, the Gym-Malware Generator, which leverages reinforcement learning, is able to generate new malware samples in just 5.73 seconds with an evasion rate of 58.35%. This demonstrates how AI is not only enhancing traditional obfuscation methods but also accelerating their deployment at scale. Additionally, there is increasing evidence that AI is being used to develop entirely new malware, such as BlackMamba (a proof-of-concept malware) and custom encryptors by hacker groups like FunkSec ransomware group, which showcase AI’s role in enabling more advanced and evasive cyberattacks.
The integration of AI also facilitates more sophisticated techniques, such as polymorphism, real-time adaptation, and impersonation, while preserving the malicious intent of the malware. As a result, AI-driven malware can continually alter its structure and behavior, making it even harder for security systems to detect and analyze.
This evolution in malware tactics underscores the need for organizations to adopt AI-driven defense mechanisms. These systems are essential to identifying the subtle, abnormal patterns in network traffic and system behaviors that traditional tools may miss, enabling organizations to stay ahead in the face of increasingly sophisticated threats.
2. Phishing and Social Engineering
AI can generate high-quality, well-written phishing emails that are grammatically correct, culturally localized, and emotionally persuasive, significantly increasing their success rate.
Phishing remains one of the most common and effective entry points for attackers and AI is making it more scalable and convincing.
According to the 2024 Phishing Threats Trends report by Egress, 74.8% of phishing kits their researchers examined referenced AI-based components and 82% mentioned deepfakes.
Malicious large language models (LLMs) like GhostGPT have emerged, capable of generating automated phishing templates, effectively lowering the technical barrier to entry for threat actors without technical expertise.
Despite the surge of AI in phishing attacks, several studies suggest that the combination of AI with human expertise has really amplified the effectiveness of these campaigns.
This hybrid approach has proven to dramatically increase engagement and success rates, making these campaigns much harder to detect for both users and for automated security systems.
3. Finding and Exploiting Vulnerabilities
AI is not only transforming the discovery of vulnerabilities, but is also enabling attackers to exploit them more efficiently. By automating traditionally labor-intensive tasks, AI is accelerating the process of identifying both known and previously undiscovered vulnerabilities, and in some cases, even autonomously exploiting them.
This growing capability presents significant challenges for defenders, as AI can now be leveraged to target both zero-day and one-day vulnerabilities at an unprecedented speed.
- Threat groups such as Emerald Sleet have been observed using large language models (LLMs) to analyze publicly reported vulnerabilities (CVEs) and understand how to exploit them more effectively.
- New tools, such as Vulnhunter, are demonstrating the power of AI in identifying both known and zero-day vulnerabilities automatically. Vulnhunter is a Python-based static code analyzer that uses LLMs to find and explain complex, multistep vulnerabilities in open-source projects. Recently, it uncovered over a dozen remotely exploitable zero-day vulnerabilities in AI-related open-source projects with more than 10,000 GitHub stars in just a few hours. These discoveries included full Remote Code Execution vulnerabilities, underscoring AI’s role in rapidly identifying critical security issues.
- Additionally, Big Sleep, a collaboration between Google Project Zero and Google DeepMind, represents a major leap in AI-assisted vulnerability research. Big Sleep’s first real-world success was the discovery of an exploitable stack buffer underflow in SQLite, a widely used open-source database engine. This vulnerability, which had the potential for serious exploitation, was identified and reported to the developers in October, who fixed it on the same day. This is believed to be the first public example of an AI agent discovering a previously unknown, exploitable memory-safety issue in widely used software.
Although threat actors may often focus on zero-day vulnerabilities due to the potential benefits of exploiting issues without patches, a 2014 study reveals that LLMs can also be used to exploit one-day vulnerabilities. This makes it possible for attackers to exploit vulnerabilities in environments that have not yet been patched or have limited available information. The study found that LLM agents, particularly GPT-4, could autonomously exploit 87% of one-day vulnerabilities using only public CVE descriptions. This highlights how AI can play a role not only in discovering new vulnerabilities but also in exploiting known issues, speeding up the attack process in environments where patches have not yet been applied.
These advancements in AI-assisted vulnerability discovery are not only speeding up the research process but also lowering the barriers for attackers to exploit previously unknown vulnerabilities.
4. Automated Reconnaissance
Reconnaissance is critical in the early stages of any cyberattack, and AI is enabling highly efficient, automated data collection and analysis.
AI tools can extract large volumes of public and breached data to build detailed victim profiles in seconds.
It can simulate legitimate user behavior, craft pretexts for social engineering, and even identify potential points of entry based on digital footprints.
Additionally, attackers are developing AI-assisted reconnaissance tools tailored for stealthy, scalable network and infrastructure mapping.
5. Scaling Operations Faster
AI allows threat actors to operate at a scale that was previously unfeasible.
It improves the quality and reliability of scripts, reduces errors in multilingual campaigns, and enhances the speed of content generation.
AI assists in pattern recognition, enabling the creation of more effective social engineering lures, tailored messaging, and advanced command generation. This scalability enables smaller threat actors to operate with capabilities similar to large, organized groups.
6. Scams and Impersonation
AI is fueling a new generation of scams by making impersonation more realistic and harder to detect.
- Voice cloning and deepfake video calls are being used to impersonate executives or trusted individuals, often to authorize fraudulent financial transactions or request sensitive information.
- AI-generated fake job offers, investment opportunities, and romance scams are increasingly convincing, leading to identity theft or financial loss.
- Generative AI tools are increasingly being used to forge documents, create fake testimonials, and craft the illusion of legitimacy around fraudulent products, services, or applications. These tools can also spoof individuals by generating false personal information, such as fake identification documents, social media profiles, or fabricated histories.
With this counterfeit data, attackers can impersonate legitimate individuals and use these fake identities to apply for loans or credit in their victims’ names. This identity impersonation enables criminals to secure funds, which they then steal, while the real person remains unaware of the fraudulent activity.
As AI-generated documents and profiles become increasingly realistic, detecting these types of fraud becomes more challenging. Consequently, cybercriminals are finding it easier to deceive both individuals and institutions, making it harder to prevent or catch such fraudulent activities. These tactics exploit trust, emotion, and urgency, and AI makes them scalable and alarmingly convincing.
7. Disinformation and Psychological Operations
AI is also being used in influence campaigns and information warfare.
- Synthetic media (videos, articles, voiceovers) can be generated to spread disinformation during elections, geopolitical events, or public crises.
- AI can be used to spoof public figures, brands, or institutions for reputation damage, blackmail, or fraud.
- Malicious actors can now launch coordinated campaigns that blend fake social media accounts, deepfake content, and AI-generated narratives to manipulate public opinion at scale.
These applications blur the line between cybercrime and psychological manipulation, making defense as much about digital literacy as it is about technology.
How Can We Protect Ourselves?
While AI has enhanced cybercriminal tactics, traditional cybersecurity best practices remain highly effective when consistently applied. The attack techniques being used are largely the same — they’ve simply evolved. Therefore, the foundation of a strong cybersecurity posture remains rooted in proven, well-established strategies, augmented by modern, AI-powered tools.
Foundational Best Practices
- Enable multi-factor authentication (MFA) across systems and applications: AI-driven phishing attacks have evolved significantly. Traditional phishing emails can often be detected by spam filters or antivirus software, but AI-generated attacks can craft messages that are personalized, grammatically perfect, and tailored to bypass automated detection systems. MFA becomes a critical layer of defense, as it requires additional verification, even if attackers manage to steal login credentials.
- Back up data regularly and securely: With AI-enhanced threats such as ransomware capable of rapidly encrypting large volumes of data, having regular and secure backups is more critical than ever. AI can modify ransomware payloads in real-time to bypass traditional detection systems. Having secure backups ensures that even if data is encrypted or compromised, critical systems can be restored quickly, minimizing the impact of an attack.
- Maintain system and software updates: AI tools are increasingly used to scan for vulnerabilities and exploit them autonomously. By keeping systems updated, you close the door to known exploits that attackers, including AI-powered ones, might leverage.
- Adopt a Zero Trust Security Model: AI allows cybercriminals to operate with the speed and agility of well-organized groups. A Zero Trust model assumes no one inside or outside the network can be trusted, helping prevent lateral movement within systems in case of a breach, especially against AI-powered attackers who may quickly scale operations.
- Implement email filtering and anti-phishing tools: With AI-generated phishing becoming more sophisticated, email filtering alone may not be enough to catch these attacks. AI-enhanced email filtering can help detect subtle patterns in messages and links that would typically go unnoticed by traditional tools, providing an extra layer of defense.
- Conduct regular security awareness training for all employees: AI-driven phishing campaigns often use highly convincing and personalized methods, making it harder for employees to differentiate between real and malicious emails. Training employees to recognize AI-enhanced threats and respond appropriately can drastically reduce the risk of a successful breach.
- Monitor for anomalies and behavioral deviations in networks: AI-assisted malware often uses evasive techniques to remain undetected. Anomaly detection powered by AI can track deviations from normal behavior patterns, alerting you to potential threats before they can cause significant damage.
- Segment critical networks and data to reduce exposure: AI-driven attackers can automate large-scale attacks with precision. By segmenting networks, you reduce the impact of any breach, as attackers will be contained within a smaller section of your network, limiting the extent of their access.
- Apply strict access control and security policies: Even with AI-driven automation, access control remains a cornerstone of cybersecurity. Limiting access to only the necessary resources ensures that AI-powered attacks can’t escalate to highly sensitive data or systems, minimizing potential damage.
These foundational measures are essential in the fight against AI-enhanced threats. While AI may change the speed and sophistication of attacks, consistent implementation of these practices helps build a robust defense. To defend effectively against AI-powered threats, it’s not just about reacting quickly but proactively anticipating and mitigating risks before they manifest.
Leveraging AI for Defense
As attackers adopt AI, defenders must do the same. AI-enhanced cybersecurity solutions offer new capabilities to strengthen detection and response:
- Advanced Threat Detection: AI-powered solutions such as XDR (Extended Detection and Response) and EDR (Endpoint Detection and Response) use machine learning to spot anomalies and detect threats that evade traditional detection systems.
- Predictive Analytics: AI’s ability to analyze historical data and predict future threats is crucial. Tools like SOAR (Security Orchestration, Automation, and Response) use predictive analytics to anticipate and prevent cyberattacks before they happen.
- Automated Incident Response: SOAR platforms can automate the response to cyber incidents, reducing the time between attack detection and mitigation.
- Vulnerability scanning: AI is also being used to improve vulnerability scanning and penetration testing, identifying security gaps before attackers can exploit them.
By integrating these technologies, organizations can increase resilience and agility in the face of AI-augmented threats.
Conclusion
AI is reshaping both sides of the cybersecurity equation. In the hands of cybercriminals, it enhances phishing attacks, enables sophisticated malware obfuscation, and lowers the barrier to entry for less technically skilled actors. AI supports everything from vulnerability exploitation to large-scale scams and disinformation, making attacks faster, smarter, and more difficult to detect.
On the defensive side, AI is proving invaluable in anomaly detection, predictive analysis, and rapid incident response. It empowers organizations to detect threats earlier and respond more effectively. However, AI’s full potential on the defense side can only be realized when paired with strong foundational security practices. While traditional security measures still play a vital role, AI-driven Security Operations Centers (SOCs) are increasingly essential in combating modern threats. These advanced solutions, like CyberProof’s SOAR platform, enhance an organization’s ability to detect, analyze, and respond to AI-powered threats in real-time, improving overall security posture.
Ultimately, AI is not introducing completely new types of attacks, but it is enhancing existing ones to be faster, more scalable, and more elusive. While AI introduces unprecedented speed and sophistication in cyberattacks, traditional security principles — when paired with AI-driven defense tools — form the most powerful defense. Organizations must maintain strong foundational controls — like multi-factor authentication, anomaly detection, and employee training — while embracing intelligent defense technologies to stay ahead in the evolving threat landscape.
By integrating AI-enhanced SOC capabilities and prioritizing a multi-layered defense strategy, businesses can better protect themselves from the increasing sophistication of cyber threats. Investing in AI-driven SOCs and leveraging CyberProof’s solutions, businesses can strengthen their security and protect themselves more effectively from the increasing sophistication of cyber threats.
References
- https://www.kaspersky.com/about/press-releases/kaspersky-reports-nearly-900-million-phishing-attempts-in-2024-as-cyber-threats-increase
- https://cloud.google.com/blog/topics/threat-intelligence/adversarial-misuse-generative-ai
- https://ieeexplore.ieee.org/document/10466545/figures#figures
- https://arxiv.org/pdf/2412.00586
- https://research.checkpoint.com/2025/funksec-alleged-top-ransomware-group-powered-by-ai
- https://www.appsoc.com/blog/testing-the-deepseek-r1-model-a-pandoras-box-of-security-risks
- https://unit42.paloaltonetworks.com/jailbreaking-deepseek-three-techniques
- https://arxiv.org/pdf/2308.09958
- https://www.hyas.com/hubfs/Downloadable%20Content/HYAS-AI-Augmented-Cyber-Attack-WP-1.1.pdf
- https://arxiv.org/pdf/2306.15559
- https://research.checkpoint.com/2025/funksec-alleged-top-ransomware-group-powered-by-ai
- https://openai.com/index/disrupting-malicious-uses-of-ai-by-state-affiliated-threat-actors
- https://cloud.google.com/blog/topics/threat-intelligence/adversarial-misuse-generative-ai
- https://www.bbc.com/news/articles/c8003dd8jzeo
- https://cybelangel.com/rise-ai-phishing
- https://pages.egress.com/whitepaper-phishing-trends-threat-report-10-24.html
- https://unit42.paloaltonetworks.com/dynamics-of-deepfake-scams
- https://pushsecurity.com/blog/considering-the-impact-of-computer-using-agents
- https://therecord.media/google-llm-sqlite-vulnerability-artificial-intelligence
- https://dfpi.ca.gov/news/insights/ai-investment-scams-are-here-and-youre-the-target
- https://www.microsoft.com/es-es/security/security-insider/meet-the-experts/emerging-AI-tactics-in-use-by-threat-actors
- https://arxiv.org/pdf/2404.08144
- https://statics.teams.cdn.office.net/evergreen-assets/safelinks/1/atp-safelinks.html
- https://www.thebusinessresearchcompany.com/report/ai-voice-cloning-global-market-report
