In short
AI changes cyber security for Australian organisations in three ways: attackers use it to write convincing phishing, clone voices and automate reconnaissance, the AI systems organisations deploy become a new attack surface open to prompt injection, and defenders use it to sort alerts and test faster. ASD has described AI as the defining force in the cyber threat landscape.
Artificial intelligence has moved from a future risk to the present one. In its 2025-26 Annual Cyber Threat Report, the Australian Signals Directorate (ASD) described AI as the defining force in the cyber threat landscape, and Australians lodged more than 82,000 cybercrime reports over the year, one every six minutes. For Australian organisations, AI and cyber security now intersect in three ways: attackers are using AI against you, the AI you deploy is a new target, and defenders can use AI to keep up.
This guide explains each, in plain language, and what to do about them.
How Attackers Use AI
AI has not invented new crimes so much as made old ones cheaper, faster and more convincing.
- Phishing that reads like a colleague wrote it. Language models produce fluent, personalised emails in seconds, removing the spelling mistakes and awkward phrasing staff were trained to spot.
- Deepfake voice and video fraud. A few seconds of audio is enough to clone a voice. Finance teams are receiving calls and video meetings that appear to come from executives approving urgent payments.
- Faster reconnaissance. AI tools map an organisation's staff, suppliers and exposed systems far faster than a human researcher, so attackers find the weak point sooner.
- Accelerated exploitation. AI helps attackers understand newly disclosed vulnerabilities and adapt exploit code quickly, shrinking the gap between a patch being released and attacks beginning.
- Scale. One operator can now run campaigns that once needed a team, which means small and mid-sized Australian businesses are targeted as routinely as large enterprises.
The end goals have not changed: stolen credentials, business email compromise, ransomware and data theft. That matters, because the defences that stop them, such as phishing-resistant multi-factor authentication, payment verification procedures and patching, still work. AI simply reduces the margin for error.
The AI You Deploy Is a New Attack Surface
The second shift gets less attention. Every chatbot, assistant and AI agent an organisation deploys can be talked to, and it treats text as instructions. That creates weaknesses traditional security testing was never designed to find.
- Prompt injection. Instructions hidden in a chat message, document, email or web page can override an AI system's rules. It is the most common AI vulnerability and cannot yet be fully prevented, only contained.
- Data leakage. AI assistants can surface anything they can reach. Microsoft 365 Copilot, for example, can find any file a user technically has access to, which turns years of oversharing into a searchable risk.
- Excessive agency. AI agents that can send email, change records or call other systems become powerful tools for an attacker who hijacks them.
- Poisoned knowledge. Systems that answer from internal documents can be manipulated by planting content in those documents.
These risks are catalogued in the OWASP Top 10 for LLM Applications and MITRE ATLAS, and they are found the same way other weaknesses are: by attacking the system under controlled conditions. That is what AI red teaming and LLM penetration testing does.
How Defenders Use AI
AI also helps defenders. Security teams use it to triage alerts, summarise incidents, detect unusual behaviour and search logs in plain language. In offensive security, AI-augmented tooling maps attack surfaces continuously and helps testers explore more attack paths in the same time, which is how we use it in AI penetration testing.
The catch is that AI output still needs verification. Automated tools produce false positives and miss context that a skilled human sees immediately. The strongest results come from pairing machine speed with human judgement: let AI widen the coverage, and let experienced people decide what is real and what matters.
What Australian Organisations Should Do Now
You do not need a separate AI security program to make progress. These steps address most of the risk.
- Harden the basics attackers exploit with AI. Phishing-resistant MFA, verified payment change procedures, timely patching of internet-facing systems and the Essential Eight remain the most effective defences against AI-assisted attacks.
- Train for AI-era social engineering. Teach staff that a convincing email, voice or video is no longer proof of identity, and agree verification steps for payments and access requests.
- Inventory your AI. List the AI tools and features in use, including those switched on inside existing software, and who owns each one.
- Fix permissions before enabling assistants. Clean up overshared files and sites before rolling out Copilot or similar tools.
- Limit what AI can do. Give AI systems the minimum access they need and require human approval for sensitive actions.
- Test your AI before attackers do. Adversarial testing shows whether prompt injection, data leakage or agent abuse is possible in your environment, and gives your board evidence that controls work.
Where to Start
If you are unsure where your organisation stands, start with a free Cyber Security Snapshot, a read-only check of your internet exposure, email spoofing protection, breached accounts and Microsoft 365 settings. If you are deploying AI, talk to us about AI red teaming. StrikeCyber is an Australian offensive security firm headquartered in Brisbane; get in touch or call 1300 654 898.
Frequently asked questions
How is AI changing cyber security?
AI changes both sides. Attackers use it to write convincing phishing, clone voices, find vulnerabilities and automate reconnaissance at a scale that used to need a team. Defenders use it to sort alerts, spot anomalies and test systems faster. And every AI system an organisation deploys becomes something new to protect. The Australian Signals Directorate described AI as the defining force in the cyber threat landscape in its 2025-26 Annual Cyber Threat Report.
What is an AI cyber attack?
An AI cyber attack is any attack where artificial intelligence does part of the work, such as writing tailored phishing emails, generating deepfake voice or video to authorise payments, automating vulnerability discovery, or manipulating an organisation's own AI system through prompt injection. Most AI-powered attacks still end in familiar outcomes: stolen credentials, fraudulent payments and data theft.
Can AI systems be hacked?
Yes. AI applications can be manipulated through prompt injection, tricked into leaking data they can access, poisoned through the documents they learn from, and abused through the tools and permissions they are given. These weaknesses are listed in the OWASP Top 10 for LLM Applications and can be found through AI red teaming before attackers find them.
How do we secure the AI we use at work?
Start with an inventory of AI tools and features in use, including those built into existing software. Limit what each system can access, fix overshared files before enabling assistants such as Microsoft 365 Copilot, require human approval for sensitive actions, log what AI systems do, and test them adversarially before and after rollout.
Is AI good or bad for cyber security?
Both. AI gives attackers speed and scale, but it also helps defenders cover more ground and respond faster. The organisations that come out ahead are the ones that use AI to strengthen their own testing and monitoring, and that treat their AI systems as part of the attack surface rather than as a black box.
