Harish Shenoy, Author at XTHE - Page 135 of 142

Home
In late 2025, the Google Threat Intelligence Group (GTIG) observed more threat actors using artificial intelligence (AI) to accelerate attacks, particularly in reconnaissance, social engineering, and malware development. This report updates our November 2025 findings on how threat actors are using AI tools.

By identifying these early indicators and offensive proofs of concept, GTIG aims to arm defenders with the intelligence needed to anticipate the next phase of AI-enabled threats, proactively thwart malicious activity, and continually strengthen our classifiers and models.

Executive Summary

Google DeepMind and GTIG have observed more attempts at model extraction, also known as distillation attacks, which constitute a form of intellectual property theft and violate Google's Terms of Service. In this report, we describe the steps we have taken to stop this activity, including detection, disruption, and model extraction. We have not seen direct attacks on our most advanced models or generative AI products from Endurance-Advanced Enduring actors. We have stopped many attempts by private companies and researchers worldwide to extract our proprietary logic.

Government-backed threat actors now use large language models as key tools for technical research, targeting, and quickly creating more convincing phishing messages. This report shows how groups from the Democratic People's Republic of Korea, Iran, the People's Republic of China, and Russia used AI in late 2025. It also helps us understand how misuse of generative AI appears in real-life campaigns we have stopped so far. GTIG has not seen APT or information operations (IO) actors reach new capabilities that would change the overall threat landscape.

This report looks at the following areas:
- Model extraction attacks: Over the past year, distillation attacks have become a more common means of stealing intellectual property.
- AI-augmented operations: Real-world examples show how groups are making reconnaissance and phishing more efficient by leveraging AI to build trust.
- Agentic AI: Threat actors are beginning to explore building Agentic AI tools to support malware and tool development.
- AI-integrated malware: New malware families like Honest Queue are testing Gemini's application programming interface (API) to create code that can download and run second-stage malware.
- Underground jailbreak ecosystem: Malicious services such as Xanthorox are appearing in underground markets. They claim to be independent models but actually use jailbroken commercial APIs and open-source model context protocol (MCP) servers.
At Google, we are committed to developing AI boldly and responsibly. This means we act to stop malicious activity by turning off projects and accounts linked to bad actors, and we keep improving our models to make them harder to misuse. We also share best practices with the industry to help defenders and strengthen protections across the ecosystem.

In this report, we describe the steps we have taken, including disabling assets and leveraging intelligence to enhance the security of our classifiers and models. You can find more details about how we protect Gemini in the white paper, Advancing Gemini's Security Safeguards.

Direct Model Risks: Disrupting Model Extraction Attacks

As more organizations use LLMs in their main operations, the unique logic and training behind these models have become valuable targets. In the past, attackers would break into computer systems and steal trade secrets. Now that many LLMs are available as services, attackers can use regular API access to try to replicate specific AI model features.

In 2025, we did not see any direct attacks on advanced models from known APT or Information Operations groups. However, we noticed model-extraction attacks, also known as distillation attacks, on our AI models. These attacks aim to learn how a model reasons and makes decisions.

What Are Model Extraction Attacks

Model extraction attacks occur when someone uses authorized access to carefully test a machine learning model and gather information to train a new one. Attackers use a method called knowledge distillation (KD) to transfer knowledge from one model to another. That is why the MEA is often called a distillation attack.

Model extraction and knowledge distillation enable attackers to speed up AI model development and reduce costs. This is a type of intellectual property (IP) theft.

Knowledge distillation (KD) is becoming a common machine learning method for training student models using existing teacher models. This usually means asking the teacher model questions in a certain area, then fine-tuning the results or using them in other training steps to create the student model. Distillation has valid uses, and Google Cloud offers tools for it. But using distillation on Google's Gemini models without permission breaks our Terms of Service. Google is working on ways to spot and stop these attempts.

Source:GTIG AI Threat Tracker: Distillation, Experimentation, and (Continued) Integration of AI for Adversarial Use
About US
In late 2025, the Google Threat Intelligence Group (GTIG) observed more threat actors using artificial intelligence (AI) to accelerate attacks, particularly in reconnaissance, social engineering, and malware development. This report updates our November 2025 findings on how threat actors are using AI tools.

By identifying these early indicators and offensive proofs of concept, GTIG aims to arm defenders with the intelligence needed to anticipate the next phase of AI-enabled threats, proactively thwart malicious activity, and continually strengthen our classifiers and models.

Executive Summary

Google DeepMind and GTIG have observed more attempts at model extraction, also known as distillation attacks, which constitute a form of intellectual property theft and violate Google's Terms of Service. In this report, we describe the steps we have taken to stop this activity, including detection, disruption, and model extraction. We have not seen direct attacks on our most advanced models or generative AI products from Endurance-Advanced Enduring actors. We have stopped many attempts by private companies and researchers worldwide to extract our proprietary logic.

Government-backed threat actors now use large language models as key tools for technical research, targeting, and quickly creating more convincing phishing messages. This report shows how groups from the Democratic People's Republic of Korea, Iran, the People's Republic of China, and Russia used AI in late 2025. It also helps us understand how misuse of generative AI appears in real-life campaigns we have stopped so far. GTIG has not seen APT or information operations (IO) actors reach new capabilities that would change the overall threat landscape.

This report looks at the following areas:
- Model extraction attacks: Over the past year, distillation attacks have become a more common means of stealing intellectual property.
- AI-augmented operations: Real-world examples show how groups are making reconnaissance and phishing more efficient by leveraging AI to build trust.
- Agentic AI: Threat actors are beginning to explore building Agentic AI tools to support malware and tool development.
- AI-integrated malware: New malware families like Honest Queue are testing Gemini's application programming interface (API) to create code that can download and run second-stage malware.
- Underground jailbreak ecosystem: Malicious services such as Xanthorox are appearing in underground markets. They claim to be independent models but actually use jailbroken commercial APIs and open-source model context protocol (MCP) servers.
At Google, we are committed to developing AI boldly and responsibly. This means we act to stop malicious activity by turning off projects and accounts linked to bad actors, and we keep improving our models to make them harder to misuse. We also share best practices with the industry to help defenders and strengthen protections across the ecosystem.

In this report, we describe the steps we have taken, including disabling assets and leveraging intelligence to enhance the security of our classifiers and models. You can find more details about how we protect Gemini in the white paper, Advancing Gemini's Security Safeguards.

Direct Model Risks: Disrupting Model Extraction Attacks

As more organizations use LLMs in their main operations, the unique logic and training behind these models have become valuable targets. In the past, attackers would break into computer systems and steal trade secrets. Now that many LLMs are available as services, attackers can use regular API access to try to replicate specific AI model features.

In 2025, we did not see any direct attacks on advanced models from known APT or Information Operations groups. However, we noticed model-extraction attacks, also known as distillation attacks, on our AI models. These attacks aim to learn how a model reasons and makes decisions.

What Are Model Extraction Attacks

Model extraction attacks occur when someone uses authorized access to carefully test a machine learning model and gather information to train a new one. Attackers use a method called knowledge distillation (KD) to transfer knowledge from one model to another. That is why the MEA is often called a distillation attack.

Model extraction and knowledge distillation enable attackers to speed up AI model development and reduce costs. This is a type of intellectual property (IP) theft.

Knowledge distillation (KD) is becoming a common machine learning method for training student models using existing teacher models. This usually means asking the teacher model questions in a certain area, then fine-tuning the results or using them in other training steps to create the student model. Distillation has valid uses, and Google Cloud offers tools for it. But using distillation on Google's Gemini models without permission breaks our Terms of Service. Google is working on ways to spot and stop these attempts.

Source:GTIG AI Threat Tracker: Distillation, Experimentation, and (Continued) Integration of AI for Adversarial Use
Tech Reviews
In late 2025, the Google Threat Intelligence Group (GTIG) observed more threat actors using artificial intelligence (AI) to accelerate attacks, particularly in reconnaissance, social engineering, and malware development. This report updates our November 2025 findings on how threat actors are using AI tools.

By identifying these early indicators and offensive proofs of concept, GTIG aims to arm defenders with the intelligence needed to anticipate the next phase of AI-enabled threats, proactively thwart malicious activity, and continually strengthen our classifiers and models.

Executive Summary

Google DeepMind and GTIG have observed more attempts at model extraction, also known as distillation attacks, which constitute a form of intellectual property theft and violate Google's Terms of Service. In this report, we describe the steps we have taken to stop this activity, including detection, disruption, and model extraction. We have not seen direct attacks on our most advanced models or generative AI products from Endurance-Advanced Enduring actors. We have stopped many attempts by private companies and researchers worldwide to extract our proprietary logic.

Government-backed threat actors now use large language models as key tools for technical research, targeting, and quickly creating more convincing phishing messages. This report shows how groups from the Democratic People's Republic of Korea, Iran, the People's Republic of China, and Russia used AI in late 2025. It also helps us understand how misuse of generative AI appears in real-life campaigns we have stopped so far. GTIG has not seen APT or information operations (IO) actors reach new capabilities that would change the overall threat landscape.

This report looks at the following areas:
- Model extraction attacks: Over the past year, distillation attacks have become a more common means of stealing intellectual property.
- AI-augmented operations: Real-world examples show how groups are making reconnaissance and phishing more efficient by leveraging AI to build trust.
- Agentic AI: Threat actors are beginning to explore building Agentic AI tools to support malware and tool development.
- AI-integrated malware: New malware families like Honest Queue are testing Gemini's application programming interface (API) to create code that can download and run second-stage malware.
- Underground jailbreak ecosystem: Malicious services such as Xanthorox are appearing in underground markets. They claim to be independent models but actually use jailbroken commercial APIs and open-source model context protocol (MCP) servers.
At Google, we are committed to developing AI boldly and responsibly. This means we act to stop malicious activity by turning off projects and accounts linked to bad actors, and we keep improving our models to make them harder to misuse. We also share best practices with the industry to help defenders and strengthen protections across the ecosystem.

In this report, we describe the steps we have taken, including disabling assets and leveraging intelligence to enhance the security of our classifiers and models. You can find more details about how we protect Gemini in the white paper, Advancing Gemini's Security Safeguards.

Direct Model Risks: Disrupting Model Extraction Attacks

As more organizations use LLMs in their main operations, the unique logic and training behind these models have become valuable targets. In the past, attackers would break into computer systems and steal trade secrets. Now that many LLMs are available as services, attackers can use regular API access to try to replicate specific AI model features.

In 2025, we did not see any direct attacks on advanced models from known APT or Information Operations groups. However, we noticed model-extraction attacks, also known as distillation attacks, on our AI models. These attacks aim to learn how a model reasons and makes decisions.

What Are Model Extraction Attacks

Model extraction attacks occur when someone uses authorized access to carefully test a machine learning model and gather information to train a new one. Attackers use a method called knowledge distillation (KD) to transfer knowledge from one model to another. That is why the MEA is often called a distillation attack.

Model extraction and knowledge distillation enable attackers to speed up AI model development and reduce costs. This is a type of intellectual property (IP) theft.

Knowledge distillation (KD) is becoming a common machine learning method for training student models using existing teacher models. This usually means asking the teacher model questions in a certain area, then fine-tuning the results or using them in other training steps to create the student model. Distillation has valid uses, and Google Cloud offers tools for it. But using distillation on Google's Gemini models without permission breaks our Terms of Service. Google is working on ways to spot and stop these attempts.

Source:GTIG AI Threat Tracker: Distillation, Experimentation, and (Continued) Integration of AI for Adversarial Use
- AI
- Troubleshooting
- Smartwatches
- Smartphones
- Camera
- Gadgets
- Games
- Laptops
- Seasonal Sales
Phone Features
In late 2025, the Google Threat Intelligence Group (GTIG) observed more threat actors using artificial intelligence (AI) to accelerate attacks, particularly in reconnaissance, social engineering, and malware development. This report updates our November 2025 findings on how threat actors are using AI tools.

By identifying these early indicators and offensive proofs of concept, GTIG aims to arm defenders with the intelligence needed to anticipate the next phase of AI-enabled threats, proactively thwart malicious activity, and continually strengthen our classifiers and models.

Executive Summary

Google DeepMind and GTIG have observed more attempts at model extraction, also known as distillation attacks, which constitute a form of intellectual property theft and violate Google's Terms of Service. In this report, we describe the steps we have taken to stop this activity, including detection, disruption, and model extraction. We have not seen direct attacks on our most advanced models or generative AI products from Endurance-Advanced Enduring actors. We have stopped many attempts by private companies and researchers worldwide to extract our proprietary logic.

Government-backed threat actors now use large language models as key tools for technical research, targeting, and quickly creating more convincing phishing messages. This report shows how groups from the Democratic People's Republic of Korea, Iran, the People's Republic of China, and Russia used AI in late 2025. It also helps us understand how misuse of generative AI appears in real-life campaigns we have stopped so far. GTIG has not seen APT or information operations (IO) actors reach new capabilities that would change the overall threat landscape.

This report looks at the following areas:
- Model extraction attacks: Over the past year, distillation attacks have become a more common means of stealing intellectual property.
- AI-augmented operations: Real-world examples show how groups are making reconnaissance and phishing more efficient by leveraging AI to build trust.
- Agentic AI: Threat actors are beginning to explore building Agentic AI tools to support malware and tool development.
- AI-integrated malware: New malware families like Honest Queue are testing Gemini's application programming interface (API) to create code that can download and run second-stage malware.
- Underground jailbreak ecosystem: Malicious services such as Xanthorox are appearing in underground markets. They claim to be independent models but actually use jailbroken commercial APIs and open-source model context protocol (MCP) servers.
At Google, we are committed to developing AI boldly and responsibly. This means we act to stop malicious activity by turning off projects and accounts linked to bad actors, and we keep improving our models to make them harder to misuse. We also share best practices with the industry to help defenders and strengthen protections across the ecosystem.

In this report, we describe the steps we have taken, including disabling assets and leveraging intelligence to enhance the security of our classifiers and models. You can find more details about how we protect Gemini in the white paper, Advancing Gemini's Security Safeguards.

Direct Model Risks: Disrupting Model Extraction Attacks

As more organizations use LLMs in their main operations, the unique logic and training behind these models have become valuable targets. In the past, attackers would break into computer systems and steal trade secrets. Now that many LLMs are available as services, attackers can use regular API access to try to replicate specific AI model features.

In 2025, we did not see any direct attacks on advanced models from known APT or Information Operations groups. However, we noticed model-extraction attacks, also known as distillation attacks, on our AI models. These attacks aim to learn how a model reasons and makes decisions.

What Are Model Extraction Attacks

Model extraction attacks occur when someone uses authorized access to carefully test a machine learning model and gather information to train a new one. Attackers use a method called knowledge distillation (KD) to transfer knowledge from one model to another. That is why the MEA is often called a distillation attack.

Model extraction and knowledge distillation enable attackers to speed up AI model development and reduce costs. This is a type of intellectual property (IP) theft.

Knowledge distillation (KD) is becoming a common machine learning method for training student models using existing teacher models. This usually means asking the teacher model questions in a certain area, then fine-tuning the results or using them in other training steps to create the student model. Distillation has valid uses, and Google Cloud offers tools for it. But using distillation on Google's Gemini models without permission breaks our Terms of Service. Google is working on ways to spot and stop these attempts.

Source:GTIG AI Threat Tracker: Distillation, Experimentation, and (Continued) Integration of AI for Adversarial Use
Buying Guides
In late 2025, the Google Threat Intelligence Group (GTIG) observed more threat actors using artificial intelligence (AI) to accelerate attacks, particularly in reconnaissance, social engineering, and malware development. This report updates our November 2025 findings on how threat actors are using AI tools.

By identifying these early indicators and offensive proofs of concept, GTIG aims to arm defenders with the intelligence needed to anticipate the next phase of AI-enabled threats, proactively thwart malicious activity, and continually strengthen our classifiers and models.

Executive Summary

Google DeepMind and GTIG have observed more attempts at model extraction, also known as distillation attacks, which constitute a form of intellectual property theft and violate Google's Terms of Service. In this report, we describe the steps we have taken to stop this activity, including detection, disruption, and model extraction. We have not seen direct attacks on our most advanced models or generative AI products from Endurance-Advanced Enduring actors. We have stopped many attempts by private companies and researchers worldwide to extract our proprietary logic.

Government-backed threat actors now use large language models as key tools for technical research, targeting, and quickly creating more convincing phishing messages. This report shows how groups from the Democratic People's Republic of Korea, Iran, the People's Republic of China, and Russia used AI in late 2025. It also helps us understand how misuse of generative AI appears in real-life campaigns we have stopped so far. GTIG has not seen APT or information operations (IO) actors reach new capabilities that would change the overall threat landscape.

This report looks at the following areas:
- Model extraction attacks: Over the past year, distillation attacks have become a more common means of stealing intellectual property.
- AI-augmented operations: Real-world examples show how groups are making reconnaissance and phishing more efficient by leveraging AI to build trust.
- Agentic AI: Threat actors are beginning to explore building Agentic AI tools to support malware and tool development.
- AI-integrated malware: New malware families like Honest Queue are testing Gemini's application programming interface (API) to create code that can download and run second-stage malware.
- Underground jailbreak ecosystem: Malicious services such as Xanthorox are appearing in underground markets. They claim to be independent models but actually use jailbroken commercial APIs and open-source model context protocol (MCP) servers.
At Google, we are committed to developing AI boldly and responsibly. This means we act to stop malicious activity by turning off projects and accounts linked to bad actors, and we keep improving our models to make them harder to misuse. We also share best practices with the industry to help defenders and strengthen protections across the ecosystem.

In this report, we describe the steps we have taken, including disabling assets and leveraging intelligence to enhance the security of our classifiers and models. You can find more details about how we protect Gemini in the white paper, Advancing Gemini's Security Safeguards.

Direct Model Risks: Disrupting Model Extraction Attacks

As more organizations use LLMs in their main operations, the unique logic and training behind these models have become valuable targets. In the past, attackers would break into computer systems and steal trade secrets. Now that many LLMs are available as services, attackers can use regular API access to try to replicate specific AI model features.

In 2025, we did not see any direct attacks on advanced models from known APT or Information Operations groups. However, we noticed model-extraction attacks, also known as distillation attacks, on our AI models. These attacks aim to learn how a model reasons and makes decisions.

What Are Model Extraction Attacks

Model extraction attacks occur when someone uses authorized access to carefully test a machine learning model and gather information to train a new one. Attackers use a method called knowledge distillation (KD) to transfer knowledge from one model to another. That is why the MEA is often called a distillation attack.

Model extraction and knowledge distillation enable attackers to speed up AI model development and reduce costs. This is a type of intellectual property (IP) theft.

Knowledge distillation (KD) is becoming a common machine learning method for training student models using existing teacher models. This usually means asking the teacher model questions in a certain area, then fine-tuning the results or using them in other training steps to create the student model. Distillation has valid uses, and Google Cloud offers tools for it. But using distillation on Google's Gemini models without permission breaks our Terms of Service. Google is working on ways to spot and stop these attempts.

Source:GTIG AI Threat Tracker: Distillation, Experimentation, and (Continued) Integration of AI for Adversarial Use
Comparison
In late 2025, the Google Threat Intelligence Group (GTIG) observed more threat actors using artificial intelligence (AI) to accelerate attacks, particularly in reconnaissance, social engineering, and malware development. This report updates our November 2025 findings on how threat actors are using AI tools.

By identifying these early indicators and offensive proofs of concept, GTIG aims to arm defenders with the intelligence needed to anticipate the next phase of AI-enabled threats, proactively thwart malicious activity, and continually strengthen our classifiers and models.

Executive Summary

Google DeepMind and GTIG have observed more attempts at model extraction, also known as distillation attacks, which constitute a form of intellectual property theft and violate Google's Terms of Service. In this report, we describe the steps we have taken to stop this activity, including detection, disruption, and model extraction. We have not seen direct attacks on our most advanced models or generative AI products from Endurance-Advanced Enduring actors. We have stopped many attempts by private companies and researchers worldwide to extract our proprietary logic.

Government-backed threat actors now use large language models as key tools for technical research, targeting, and quickly creating more convincing phishing messages. This report shows how groups from the Democratic People's Republic of Korea, Iran, the People's Republic of China, and Russia used AI in late 2025. It also helps us understand how misuse of generative AI appears in real-life campaigns we have stopped so far. GTIG has not seen APT or information operations (IO) actors reach new capabilities that would change the overall threat landscape.

This report looks at the following areas:
- Model extraction attacks: Over the past year, distillation attacks have become a more common means of stealing intellectual property.
- AI-augmented operations: Real-world examples show how groups are making reconnaissance and phishing more efficient by leveraging AI to build trust.
- Agentic AI: Threat actors are beginning to explore building Agentic AI tools to support malware and tool development.
- AI-integrated malware: New malware families like Honest Queue are testing Gemini's application programming interface (API) to create code that can download and run second-stage malware.
- Underground jailbreak ecosystem: Malicious services such as Xanthorox are appearing in underground markets. They claim to be independent models but actually use jailbroken commercial APIs and open-source model context protocol (MCP) servers.
At Google, we are committed to developing AI boldly and responsibly. This means we act to stop malicious activity by turning off projects and accounts linked to bad actors, and we keep improving our models to make them harder to misuse. We also share best practices with the industry to help defenders and strengthen protections across the ecosystem.

In this report, we describe the steps we have taken, including disabling assets and leveraging intelligence to enhance the security of our classifiers and models. You can find more details about how we protect Gemini in the white paper, Advancing Gemini's Security Safeguards.

Direct Model Risks: Disrupting Model Extraction Attacks

As more organizations use LLMs in their main operations, the unique logic and training behind these models have become valuable targets. In the past, attackers would break into computer systems and steal trade secrets. Now that many LLMs are available as services, attackers can use regular API access to try to replicate specific AI model features.

In 2025, we did not see any direct attacks on advanced models from known APT or Information Operations groups. However, we noticed model-extraction attacks, also known as distillation attacks, on our AI models. These attacks aim to learn how a model reasons and makes decisions.

What Are Model Extraction Attacks

Model extraction attacks occur when someone uses authorized access to carefully test a machine learning model and gather information to train a new one. Attackers use a method called knowledge distillation (KD) to transfer knowledge from one model to another. That is why the MEA is often called a distillation attack.

Model extraction and knowledge distillation enable attackers to speed up AI model development and reduce costs. This is a type of intellectual property (IP) theft.

Knowledge distillation (KD) is becoming a common machine learning method for training student models using existing teacher models. This usually means asking the teacher model questions in a certain area, then fine-tuning the results or using them in other training steps to create the student model. Distillation has valid uses, and Google Cloud offers tools for it. But using distillation on Google's Gemini models without permission breaks our Terms of Service. Google is working on ways to spot and stop these attempts.

Source:GTIG AI Threat Tracker: Distillation, Experimentation, and (Continued) Integration of AI for Adversarial Use
News
In late 2025, the Google Threat Intelligence Group (GTIG) observed more threat actors using artificial intelligence (AI) to accelerate attacks, particularly in reconnaissance, social engineering, and malware development. This report updates our November 2025 findings on how threat actors are using AI tools.

By identifying these early indicators and offensive proofs of concept, GTIG aims to arm defenders with the intelligence needed to anticipate the next phase of AI-enabled threats, proactively thwart malicious activity, and continually strengthen our classifiers and models.

Executive Summary

Google DeepMind and GTIG have observed more attempts at model extraction, also known as distillation attacks, which constitute a form of intellectual property theft and violate Google's Terms of Service. In this report, we describe the steps we have taken to stop this activity, including detection, disruption, and model extraction. We have not seen direct attacks on our most advanced models or generative AI products from Endurance-Advanced Enduring actors. We have stopped many attempts by private companies and researchers worldwide to extract our proprietary logic.

Government-backed threat actors now use large language models as key tools for technical research, targeting, and quickly creating more convincing phishing messages. This report shows how groups from the Democratic People's Republic of Korea, Iran, the People's Republic of China, and Russia used AI in late 2025. It also helps us understand how misuse of generative AI appears in real-life campaigns we have stopped so far. GTIG has not seen APT or information operations (IO) actors reach new capabilities that would change the overall threat landscape.

This report looks at the following areas:
- Model extraction attacks: Over the past year, distillation attacks have become a more common means of stealing intellectual property.
- AI-augmented operations: Real-world examples show how groups are making reconnaissance and phishing more efficient by leveraging AI to build trust.
- Agentic AI: Threat actors are beginning to explore building Agentic AI tools to support malware and tool development.
- AI-integrated malware: New malware families like Honest Queue are testing Gemini's application programming interface (API) to create code that can download and run second-stage malware.
- Underground jailbreak ecosystem: Malicious services such as Xanthorox are appearing in underground markets. They claim to be independent models but actually use jailbroken commercial APIs and open-source model context protocol (MCP) servers.
At Google, we are committed to developing AI boldly and responsibly. This means we act to stop malicious activity by turning off projects and accounts linked to bad actors, and we keep improving our models to make them harder to misuse. We also share best practices with the industry to help defenders and strengthen protections across the ecosystem.

In this report, we describe the steps we have taken, including disabling assets and leveraging intelligence to enhance the security of our classifiers and models. You can find more details about how we protect Gemini in the white paper, Advancing Gemini's Security Safeguards.

Direct Model Risks: Disrupting Model Extraction Attacks

As more organizations use LLMs in their main operations, the unique logic and training behind these models have become valuable targets. In the past, attackers would break into computer systems and steal trade secrets. Now that many LLMs are available as services, attackers can use regular API access to try to replicate specific AI model features.

In 2025, we did not see any direct attacks on advanced models from known APT or Information Operations groups. However, we noticed model-extraction attacks, also known as distillation attacks, on our AI models. These attacks aim to learn how a model reasons and makes decisions.

What Are Model Extraction Attacks

Model extraction attacks occur when someone uses authorized access to carefully test a machine learning model and gather information to train a new one. Attackers use a method called knowledge distillation (KD) to transfer knowledge from one model to another. That is why the MEA is often called a distillation attack.

Model extraction and knowledge distillation enable attackers to speed up AI model development and reduce costs. This is a type of intellectual property (IP) theft.

Knowledge distillation (KD) is becoming a common machine learning method for training student models using existing teacher models. This usually means asking the teacher model questions in a certain area, then fine-tuning the results or using them in other training steps to create the student model. Distillation has valid uses, and Google Cloud offers tools for it. But using distillation on Google's Gemini models without permission breaks our Terms of Service. Google is working on ways to spot and stop these attempts.

Source:GTIG AI Threat Tracker: Distillation, Experimentation, and (Continued) Integration of AI for Adversarial Use
Contact
In late 2025, the Google Threat Intelligence Group (GTIG) observed more threat actors using artificial intelligence (AI) to accelerate attacks, particularly in reconnaissance, social engineering, and malware development. This report updates our November 2025 findings on how threat actors are using AI tools.

By identifying these early indicators and offensive proofs of concept, GTIG aims to arm defenders with the intelligence needed to anticipate the next phase of AI-enabled threats, proactively thwart malicious activity, and continually strengthen our classifiers and models.

Executive Summary

Google DeepMind and GTIG have observed more attempts at model extraction, also known as distillation attacks, which constitute a form of intellectual property theft and violate Google's Terms of Service. In this report, we describe the steps we have taken to stop this activity, including detection, disruption, and model extraction. We have not seen direct attacks on our most advanced models or generative AI products from Endurance-Advanced Enduring actors. We have stopped many attempts by private companies and researchers worldwide to extract our proprietary logic.

Government-backed threat actors now use large language models as key tools for technical research, targeting, and quickly creating more convincing phishing messages. This report shows how groups from the Democratic People's Republic of Korea, Iran, the People's Republic of China, and Russia used AI in late 2025. It also helps us understand how misuse of generative AI appears in real-life campaigns we have stopped so far. GTIG has not seen APT or information operations (IO) actors reach new capabilities that would change the overall threat landscape.

This report looks at the following areas:
- Model extraction attacks: Over the past year, distillation attacks have become a more common means of stealing intellectual property.
- AI-augmented operations: Real-world examples show how groups are making reconnaissance and phishing more efficient by leveraging AI to build trust.
- Agentic AI: Threat actors are beginning to explore building Agentic AI tools to support malware and tool development.
- AI-integrated malware: New malware families like Honest Queue are testing Gemini's application programming interface (API) to create code that can download and run second-stage malware.
- Underground jailbreak ecosystem: Malicious services such as Xanthorox are appearing in underground markets. They claim to be independent models but actually use jailbroken commercial APIs and open-source model context protocol (MCP) servers.
At Google, we are committed to developing AI boldly and responsibly. This means we act to stop malicious activity by turning off projects and accounts linked to bad actors, and we keep improving our models to make them harder to misuse. We also share best practices with the industry to help defenders and strengthen protections across the ecosystem.

In this report, we describe the steps we have taken, including disabling assets and leveraging intelligence to enhance the security of our classifiers and models. You can find more details about how we protect Gemini in the white paper, Advancing Gemini's Security Safeguards.

Direct Model Risks: Disrupting Model Extraction Attacks

As more organizations use LLMs in their main operations, the unique logic and training behind these models have become valuable targets. In the past, attackers would break into computer systems and steal trade secrets. Now that many LLMs are available as services, attackers can use regular API access to try to replicate specific AI model features.

In 2025, we did not see any direct attacks on advanced models from known APT or Information Operations groups. However, we noticed model-extraction attacks, also known as distillation attacks, on our AI models. These attacks aim to learn how a model reasons and makes decisions.

What Are Model Extraction Attacks

Model extraction attacks occur when someone uses authorized access to carefully test a machine learning model and gather information to train a new one. Attackers use a method called knowledge distillation (KD) to transfer knowledge from one model to another. That is why the MEA is often called a distillation attack.

Model extraction and knowledge distillation enable attackers to speed up AI model development and reduce costs. This is a type of intellectual property (IP) theft.

Knowledge distillation (KD) is becoming a common machine learning method for training student models using existing teacher models. This usually means asking the teacher model questions in a certain area, then fine-tuning the results or using them in other training steps to create the student model. Distillation has valid uses, and Google Cloud offers tools for it. But using distillation on Google's Gemini models without permission breaks our Terms of Service. Google is working on ways to spot and stop these attempts.

Source:GTIG AI Threat Tracker: Distillation, Experimentation, and (Continued) Integration of AI for Adversarial Use

Harish Shenoy

Total Post: 1417

Harish Shenoy | Technology Journalist Harish Shenoy is a veteran technology journalist at Xthe with over 21 years of experience spanning tech, health, finance, and future technology. Specializing in semiconductor roadmaps, enterprise AI infrastructure, and consumer hardware, his reporting is driven by primary-source verification and official corporate disclosures. With more than a decade of focused writing across the US, UK, Australia, Canada, and India, Harish maintains a strict commitment to factual, non-sensationalized reporting on the economic and logistical shifts within the global tech sector. His current work centers on the technical evolution of "agentic AI" and next-generation silicon.

The Model Extraction Alert: Google Disrupts State-Backed Campaign to Steal AI Logic

In late 2025, the Google Threat Intelligence Group (GTIG) observed more threat actors using artificial intelligence (AI) to accelerate attacks, particularly in...

What is Intel’s Z Angle breakthrough that prevents laptops from overheating during AI tasks?

To sum up: Intel has introduced the prototype of Z-Angle Memory (ZAM), which it is developing together with Saimemory, a SoftBank subsidiary. According...

Beyond The Chatbot: Why Samsung’s February 2025 San Francisco Launch Is A Pivot To Personal AI

In February 2025, Samsung is focusing its strategy on the Galaxy S25 series, shifting from offering AI as separate features...

OpenAI Officially Retires GPT-4O Today – How To Preserve Your Workflows As GPT Shifts To GPT-5.2

OpenAI plans to retire GPT-4o and several related models on Feb 13, 2026. After this change, GPT-5.2 will become the primary model...

Protecting The Grid: Why Tech Giants Are Racing To Implement Post-Quantum Cryptography Now

Tech giants and critical infrastructure operators are moving quickly to adopt post-quantum cryptography (PQC). Even though large-scale quantum computers may...

Microsoft AI 360 vs Co-Pilot: What Changed in the New Unified Governance Era?

Microsoft is now managing AI as a connected, secure ecosystem rather than as separate tools like Co-Pilot. The main challenge...

The End of the TV Camera: How 26 Smartphones Just Powered a Global Sports Broadcast

Samsung Electronics, a global Olympic and Paralympic partner, shared its plans to make the Milano-Cortina Winter Games more connected by...

NASA Opens Exclusive Patent Licensing for New 2026 Inventions: A Private Sector Tech Gold Rush

In early 2026, NASA ramped up its push to move space technology into the private sector by issuing notices of...

Samsung’s First Home AI Orchestrator Enters Mass Production

The physical AI home, a key trend at CES 2026, signals a shift as artificial intelligence moves from digital screen-based...

OpenAI Begins Testing Ads In ChatGPT: What The New Promoted Responses Look Like

After several weeks of hints, OpenAI is now testing ads in ChatGPT for users in the US. This is a...