[Patent 5] GenAI Security Blind Spots

“Using AI to Guard AI”—It’s Not a Slogan, It’s Architectural Design

During the process of introducing Generative AI to the bank, the issue that kept me awake at night wasn’t model performance, but security.

If an AI chatbot capable of querying customer data is manipulated by malicious actors through tailored prompts, the consequences are unimaginable.

What’s more troublesome is that static rules of traditional cybersecurity simply cannot catch up with the creativity of prompt injections. Attackers change their wording, and the rules fail entirely.

This prompted me to design a generative adaptive security strategy, which ultimately earned the patent M674713 “Data Query System”.

---

System Architecture: Teaching AI to “Understand Malice”

The system is divided into two core layers:

Layer 1: User Behavior and Intent Analysis Model

• Behavior Analysis Module: Analyzes the user’s query frequency, time, IP location, and historical query patterns.
• Real-Time Intent Analysis Module: Uses NLP to interpret the underlying intent of every query.
• Anomalous Operation Detection Module: Identifies operations that deviate from normal patterns.
• Attack Behavior Judgment Module: Synthesizes the above signals to output a risk assessment.

Layer 2: Security Strategy Generation Model

• Dynamic Access Control Module: Instantly adjusts the user’s database access permissions based on their query patterns.
• Smart Blocking Module: Decides whether to intercept the query based on the risk level.
• Strict Response Control Module: Generates “strategic responses” to high-risk queries (instead of direct error messages, avoiding system info leakage).

There is also a critical design feature: the Continuous Optimization Module. When the system detects an attack, it uses GANs to generate simulated attack scenarios, continually training the model. The more it’s attacked, the smarter it gets.

---

Why Is This a Core Issue in GenAI Product Design?

As a GenAI Product Manager, security isn’t something to “hand over to the infosec department”; it must be part of the product architecture.

To design this system, I learned several key insights:

1. Static rules can never outpace dynamic attacks Every guardrail you set might be bypassed by a creative user. The system itself needs the “ability to learn.”

2. Security and user experience are not a zero-sum game Good security design should be invisible to regular users, only triggering on abnormal behavior. This requires extremely precise risk grading.

3. AI access control should be dynamic It’s not a binary judgment of “do you have permission,” but rather “in this context, for this query, what tier of information is appropriate for you to see right now?”

---

A GenAI PO’s Checklist: Is Your AI System Secure Enough?

• Can the system detect abnormal query patterns?
• Is there a real-time intent analysis mechanism?
• Is access control based on static roles or dynamic context awareness?
• Does the system get smarter after an attack, or stay exactly the same?

Security is the final gate before a GenAI product goes live, and it’s the first wall of user trust.

---

M674713 Data Query System (Generative Adaptive Security Strategy) | Grant Date: 2025/09/11 | Sole Inventor: Nils Liu

💬 Read more: 2025 Year in Review (English)