Building Secure and Responsible LLM Applications Training Course

Overview of LLM Architecture and Attack Surface

• Understanding how LLMs are built, deployed, and accessed via APIs
• Key components in LLM application stacks (e.g., prompts, agents, memory, APIs)
• Identification of where and how security issues arise in real-world scenarios

Prompt Injection and Jailbreak Attacks

• Definition and dangers of prompt injection
• Direct and indirect prompt injection scenarios
• Jailbreaking techniques used to bypass safety filters
• Strategies for detection and mitigation

Data Leakage and Privacy Risks

• Unintentional data exposure through model responses
• PII leaks and misuse of model memory
• Designing privacy-conscious prompts and retrieval-augmented generation (RAG) systems

LLM Output Filtering and Guarding

• Utilizing Guardrails AI for content filtering and validation
• Defining output schemas and constraints
• Monitoring and logging unsafe outputs

Human-in-the-Loop and Workflow Approaches

• Identifying points for human oversight
• Managing approval queues, scoring thresholds, and fallback handling
• Calibrating trust and the role of explainability

Secure LLM App Design Patterns

• Implementing least privilege and sandboxing for API calls and agents
• Applying rate limiting, throttling, and abuse detection
• Ensuring robust chaining with LangChain and prompt isolation

Compliance, Logging, and Governance

• Ensuring auditability of LLM outputs
• Maintaining traceability and prompt/version control
• Aligning with internal security policies and regulatory requirements

Summary and Next Steps