Grazie per aver inviato la tua richiesta! Uno dei nostri team membri ti contatterà a breve.
Grazie per aver inviato il tuo prenotazione! Uno dei nostri team membri ti contatterà a breve.
Struttura del corso
Introduction to LLMOps
- LLMOps vs MLOps: unique challenges of operating LLMs
- The LLM application lifecycle: prompt, evaluate, deploy, monitor
- Production readiness checklist for GenAI applications
Prompt Management and Versioning
- Prompt templating systems and variable injection
- Semantic versioning for prompts with automated regression testing
- Prompt registries and collaboration workflows
LLM Evaluation at Scale
- Evaluation dimensions: accuracy, relevance, safety, groundedness
- LLM-as-judge metrics and human evaluation pipelines
- Automated eval frameworks: RAGAS, DeepEval, and custom evaluators
- Quality gates in CI/CD for LLM deployments
Safety Guardrails and Content Governance
- Input and output guardrails: NeMo Guardrails and Guardrails AI
- PII detection, toxicity filtering, and topic boundaries
- Jailbreak and prompt injection defense strategies
- Red-teaming LLM applications for safety assurance
LLM Observability and Monitoring
- Telemetry: token usage, latency, cost, and quality metrics
- Drift detection in LLM outputs and embedding spaces
- Session-level tracing for multi-turn agent conversations
- Dashboards and alerting with LangSmith, Arize, and OpenTelemetry
AI Gateway and Model Orchestration
- Multi-provider routing with LiteLLM and Portkey
- Fallback strategies, retry logic, and circuit breakers
- Cost-aware model selection and load balancing
- Rate limiting, quota management, and API key governance
Performance Optimization
- Semantic caching with vector stores and exact-match strategies
- Structured output enforcement with constrained decoding
- Batching, streaming, and concurrency patterns
- Latency optimization across model providers
Governance, Compliance, and Audit
- LLM audit trails: prompt logs, response logs, and decision provenance
- Data residency and privacy considerations for LLM APIs
- Policy-as-code for LLM usage within organizations
- Building an internal LLM operations playbook
Requisiti
- Experience building or integrating LLM-powered applications.
- Familiarity with Python and REST APIs.
- Basic understanding of prompt engineering concepts.
Audience
- ML engineers and MLOps practitioners transitioning to LLM operations.
- Platform engineers responsible for LLM infrastructure.
- Technical leads managing production GenAI deployments.
14 ore
Recensioni (2)
Lo stile interattivo, gli esercizi
Tamas Tutuntzisz
Corso - Introduction to Prompt Engineering
Traduzione automatica
Un ottimo archivio di risorse per un utilizzo futuro, stile dell'insegnante (pieno di buon umorismo e grande attenzione ai dettagli)
Adam - GE Aerospace Poland Sp. z o.o.
Corso - Prompt Engineering for ChatGPT
Traduzione automatica
