Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 14 hours
Course Outline
Foundations of Multimodal LLMs in Vertex AI
- An overview of multimodal capabilities offered by Vertex AI
- Introduction to Gemini models and the modalities they support
- Exploration of enterprise and research use cases
Preparing the Development Environment
- Configuring Vertex AI for multimodal operations
- Managing datasets spanning multiple modalities
- Practical lab: setting up the environment and preparing datasets
Long Context Windows and Sophisticated Reasoning
- Concepts behind long-context workflows
- Applications in planning and decision-making processes
- Practical lab: implementing long-context analysis techniques
Architecting Cross-Modal Workflows
- Synthesizing text, audio, and image analysis
- Orchestrating multimodal steps within pipeline sequences
- Practical lab: designing a comprehensive multimodal pipeline
Managing Gemini API Parameters
- Setting up multimodal input and output configurations
- Strategies for optimizing inference speed and efficiency
- Practical lab: fine-tuning Gemini API parameters
Advanced Implementations and System Integrations
- Developing interactive multimodal agents and assistants
- Connecting external APIs and third-party tools
- Practical lab: constructing a fully functional multimodal application
Performance Evaluation and Refinement
- Methods for testing multimodal system performance
- Evaluating metrics related to accuracy, alignment, and data drift
- Practical lab: assessing the effectiveness of multimodal workflows
Recap and Future Directions
Requirements
- Strong proficiency in Python programming
- Practical experience in developing machine learning models
- A solid understanding of multimodal data types, including text, audio, and images
Target Audience
- AI researchers
- Senior developers
- Machine Learning scientists
