Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 14 hours
Course Outline
Foundations of Multimodal LLMs in Vertex AI
- An overview of multimodal features available in Vertex AI
- Introduction to Gemini models and their supported modalities
- Key use cases in enterprise and research settings
Preparing the Development Environment
- Setting up Vertex AI for multimodal processing workflows
- Managing datasets across different modalities
- Practical lab: establishing the environment and preparing datasets
Long Context Windows and Complex Reasoning
- Concepts behind long-context workflow design
- Applications in planning and decision-making processes
- Practical lab: executing long-context analysis
Architecting Cross-Modal Workflows
- Merging text, audio, and image analysis techniques
- Sequencing multimodal steps within pipelines
- Practical lab: constructing a comprehensive multimodal pipeline
Managing Gemini API Parameters
- Configuring multimodal input and output settings
- Strategies for optimizing inference speed and efficiency
- Practical lab: adjusting and tuning Gemini API parameters
Advanced Implementations and System Integrations
- Developing interactive multimodal agents and assistants
- Connecting external APIs and third-party tools
- Practical lab: building a fully functional multimodal application
Performance Evaluation and Iterative Improvement
- Methods for testing multimodal model performance
- Evaluating metrics related to accuracy, alignment, and drift
- Practical lab: assessing and refining multimodal workflows
Conclusion and Future Directions
Requirements
- Strong proficiency in Python programming
- Practical experience in developing machine learning models
- Working familiarity with multimodal data types, including text, audio, and images
Target Audience
- AI researchers
- Senior software developers
- Machine learning scientists
