Get in Touch

Course Outline

Introduction to Open-Source LLMs

  • What open-weight models are and their significance
  • Overview of LLaMA, Mistral, Qwen, and other community-driven models
  • Use cases for private, on-premise, or secure deployments

Environment Setup and Tools

  • Installing and configuring Transformers, Datasets, and PEFT libraries
  • Selecting appropriate hardware for fine-tuning
  • Loading pre-trained models from Hugging Face or other repositories

Data Preparation and Preprocessing

  • Dataset formats (instruction tuning, chat data, text-only)
  • Tokenization and sequence management
  • Creating custom datasets and data loaders

Fine-Tuning Techniques

  • Standard full fine-tuning versus parameter-efficient methods
  • Applying LoRA and QLoRA for efficient fine-tuning
  • Using the Trainer API for rapid experimentation

Model Evaluation and Optimization

  • Assessing fine-tuned models with generation and accuracy metrics
  • Managing overfitting, generalization, and validation sets
  • Performance tuning tips and logging

Deployment and Private Use

  • Saving and loading models for inference
  • Deploying fine-tuned models in secure enterprise environments
  • On-premise vs. cloud deployment strategies

Case Studies and Use Cases

  • Examples of enterprise use of LLaMA, Mistral, and Qwen
  • Handling multilingual and domain-specific fine-tuning
  • Discussion: Trade-offs between open and closed models

Summary and Next Steps

Requirements

  • Familiarity with large language models (LLMs) and their architecture
  • Proficiency in Python and PyTorch
  • Basic understanding of the Hugging Face ecosystem

Target Audience

  • Machine Learning practitioners
  • AI developers
 14 Hours

Number of participants


Price per participant

Upcoming Courses

Related Categories