Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 7 hours
Course Outline
Transitioning from Text Generation to Typed Decisions
- Nature and output of decision model responses.
- Contrast between System One decisions and reasoning LLMs.
- Analysis of cost and latency profiles for bounded decisions.
The Landscape of Open-Weight Decision Models
- AWS Labs' Strands Decider 2B.
- The JEV and GEV open-weight model families.
- Overview of Pplx-Decider, Liquid D1, and Clef.
- Differences in licensing and deployment strategies.
Architectural Insights into Decision Models
- Utilizing an LLM torso without a standard language modelling head.
- Scoring candidate options via the pointer head mechanism.
- Origin and calculation of confidence scores.
Executing Initial Decisions
- Local setup on CPU and consumer-grade GPUs.
- Structuring questions and defining option sets.
- Processing multiple queries in a single pass.
- Managing thresholds and fallback behaviors.
Evaluating Decision Quality
- JevBench and other relevant decision benchmarks.
- Trade-offs between accuracy and calibration.
- Assessing confidence reliability using Brier scores.
- Identifying tasks where decision models are unsuitable.
Requirements
- Fundamental knowledge of large language models and prompt-driven applications.
- Proficiency with command-line interfaces and Python scripting.
- No prior experience in machine learning training is necessary.
Target Audience
- AI engineers assessing decision models for agent workflows.
- Software developers looking to substitute LLM calls with bounded decisions.
- Solution and platform architects designing agent-based systems.
- Technical product managers evaluating decision model capabilities.
