Router Configuration
Configure routers to optimize for your specific needs.
ModelPilot routers intelligently select the best AI model for each request based on your optimization preferences. Configure routers to balance quality, cost, speed, and environmental impact.
Router Modes
Automatically selects the best model for each request using Thompson Sampling and Bayesian optimization. Learns from production data to improve selection over time.
- Multi-objective optimization (quality, cost, latency, carbon)
- Adapts to different prompt types and complexity levels
- Continuous learning from real usage
Routes all requests to one specified model. Useful for testing, debugging, or when you need consistent model behavior.
When to use: Testing specific models, legacy integrations, compliance requirements
Optimization Weights
Smart Router mode only. Configure how the router balances different objectives using sliders in the dashboard. Weights must sum to 100%.
Default Configuration
- • Quality: 40% - Focus on response accuracy
- • Cost: 30% - Balance affordability
- • Speed: 20% - Ensure timely responses
- • Environment: 10% - Consider carbon impact
Prioritizes response quality and accuracy. Higher values prefer premium models even if more expensive or slower.
Prioritizes low-cost models. Higher values prefer cheaper models even if slightly lower quality or slower.
Prioritizes fast response times. Higher values prefer models with lower latency.
Prioritizes environmental impact. Higher values prefer models with lower CO₂e emissions.
Example Configurations
Use the dashboard sliders to set:
Best for: High-volume applications, background processing, cost-sensitive projects
Use the dashboard sliders to set:
Best for: Critical applications, content creation, research tasks
Use the dashboard sliders to set:
Best for: Chat applications, live assistance, interactive tools
This is the default configuration when you create a new router:
Best for: General applications, getting started, balanced needs
Hard Constraints
In the router configuration interface, you can set:
Capabilities
- • Function Calling: Require models that support function/tool calling
- • Multimodal: Require image/audio processing capabilities
- • Streaming: Require streaming response support
- • Structured Output: Require JSON output formatting
Performance Limits
- • Max Latency: Set maximum acceptable response time (e.g., 2000ms)
- • Max Cost per Token: Set cost ceiling per token (e.g., $0.00001)
- • Max Tokens: Limit response length for cost control
Models that don't meet these requirements will be automatically filtered out before the router makes its selection.
Available Models
Important: For Smart Router mode, it's recommended to leave the model selection at the default "All Available Models" setting. This allows the router to automatically access new models as they're added and make optimal selections from the complete catalog.
When to Limit Models
Only specify specific models if you have:
- • Compliance requirements - Data residency or regulatory constraints
- • Testing needs - Compare specific models side-by-side
- • Technical requirements - Models must support specific features
- • Cost controls - Exclude expensive premium models
💡 Tip: The dashboard shows real-time model availability and performance metrics to help you make informed decisions.
Best Practices
- ✓Start with default configuration
Use the balanced default weights initially, then adjust based on your specific needs through the dashboard
- ✓Use Smart Router for production
Let the router optimize and learn from real usage patterns
- ✓Monitor router analytics
Check the dashboard regularly to track cost, quality, and latency metrics for fine-tuning
- ✓Create separate routers per environment
Use different configurations for development, staging, and production through the dashboard
- ✓Test configuration changes
Use the router's test feature to validate changes before applying to production