- Batch inferencing
- Involves processing multiple inputs simultaneously
- Typically used for Non-time critical applications.
- It’s more efficient for large-scale processing
- Real-time inferencing
- It processes one input at a time
- Provide immediate responses
- Used on crucial applications like Autonomous vehicles