WORKSHOP
The Open Source Jarvis: Build Your Own Private Voice Assistant (WS10)
Date: 08 October 2026 | Time: 03:30 06:00 PM
Venue: Workshop Room 1, NIMHANS Convention Centre, Bangalore
FEES:
• Rs.299 for Leaders Pass holders
• Rs.2009 for Professionals Pass holders
• Rs.2249 for Knowledge Pass holders
• Rs.2699 for Community++ or Student Pass holders
• Rs.2999 for all others
(Limited seats available)
Show attendees how to build an offline AI voice assistant from scratch using open-source models. By chaining open speech-to-text, local LLM reasoning, and text-to-speech, participants will build a private assistant capable of running daily automations without third-party APIs. Attendees will get hands-on experience running inference directly against an on-site NVIDIA DGX Spark, giving them a first-hand look at how a compact AI supercomputer powers local reasoning in real time.
Sayed Imran
Site Reliability Engineer, 66degrees
- Developers and open-source enthusiasts who want to build private, local AI tools.
- 1. The Open-Source Voice Architecture & Core Theory
- 1.1 Overview of the pipeline: Voice Input to Audio Capture to STT to Local LLM to Action to TTS.
- 1.2 Architecture trade-offs: Laptop-level inference vs. running on a dedicated desktop AI supercomputer.
- 1.3 LLM Serving Theory: Understanding the architectural differences between llama.cpp (C/C++ engine, GGUF quantization, slot-based batching) and vLLM (Python engine, PagedAttention memory management, continuous batching).
- 1.4 TTS Theory: How modern neural text-to-speech works, focusing on the Kokoro-82M architecture and how OpenAI-compatible API endpoints stream expressive voices without high overhead.
- 2. Capturing Audio and Speech-to-Text (STT)
- 2.1 Setting up fast, lightweight speech recognition using Whisper.cpp or faster-whisper.
- 2.2 Optional: Detecting speech cleanly with Voice Activity Detection (VAD).
- 3. Running the Brain on DGX Spark
- 3.1 Introduction to the NVIDIA DGX Spark architecture for local LLM inference.
- 3.2 Serving strategies and benefits: When to deploy llama.cpp (best for memory efficiency and single-user low latency) vs. vLLM (best for maximum throughput and concurrent multi-user requests).
- 3.3 Connecting the pipeline: How attendees will configure and point their local AI agents to utilize the LLMs running on the centralized DGX Spark over the local network.
- 3.4 Teaching the model to trigger system tasks (creating files, launching scripts, checking tasks).
- 4. Speech Output: Fast Local TTS
- 4.1 Generating clean, low-latency audio using Kokoro-FastAPI.
- 4.2 Exploring the benefits of a Dockerized, OpenAI-compatible wrapper for the Kokoro-82M model.
- 4.3 Streaming audio responses and utilizing multi-speaker voice mixing so the bot speaks naturally without lag.
- 5. Putting It All Together
- 5.1 Connecting STT, the DGX Spark LLM backend, and Kokoro-FastAPI into a single Python app.
- 5.2 Live build: Automating a daily workflow.
Benefits/Takeaways of this workshop for the attendees (What will attendees do after attending the workshop which they were not able to do before attending this)
- First-hand experience running real-time model inference against a physical NVIDIA DGX Spark supercomputer.
- A working Python repository you can adapt for both lightweight laptop setups and dedicated hardware.
- Hands-on experience connecting speech pipelines and local LLMs to system tools and custom scripts.
- Practical knowledge of latency, memory management, and network orchestration for offline AI systems.
About Speakers
Sayed Imran
Highly motivated Cloud & DevOps Engineer with a passion for streamlining software delivery and infrastructure management. Possess certifications in Kubernetes, AWS, and Google Cloud Platform, demonstrating a strong foundation in various cloud environments.
He has deep expertise in Kubernetes and Istio Service Mesh, actively involved in developing Kubernetes Operators for automation and enhancing infrastructure efficiency. Proven ability to automate tasks and implement solutions that foster faster deployments and smoother operations.