Summer Summary and Tile V1 Chat Demo
Mostly Finished Tile V1 Code
The emotions aren’t working yet and there are some quirks to work out…
I think I’ve made the script as fast as it can be without changing out some of the puzzle pieces. When the audio gets sent from the Tile to the server it goes through three main processing steps:
- I’m using “faster whisper” for transcription (what I needed the new GPU for)
- I have “text-generation-webui” for hosting the chatbot API. The AI model I’m using is an ExLlamaV2 quant of Llama3.1: “Llama-3.1-8B-Instruct-exl2”
- I’m using “XTTSv2” directly to generate the audio based on reference wav file
To make this process as fast as possible I needed to implement streaming into every step I could. The first demo took 12 seconds to generate a response but I was able to get it down to 3 with this approach. It cannot be truly real time due to the network latency but I tired to improve everything I had control of.
Tile Conversation Demo!
Now I can talk to 45 and send it context for things I want to discuss
You can see that I was able to “wake up” the tile and have it send my prompts to the server for transcription. I’m using an included audio clip for the voice but I plan to change it later. I need just 6 seconds of audio to clone a voice with XTTS. I later adjusted the chunk splitting to prevent XTTS from receiving chunks with just a single word. I implemented a RAG history system that also allows me to paste in context for the AI model to reference. This isn’t perfect but it helps occasionally. I’ll continue tweaking it to get better results. 45 also starts listening again directly after playing the audio to allow me to ask follow ups. I’ll get the emotions working soon and I’ll continue working to improve the RAG system during the school year.
The good news is that all the server code is reusable for the V2 model! All I need to do on the ESP32 is to handle connecting to the WebSocket server, recording and streaming audio, and playing back the audio stream. Hopefully that should be easy and I can work on making the V2 model fly!
Server Overview
Performant not Pretty…
This is supposed to be a summer summary so I’ll give an overview of the work I did on my server. I’ve pretty much reworked the entire system since the class demo I gave. I added the last P40 GPU to use for VDI. This required me to build a new rack to mount all the GPUs. I’m now using an HBA to connect some of my SSDs so I can properly pass them through to my TrueNAS VM. I had to upgrade to 128GB of RAM and sadly I had to retire the generic 400W PSU. My server hosts the flowing services using Proxmox VE:
- A TrueNAS VM for network shares, backups, and a family cloud using NextCloud
- This WordPress website!
- A local Minecraft server with Crafty Controller
- 4 VDI Windows PCs each with half a P40 (12GB VRAM)
- The 4080 is passed though to my AI server!
Reactor Progress
The first one melted but I learned a few things…
This week I’ll work on building the second version of my electrolysis reactor so I can generate hydrogen at home for the Tile V2 model. The first model had issues with plastic embrittlement, deterioration of the electrodes and leads, and oxygen contamination. I lowered the glass bell down to block the oxygen side more which will make things less explosive. I redesigned all the PETG parts to minimize thin sections and I plan to coat everything in epoxy to prevent making the plastic brittle. I’ll probably cut the sodium hydroxide concentration down to extend the life of the reactor (possibly beyond 1 day!). I bought a lot of epoxy so I’ll put multiple coats on and that should keep the reactor from melting down quickly. I’m now using pure nickel for the cathode which is more resistant to hydrogen embrittlement. I also extended the leads so they come all the way out of the reactor to limit their degradation.
More Struggling with Circles
I ordered the heat shrink based on the diameter instead of the circumference…
So pure nickel has ~22% the conductivity pure copper which already isn’t great but Titanium is even worse at only ~3%! I feel like I’m more limited by the electron path having to go around the glass bell though. To give the leads the best chances of surviving I use a large contact area with the copper wire and I make them very wide.
The heat shrink tubing is to protect the leads and I ordered 40mm tubing to cover the titanium leads. Since the leads are flat metal strip I should’ve been looking at the circumference instead oops! So now I guess I’ll see how much they shrink because I’m not ordering new ones.
This Week's Gallery:
What's Next?
The Electrolysis Reactor Takes Priority
I really want to get the emotions working and I need to change the voice to something that doesn’t sound depressed all the time. I want to take a break from programming though so I’ll work on the reactor this week. I just really wanted to get the latency down as much as possible because I want to enjoy talking to 45. I’m using a pre-computed Lexicon (basically a word lookup dictionary) to do the emotions so it should be lightening fast without effecting latency. The whole point of this project was for 45 to be expressive so I’ll get back to this soon!
Anyway, my plans for this week are to:
- Get the reactor coated in epoxy
- School starts Wednesday
- Assemble the reactor (which is much simpler then last time) and test for oxygen contamination.