Loading…
20
Hit 3 seconds of lag on my home assistant running a 7B model, can local AI ever beat cloud?
Just set up a 7B model on my old gaming PC and the delay is exactly 3 seconds before it replies, which feels slow for basic voice commands. Some folks say local privacy is worth the wait, but others swear cloud speed is the only thing that matters for real use. Where do you draw the line between waiting a few seconds and giving up your data?
1 comments
Log in to join the discussion
Log In1 Comment
bettymurphy5d ago
That 3 second delay on a 7B model is pretty normal for older hardware, so dont feel like you did something wrong. What helped me was cutting the model down to a smaller one for basic voice stuff and saving the big model for when I actually need it. Local AI wont beat cloud speed on the same box, but if you split the work by task the wait stops bugging you so much.
6