How to Run AI Roleplay Completely Locally — LLM, Images and Voice

Local AI roleplay in 2026 no longer means:

Open a terminal, install 48 Python packages, break CUDA, cry, reinstall Windows.

Okay. Sometimes it still does.

But today you can put together a system that runs entirely on your own computer:

The character thinks locally → responds locally → generates a scene image locally → speaks with a local voice.

Once the models are downloaded, you don't need the internet for any of this. No cloud APIs. The local components still communicate through APIs on your computer.

No token charges. No message quotas. No sending your chat history to somebody else's server.

And perhaps the most interesting part: the individual pieces of this stack can already produce something that doesn't feel like a technical demo for people who enjoy watching VRAM usage.

In this article, I'll put together a practical stack with four parts: LLM → frontend → image generation → TTS. And we'll look at what makes sense for 6, 8, 12 and 16+ GB of VRAM.

What are we actually building?

If you want to see the game around this stack first, the Multiic AI roleplay overview connects character cards, group scenes, memory and visual storytelling.

A simplified local AI character looks like this:

                 ┌─────────────────┐
Your message →   │       LLM       │
                 │ KoboldCpp/GGUF  │
                 └────────┬────────┘
                          │
                  Character response
                          │
             ┌────────────┼────────────┐
             ▼            ▼            ▼
         Interface      ComfyUI        TTS
                         Image        Voice
             │            │            │
             └────────────┴────────────┘
                          │
                        Scene

An important point: this is not one enormous AI model. It usually makes more sense to use several specialized models.

The LLM handles:

• Dialogue.
• Personality.
• Memory.
• Scene logic.

The image model handles:

• The character's appearance.
• Pose.
• Surroundings.
• Lighting.
• Camera.

TTS handles:

• Voice.
• Intonation.
• Emotion.
• Pronunciation.

The frontend connects everything. This modularity is what makes a local stack interesting. Don't like the voice? Change the TTS. Don't like the image style? Change the checkpoint.

Found a new RP model on Hugging Face at three in the morning? Drop in a new GGUF.

Ten minutes later, you're convincing yourself again that downloading a 9 GB model counts as scientific research.

My recommended stack

If you want the short answer first:

You don't have to use these exact programs. But it's a good starting point.

Part 1. A local LLM

Let's start with the brain.

For local RP, models in GGUF format are a convenient option. With a compatible runtime, you can run them:

• Entirely on GPU.
• Partly on GPU.
• With some weights in system RAM.
• Entirely on CPU, if you have the patience of a monk.

On Windows, KoboldCpp is a straightforward runtime for this.

KoboldCpp

KoboldCpp has grown from a simple llama.cpp launcher into something rather monstrous. It is a standalone executable that runs compatible GGUF models on CPU/GPU with full or partial GPU offloading and makes generation available through an API. The project also supports image generation, speech recognition, TTS and several compatible APIs. On Windows, the process goes roughly like this:

1. Download koboldcpp.exe.
2. Download a .gguf model.
3. Select the model.
4. Choose the context size.
5. Enable CUDA for a compatible NVIDIA GPU.
6. Launch.

No separate Python installation, conda, or ritual sacrifices to requirements.txt. That's one reason I recommend it to people who want to play, rather than earn a second degree in dependency management. KoboldCpp also provides an OpenAI-compatible API under /v1, so the frontend doesn't necessarily need to know that a local model is running underneath. A typical local address looks like this:

http://127.0.0.1:5001

The frontend sends a prompt there and receives text.

Which LLM should you choose?

I have a separate guide to local roleplay models, so I'll keep this part brief.

6 GB VRAM

Stheno 8B Q4 for English RP. Or a smaller quantization of Qwen3.5-9B if intelligence and multilingual capabilities matter more.

8 GB

Qwen3.5-9B Q4 as a general-purpose option.

For Russian, you can run: Saiga Nemo 12B Q4_K_M, but the model file itself is around 7.5 GB, so you'll need a small context or partial CPU execution.

12 GB

My favorite for Russian: Saiga Nemo 12B Q5/Q6.

For English: Rocinante-X / Impish Bloodmoon / other good Nemo RP fine-tunes.

16 GB+

Here you can start considering 20–30B models and MoE: Gemory 26B-A4B, Gemma 4, larger Qwen models.

The quantization and total weight size still matter. In an MoE model, the number of active parameters does not tell you how much memory all its weights require.

How much context should you use?

Don't automatically choose the maximum listed on the model page. If a model supports 128k, that doesn't mean:

Excellent, I'll set it to 128000.

Context uses memory through the KV cache. For ordinary roleplay, I'd start around:

8192–16384

And increase it only when needed. 16k of usable context is usually more helpful than 128k that pushes half the model into RAM and makes it respond at the speed of a contemplative turtle.

CPU execution isn't defeat

There's a strange idea that:

The whole model must fit on the GPU.

It doesn't have to. KoboldCpp lets you keep some layers in RAM and execute them on CPU. It will be slower. But RP doesn't require 100 tokens/s.

If the character responds at 8–20 tokens/s, that can already be comfortable. Especially with streaming enabled. The text appears gradually while the rest of the reply is being generated. Starting speech in the same way requires separate support for streaming TTS.

Part 2. The frontend

An LLM on its own doesn't manage:

• Character cards.
• Relationships.
• Persistent memory.
• Appearance.
• Images.
• Voices.
• The world.

The frontend usually handles these things. Two options:

• SillyTavern: if you want to assemble the setup yourself.
• Multiic: if you want text, images, characters, voice and a sandbox connected within one system.

SillyTavern

SillyTavern has become a standard frontend for local character chat. It supports:

• Character cards.
• Group chats.
• Lorebooks.
• Personas.
• Memory features.
• TTS.
• Image generation.
• KoboldCpp connections.
• API connections.

Its documentation describes KoboldCpp as a self-contained API for GGML/GGUF and includes an NVIDIA quickstart. If your goal is simply:

I want to chat with a character and assemble the whole stack myself.

It's a good option.

Multiic

Multiic takes a slightly different direction. The idea is to connect: LLM + images + voice + characters + scene state + sandbox. The LLM responds within a visual scene.

You can use:

• Local GGUF models.
• External APIs.
• A local image backend.
• ComfyUI.
• Local TTS.

The practical advantage becomes clear when several modalities work together. For example:

Player:
I walk to the window and ask Dina what she saw.

↓

LLM:
Generates Dina's reaction.

↓

Scene director:
Determines character position, expression and camera.

↓

ComfyUI:
Generates the scene.

↓

TTS:
Voices the line.

This is where AI roleplay starts to feel more like an endless visual novel than a chatbot.

Part 3. Local image generation

The next layer is images. If you want to tailor image generation to your characters and scenes, I'd choose ComfyUI. Not because its interface is especially friendly.

It looks like a nuclear reactor wiring diagram that somebody accidentally connected to an anime girl generator.

But it is incredibly flexible. That flexibility takes some learning: understanding the nodes, choosing compatible models, and building or configuring a workflow. You'll need to spend some time on it.

If you want to get started quickly and generate images in the game right away, KoboldCpp is the simpler option. It supports image generation and is already integrated into Multiic, so you can start without learning ComfyUI or assembling a node graph. You'll still need to download and select a compatible image model. I'd start with KoboldCpp and move to ComfyUI when I wanted more control: custom LoRAs, character references, poses and more elaborate workflows.

Why ComfyUI?

Instead of a fixed chain:

Prompt → image

You build a workflow:

Prompt
  ↓
Character reference
  ↓
Pose
  ↓
Checkpoint
  ↓
LoRA
  ↓
ControlNet
  ↓
Sampler
  ↓
Face detailer
  ↓
Upscale
  ↓
Image

The frontend can supply data to this workflow automatically. That's why ComfyUI is useful for RP. Build a workflow once, then let the game change:

• Prompt.
• Character.
• Seed.
• Reference.
• Pose.
• Clothing.
• Surroundings.

Which image model should you use?

Things change even faster here than with LLMs. In 2026, we can roughly divide image models into several groups.

Good old SD1.5

Advantages:

• Lightweight.
• A huge LoRA ecosystem.
• Relatively accessible character training.
• Fast generation.
• Usable on less powerful GPUs.

Disadvantages:

• Weaker anatomy.
• Weaker understanding of complex prompts.
• Resolution and composition lag behind modern models.

But with 4–6 GB of GPU memory, SD1.5 can still make sense. Especially if you already have good character LoRAs.

SDXL

SDXL is something of a classic now too. But it remains convenient for local RP. Advantages:

• Many checkpoints.
• Many LoRAs.
• Relatively moderate requirements.
• A useful speed/quality balance.
• Options for maintaining character consistency.

On 8 GB, it's one of the more approachable options, depending on workflow and other loaded models.

FLUX

FLUX improved understanding of ordinary text prompts. It's easier to describe:

A girl stands by a window, looking over her shoulder toward the camera. Her left hand touches the glass, with an evening city outside.

Instead of trying to write:

1girl, window, looking_back, hand_on_window,
night_city, masterpiece, best_quality...

Some newer FLUX variants are lighter. For example, the official ComfyUI guide for FLUX.2 Klein 4B reports around 8.4 GB VRAM for the distilled version on its test RTX 5090 and describes the model as intended for fast interactive workflows. That's an interesting direction for RP: Fast generation matters more than producing one perfect picture in two minutes.

Qwen Image and other instruction-based image models

Another important trend: image models are getting better at understanding ordinary sentences, references and editing instructions. For games, that's more convenient than old-style tag prompting. An LLM can produce:

Fiona is sitting sideways on a worn sofa, looking annoyed at the player. Warm light from the kitchen, medium shot, eye-level camera.

And send it to the image model. The LLM becomes a small scene director too. That's how I'd build the integration today.

Character consistency

Here's where the real pain of AI roleplay begins. Drawing a beautiful girl is easy. Drawing the same girl twenty times is much harder. Options:

Character LoRA

An established and still useful approach. Train a LoRA for the character and add it to the workflow. Advantages:

• Helps preserve identity.
• Small file.
• Fast to use.

Disadvantages:

• You need to train a LoRA for each character.

Reference adapters

More workflows and models let you supply one or several reference images. That's convenient for user-created characters. Save a few portraits, then use them when generating a new scene.

Image editing

An even more interesting option is editing an existing frame rather than generating everything again. For example:

Original image:
Dina is sitting on a sofa.

New scene:
Make her stand up and look toward the door.

In the long run, I think this could be the most convenient approach for AI visual novels. It can help preserve:

• The person.
• Clothing.
• The room.
• Lighting.

Part 4. A local voice

There has been a big leap here over the last two years. Local TTS used to mean:

Technically human speech.

Now there are models good enough for games. I'd divide them into three groups.

Option 1: Supertonic 3 — speed

If you need very fast local TTS, Supertonic 3 is still interesting. It uses ONNX and was designed for fast on-device inference. Supertonic 3 was released in April 2026 with 31 languages, improved reading accuracy and fewer repetitions or skipped words. Its main advantage:

Latency.

That's important for roleplay. If a character writes a line and you wait five seconds for audio to begin, the magic dies quickly. Fast TTS helps playback start sooner.

Important update

The original open-source Supertonic project was archived in September 2026, and official development and support have ended. The models and ONNX assets remain available and can still run locally, but you shouldn't count on ongoing official support. That doesn't make the model worse. But I'd account for it when starting a new project.

Option 2: Kokoro — small and fast

Kokoro is another option when you care about:

• Small size.
• CPU inference.
• Low latency.

Kokoro has just 82M parameters, and local implementations can work offline after downloading the necessary files. SillyTavern supports local Kokoro through kokoro.js, including WebGPU in supported browsers. For a simple character voice, it's an inexpensive option in terms of resources.

Option 3: Qwen3-TTS — for voice cloning

This is where things get more interesting. Qwen3-TTS was released in January 2026 in 0.6B and 1.7B variants. The official models support ten languages:

• Chinese.
• English.
• Japanese.
• Korean.
• German.
• French.
• Russian.
• Portuguese.
• Spanish.
• Italian.

Across its model variants, the family offers:

• Streaming speech.
• Voice cloning.
• Voice design.
• Voice control through natural-language instructions.

The most interesting feature for RP is voice cloning. The official 0.6B Base checkpoint supports cloning from roughly three seconds of reference audio. You can give each character a voice without training a full model.

Why this matters for RP

The older TTS approach:

Dina → voice 7
Fiona → voice 11
Laura → voice 3

The newer approach:

Dina → reference_dina.wav
Fiona → reference_fiona.wav
Laura → reference_laura.wav

TTS then tries to preserve that vocal identity. With a suitable Qwen3-TTS variant, you can also give directions such as:

Speak quietly, slightly irritated,
trying not to show that she is scared.

The 1.7B CustomVoice and VoiceDesign variants support instruction-based style control; the Base variant uses a different voice-cloning mode. For an audiobook, that's cool. For roleplay, it's very cool. Because the LLM already has information about the emotional state.

You can build:

LLM:
emotion = angry
intensity = 0.65

↓

TTS:
Speak in a restrained but clearly angry voice.

The character does more than simply read the text.

What about XTTS-v2?

XTTS-v2 still exists and still works. Its main advantage: The surrounding infrastructure already exists. There are API servers, a SillyTavern integration and plenty of guides.

SillyTavern documents a local XTTSv2 server and voice cloning using roughly ten-second WAV samples. But today I'd look at Qwen3-TTS first. XTTS is more of a:

I need an established pipeline I already know.

TTS comparison

The stars are my approximate RP assessment, not a shared benchmark. Speed and quality depend on hardware, language and integration.

The important part: they all compete for one GPU

Now for the part many tutorials forget. You have:

LLM
Image model
TTS

All three want VRAM.

The first two look at each other like two cats staring at the only cardboard box.

For example, an RTX 3060 with 12 GB

Imagine Saiga Nemo Q5 taking roughly:

~9 GB

And SDXL/FLUX wanting several more gigabytes. They no longer fit comfortably together. What can you do?

Option 1: Load and unload models dynamically

This makes sense if your integration can manage model loading. Two separate servers won't automatically unload models for each other.

Chat:
LLM is on the GPU.

↓

An image is needed:
Unload some or all of the LLM.
Load the image model.
Generate the image.
Release the image model.

↓

Bring the LLM back.

Not ideal for latency. But it lets you use capable models on an ordinary consumer GPU.

Option 2: Keep part of the LLM in RAM

For roleplay, this can be a useful compromise. For example:

• 70% of LLM layers → GPU.
• The rest → CPU/RAM.
• Some VRAM remains available for ComfyUI.

Text generation becomes slower. But images may not require reloading the whole setup every time.

Option 3: Two GPUs

If you have a second NVIDIA card lying around, a local multimodal stack becomes more convenient. For example:

GPU 0:
LLM

GPU 1:
ComfyUI

TTS can stay on CPU or use one of the GPUs. This can reduce waiting between text and images.

Option 4: Generate images on request

An image does not need to be generated after every line. That can actually get annoying. Generate one when:

• The location changes.
• Clothing changes.
• A different character appears.
• An important action happens.
• The player asks for an image.

An LLM can help identify these moments automatically. This can cut the load substantially.

How much hardware do you really need?

Very roughly. These are planning directions, not guarantees that every model will fit in VRAM simultaneously:

VRAM isn't everything. I'd also want:

RAM

16 GB as a starting minimum. 32 GB is more comfortable. 64 GB helps with larger GGUF models and partial CPU execution.

SSD

AI models consume disk space as if it were free. One LLM: 5–20 GB. Image models + text encoders + VAE:

Another few dozen gigabytes. LoRAs: another few dozen as your collection grows. ComfyUI output after a couple of months:

Where did these 180 GB of pictures come from?

So a dedicated 500 GB–1 TB for models can disappear surprisingly quickly.

Can everything run without internet?

Yes. With an important qualification: After all the necessary models and files are downloaded. KoboldCpp runs locally.

ComfyUI with local checkpoints runs locally. Supertonic/Kokoro/Qwen3-TTS can run locally. The frontend can communicate with them through:

localhost
127.0.0.1

The data doesn't have to leave the computer.

How to check that your setup works offline

The simplest test:

Disconnect the internet.

Seriously. It's a more direct test of offline operation than reading twenty privacy policies. If afterwards:

• The LLM responds.
• ComfyUI generates images.
• TTS speaks.

Then the main pipeline can work locally. To check network requests while connected, inspect network activity separately. Make sure the programs aren't waiting to download models on their first request.

An example of a fully local stack

If I were assembling a simple Windows + NVIDIA setup, it would look roughly like this:

Frontend:
Multiic / SillyTavern

↓

LLM:
KoboldCpp
localhost:5001

Model:
Saiga Nemo 12B Q4/Q5
or Qwen3.5-9B

↓

Images:
ComfyUI
localhost:8188

Model:
SDXL / FLUX / Qwen Image workflow

↓

Voice:
Qwen3-TTS
or Supertonic/Kokoro

localhost:<TTS port>

That's it. From the outside, it looks like one AI character. Inside, three completely different neural networks are working and pretending to be one person.

Honestly, quite relatable.

What one turn looks like

For example, you write:

Dina, you knew he was coming, didn't you?

Step 1

The frontend assembles the prompt:

System
Character card
Memory
Current scene
Recent dialogue
User message

Step 2

KoboldCpp → Saiga Nemo:

Dina looks away.

“Maybe. What difference does it make now?”

Step 3

The frontend extracts scene state:

character: Dina
emotion: defensive
location: kitchen
pose: looking away
camera: medium shot

Step 4

It builds an image prompt:

Dina standing in a dim kitchen at night,
looking away from the viewer,
defensive expression,
arms slightly crossed,
medium shot,
warm practical lighting

Step 5

ComfyUI generates the image.

Step 6

The TTS integration receives:

text:
Maybe. What difference does it make now?

voice:
Dina

emotion:
restrained, defensive, slightly angry

It translates these fields into whatever the selected voice model supports.

Step 7

Audio starts playing. That's already quite close to a small procedural visual novel.

Character memory

Another mistake: Trying to store the character's entire life directly in context. That tends to work poorly. It's better to separate:

Character card

Things that rarely change:

Personality
Biography
Speaking style
Relationships

Scene state

What's happening now:

Location
Clothes
Emotion
Present characters
Time

Long-term memory

Important events:

Player lied to Dina about Josephine.
Dina discovered it later.
Trust decreased.

Recent messages

The ordinary context of recent exchanges. This gives the LLM a more useful prompt without turning context into an 80k-token dumpster.

Images need memory too

This is especially important. If the LLM knows:

Dina is wearing a red jacket today.

But the image prompt is built only from the current line:

Dina looks angry.

A minute later, she has:

• A new face.
• A green dress.
• Three extra centimeters of cleavage.
• Possibly a tail.

The image director should receive structured state, not just the response text. For example:

{
  "character": "Dina",
  "hair": "curly dark hair",
  "clothes": "oversized red jacket",
  "location": "kitchen",
  "emotion": "annoyed"
}

This helps improve visual consistency.

Keep TTS directions separate from dialogue

Don't ask the LLM to write:

[sad voice]
I don't know...

And then feed the bracketed directions directly into TTS. Get separate metadata instead:

text: "I don't know."
emotion: sad
energy: low
speed: 0.9

Let the integration translate this into supported voice controls. Suitable Qwen3-TTS variants allow natural-language instructions for style control.

Could you do everything through KoboldCpp?

Funny enough, you can do quite a lot. Modern KoboldCpp supports more than GGUF text generation, including:

• Image generation.
• Image editing.
• Whisper.
• TTS.
• Vision.
• Several API-compatible endpoints.

A minimalist setup can be built largely around one backend. That's appealing for a beginner. But I still prefer: KoboldCpp for the LLM + ComfyUI for images + separate TTS.

Why? Because the components evolve at different speeds. If an excellent image model appears tomorrow, I want to replace one workflow instead of waiting for an entire monolithic backend to support it.

The simplest solution

If, after reading all this, you're thinking:

Great. Now I just need to install five programs and qualify as a system administrator.

Yes. Fair point. That's why projects connect these components for you. Multiic, for example, brings together:

• Local GGUF models.
• KoboldCpp.
• Local image generation.
• ComfyUI workflows.
• Local TTS.
• Character memory.
• Visual scenes.

Through one interface. Connection settings are covered in Multiic's help. Instead of assembling:

SillyTavern
 + KoboldCpp
 + ComfyUI
 + TTS server
 + Your own scripts

You can use a connected setup. The philosophy underneath is the same. Several specialized models work together.

What I'd install today

Less powerful PC / 6 GB VRAM

LLM:
Stheno 8B Q4

Images:
SD1.5

Voice:
Kokoro / Supertonic 3

Frontend:
Multiic or SillyTavern

A relatively lightweight stack.

8 GB VRAM

LLM:
Qwen3.5-9B Q4
or Saiga Nemo Q4 with partial CPU execution

Images:
SDXL

Voice:
Qwen3-TTS 0.6B / Kokoro

Frontend:
Multiic / SillyTavern

In my view, a useful configuration—with the memory planning discussed above.

12 GB VRAM

My favorite sweet spot:

LLM:
Saiga Nemo 12B Q5/Q6

Images:
SDXL / modern ComfyUI workflow

Voice:
Qwen3-TTS

Frontend:
Multiic

Plan to load and unload the LLM and image models dynamically when needed.

16–24 GB

More room to experiment:

LLM:
20–30B / MoE model

Images:
FLUX / Qwen Image / modern editing workflow

Voice:
Qwen3-TTS 1.7B

Character:
Multiple references / LoRA

Context:
16–32k+

This is where the results can get interesting. Exact choices still depend on combined memory use.

How much does it cost?

If you already have the computer:

$0 in cloud inference fees per message.

After downloading the models. That's one of local AI's main advantages. During hours of RP, a cloud LLM repeatedly processes:

System prompt
Character
Memory
History
Current scene

Those input tokens can contribute to the bill. Then there's image generation. Then TTS. Locally, the cost becomes roughly:

Electricity + hardware + time spent explaining to CUDA who's in charge.

Is the quality worse than the cloud?

Sometimes, yes.

A large frontier cloud model will often outperform a small 8B model in:

• Reasoning.
• Following complex instructions.
• Managing complicated plots.

Cloud image models can also produce excellent results with little setup. But a local stack offers other advantages:

Freedom

You can change the components.

Privacy

With local connections and storage, history can stay on your machine.

Cost

You're not counting every message against an inference bill.

Latency

Lightweight TTS can be particularly responsive.

Customization

LoRAs, fine-tunes, character voices and your own workflows.

Reproducibility

An already downloaded model won't disappear because a company replaces its hosted version. A downloaded .gguf can't undergo a rebrand.

And that's wonderful.

Moving away from Character.AI? Here's a comparison of nine Character.AI alternatives — from hosted services to fully local setups.

Wrapping up

In 2026, fully local AI roleplay is something you can actually build. You can put together a local LLM + a persistent character + images + voice cloning on an ordinary gaming computer, with suitable model choices and resource management.

You don't need a server cluster. You don't need a cloud API key. You don't need a subscription for local inference. A starting point:

KoboldCpp
+
8–12B GGUF
+
ComfyUI
+
Lightweight TTS

From there, you can keep improving the system. Add:

• A character LoRA.
• Reference images.
• Memory.
• Emotion detection.
• Voice cloning.
• Image editing.
• Multiple characters.
• World state.
• Local video generation.

At some point, you realize you were assembling a chatbot and accidentally built a small procedural game engine.

That's roughly how Multiic came about.