After You Export

You trained a model and exported it. Now what? This guide covers your options for using it inside TuneSalon, on your local machine, or deployed to a live website or service.

Your Two Exports

When training finishes on TuneSalon, you have two ways to export your work. Each serves a different purpose.

Think of the .adapter as a lens that clips onto a camera. It only works with the right camera body (the base model). The .gguf is more like a fully assembled camera, ready to use on its own.

Using Within TuneSalon

If you just want to use your model for chatting or testing, you don't need to set up anything external.

Chat Tab (Web)

Go to the Chat tab, load the base model, and attach your .adapter file. You can load adapters from your Library or upload from your computer. You can even stack up to 5 adapters at once to combine different skills.

TuneSalon Desktop (Local)

The free desktop app lets you run your .gguf file entirely on your own hardware. Nothing leaves your computer. This is the best option if privacy is a priority and you have a GPU with enough memory to run the model.

For most people using their model personally or testing it before deployment, these two options are all you need. The rest of this guide is for when you want your model running on a live website or service.

Deploying to Your Website or Service

Let's say you've fine-tuned a customer support model and you want it answering questions on your website, 24 hours a day. Or you've built a legal writing assistant and want it available as an API that your team's tools can call.

To make this work, your model needs to run on a server that's always on and fast enough to generate responses in real time. This requires a cloud GPU, because AI models need specialised hardware to run at a reasonable speed.

The good news is that you don't need to build this from scratch. There are platforms designed specifically for hosting AI models. You upload your .gguf file, and they give you an API endpoint: a URL that your website or app can send questions to and receive answers from.

Option A: Hosted Platforms

This is the easiest path. These platforms handle the GPU hardware, server management, and scaling for you. You upload your model file and get an API endpoint.

Here are some of the platforms that support custom GGUF model hosting. Each has its own pricing, features, and setup process. We'd encourage you to explore them and pick what fits your needs.

These are some of the well-known options, but the space is growing quickly. Search for "GGUF model hosting" or "custom LLM inference hosting" to find more.

Option B: Self-Hosted (Advanced)

If you have technical experience and want full control over your infrastructure, you can rent a cloud GPU and run the inference server yourself. This gives you more flexibility over configuration, cost optimisation, and scaling, but requires hands-on setup.

The general approach is:

1. Rent a cloud GPU from a provider like Modal, RunPod, Lambda Labs, Vast.ai, or a major cloud (AWS, GCP, Azure).
2. Set up an inference server using software like llama.cpp (which natively supports GGUF files) or vLLM.
3. Expose an API endpoint that your website or service can call.

TuneSalon itself uses Modal for cloud GPU workloads. Modal lets you define GPU functions in Python and deploy them as serverless endpoints, meaning you only pay when your model is actually processing requests.

A Note on Model Size

Larger models produce higher-quality responses but require more powerful (and more expensive) GPU hardware to run. When choosing where to deploy, you'll need a GPU with enough memory to hold your model.

The Model Guide lists the memory requirements for each model at full precision. For deployment, you can use quantization to reduce the memory needed. Quantization compresses the model into a smaller format, which lowers the GPU requirement. The trade-off is that model quality may decrease slightly, especially at higher compression levels.

Most hosting platforms let you choose a GPU tier when you deploy. Pick a GPU that comfortably fits your model. The platform's documentation will help you match your model size to the right hardware.

Connecting to Your Website

Once your model is deployed on a hosting platform (either Option A or B), you'll have an API endpoint: a URL that accepts messages and returns your model's responses.

The pattern is straightforward: your website sends a request to the endpoint with the user's message, waits for the model to generate a response, and displays it. Most hosting platforms provide an OpenAI-compatible API, which means the way you send and receive messages follows a widely-used standard.

Each platform provides its own integration guide with code examples for connecting from a website, mobile app, or backend service. Here are the relevant docs:

• Replicate: Get started with Node.js
• Hugging Face: JavaScript inference client
• OpenAI Chat API reference (most platforms follow this same format)

Next Steps

You've trained a custom AI, exported it, and now know how to take it from TuneSalon to the rest of the world. Here are some related resources:

• Model Guide — Find the right model for your use case and check memory requirements.
• GPU Guide — Understand the difference between A100 and H200 cloud GPUs on TuneSalon.
• Our Method — How LoRA adapters work and why they're efficient.
• Train a model — Ready to build something new? Start training.