Choose where the system runs.
On premises means running the agreed components on hardware at your organisation’s site. A customer-controlled private cloud uses infrastructure in an account you control, with agreed network, identity, region and provider arrangements. The choice affects hardware access, connectivity, support and day-to-day ownership.
Map the complete data flow: who can submit a request, which documents each person can retrieve, what is logged, where backups go and what administrators or support staff can access. Any external connector or hosted component needs its own agreed boundary. Deployment location alone does not establish privacy or security.
Connect the model to a useful task.
An internal document assistant can retrieve authorised material, show the sources used and help staff prepare an answer. Source permissions need to carry through retrieval, and the workflow should handle missing information and uncertain answers with review or escalation.
For customer support, a first version can draft a reply, suggest a category or route a request to the right person. Agree which actions require approval and how a person takes over. Text, images, audio and document inputs depend on the selected model and any OCR, speech or other processing components; each supported input path needs testing.
Select, size and test for your workload.
Bring a sample or redacted task, the data types and document sizes, expected request volume, peak concurrent users and a useful response-time target. We assess model memory, GPU capacity, storage, networking and the supporting software against that workload. Existing hardware compatibility is established through assessment and testing.
Kimi K3 and specific GLM releases are examples to assess only when the exact model’s current licence, commercial-use terms, supported inputs, hardware and tested quality fit. Open weights do not by themselves establish an open-source licence or unrestricted commercial use. Kimi K3’s published serving recipe requires substantial GPU infrastructure, with multi-node configurations for production traffic.
Evaluate representative tasks before rollout: answer quality, source accuracy, language coverage, latency, concurrency, failure handling and the review effort your team needs. Keep acceptance examples separate from configuration examples and agree when the system should hand work to a person.
Official model and serving references:
Plan the full cost and the ongoing work.
Self-hosted components can reduce dependence on a third-party inference API’s per-token prices and policies. Connected hosted services can still carry outside dependencies, charges and terms. Compare the full cost of hardware or cloud rental, licences, power and networking, storage, backups, patching, monitoring, staff and ongoing evaluation; savings are not automatic.
Agree who manages access, monitors usage and failures, tests model updates, applies patches, restores backups and handles incidents. Define retention and support access, review the system as documents and workloads change, and budget for those operating responsibilities.