Self-hosting

Run Myles on servers, laptops and GPUs you operate, including disconnected networks.

Myles runs on hardware you operate: servers, laptops, phones and GPUs, including disconnected and air-gapped networks. Nothing is sent to Mellix. Myles-G runs behind the same Decision API as the hosted endpoint, so the same POST /v1/decide request works on both. The Batch API is hosted only.

Choosing hardware

Model	Hardware	Median decision
Myles-G 31B	One data-centre GPU	48 ms
Myles-G 12B	One workstation GPU	31 ms
Myles-G Edge	Laptop- and phone-class GPUs	21 ms
Myles-400M	Any server CPU, one thread per decision	about 500 ms
Myles-150M	A laptop CPU	196 ms
Myles-22M	A browser or a phone	27 ms

Median decision: one request at a time on an otherwise idle machine, Myles-G on one GPU and the others on one CPU thread of a desktop processor. Throughput under load, with several requests at once, is in GPU sizing.

Myles-G 31B is available today as a ready-to-run package, described on the rest of this page. Myles-400M, Myles-150M and Myles-22M are requested and downloaded in the console. Myles-G 12B and Myles-G Edge are available on request: ask us and we will tell you what we can supply for your hardware.

The Myles-G 31B package

The package runs Myles-G 31B on your own GPUs, in your data centre or your own cloud account, with no connection to Mellix or anyone else. The current release is package 0.2.0, model build myles-g-31b-20261025-bf16.

Part	What it is
Container image	An OCI image holding the inference engine and the Decision API. It contains no model weights and no secret keys. Delivered as an archive you load with `docker load` or push to your own registry.
Model bundle	The model weights and the readout maps (allow about 75 GB of disk), downloaded from the Licences page of the console.
Licence file	`licence.json`, signed by Mellix, naming your organisation, the build it covers and its dates.
Checksum list	`SHA256SUMS`, the checksum of every file (bundle, image and documents), and `SHA256SUMS.sig`, the Mellix signature over that list. You check both offline before installing.
Documents	Installation, operations and security guides, a model card and third-party notices, also inside the image.

The container serves the same Decision API as the hosted endpoint: POST /v1/decide, GET /v1/models and GET /health on port 8010, with Prometheus metrics on port 9400. Every request carries an operator key you create; optionally, a key file gives each client application its own key, rate limit, daily cap and expiry.

GPU sizing

GPUs	Precision	Throughput under load (four requests at a time)
One 48 GB Ada GPU (L40, L40S or RTX 6000 Ada)	fp8 weights and fp8 attention cache	About 5 requests a second, p95 about 1.7 s per request (measured on one L40)
One 80 GB GPU (H100 or A100 80 GB)	bf16	Higher; not yet measured for this package
Two L40 or L40S GPUs	bf16, split across both GPUs	Higher; not yet measured for this package

Precision is chosen for you. The container uses bf16 when its GPUs together have at least 79,000 MiB of memory; otherwise fp8, when they have at least 44,000 MiB and are Ada, Hopper or newer (compute capability 8.9 or above). On anything smaller it stops with a message naming what is missing. You can force either precision; the same checks apply.
An 80 GB GPU must be free of other work. In bf16 the full context needs 96% of its memory.
Host: x86-64 Linux; NVIDIA driver R580 or newer, or an older data-centre driver with the CUDA forward-compatibility package; the NVIDIA Container Toolkit; Docker 24 or newer, or any OCI runtime with GPU support.
Memory and disk: 32 GB of RAM or more, with --shm-size 16g for the container; about 75 GB of fast local disk for the bundle (it is read in full at every start) and about 15 GB for the image, plus the same again while you unpack the image archive.
In the cloud: any instance that meets the above works, for example a cloud GPU instance with one 48 GB L40S.

Start-up takes a few minutes: the container finds the GPUs, picks the precision, checks the readout map and the licence, loads the model, then logs READY and answers /health.

Deployment options

Option	What the package gives you
Docker	One `docker run` command, with the bundle and the operator key mounted read-only and the API published on the host's loopback for your own proxy.
Docker Compose	A Compose file that puts the model on an internal network with no route out of the host, behind a TLS proxy as the only way in.
Kubernetes	Manifests for a Deployment, a Service, a NetworkPolicy with no egress, and an example Prometheus scrape. Requires the NVIDIA GPU Operator or device plugin. Each replica requests one GPU, so a cluster with a GPU node pool scales by adding replicas.
Managed endpoint platforms	A `serve` mode that answers `GET /ping` and `POST /invocations` on port 8080, for platforms that host containers behind their own authentication.

Put the container behind your own load balancer or gateway for TLS and authentication; it speaks plain HTTP and its ports should never face the internet. Upgrades are side by side: each build is pinned, you start a new build beside the old one, compare, then switch, and rollback is starting the previous build again. We support the latest two self-hosted builds and give 90 days' notice before one is retired.

Installing on an air-gapped network

On a connected machine, sign in to the console and open your licence on the Licences page. Download the build with its download script (it resumes after interruptions and checks every file), or file by file. Download the Mellix release key from the same page, never from the package itself.
Check the signature on SHA256SUMS with the release key, then every file against the list. Stop if either check fails.
Move the folder to the target machine by your usual approved route.
Load the image, place the bundle and licence file on local disk, create an operator key, and start the container. No step from here on needs a network.

Network and licence

Nothing calls home. The package sends no telemetry, reports no usage, checks for no updates and contacts no licence server. Every release is started with no network at all, in both precisions, before it is delivered, and reaches READY that way.
The licence is checked offline. licence.json is signed by Mellix and checked against a public key built into the image, at start and every 60 seconds: the signature, the build it names, and its dates against the host clock. To renew, replace the file; a running container picks it up without a restart. It stops within a minute of the licence ending, and a metric shows the time left so you can alert well before.
Your data stays with you. States, questions, images and answers are held in memory only while a request is answered, and are never logged or written to disk. Logs carry the method, path, status, time and a request id; metrics carry counts and timings only.
The only outbound option is one you choose. The container can fetch its bundle at first start if you configure that; mounting the bundle from disk, as above, needs no network.

Limits of the packaged build

Limit	Value
Context window	16,384 tokens on build `myles-g-31b-20261025-bf16`. A longer state is never refused: it is trimmed to fit, keeping its beginning and its end, and the response says so with `"truncated": true`, `state_chars` and the `X-State-Truncated` header. This differs from the hosted Myles-G 31B, which has 24,576 tokens and refuses a state that does not fit (see State).
Questions and options	256 questions a request, 256 options a question, 4,096 options a request
Images	Up to 8 a question, PNG or JPEG
Request body	32 MiB
Concurrency	8 requests in the model at once on each replica; further requests queue

Licensing

One licence per deployment, monthly or annual, with no per-decision fees. See Enterprise for how licensing works.

Talk to us about self-hostingTell us about your hardware and your decisions, and we will tell you what fits.Use the hosted API insteadMyles-G 12B and Myles-G 31B, hosted by us. Buy credit in the console and get a key.

Security reviews

Ask us for the lineage record, our evaluation method and results, the software bill of materials for the container image, or send your security questionnaire: security questions. See also the assurance facts on the Enterprise page.

Myles-G 31B is fine-tuned by Mellix from an openly licensed base model (Apache License 2.0), which its lineage record names. The container image is built on the open-source vLLM inference engine (Apache License 2.0). The package's third-party notices list every component and its terms.