State

The state is everything Myles needs to know to answer: the message, the record, the policy and the case. Send it as text, or as JSON, which is passed to the model as indented JSON.

What to include

• The thing being decided about. The message, the application or the action an agent wants to take.
• The rule, if there is one. A refund policy, an eligibility rule or a routing guide. Put it in the state, not the question.
• Only what matters. Every token is read and, on the hosted API, paid for. Leave out signatures, footers and quoted history.

{
  "model": "myles-g-12b",
  "state": {
    "policy": "Refunds within 30 days with proof of purchase.",
    "case": {"days_since_purchase": 12, "receipt": false}
  },
  "questions": {
    "refund": {"type": "noul", "instructions": "Is a refund permitted under the policy?"}
  }
}

How long it can be

• The model reads the whole state whenever the request fits its context window: 24,576 tokens on the hosted Myles-G 31B and 32,768 on Myles-G 12B.
• On Myles-G 31B, a state too long for the context window is refused with 422 context_length_exceeded (detail names body.state), never trimmed. It never decides on part of a state, so send a shorter state or split it.
• On Myles-G 12B, a state too long for the context window is trimmed to fit, never refused. The model reads its beginning (about two thirds of what fits) and its end, with a marker where the middle was left out. The response says so every time, with "truncated": true, state_chars (the characters sent and the characters kept) and the X-State-Truncated header. Put what matters at the beginning or the end.
• The whole request body can be up to 32 MiB.
• A question and its options must fit the context window on their own. Otherwise the request returns 422 context_length_exceeded.

Images in the state

On Myles-G 12B and Myles-G 31B the state can include images. Send each image in assets under a reference, then write the reference where the image belongs. In a JSON state, put it in as a value, because keys are not searched:

{
  "model": "myles-g-12b",
  "state": {
    "policy": "Refunds within 30 days with proof of purchase.",
    "photo": "photo_1"
  },
  "questions": {
    "damaged": {"type": "noul", "instructions": "Does the item in the photo look damaged?"}
  },
  "assets": {
    "photo_1": {"format": "jpeg", "data_base64": "..."}
  }
}

An image in the state is billed once per request, however many questions read it, so the state is the best place for a picture that several questions are about. A state with images is the exception to the length rule above. It is read up to its first 20,000 characters (a JSON state counts as its indented JSON), even when more would fit the context window, and a longer one is cut there with the response saying so ("truncated": true). A reference must fall within those 20,000 characters, so in a long state put image references early. One beyond that point is refused with 400 image_reference_truncated rather than answered without the image. See Images for formats, limits and billing.

Writing the question

Write instructions as you would ask a careful colleague: one question, plain words, no answer hidden in it. Describe each option in criteria when the labels alone are ambiguous. More in Writing questions.