> ## Documentation Index
> Fetch the complete documentation index at: https://docs.apiyi.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Long-form Output Practices

> How to reliably get results for long-output workloads — drama scripts, fiction, 10k-word articles: stream, size the read timeout to the inter-event gap, give max_tokens room, check stop_reason, split very long outputs, and resume dropped streams. With a Claude native example and per-language timeout settings.

> [!NOTE]
>
> **In one line**: when you ask a model to produce tens of thousands of characters in a single call (episode outlines, long fiction, large translations, big code), **stream the response, do not use non-streaming**; set the client read timeout to the gap *between data events* (tens of seconds — 90–120s is a safe value), not to the *total* generation time; give `max_tokens` room; and check `stop_reason` before using the text. Do these four things and long-output calls stop "returning nothing." Beyond 20k–30k characters, add one more: split the work into several calls instead of holding one connection open for 20 minutes.

This page applies to all large models (OpenAI, Claude, Gemini, Grok, and others). Code examples use Claude's native `/v1/messages`; differences for the OpenAI-compatible format are called out separately.

## Three things to know first

1. **For a 10k-word output, 10–20 minutes of real generation is normal.** The model emits tens of thousands of characters token by token, plus a reasoning/thinking phase — end-to-end latency is genuinely long. This is not the gateway being slow; generation itself is slow.

2. **Non-streaming buffers the whole thing before sending.** With non-streaming (`stream` omitted or `false`), the server must wait for the model to finish the entire generation, then send the whole body back at once. During those minutes your client read timeout is racing against it, and the longer the generation the more likely you disconnect before the result arrives — and the exception is often empty (`httpx.ReadError`'s `str(e)` is blank), so you cannot see the cause.

3. **A dropped connection is still billed, so blind retries double-charge.** Once the server has generated the output, the call is billed even if the result never reached you. Retrying after you have already received part of the body means the model runs again and you pay again.

## Stream, do not use non-streaming

With streaming (`stream: true`), the first byte arrives within seconds, and after that a data event arrives every few tens of seconds. Your read timeout only has to cover the gap *between events*, not a generation that runs for many minutes — that is the whole reason streaming reliably delivers long output.

The two protocols have **different terminators** — do not mix them up:

| Protocol | Terminator | How to read the text |
| - | - | - |
| Claude native `/v1/messages` | `event: message_stop` (**no `[DONE]`**) | the `delta.text` of a `content_block_delta` whose `delta.type == "text_delta"` |
| OpenAI-compatible `/v1/chat/completions` | `data: [DONE]` | `choices[0].delta.content` |

With adaptive thinking on, Claude native emits a `type: "thinking"` block **first** (its increments are `thinking_delta`), then the `text` block. When rendering, route `thinking_delta` and `text_delta` separately and do not concatenate thinking into the body text.

Minimal Claude native `/v1/messages` streaming example (plain httpx, line-by-line SSE parsing):

```python theme={null}
import json
import httpx

def generate_long_text(prompt, api_key, model="claude-opus-5", max_tokens=64000):
    url = "https://api.apiyi.com/v1/messages"
    headers = {
        "x-api-key": api_key,
        "anthropic-version": "2023-06-01",
        "content-type": "application/json",
        "accept": "text/event-stream",
    }
    payload = {
        "model": model,
        "max_tokens": max_tokens,
        "stream": True,                        # key: long output must stream
        "thinking": {"type": "adaptive"},      # adaptive thinking; the model decides depth
        "messages": [{"role": "user", "content": prompt}],
    }
    # Three-part timeout: read only covers the inter-event gap, not the whole generation
    timeout = httpx.Timeout(connect=30, write=120, read=120, pool=30)

    text, stop_reason = [], None
    with httpx.Client(timeout=timeout) as c:
        with c.stream("POST", url, json=payload, headers=headers) as r:
            if r.status_code != 200:
                raise RuntimeError(f"HTTP {r.status_code}: {r.read()[:400]}")
            event, data = None, []
            for line in r.iter_lines():
                if line == "":                 # events are separated by a blank line
                    if data:
                        d = json.loads("\n".join(data))
                        t = d.get("type") or event
                        if t == "content_block_delta" and d.get("delta", {}).get("type") == "text_delta":
                            text.append(d["delta"]["text"])
                        elif t == "message_delta":
                            stop_reason = d.get("delta", {}).get("stop_reason") or stop_reason
                    event, data = None, []
                    continue
                if line.startswith("event:"):
                    event = line[6:].strip()
                elif line.startswith("data:"):
                    data.append(line[5:].strip())
    # A Claude stream ends on message_stop, never [DONE]
    if stop_reason == "max_tokens":
        raise RuntimeError(f"truncated by max_tokens after {len(''.join(text))} chars; raise max_tokens and retry")
    return "".join(text).strip()
```

> [!TIP]
>
> If you use the official anthropic SDK, point `base_url` at `https://api.apiyi.com` and use `client.messages.stream(...).get_final_message()` — the SDK handles SSE parsing, timeouts, and `stop_reason` for you. The httpx version above is for when you would rather not pull in the SDK.

## Size the read timeout to the inter-event gap, not the total time

Many people set the read timeout to one huge value meant to cover the whole generation (say 1800 seconds) and still time out — because with non-streaming that value has to race the entire generation and any hiccup breaks it. The right approach is streaming plus a read timeout sized to the inter-event gap.

Measured reference (`claude-opus-5` producing a \~20k-character episode outline from a \~15k-character input):

| Metric | Measured |
| - | - |
| Time to first byte | 3 – 130 s |
| Max silent gap during thinking | \~42 s (with keepalive pings in between) |
| End-to-end total | 9 – 12 minutes |

So a read timeout of **90–120 seconds** covers the largest inter-event gap with headroom — no need for a value of many minutes. A three-part timeout splits the phases and sizes each one:

```python theme={null}
# connect: establish; write: upload the request body; read: max gap between reads
timeout = httpx.Timeout(connect=30, write=120, read=120, pool=30)
```

### Setting it in each client

Many client libraries have both a *per-read* timeout and a *total* timeout, often with similar names. For long output you want the former; leave the latter unset or unlimited:

| Client | Set this (max gap between reads, 120 s recommended) | Avoid |
| - | - | - |
| Python httpx / OpenAI SDK / anthropic SDK | `httpx.Timeout(connect=30, write=120, read=120, pool=30)` | a single big number like `timeout=3600`, which also makes the read timeout one hour |
| Python requests | `timeout=(30, 120)` — the second value is the read timeout | `timeout=3600` |
| Java Hutool | `setConnectionTimeout(30000)` + `setReadTimeout(120000)` | `setReadTimeout(3600000)` or `timeout(3600000)` (the latter sets both connect and read) |
| Java OkHttp | `readTimeout(120, TimeUnit.SECONDS)` | keeping the 10 s default (silence during thinking trips it); `callTimeout` is a total limit — leave it unset |
| Node.js fetch | `undici`'s `new Agent({ bodyTimeout: 120_000 })`, passed as the `dispatcher` | `AbortSignal.timeout(...)` is a total limit and will cut long output off |
| Go net/http | your own idle timer on the body: reset it on every chunk, cancel the `context` after 120 s without data | `http.Client{Timeout: ...}` is a total limit that includes reading the whole body |

## Streams can stall too: the connection stops receiving mid-way

Streaming fixes the "buffer everything before sending" problem, but there is another failure mode to guard against: **a connection that stalls mid-stream**.

* **Symptom**: text arrives normally at first, then nothing more comes — yet the connection neither errors nor closes. The client hangs until its own timeout: if that is one hour, it waits a full hour, and ends up with only the first small piece.
* **Cause**: on cross-region and cross-border network paths, a connection held open for a long time can get stuck at an intermediate network device. The server finished generating and closed the connection long ago, but the close never reaches you. **The longer a connection lasts, the more likely this is** — a 20-minute connection is hit more often than a 3-minute one.
* **Billing**: the server finishes generating and bills as usual (see point 3 of "Three things to know first").

What to do:

1. **Size the read timeout to the inter-event gap** (previous section). With a 120 s read timeout a stall costs you two minutes before you can retry; with a one-hour read timeout it costs a full hour. This is the most important one.
2. **Shorten each connection**: split very long outputs into several calls — see "Split very long outputs into several calls" below.
3. **Resume a dropped stream from where it stopped** — see "Retry strategy" below.

> [!WARNING]
>
> Do not set the read timeout to "total generation time plus headroom" (1800 s, 3600 s, and so on). It makes no difference while things work, but when a connection stalls you wait that long before you even learn it failed.

## Give max\_tokens room, and check stop\_reason

Long output easily hits the `max_tokens` ceiling and gets truncated. This is especially true for models like Claude with **thinking on — thinking itself consumes the `max_tokens` budget**, and a long piece can exhaust it.

* **Start `max_tokens` at 64000** (give even more at high effort / deep thinking; `claude-opus-5` supports 128K output).
* **Check `stop_reason` before using the response:**
  * `end_turn` — finished normally, the text is complete; this is the only success.
  * `max_tokens` — truncated, the text may be incomplete or even empty. This is **truncation**, not an "empty result" — raise `max_tokens` and retry.
  * `refusal` — declined by a safety policy; handle separately.

Judging success by `str(e)` alone or by "the text is empty" is misleading — an empty body is usually `max_tokens` truncation.

## Split very long outputs into several calls

Asking for 50k characters in one call that runs for 20 minutes is the most fragile pattern, even with streaming: any network hiccup along the way voids the whole piece, and the whole piece is billed. Once the output exceeds 20k–30k characters, split it:

1. Generate the **outline / episode list / chapter list** first — a short call.
2. Then generate **one chapter or episode per call**, keeping each under 10k–20k characters. Include the outline and the context each part needs (character settings, how the previous part ended) so the pieces stay consistent.
3. Once the outline is fixed, the parts are independent and can run **in parallel**.

| | One call | Split |
| - | - | - |
| Duration of each connection | 10 – 20 minutes | 3 – 5 minutes |
| Cost of a stall or drop | redo and pay for the whole piece | retry one part |
| Total time | one long serial run | parts run in parallel, usually faster |
| Risk of `max_tokens` truncation | high | low |

## Retry strategy

Be conservative with retries on long output, so "retry on failure" does not become "double billing plus a second long run":

* **Retry only on failures before the response headers arrive, and on `5xx` / `429`** (with backoff, at most twice). Those are connection or transient issues where a retry makes sense.
* **If a stream drops after you already received part of the body, resume from where it stopped instead of starting over.** The server has generated and billed it; restarting runs it again and pays again. Keep the text you received, include it in a new call as what has been written so far, and let the model continue — you pay only for the remainder.
* **Treat a read timeout (a stalled connection) the same way**: if you already have a good amount of text, resume; if you received almost nothing, retry the whole call.
* Log the request id from the response headers for reconciliation and troubleshooting.

A resume request looks like this (works with any model):

```python theme={null}
messages = [{
    "role": "user",
    "content": (
        f"{original_prompt}\n\n"
        f"Here is what has been written so far:\n\n{partial_text}\n\n"
        "Continue directly from the last sentence above. Do not repeat existing content or add any commentary."
    ),
}]
```

## Scenario cheat sheet

| Scenario | Typical output | max\_tokens | read timeout |
| - | - | - | - |
| Drama / short-drama episode outline | 10k – 30k chars | 64000 | 90 – 120 s |
| Fiction (novels / chapters) | 10k – 50k chars | 64000 – 128000 | 90 – 120 s |
| Long translation | scales with source | \~1.5x source tokens | 90 – 120 s |
| Large code generation | thousands of lines | 32000 – 64000 | 90 – 120 s |

For a single piece over 30k characters, split it by chapter or episode (see "Split very long outputs into several calls"). Always stream. Use `api.apiyi.com` (or the backup `b.apiyi.com`), and **not `api-cf.apiyi.com`** (the CDN node returns `524` at \~100 seconds and cannot carry long requests).

## Related

> **[How to avoid API timeouts](/en/faq/timeout-configuration)**
>
> Timeout values by scenario

> **[Streaming vs non-streaming](/en/faq/streaming-vs-non-streaming)**
>
> The tradeoffs and how to choose

> **[Claude thinking & effort](/en/api-capabilities/claude-effort-thinking)**
>
> Adaptive thinking, effort tiers, max\_tokens and truncation

> **[Claude response handling](/en/api-capabilities/claude-response-handling)**
>
> Native response shape, SSE events, stop\_reason


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.