Deep Dive - Tinker

At 09:12, your clamp run reports lower loss, but the next sample still returns return x. The local grader sees reward 0. The remote worker accepted forward_backward and optim_step. Before changing optimizer settings, ask three narrower questions: did the loop send the intended tokens, did the worker update the intended adapter, and did the sampler read that update?

That debugging split is Tinker's design. Your Python process renders prompts, runs the verifier, and decides what evidence to submit. Hosted workers execute model math, hold adapter and optimizer state, and produce samples. The boundary gives you handles and results, not the worker's placement decisions or its hidden gradients.

Tinker is Thinking Machines Lab's managed hosted low-rank adaptation (LoRA) service for this job. A local Python process builds prompts, environments, rewards, and loss inputs. Remote workers run forward and backward passes, apply optimizer updates, and generate samples. The public software development kit (SDK) and Tinker Cookbook are Apache-2.0 open-source code. The hosted service, model catalog, data handling, and API access are a separate product with their own terms.^{[1]Reference 1Announcing Tinkerhttps://thinkingmachines.ai/news/announcing-tinker/[2]Reference 2Tinker SDK 0.27.0 Sourcehttps://github.com/thinking-machines-lab/tinker/tree/0ce5bf2484891570b59f8d330a2daf793646917e[3]Reference 3Tinker Cookbook Source (600b802)https://github.com/thinking-machines-lab/tinker-cookbook/tree/600b802d09be13e03756bfdc31e8c864fed90e7b}

Follow one four-completion clamp group across forward_backward, optim_step, sample, and save_state. The source walkthrough uses SDK 0.27.0, commit 0ce5bf2484891570b59f8d330a2daf793646917e, and Cookbook commit 600b802d09be13e03756bfdc31e8c864fed90e7b, checked September 2, 2026. The local examples need no API key and submit no hosted jobs. A pinned client fixes local code, not the remote service implementation.^{[2]Reference 2Tinker SDK 0.27.0 Sourcehttps://github.com/thinking-machines-lab/tinker/tree/0ce5bf2484891570b59f8d330a2daf793646917e[3]Reference 3Tinker Cookbook Source (600b802)https://github.com/thinking-machines-lab/tinker-cookbook/tree/600b802d09be13e03756bfdc31e8c864fed90e7b}

Start with the boundary

Before reading an API call, predict who can answer each failure: a malformed Datum, an unavailable model, or a missing checkpoint. Tinker has three names that look like one product but describe different surfaces.

The hosted service isn't the same thing as the GitHub repository. Cloning the SDK gives you client code, not the worker pool, model weights, scheduler, storage, or failure-recovery system. Using the service also doesn't force a fixed high-level RL trainer. The low-level calls are intentionally small.^{[4]Reference 4Tinker Documentationhttps://tinker-docs.thinkingmachines.ai/[2]Reference 2Tinker SDK 0.27.0 Sourcehttps://github.com/thinking-machines-lab/tinker/tree/0ce5bf2484891570b59f8d330a2daf793646917e}

Treat the model catalog as a capability response, not as a constant in your Python package. Current docs describe LoRA fine-tuning for dense and mixture-of-experts (MoE) models from about 1B to 1T+ parameters, including vision models, and publish a live models page.^{[4]Reference 4Tinker Documentationhttps://tinker-docs.thinkingmachines.ai/}

The model page separates current training IDs, inference offerings, and retired models. A Hugging Face checkpoint isn't automatically a valid Tinker training ID. Check get_server_capabilities and the current model documentation before selecting a base model; separately verify its loss, modality, export, and account constraints. A list of model names doesn't encode every capability.^{[5]Reference 5Tinker: Supported Modelshttps://tinker-docs.thinkingmachines.ai/tinker/models/}

A model may fit the service's architecture adapter and still be absent from your account, restricted by third-party terms, or unavailable for a particular loss or input modality. Treat the capability response and live model page as run-start evidence, not as static metadata in a blog post.

Local control plane, remote data plane

Start with one local question: what should happen to the next completion? The control plane is the code that answers it. Your process loads data, renders a chat template, calls an environment, grades clamp, computes group-relative advantages, selects a loss, and decides when to checkpoint. It can also start many requests concurrently with asyncio.

The data plane answers a different question: how does the model produce that completion? Hosted workers receive serialized model inputs and loss tensors, run distributed forward and backward computation, apply the optimizer state, and return compact outputs. Sampling workers generate token IDs and optional log probabilities. Storage workers make named checkpoints addressable through tinker:// paths.

This split lets you change a reward function without changing distributed model code. It also tells you where to look when the run fails. Local code can produce a malformed target tensor; remote workers can reject a model or run out of capacity; a sampler can use an older checkpoint than the trainer; and a retry can duplicate an operation unless request and checkpoint names are idempotent.

When a result looks wrong, ask, "Where does this state live?" Your loop constructs prompt rows, reward labels, and experiment configuration; requests then send selected data to the service. LoRA parameters, optimizer moments, and worker placement stay hosted. A future is a handle to completion, not a download of model-parameter gradients. Custom losses can differentiate returned log probabilities locally, which is a different gradient boundary.

What does cloning tinker give you?

Answer

Cloning provides the client, types, protocol conversion, tests, and CLI. Hosted GPU workers, model catalog, storage service, and API authorization remain outside the checkout.

The four primitives

When clamp produces a bad sample, don't start with endpoint names. Predict which state transition should have changed: gradient accumulation, adapter weights, sampler weights, or a recovery artifact. These four primitives make that transition observable.

The SDK also exposes save_weights_for_sampler and save_weights_and_get_sampling_client. Those operations publish a sampler-compatible adapter snapshot. save_state restores training state; a sampler checkpoint evaluates or serves a chosen weight version. Keep those consumers separate when you name and inspect artifacts.

The synchronous entry points return future-like objects. Their _async variants don't all have the same return contract: awaiting forward_backward_async gives an APIFuture that still needs result_async(), while awaiting sample_async gives the completed SampleResponse. Neither changes the server's ordered training schedule into an unbounded asynchronous parameter server.^{[2]Reference 2Tinker SDK 0.27.0 Sourcehttps://github.com/thinking-machines-lab/tinker/tree/0ce5bf2484891570b59f8d330a2daf793646917e}

forward_backward receives a Datum

Before sending one datum, predict what the remote loss can validate: model input plus tensors named by that loss. A Datum has model_input and loss_fn_inputs. model_input is a token or multimodal sequence. For SFT, loss_fn_inputs commonly contains target_tokens and weights. For RL, it can include sampled logprobs, advantages, target_tokens, and an action mask before cookbook code removes or consumes the mask.

Large lists may become multiple requests. The SDK's client-config defaults are 1,024 datums and 5,000,000 estimated pre-compression protobuf bytes per chunk; server-provided configuration can change them. The chunker doesn't split an individual oversized datum, so the byte value isn't a universal hard cap. Ordered sequence IDs preserve logical request order even when chunks are submitted concurrently.

The loss name is a protocol value, not a promise that a run will accept it. The pinned SDK declares cross_entropy, importance_sampling, ppo, cispo, and dro as supported loss identifiers. A model or service deployment may enable fewer options, so validate against current server capability and docs when a run starts.^{[2]Reference 2Tinker SDK 0.27.0 Sourcehttps://github.com/thinking-machines-lab/tinker/tree/0ce5bf2484891570b59f8d330a2daf793646917e[4]Reference 4Tinker Documentationhttps://tinker-docs.thinkingmachines.ai/}

optim_step advances adapter state

If loss improves but a resume behaves differently, inspect the update boundary first. optim_step consumes gradients accumulated by preceding forward/backward work and updates trainable parameters with AdamW-equivalent behavior. SDK documentation calls out a default weight decay of zero, which differs from some PyTorch defaults.^{[2]Reference 2Tinker SDK 0.27.0 Sourcehttps://github.com/thinking-machines-lab/tinker/tree/0ce5bf2484891570b59f8d330a2daf793646917e}

The optimizer state stays remote, so you don't download a Python optimizer object after every step. That keeps the loop small but makes checkpoint choice consequential. Loading weights alone resets optimizer history; loading with optimizer restores saved moments when the checkpoint contains them.

The operation ordering is part of correctness. An optimizer request submitted before its matching forward/backward result can still be valid because the service sequences requests for that training client. A separate client, a retry, or an accidental concurrent loop can violate the intended order. Give one component ownership of a training client, and log request IDs with step numbers in the local loop.

sample returns evidence for learning

Before trusting a reward, predict what evidence the sampler must return. SamplingClient.sample accepts a ModelInput, number of completions, and SamplingParams. Each returned sequence includes generated token IDs and a stop reason. If requested, it includes one log probability per generated token. compute_logprobs can score prompt tokens for a separate comparison.

The decoded string answers the verifier's question; token identifiers (IDs) and log probabilities answer the learner's. If you decode and re-tokenize before constructing an RL Datum, a different tokenizer version or normalization rule can shift token boundaries. The unit test may still pass while the policy-gradient target points at the wrong positions.

num_samples is a group primitive. Four independent clamp completions give a group over which the Cookbook centers rewards. Four separate calls can work, but one grouped call makes the comparison and batching explicit. Concurrent calls across different prompts add another axis of parallelism.^{[4]Reference 4Tinker Documentationhttps://tinker-docs.thinkingmachines.ai/[3]Reference 3Tinker Cookbook Source (600b802)https://github.com/thinking-machines-lab/tinker-cookbook/tree/600b802d09be13e03756bfdc31e8c864fed90e7b}

save_state creates a recovery point

After an update, ask which consumer needs the bytes. save_state("step-0042") stores a named training checkpoint. load_state restores model weights but not optimizer state. load_state_with_optimizer restores both when available. A TTL can make a checkpoint expire, and overwrite controls whether the name can be replaced.^{[2]Reference 2Tinker SDK 0.27.0 Sourcehttps://github.com/thinking-machines-lab/tinker/tree/0ce5bf2484891570b59f8d330a2daf793646917e}

For evaluation, save sampler-compatible weights instead. Export is another path: for models with downloadable compatible weights, Cookbook utilities can build a parameter-efficient fine-tuning (PEFT) adapter or merge it into a Hugging Face model. Not every hosted model necessarily permits the same export. Runtime compatibility, base-model availability, and applicable terms still govern deployment.

LoRA is the adaptation boundary

Those four calls describe movement. The next question is what the optimizer can change.

Tinker is a LoRA training service, not a full-parameter training service. LoRA keeps the frozen base matrix $W$ and learns a low-rank update. Thinking Machines Lab writes the adapted matrix as

$$W' = W + \gamma BA,$$

where $A$ and $B$ are small trainable matrices and $\gamma$ is a constant scale. Many implementations, including the original LoRA paper, take $\gamma = \alpha / r$ with rank $r$ and a scale hyperparameter $\alpha$. The service stores and updates the adapter path while sharing the base model across runs. That's how a multi-tenant worker pool can serve many experiments without copying a full model and optimizer state for each run.^{[6]Reference 6LoRA: Low-Rank Adaptation of Large Language Models.https://arxiv.org/abs/2106.09685[1]Reference 1Announcing Tinkerhttps://thinkingmachines.ai/news/announcing-tinker/[7]Reference 7LoRA Without Regrethttps://thinkingmachines.ai/blog/lora/}

LoRA doesn't mean "small model." Tinker has documented support for large dense and MoE models, including trillion-parameter examples in its announcements. The memory savings come from trainable state and optimizer path, not from making every base-model weight cheap to move. A huge model still needs a compatible hosted topology, checkpoint format, and model-specific adapter implementation.^{[8]Reference 8Tinker General Availabilityhttps://thinkingmachines.ai/news/tinker-general-availability/[4]Reference 4Tinker Documentationhttps://tinker-docs.thinkingmachines.ai/}

Rank is a capacity choice, not a quality guarantee. A larger rank can express a broader update but costs more adapter parameters and optimizer state. Training only attention, only a multilayer perceptron (MLP), or the unembedding layer changes which behavior can move. Thinking Machines Lab's LoRA study found attention-only LoRA underperformed MLP LoRA even after matching parameter count, and that RL matched full fine-tuning at very low ranks in their math setup. The create_lora_training_client options make those choices explicit. Compare runs with target modules held constant before attributing a change to rank.^{[7]Reference 7LoRA Without Regrethttps://thinkingmachines.ai/blog/lora/}

LoRA also changes what "full control" means. You control the algorithm, data, reward, sampling schedule, and loss inputs. You don't control arbitrary base-weight tensors, kernel fusion, placement, or collective algorithms. That boundary is productive for post-training research, but it isn't equivalent to owning the full training stack.

Why can't a Tinker LoRA run answer a question about full-parameter stability?

Answer

The trainable matrices are low-rank adapters while the base weights remain frozen. Results can reveal post-training behavior under LoRA, but they don't measure a full-parameter optimizer path or its optimizer-state memory.

SFT first: a token-level contract

A low-rank update still needs a token-level contract. Start with supervised clamp before adding rewards. Before you submit the datum, predict which positions should be allowed to change the adapter.

A rendered conversation becomes an input sequence and a target sequence. A weight tensor marks which target positions contribute to cross-entropy. Prompt positions usually receive zero weight; assistant positions receive one or a chosen fractional weight.

For one datum, the core alignment is:

These are toy IDs, not a real tokenizer vocabulary. The full sequence is [10, 11, 20, 21]: inputs omit its last token, targets omit its first token, and weights follow the target role. Masking by the input role would incorrectly drop the first answer token at position 1.

The arrays must share the expected shifted length. The Cookbook's conversation_to_datum helper handles renderer-specific role markers, target shifting, and response masks. Read that helper when a loss curve looks impossible. A one-token offset can produce a decreasing loss while training the model on the wrong targets.

The smallest SFT loop has one forward_backward and one optim_step. The Cookbook submits both before waiting so they can share a worker clock cycle. It repeats the pair while recording loss and evaluation metrics. A decrease in loss is evidence that the chosen weighted tokens are easier for the adapter to predict. It isn't evidence that the unit test started passing. Run that test separately.

RL: preserve tokens, logprobs, and credit

Loss going down doesn't mean the grader is happier. RL adds a second model interaction: the current policy samples a completion, an environment or verifier assigns reward, and the loop converts that evidence into a training datum. The reward function can be local and deterministic, a remote grader, a unit-test sandbox, or another model. Tinker doesn't choose it for you.

Keep the clamp group of four completions:

The group mean is $\bar{R} = 0.5$. Centered advantages are $A = (0.5, -0.5, 0.5, -0.5)$. The reward objective encourages completions 0 and 2 and discourages 1 and 3; shared parameters mean no individual probability change is guaranteed. If all rewards were 1, every centered advantage would be zero. A KL penalty or another auxiliary objective could still contribute gradients.

The Cookbook's basic Group Relative Policy Optimization (GRPO) path turns that prediction into a sequence:

1. Select a group of prompts.
2. Sample several completions per prompt from current sampler weights.
3. Grade every completion.
4. Center rewards within each group: $A_i = R_i - \operatorname{mean}(R_{group})$.
5. Optionally filter constant-reward groups. The helper retains one group with a warning if all are uniform; that fallback avoids an empty batch but doesn't create reward signal. The pinned training config disables this filtering by default.
6. Build Datum objects with sampled log probabilities and token-aligned advantages.
7. Submit forward_backward(..., loss_fn="importance_sampling").
8. Submit optim_step with the chosen Adam parameters.
9. Save and evaluate a sampler checkpoint.

The group baseline is a variance-reduction choice, not a universal reward definition. Code tasks might use unit-test pass rate, while tool-use environments can combine success, invalid-call penalties, and latency budgets. Preference models may return dense scores. The model sees a token-level signal only after your loop maps those outcomes into advantages and masks. That mapping is where an evaluation decision becomes a training decision.

The RL datum shape

Before looking at fields, predict what should happen when a trajectory branches. The Cookbook's trajectory converter emits one or more datums per trajectory. One trajectory can split when a later observation isn't a prefix of the sequence accumulated so far. That happens naturally with tool calls, branches, or context compaction.

The resulting loss_fn_inputs include:

Observation positions receive zero logprob, zero advantage, and zero mask. Action positions receive the sampled logprob and trajectory advantage. A parse-error policy can set an action mask to zero while retaining the rest of the trajectory. The invalid turn stays visible in metrics without teaching the model to reproduce malformed output.

The named loss consumes only the fields it supports. importance_sampling compares current log probabilities with sampled behavior-policy log probabilities. Proximal Policy Optimization (ppo) adds clipping behavior. The cispo and dro losses use other importance and truncation choices. A custom loss can run on log probabilities through forward_backward_custom, but the SDK still translates your derivative into a backend-supported weighted cross-entropy pass.^{[2]Reference 2Tinker SDK 0.27.0 Sourcehttps://github.com/thinking-machines-lab/tinker/tree/0ce5bf2484891570b59f8d330a2daf793646917e[3]Reference 3Tinker Cookbook Source (600b802)https://github.com/thinking-machines-lab/tinker-cookbook/tree/600b802d09be13e03756bfdc31e8c864fed90e7b}

Logprob alignment is the hidden invariant

Suppose a sampler returns four response tokens and four log probabilities for one clamp completion. Predict where the first score belongs before building the datum. The training input usually contains prompt context plus the response prefix, while the target array contains response tokens shifted one position to the left. Prompt padding is deliberate: it keeps positions aligned while assigning zero learning signal to context.

If the arrays differ by one token, an operation can fail loudly. Worse, accidental padding can make lengths agree while applying the wrong credit to every token. Log a short prefix of tokens, log probabilities, advantages, and masks for one group before launching a long run.

Make the reward reproducible enough to debug. Save the prompt ID, sampler checkpoint path, sampled tokens, stop reason, reward components, environment version, and advantage statistics. A reward value without that evidence can't tell you whether the model improved or the grader changed.

The pipeline clock

Correct datums can still miss GPU time if the client waits between calls. Tinker worker pools execute training clients in lock-step clock cycles. A cycle can include forward/backward work and an optimizer step for one or more LoRA models sharing the pool. Multi-tenancy keeps small batches from leaving the pool idle, but it also means a small request can observe latency shaped by the pool's schedule.^{[4]Reference 4Tinker Documentationhttps://tinker-docs.thinkingmachines.ai/}

Submit work early, then await its results. In the documentation's illustrative schedule, waiting for a forward/backward result misses the next admission boundary and makes the pair span three cycles. That cycle count is a scheduling example, not a measured latency guarantee. Both snippets assume an initialized training client, batch, and optimizer parameters; they aren't offline examples.^{[9]Reference 9Tinker: Under the Hoodhttps://tinker-docs.thinkingmachines.ai/tinker/under-the-hood/}

The pipelined loop submits both operations before waiting:

The service can place both in one cycle because the training client preserves request order. For several batches, enqueue the next forward/backward plus optimizer pair before consuming the current result. The Cookbook's train_step uses this pattern and returns training log probabilities for diagnostics.^{[4]Reference 4Tinker Documentationhttps://tinker-docs.thinkingmachines.ai/[3]Reference 3Tinker Cookbook Source (600b802)https://github.com/thinking-machines-lab/tinker-cookbook/tree/600b802d09be13e03756bfdc31e8c864fed90e7b}

Pipeline depth isn't free. A deeper queue can hide network latency and keep GPUs busy, but it can also increase local data waiting for a result and complicate failure handling. In RL, over-enqueueing samples can also increase policy lag. Keep a bounded number of in-flight groups and log sampler checkpoint version against trainer step.

The clock isn't the same as GPU kernel overlap inside a model. A worker may overlap collective communication and matrix operations internally while your client still misses a cycle by waiting before submitting the next request. Read both layers when tuning throughput.

Official docs warn against adding client timeout-and-resubmit loops around Tinker calls. Local cancellation doesn't establish that remote work stopped. If an application deadline expires, record the outstanding request, stop scheduling replacement work, and reconcile its status through the supported SDK behavior. Bound concurrency and total run duration without blindly duplicating expensive requests.

Checkpoints and export paths

Even a fast loop needs a named artifact you can resume or serve. Before saving, decide which question the artifact must answer: can training continue, can a sampler evaluate it, or can another runtime load it? Tinker has three artifact states.

The checkpoint name is part of your experiment schema. Include a run ID, step, dataset revision, and reward version in your own metadata even if the service accepts a shorter name. A checkpoint path says where bytes are; it doesn't explain why that version should be trusted.

save_state can preserve a training resume point. save_weights_for_sampler makes a sampler path. save_weights_and_get_sampling_client combines save and client creation for a convenient evaluation step. The Cookbook's weights utilities download the adapter, build an HF model, build a PEFT adapter, or publish a checkpoint to a hub. Each export choice creates a new license and security review surface.^{[4]Reference 4Tinker Documentationhttps://tinker-docs.thinkingmachines.ai/[3]Reference 3Tinker Cookbook Source (600b802)https://github.com/thinking-machines-lab/tinker-cookbook/tree/600b802d09be13e03756bfdc31e8c864fed90e7b}

Merging $W' = W + \gamma BA$ turns an adapter into a standalone model directory. It doesn't make the base-model license disappear or preserve remote optimizer state. Keep the original tinker:// path and training metadata next to the exported model so a future evaluator can reproduce the exact adapter, base revision, tokenizer, and renderer.

Evaluation should use a fresh sampler client for the saved path or a path explicitly tied to the training client. A long-lived sampler doesn't automatically see the latest adapter. A serving client built from an older path can produce plausible text from a stale policy and make a reward curve look flat, so record the policy version with every trajectory.

API, service, and data boundaries

Both pinned source snapshots use Apache-2.0. That license lets you inspect, modify, and redistribute client and recipe code under its terms. It doesn't grant rights to Thinking Machines Lab's hosted service, third-party model weights, or every dataset copied into a recipe.^{[2]Reference 2Tinker SDK 0.27.0 Sourcehttps://github.com/thinking-machines-lab/tinker/tree/0ce5bf2484891570b59f8d330a2daf793646917e[3]Reference 3Tinker Cookbook Source (600b802)https://github.com/thinking-machines-lab/tinker-cookbook/tree/600b802d09be13e03756bfdc31e8c864fed90e7b}

The service terms checked on September 2, 2026 distinguish customer datasets, hosted models, fine-tuned models, third-party terms, company models, and available weights. The service may add or remove model access. A hosted model's presence in the catalog doesn't establish permission or technical support for downloading its weights.^{[10]Reference 10Tinker Terms of Servicehttps://thinkingmachines.ai/legal/terms/}

A hosted run has two budgets. Samples, training tokens, retries, and checkpoint storage consume service capacity or storage; missing token receipts, version IDs, and reward components consume evidence. Put both in the run manifest before scaling. A lower loss is not a cost report, and a reward curve without its policy version is not a reliable evaluation.

Your dataset isn't automatically safe because the SDK serializes it. You are responsible for having the rights to send prompts, images, tool traces, and labels to a hosted service. The terms grant the provider a limited license to access and process customer datasets to provide the service and comply with law. They also commit that Thinking Machines Lab won't use customer content to train or improve their own models. Personal-data deletion and export obligations still depend on the applicable agreement and processing terms.^{[10]Reference 10Tinker Terms of Servicehttps://thinkingmachines.ai/legal/terms/}

Keep secrets out of the dataset and logs. API keys belong in environment variables or a secret manager. Reward traces can contain private source code, credentials emitted by a tool, or user data. Redact before logging and set checkpoint retention explicitly. A sampler checkpoint can be as sensitive as the training data because the adapter may memorize rare examples.

Data residency, subprocess behavior, storage TTL, deletion, and enterprise commitments are product questions. Don't infer them from an open-source test or from a local tinker:// path. Ask the current service and agreement for the requirement your deployment needs.

Team, contributors, and governance

Tinker is built by Thinking Machines Lab. The company maintains the hosted service, SDK repository, Cookbook, documentation, model integrations, and customer-facing terms. Public repositories show code and commit history, but a contributor count or top-contributor list is only a snapshot. Use the pinned commit and live repository history when attribution matters.

Use repository history for attribution and contribution guidance for proposed changes. Commit counts don't establish authority over the hosted runtime, and an accepted SDK contribution doesn't give its author control of service policy.

The Cookbook's contribution guide asks contributors to run Ruff, Pyright, and tests, and describes an open-science workflow. Its code structure separates configuration builders from heavyweight datasets, environments, rollout strategies, and training loops. Those conventions provide governance through code review and reproducibility, not through a formal foundation with independent technical steering.^{[3]Reference 3Tinker Cookbook Source (600b802)https://github.com/thinking-machines-lab/tinker-cookbook/tree/600b802d09be13e03756bfdc31e8c864fed90e7b}

For an incident, route by evidence. Service errors, capacity, and catalog access belong with the service owner. Malformed datums, reward side effects, and renderer drift belong with your loop owner. Model rights, retention, and export obligations belong with the contract owner. Don't call the SDK a community-governed replacement for the service: its inspectable client and recipe layer still depend on a company-operated API.

Strengths

The cluster is on the other side of the SDK

You can test a reward function, a custom loss, or an environment from a CPU-oriented development machine. Thinking Machines Lab handles scheduling, resource allocation, worker placement, and failure recovery for hosted runs. The first experiment is a Python loop, not a cluster bring-up.^{[1]Reference 1Announcing Tinkerhttps://thinkingmachines.ai/news/announcing-tinker/}

Low-level enough for research

forward_backward and sample expose token and logprob evidence that many high-level trainers hide. You can implement GRPO-style relative rewards, Proximal Policy Optimization (PPO)-style clipping, DPO data preparation, prompt distillation, or a custom logprob loss. The Cookbook provides a starting point without forcing every project into one environment abstraction.^{[1]Reference 1Announcing Tinkerhttps://thinkingmachines.ai/news/announcing-tinker/[4]Reference 4Tinker Documentationhttps://tinker-docs.thinkingmachines.ai/}

One path from SFT to RL

The same TrainingClient can run cross-entropy SFT and RL losses. The same SamplingClient can generate evaluation completions and behavior-policy traces. That keeps one client boundary in a staged post-training project, while still requiring separate checks for training and evaluation.

Large-model access without local replicas

LoRA lets a service share a frozen base model across training runs. The documented model range includes large dense and MoE models. The general-availability announcement added Kimi K2 Thinking as a trillion-parameter example; the current catalog has moved on, so you still have to read the live page rather than that post.^{[8]Reference 8Tinker General Availabilityhttps://thinkingmachines.ai/news/tinker-general-availability/[4]Reference 4Tinker Documentationhttps://tinker-docs.thinkingmachines.ai/} A researcher can test an idea on a small model, then check whether a large model is in the current catalog without rewriting the loop.

Export is a separate path

Sampler checkpoints can be evaluated through the SDK or an OpenAI-compatible interface. Cookbook utilities can download and merge adapters into Hugging Face layouts. After you check tokenizer, license, and runtime compatibility, you can move a successful experiment onto an existing serving stack.^{[8]Reference 8Tinker General Availabilityhttps://thinkingmachines.ai/news/tinker-general-availability/[3]Reference 3Tinker Cookbook Source (600b802)https://github.com/thinking-machines-lab/tinker-cookbook/tree/600b802d09be13e03756bfdc31e8c864fed90e7b}

Weaknesses and limits

Not full-parameter training

LoRA constrains the update space. A behavior that requires broad base-weight changes may not transfer, and a LoRA result doesn't answer full fine-tuning questions. If your research question is optimizer scaling, full-rank catastrophic forgetting, or expert-weight surgery, Tinker isn't the right primary system.

Hosted capacity and catalog are external state

Your code can be deterministic while service capacity, model availability, and scheduling latency change. A pinned SDK doesn't pin the remote runtime. Record capability responses and service metadata at run start, then treat them as part of the experiment evidence.

Network latency shapes algorithm design

Every sample, grader, checkpoint, and training request crosses a boundary. A chatty environment can spend more time waiting on transport than grading. Batch prompts, use concurrency, and keep a bounded pipeline. API acceptance doesn't make secrets or high-cardinality debug payloads safe to send.

Staleness can accumulate silently

An RL sampler can continue using an older sampler checkpoint while training updates a newer adapter. A deep local queue can hide that mismatch. Include a policy version in each trajectory, then reject or measure samples beyond your staleness budget.

Service terms still apply

An Apache-2.0 SDK doesn't turn a hosted run into a self-hosted run. Customer data, model weights, available checkpoints, and downstream use are governed by the current service agreement and third-party model licenses.^{[10]Reference 10Tinker Terms of Servicehttps://thinkingmachines.ai/legal/terms/}

Failure recovery has a boundary

The service can recover worker failures, but your environment may not be idempotent. If a tool call charges money, mutates a repository, or sends a message, a retry can repeat the side effect. Give every external action an idempotency key and store a local trajectory receipt.

A runnable contract check

Use one cheap local gate before spending remote GPU time. This standard-library example grades four known local functions on five fixtures, centers their rewards, and checks a toy trajectory. It doesn't execute untrusted generated code or call Tinker. The fixtures cover below-range, inside-range, above-range, a zero-width interval, and negative bounds; they aren't proof of correctness for arbitrary programs.

Equal lengths still can't detect scores attached to the wrong tokens. Here, constructing both arrays from one full sequence and checking the first action protects the illustrated shift. Real multimodal renderers need their own fixtures. These checks don't prove reward quality, sampler freshness, or learning improvement.

Check the actual SDK without a hosted job

The downloadable SDK contract probe uses the exact Git revision, not just its 0.27.0 version string. Installing dependencies needs network access; execution creates no ServiceClient, HTTP transport, or hosted model. From this repository's root, run:

The executed receipt records six passing checks and source hashes: wire dtypes, a protobuf roundtrip preserving the first-answer weight, count-based chunk ordering, oversized-datum behavior, empty chunk input, and the SDK's acceptance of mismatched lengths. The pinned SDK's seven native request-serialization tests also passed, including a mocked transport test. None of these checks measures remote ordering, clock-cycle latency, model quality, checkpoint recovery, or distributed training.

Production failure modes

The happy path is short. Production debugging isn't. Start with the evidence closest to the token boundary, then move outward to service state and terms.

An actionable incident record contains the SDK commit, Cookbook commit, model identifier, capability response, base revision, tokenizer revision, renderer name, loss name and config, optimizer parameters, sampler checkpoint path, trajectory sample, reward components, and checkpoint path. Without those fields, a replay is guesswork. Capture them before cleanup or checkpoint expiry removes the trail.

Retry and idempotency

The SDK retries some transport failures through its client holder. A retry can be safe for a read, but a training request or external environment call needs a sequence contract. The pinned SDK assigns sequence IDs to training requests and preserves turn ordering across chunks. Your environment must add its own idempotency around side effects.^{[2]Reference 2Tinker SDK 0.27.0 Sourcehttps://github.com/thinking-machines-lab/tinker/tree/0ce5bf2484891570b59f8d330a2daf793646917e}

For evaluation, prefer pure functions and immutable fixtures. For a tool write, persist its operation ID and use downstream idempotency or a status lookup. A local "started" record alone can't distinguish a crash before the remote effect from a crash after it. Reconcile uncertain outcomes rather than automatically replaying them.

Catalog and model terms

A model string in a tutorial isn't a production approval. The service can add or remove models. A third-party model can have a separate license, usage restriction, or attribution requirement. Query capabilities, retain the response, and link your run manifest to the model terms you accepted.^{[10]Reference 10Tinker Terms of Servicehttps://thinkingmachines.ai/legal/terms/}

Source-reading map

Read the repositories in an order that follows one request instead of scanning every file.

SDK snapshot

1. src/tinker/lib/public_interfaces/service_client.py: client creation, model capability calls, and the boundary between base and saved models.
2. src/tinker/lib/public_interfaces/training_client.py: request sequencing, request chunking, forward_backward, optim_step, save, and load behavior.
3. src/tinker/lib/public_interfaces/sampling_client.py: sample, logprob options, multiprocessing, and sampler creation.
4. src/tinker/types/datum.py: tensor conversion and serialization assumptions. Construction alone doesn't verify semantic token alignment.
5. src/tinker/proto/request_conv.py: how a Datum becomes a wire request.
6. src/tinker/lib/api_future_impl.py: how remote completion and errors reach local code.
7. src/tinker/resources/training.py and src/tinker/resources/sampling.py: HTTP endpoint names and response conversion.
8. Repository-root LICENSE and pyproject.toml: license, dependencies, and package identity.

The key path is TrainingClient._run_fwd_bwd to _take_turn to resources/training.py. That path explains why multiple chunks can be submitted concurrently while sequence IDs keep server order. The optimizer path uses the same turn counter. The sampler path is separate and can be created from a saved sampler checkpoint.

Cookbook snapshot

1. tinker_cookbook/rl/types.py: Env, trajectories, transitions, stop reasons, and group notation.
2. tinker_cookbook/rl/data_processing.py: advantage centering, trajectory splitting, masks, and datum assembly.
3. tinker_cookbook/rl/train.py: clock-cycle pipeline, loss selection, training logprob diagnostics, and checkpoint hooks.
4. tutorials/104_first_rl.py: the smallest readable GRPO-style loop.
5. tutorials/103_async_patterns.py: concurrent sampling and group batching.
6. tutorials/501_export_hf.py: adapter download, merge, and validation.
7. CONTRIBUTING.md: builder conventions, async guidance, testing, and review expectations.
8. LICENSE: Apache-2.0 grant for the cookbook source.

The SDK and Cookbook revisions are pinned at the start of this walkthrough. Their implementations can be tested locally, but no client-side test pins server availability, scheduling latency, or learning outcomes.

Tinker versus DeepEP

Both projects matter for frontier-scale post-training, but they solve different problems. DeepEP's primary interface is expert-parallel dispatch/combine; its V2 snapshot also has experimental communication primitives beyond expert parallelism. It isn't a hosted training API.^{[11]Reference 11DeepEPhttps://github.com/deepseek-ai/DeepEP/tree/dd758caf451848bd150e1046af3d0a73e5fff38d[12]Reference 12DeepEP MIT Licensehttps://github.com/deepseek-ai/DeepEP/blob/dd758caf451848bd150e1046af3d0a73e5fff38d/LICENSE}

DeepEP can sit below a self-hosted trainer that runs models like DeepSeek's MoE systems. It doesn't replace the RL loop. Tinker can help a researcher run a reward experiment on an available MoE model without assembling expert-parallel collectives. It doesn't expose DeepEP's dispatch buffers, RDMA tuning, or expert routing internals.

The projects complement each other in a curriculum. Tinker teaches the algorithm contract and service boundary. DeepEP teaches the communication contract that makes sparse expert training and inference fit a real cluster. Megatron or another distributed trainer connects both contracts into a full-parameter or adapter training runtime.

Don't benchmark them as alternatives. A hosted service's wall-clock latency and a communication kernel's tokens-per-second number answer different questions. Compare the total experiment cost and reproducibility boundary, not one operation's microbenchmark.

A practical reading exercise

Use the tiny deterministic clamp task before trying a large model.

1. Build one SFT datum with a prompt, target tokens, and response-only weights.
2. Submit one forward/backward request for that batch.
3. Submit its optimizer request before awaiting both results; inspect both outcomes.
4. Save a sampler checkpoint with a step name.
5. Sample four completions from the saved path.
6. Grade the actual completions with a sandboxed evaluator. Don't force the illustrative 1/0/1/0 reward pattern.
7. Center rewards and inspect token-aligned advantages.
8. Run the stdlib contract check above.
9. Only then add asynchronous sampling or a real environment.

The exercise should produce a run manifest alongside reward numbers. Record the model identifier, rank, target modules, renderer, loss, optimizer settings, checkpoint path, sample tokens, reward list, group mean, and mask counts. A manifest lets another reader inspect every contract boundary; screenshots omit that evidence.

For a large model, scale one dimension at a time. Increase prompt length, group size, number of concurrent environments, adapter rank, or pipeline depth separately. A reward change after four simultaneous changes can't tell you which boundary moved.

Evaluation rubric

• Trace ownership: distinguish local client handles from hosted adapter, optimizer, and sampler state, without inferring server internals from an SDK checkout.
• Audit one trajectory: derive shifted targets, preserve behavior logprobs, include the first answer token, and explain why equal lengths alone aren't sufficient.
• Design recovery: pair updates in order, bound in-flight work, choose an optimizer-preserving resume point, and record the sampler version separately.

Follow-up questions

Can an identical-reward group still update the adapter?

Centering makes its reward advantages zero, but a KL penalty or another auxiliary objective can still contribute gradients. Filtering removes the group from the update, including any terms computed on those samples; retaining it doesn't manufacture reward contrast.

Why can evaluation remain stale after a successful optimizer step?

The training adapter changed, but the sampler snapshot didn't. Publish sampler-compatible weights, select that saved path, and record it beside the trajectory. An optimizer success receipt alone doesn't prove which policy generated a sample.

Mastery quiz