ATIF 2.0 RFC Explainer
1 / 1
RFC 0004 · ATIF-v2.0 · Draft

ATIF 2.0 RFC Explainer

An interactive walkthrough of RFC 0004 for engineers familiar with ATIF 1.x: append-only event streaming (ATIF-E), consolidated trajectories (ATIF) and bundles (ATIF-B), unified tool actions, branch lineage, and lossless v1.x ↔ v2.0 migration.

ATIF-E · ATIF · ATIF-B stream consolidation unified Actions 26 canonical actions DAG lineage inline previews + blobs namespaced extensions
Keyboard shortcuts: →/Space next · ← back · Home/End · T slide index · D dark/light theme.
ATIF 1.x trajectory.json single monolithic document ATIF 2.0 ATIF-E · Events .atif-e.jsonl ATIF · Trajectory .atif.json ATIF-B · Bundle .atif-b.json / .tar.zst consolidate upgrade_v1_to_v2
Baseline

ATIF 1.x Schema Overview (v1.0–v1.8)

ATIF evolved through nine minor versions from ATIF-v1.0 to ATIF-v1.8. Select a version on the right to see the fields it introduced.

harbor/models/trajectories (v1.8)Pydantic, extra="forbid"

        
Why v2.0? Every 1.x release kept the same core structure: a single JSON file, a linear steps[] array, separate tool_calls and observation lists joined by ID, and untyped extra dicts. Supporting live streaming, non-linear execution, and cross-harness analytics required structural changes rather than additional optional fields.
Motivation

Five Limitations of ATIF 1.x

Using ATIF across 30+ agent harnesses and downstream consumers (NeMo, Phoenix, Letta, SkyRL) surfaced five structural limitations in the 1.x design.

Architecture

Three Self-Identifying Formats: ATIF-E, ATIF, and ATIF-B

ATIF 2.0 separates runtime event capture from analytical storage. Harnesses append events to an ATIF-E stream (.atif-e.jsonl), and a deterministic consolidation algorithm folds that stream into a single ATIF trajectory (.atif.json) or a multi-trajectory ATIF-B bundle (.atif-b.json).

File extensions and schema_kind: File extensions distinguish the three formats on disk. Inside the payload, schema_kind: "event" | "trajectory" | "bundle" (alongside schema_version: "ATIF-v2.0") identifies the document when read from stdin, HTTP, PubSub, or a SQL JSON column, and serves as an O(1) discriminator for Pydantic, Zod, and JSON Schema validators.
ATIF-E · Events

ATIF-E: The TrajectoryEvent Envelope

Each line in an .atif-e.jsonl file has a shared envelope and exactly one payload key: trajectory, step, or action. Events record only the fields known at emit time; derived fields (step_id, final_metrics, union type discriminators) are omitted and computed during consolidation.

session.atif-e.jsonl

        

Envelope fields

schema_kindAlways "event" on an ATIF-E line ("trajectory" / "bundle" on ATIF / ATIF-B roots).
event_idUnique within trajectory_id. Preserves the harness's native event ID (such as Claude Code's uuid).
parent_event_idOptional causal predecessor. Omitted on normal linear events (which implicitly link to the preceding event of the same trajectory_id); set explicitly when a branch or rewind jumps back to an earlier event.
response_idProvider inference ID. Events sharing a response_id always fold into the same Step.
seqOptional monotonic sequence number.
Streaming Durability

Append-Only Streaming vs. Whole-File Writes

In ATIF 1.x, a producer must either buffer the trajectory in memory until the run finishes or rewrite the entire JSON file after every step. ATIF-E appends one JSONL line per native event. Drag the slider to compare what is preserved if the container terminates mid-run.

Why this matters: In-memory buffering loses the entire trajectory when a container times out or crashes, while rewriting trajectory.json on every turn causes quadratic I/O. Appending one JSONL line per event preserves partial runs on disk and supports live rollout streaming for online RL.
Supplemental 0004-B

Mapping Native Harness Events to TrajectoryEvent

A streaming adapter maps each native harness record directly to an ATIF-E event without buffering multi-event turns. Select a harness event type below and hover or click any field to inspect its mapping.

identity / correlation key transformed copied
native field
rule
TrajectoryEvent field
resulting .atif-e.jsonl linepretty-printed

          
Stream Consolidation · Phase 1

Phase 1: Resolving the Event DAG into Linear Paths

Using the example stream from RFC 0004 §6.5: after a user prompt, one streamed inference, and a tool output (ev-4), generation ev-5 is cancelled and replaced by a retry (ev-6) that branches from ev-4. Step through the three stages of Phase 1:

Comparison with 1.x: In ATIF 1.x, rewound or retried turns either had to be dropped or left inline in steps[], breaking causal ordering. In ATIF 2.0, the active trunk remains strictly linear while discarded branches are extracted as child Trajectory objects.
Stream Consolidation · Phase 2 + 3

Phase 2: Folding Events into Steps

Phase 2 folds each linear event chain using a three-state machine (IDLE → GENERATING → OBSERVED), an in-place actions_by_id pointer map, and a steps_by_response_id index. Step through the events below to watch the consolidated trajectory build.

traj-1 (ATIF · Trajectory, in progress)

        
Field-level merge rules

Phase 2 & 3: Merging Metrics, Folding Content, and Computing Derived Fields

Harnesses such as Claude Code emit a single API response across multiple JSONL records and repeat cumulative usage on each slice. Compare summing across slices vs. replacing with the latest snapshot:

3 slices of msg_01XYZ

Adjacent folding & guards (content, output)

Parts are processed in encounter order. Adjacent text parts (and unsigned, non-redacted reasoning parts with matching summarized) concatenate in place only when neither part has truncated_fields or extensions set. Signed or redacted thinking blocks stay separate.

Phase 3: derived fields & truncation

  1. schema_kind ("trajectory" / "bundle"), and type on every ContentPart and CanonicalAction (media type inferred from source.media_type).
  2. Inline preview truncation (truncated_fields + optional BlobReference) on oversized fields not already truncated in ATIF-E.
  3. Lineage: parent_step_id, ancestor_trajectory_ids, child_trajectory_ids, trajectory_depth, using an inverted index of event and action IDs.
  4. step_id numbered 1..N in fold order; when parent_steps_excluded is true, leading prefix steps unchanged from the parent are dropped (normally starting at parent_step_id + 1, or parent_step_id on a mid-step fork).
  5. CompactionInfo.step_id_start/end resolved from event IDs.
  6. status, if no { trajectory } event set it, from steps[-1].stop_reason: completed → completed, cancelled → cancelled, errored/timed_out/refused → failed, tools/max_tokens/unset (or no steps) → running.
  7. start_time, end_time, step_count, final_metrics (every step in this Trajectory, keeping explicit vendor totals; a bundle sums only native steps, original_trajectory_id == null).
total_llm_calls counts each agent step with model content or metrics (one inference by construction) plus tool-internal Action.metrics.
ATIF · Trajectory

Trajectory Root Schema: 1.x vs. 2.0

Select a category below to filter the schema diff, or click any field to see how it changed from ATIF 1.x.

ATIF-v1.8 · trajectory.py− removed · ~ changed
ATIF-v2.0 · Trajectory+ added · ~ changed
Identity & lineage

Run Identity (session_id) vs. Path Identity (trajectory_id)

ATIF v1.7 introduced separate session_id and trajectory_id fields, but both remained optional. In ATIF 2.0 both are required on every Trajectory, making (trajectory_id, step_id) a composite primary key across JSON, Protobuf, and SQL.

Lineage fields on every child

parent_trajectory_idimmediate parentraw
parent_reasonsubagent · fork · rewind · retry · candidate · compaction · continuationraw
parent_event_id / parent_action_idwhere it branched / which tool call spawned itraw
parent_steps_excludedDelta Mode: untouched shared prefix steps omittedraw
parent_step_idresolved step in parentderived
ancestor_trajectory_idsroot…parent, so SQL needs no recursive CTEsderived
trajectory_depth, child_trajectory_idstree helpersderived
ATIF · Step

Step Structure: ATIF 1.x vs. ATIF 2.0

Step 2 from RFC 0001's GOOGL example (two parallel tool calls), converted to ATIF 2.0 via upgrade_v1_to_v2. Click the buttons below to highlight how each field group maps between versions.

ATIF-v1.5 · steps[1]
ATIF-v2.0 · steps[1]
Step.content[]

Preserving Interleaved Reasoning, Text, and Tool Calls in Step.content[]

Within a single LLM response, models frequently interleave signed or redacted thinking blocks, text responses, and parallel tool calls before any tool executes. Compare how ATIF 1.x and ATIF 2.0 represent a multi-block turn:

Step.actions[]

Unifying tool_calls and observation into Step.actions[]

ATIF 1.x stores tool invocations in tool_calls[] and tool outputs in observation.results[], requiring consumers to join them by tool_call_id == source_call_id. ATIF 2.0 combines each invocation, its output, timing, and error state into a single Action object.

ATIF 1.x step 4

tool_calls[]
observation.results[]
Click a tool call.
ATIF 2.0 step 4 · actions[]

        
Action anatomy

First-Class Action Metadata: Timing, Background Tasks, Approvals, and Subagents

In ATIF 1.x, execution status, per-call timestamps, background task handles, and human approval decisions were stored in harness-specific extra keys (with 6 incompatible key names for tool errors alone). ATIF 2.0 defines typed fields for each:

Action.canonical · Supplemental 0004-A

Normalizing Tool Semantics with Action.canonical

Action.name, arguments, and output always preserve the raw harness tool call. Action.canonical is an optional typed overlay that normalizes common operations across harnesses. Select an operation and a harness below:

→
Raw (varies per harness)
Action.canonicalidentical for every harness

        
Why keep raw tool fields alongside canonical? SFT and replay pipelines need the exact tool name and argument schema emitted by the model, while cross-harness SQL queries need a consistent schema across harnesses. Storing Action.canonical as an optional overlay (a single-key oneof, optional MCP provenance in canonical.mcp, and a derived type) supports both without mutating the raw call.
CanonicalAction

The 26 CanonicalAction Variants

Select any variant below to view its typed fields and native tool mappings. Protobuf field tags are allocated in 10-tag blocks per family.


        
Non-linear execution

Representing Rewinds and Retries: Delta Mode vs. Self-Contained Mode

Suppose a user tries steps 4–5, rewinds to step 3, and continues with a new prompt for steps 4–6. Compare how ATIF 1.x, ATIF 2.0 Delta Mode, and ATIF 2.0 Self-Contained Mode represent the run:


        
TrajectoryBundle

Flattening Multi-Trajectory Runs in TrajectoryBundle

ATIF v1.7 nested subagents recursively inside the parent trajectory. ATIF 2.0 flattens every linear path in a session (subagents, forks, Best-of-N candidates, compactions, and discarded rewind/retry branches) into TrajectoryBundle.trajectories[]:


        

Topology → parent_reason → prefix handling

TopologyreasonPrefix 1..KFirst step_id
Root—n/a1
Delegated subagent / compactionsubagent compactionfresh context, or copied when context-forked1
Rewind / 1-step retryrewind retryomitted in bundle (parent_steps_excluded: true)K+1 (or K mid-step)
Best-of-N armcandidateomitted in bundle (parent_steps_excluded: true); winner merged back with original_trajectory_idK+1 (or K mid-step)
Session forkforkcopied by default (flag omitted)1
Context continuationcontinuationfrom v1 is_copied_context1
Why represent branches as Trajectory objects? Adding a separate branches[] structure would duplicate lineage metadata and require a 3-column (trajectory_id, branch_id, step_id) primary key in SQL. Modeling every linear path as a Trajectory keeps steps[] strictly linear everywhere.
Step.compaction

In-Place Context Compaction with Step.compaction

Claude Code (compact_boundary), Codex (compacted), OpenCode, Pi, and Terminus 2 all compact context during long runs. Compare the two ATIF 1.x patterns with ATIF 2.0's first-class Step.compaction:


        
Active context tokens around the compaction step
TruncationInfo · BlobReference

Payload Truncation and Content-Addressed Blobs

Unbounded command outputs, file reads, and media produced 30–100+ MB trajectory files in 1.x, while ImageSource.path required a local filesystem. In ATIF 2.0, consolidation keeps an inline preview and records truncated_fields with an optional content-addressed BlobReference:

Recommended inline thresholds: 16 KiB for text/JSON, 0 bytes (always by reference) for binary media.

        

.atif-b.tar.zst layout

bundle.atif-b.json      # root manifest, first entry
events.atif-e.jsonl     # optional raw ATIF-E stream
blobs/sha256/4a/4a7c…e1 # <2-hex>/<64-hex>, raw bytes
blobs/sha256/c0/c09f…2b
AgentMetadata · ModelMetadata · EnvironmentMetadata

Structured Agent, Model, and Environment Metadata

ATIF 1.x lacked structured fields for system instructions, skills, MCP servers, subagent definitions, OS/sandbox settings, and VCS state, so converters placed them under custom keys in agent.extra. Select a section below to inspect the ATIF 2.0 schema:

1.x agent
2.0 agent + environment
StepMetrics · FinalMetrics

Token Metrics and Cross-Provider Normalization

ATIF 2.0 retains the v1.2 inclusive invariant: input_tokens includes cached tokens, and output_tokens includes reasoning tokens. Because model providers report token buckets differently, Supplemental 0004-C specifies exact normalization formulas per provider:

StepMetrics
1.x2.0
prompt_tokensinput_tokens
completion_tokensoutput_tokens
cached_tokenscache_read_tokens
extra.{5 aliases}cache_write_tokens new
extra.reasoning_tokens …reasoning_tokens new
Step.llm_call_countextensions["com.harborframework.v1"] moved (1 agent step = 1 LLM call)
—cost_source, retry_count, time_to_first_token_ms, streaming_duration_ms, response_id, granular[]
Extensions

Namespaced extensions and Promoted Core Fields

Across 30 ATIF 1.x converters and integrations, identical concepts were stored under conflicting extra keys (5 names for cache-write tokens; 6 for tool errors). ATIF 2.0 promotes universal fields into the core schema and requires reverse-DNS or URL namespaces in extensions:

1.x key seen in the wild

Official com.harborframework.* namespaces

com.harborframework.evaluationTrajectory / Bundlereward, correct, verdict, benchmark, task_id, rollout_rank, breakdown…
com.harborframework.rlTrajectory + Stepprompt_id, group_index, masked_out · per step reward, done, policy_version, assistant_masks, env_tokens…
com.harborframework.v1any containerschema_version, llm_call_count, message_was_string, continued_trajectory_ref, subagent_trajectory_ref, extra, tool_call_extra, observation_extra
where it lands in 2.0
Supplemental 0004-C

Lossless v1.x ↔ v2.0 Conversion (upgrade_v1_to_v2)

upgrade_v1_to_v2 converts any ATIF v1.0–v1.8 document to v2.0, and downgrade_v2_to_v1(upgrade_v1_to_v2(doc)) preserves full semantic equivalence (with timestamps normalized to UTC RFC 3339) whenever extensions["com.harborframework.v1"] is retained. Select a rule below to highlight its effect on RFC 0001's GOOGL trajectory:

trajectory.json (ATIF-v1.5)
trajectory.atif.json (ATIF-v2.0)
Supplemental 0004-D

Protobuf v3 Schema and 4-Table SQL Warehouse Projection

The TypeScript interfaces map 1:1 onto proto3 (using oneof for unions and string for open vocabularies) and onto a 4-table analytical SQL schema. Compare querying shell command failure rates and average cost by agent and model in ATIF 2.0 vs. ATIF 1.x:

trajectories PK trajectory_id parent_*, ancestor_ids agent STRUCT, final_metrics part. DATE(end_time) steps (trajectory_id, step_id) content ARRAY<STRUCT> metrics, compaction actions (trajectory_id, step_id, action_id) canonical STRUCT, output_text blobs cluster by digest urn:sha256 dedup across corpus
Because every branch is stored as its own row in trajectories, (trajectory_id, step_id) is unique across the warehouse, and excluding discarded branches requires only a filter on parent_reason.

        
Reference

ATIF 1.x → 2.0 Field Mapping Reference

Summary

Summary for Producers and Consumers

Overview of the six structural changes in ATIF 2.0 and practical checklists for trajectory producers and consumers.

If you produce trajectories (harness / adapter)

  • Emit ATIF-E (.atif-e.jsonl): one schema_kind: "event" line per native event, with native IDs as event_id. Set parent_event_id only on a branch or rewind jump.
  • Set response_id whenever the provider gives one. Leave out derived fields.
  • Normalize tokens to the inclusive invariant at emit time.
  • Fill in canonical when you know the operation's meaning.
  • Keep ATIF-E untruncated by default; Phase 3 consolidation truncates >16 KiB fields to a readable preview + truncated_fields (and optional blob). Media goes in source.
  • Put custom keys under your own reverse-DNS namespace (on Trajectory, Step, ContentPart, Action, or catalog items).

If you consume trajectories (eval / SFT / RL / UI)

  • Read consolidated ATIF (.atif.json, or Harbor's trajectory.json; dispatch on schema_kind): steps is still a contiguous, linear 1..N.
  • Iterate step.actions, not two arrays joined on IDs.
  • Filter training data on original_trajectory_id == null (previously is_copied_context).
  • Use ATIF-B (.atif-b.json) for subagents and branches, and filter on parent_reason.
  • Group on canonical.type rather than per-harness tool-name lists.
  • Run old files through upgrade_v1_to_v2; nothing is lost.