Agent lifecycle

The agent layer is the stateful floor above the stateless ObanClaude seam: one agent = one :gen_statem process whose conversational turns run as ordinary ObanClaude.Worker jobs. The process never blocks on claude -- a prompt enqueues a job and the machine parks until the worker's callbacks report back. Provider sessions live in bounded, host-named conversation arcs, so one agent can keep an operator conversation separate from scheduled and task-specific work while remaining single-turn-at-a-time.

It is opt-in: nothing runs unless ObanClaude.Agent.Supervisor is in your tree, and the core seam (ObanClaude.run/2, ObanClaude.Worker) is untouched by it.

Quickstart

Add the supervisor after your Oban instance, and run the two queues:

children = [
  MyApp.Repo,
  {Oban, repo: MyApp.Repo, queues: [agents: 2, ticks: 1], ...},
  ObanClaude.Agent.Supervisor
]

Then, from anywhere:

{:ok, _pid} = ObanClaude.Agent.start_agent("a1", args: %{"model" => "haiku"})
:processing  = ObanClaude.Agent.submit_prompt("a1", "reply with just: hi")
{:ok, :idle} = ObanClaude.Agent.await("a1", :idle, 120_000)
{:ok, log}   = ObanClaude.Agent.history("a1")

Every config key has a default; :args is any string-keyed claude args map (build it with ObanClaude.Args.defaults/1). See ObanClaude.Agent.Instance for the full config table.

The lifecycle

A :state_timeout watchdog (:job_timeout, default 60s) guards :running against a turn that never reports back. Every state change atomically updates the registry -- ObanClaude.Agent.status/1 reads the state and the gated payload (action or question) in one messageless read -- and emits [:oban_claude, :agent, :transition] telemetry.

Directives: how a turn routes the machine

The machine routes on the turn's structured output. Give the agent a json_schema arg with a "directive" field and tell it (via append_system_prompt) when to use each value:

• "ask_user" + "question" -> :waiting_for_user
• "request_permission" + "action" -> :awaiting_permission
• anything else -> :idle

No schema means every turn returns to :idle -- fine for plain conversational agents.

Approvals actually unlock things

Conversational approval is only meaningful if it changes what the turn may do. The :approved_args config (e.g. %{"permission_mode" => "accept_edits"}, or a "worktree" for isolated edits) merges over the agent's args for approve continuations only -- normal turns run with whatever standing permissions you configured, which can be none at all.

Approved work is also not allowed to silently die: if the approved turn fails (a rail stop, a crash verdict) or hits the watchdog, the machine re-gates -- back to :awaiting_permission with the same description and a fresh action id, with {:approval_incomplete, reason} in history. Because the failed turn's session id was captured, a re-approval resumes the interrupted work rather than starting over.

Pause at a safe turn boundary

Hosts that enforce a spend, token, or policy rail often learn that an agent must pause while its turn is still running. Calling emergency_pause/1 at that point is too blunt: it deliberately drops action and question gates, and a later asynchronous pause can race a postponed prompt through :idle.

pause_after_turn/3 synchronously installs a correlated latch inside the state machine before the worker callback can finish the turn:

:telemetry.attach_many(
  "daily-spend-rail",
  [[:oban_claude, :run, :stop], [:oban_claude, :run, :exception]],
  fn _event, _measurements, %{job: %{meta: %{"agent_id" => id} = job_meta}}, _config ->
    case ObanClaude.Agent.pause_after_turn(id, :daily_spend, job_meta) do
      :ok -> :ok
      {:error, reason} -> Logger.warning("safe-boundary pause refused: #{inspect(reason)}")
    end
  end,
  nil
)

The telemetry handler must call pause_after_turn/3 inline. The worker sends its terminal callback only after telemetry handlers return, which gives the latch deterministic ordering. Starting another process for the call gives up that guarantee.

Treat a returned error as a visible, fail-closed race outcome. Do not raise in the handler: telemetry detaches handlers that crash.

Pass the exact job metadata captured at enqueue time. The machine validates the agent id, instance generation, and logical turn id atomically. A foreign, stale, retired, or malformed identity returns a typed error and changes nothing. Repeating the accepted source or current/latest owner identity is idempotent; the first pause reason wins. A matching completed turn remains correlatable while it is parked in :waiting_for_user or :awaiting_permission, so a host that learns about the rail after the gate opens can still install the latch without erasing the gate. A terminal turn that already reached :idle remains retired and cannot be latched afterward.

Ordinary completion, terminal failure, and watchdog expiry now land directly in :paused, before postponed prompts can run. A permission request or user question remains gated. Approving or answering grants one continuation while the latch stays active; that continuation pauses when it finishes, or remains gated if it requests another action or question. Rejecting a latched permission gate records the denial and goes directly to :paused.

Retries remain part of the same logical turn and keep the latch. An explicit resume_agent/1 clears it. emergency_pause/1 remains immediate and clears the latch together with all gate scopes.

For lifecycle changes that do not originate in turn telemetry, use quiesce/2. It chooses the boundary atomically inside the state machine, so a host does not need a racy status/1 read followed by a pause:

case ObanClaude.Agent.quiesce("caretaker", {:config_changed, revision}) do
  :paused -> restart_with_new_config()
  :armed -> wait_until_paused_then_restart()
  :already_paused -> restart_with_new_config()
  :draining -> wait_for_turn_then_retry_quiesce()
end

An idle agent pauses immediately. A running agent finishes its turn. A gated agent retains its question or permission and allows the continuation before pausing; rejecting a pending permission pauses immediately. Existing latches are left in place, so the first reason wins. :draining is not a replacement boundary: an emergency-paused turn still owns physical work, and the host must retry after its terminal callback retires that turn. quiesce/2 emits the same pause_reason and pause_action transition fields with cause: :quiesce.

An optional config_revision on start_agent/2 identifies the immutable host configuration applied to that provider process. It is an opaque non-empty string of at most 256 bytes. The provider does not compare or compute it:

ObanClaude.Agent.start_agent("caretaker",
  args: %{"model" => "sonnet"},
  config_revision: "routine-42:7"
)

The applied revision appears in info/1, every job's metadata, and Agent transition and turn-completion telemetry. It is omitted from metadata when no revision was supplied. The host can compare the observed revision with its desired revision, call quiesce/2, stop the paused process, and start a new one. The provider never mutates a running config in place.

Prompts: sync, async, and options

• submit_prompt/3 is a call: it blocks until the machine accepts the prompt (microseconds in :idle; the whole in-flight turn if postponed) and replies :processing. Backpressure for scripts.
• cast_prompt/3 is fire-and-forget: never blocks the caller. For LiveView handlers, schedulers, anything that must not wait.

Both take options: arc_id: "issue-651" selects an opaque conversation arc; omission uses the backward-compatible "default" arc. session: :fresh clears only the selected arc at delivery time. session: :fresh_fallback records that the host deliberately recovered from a failed resume. origin: :tick marks a scheduled delivery that must never be consumed as the answer to a pending question.

Conversation arcs

Seed known provider handles when starting or restoring the process:

{:ok, _pid} =
  ObanClaude.Agent.start_agent("caretaker",
    session_arcs: %{
      "operator" => persisted_operator_session,
      "issue-651" => persisted_issue_session
    },
    max_session_arcs: 32
  )

:processing =
  ObanClaude.Agent.submit_prompt("caretaker", "continue the issue",
    arc_id: "issue-651"
  )

Each job's metadata identifies arc_id, the input session_id, and its continuation_decision / continuation_reason. info/1 returns session_arcs, active_arc_id, and the current or most recent continuation. A terminal resume classified as :session_not_found, :invalid_session, :unknown_session, or :session_rejected produces a typed outcome: :session_rejected. The host can then reconstruct a durable handoff and deliberately submit session: :fresh_fallback; no local transcript is selected implicitly.

Claude can branch a retained session without changing its source arc:

:processing =
  ObanClaude.Agent.fork_arc(
    "caretaker",
    "issue-651",
    "issue-651-experiment",
    "try the alternate design"
  )

The returned child session is stored under the target arc. Least-recently used inactive handles are evicted when max_session_arcs is reached. Arc persistence and rotation policy remain the host application's responsibility. If the host still has a compatible exact handle after eviction, it can supply it with the turn so admission and enqueue happen in one state-machine event:

ObanClaude.Agent.submit_prompt("caretaker", "operator work",
  arc_id: "operator",
  resume_session_id: persisted_operator_session
)

The handle is used only when that arc is absent from the live cache. A different live handle returns {:error, {:session_conflict, arc_id}} without enqueuing; the host must reconcile its durable record. This does not verify that an evicted provider transcript still exists. A confirmed native rejection still requires the host's explicit recovery policy. Keep gate-bound answers on the arc that opened the gate.

Retries are one logical turn

ObanClaude.Agent.Job routes terminal-aware: {:cancel, _} or a final {:error, _} reports job_finished/3; a retryable failure or {:snooze, _} reports job_retrying/3, which keeps the machine in :running and re-arms the watchdog. The default worker stays max_attempts: 1 (every retry is a paid call); opt in with a three-line delegating worker -- see ObanClaude.Agent.Job.

Every job carries an opaque instance generation and logical turn id in its metadata. The instance checks both inside the state machine before changing state, session, approval, counters, or watchdogs. Late outcomes, duplicate callbacks, and callbacks from an earlier same-id process are retained only as bounded diagnostics. Custom workers that delegate their result and error callbacks to ObanClaude.Agent.Job inherit this behavior automatically.

Scheduling: a crontab entry is an agent

ObanClaude.Agent.Tick adapts Oban.Plugins.Cron (which schedules at the worker layer) to the machine (which owns turn enqueueing): a beat delivers a prompt through the facade, never a job behind the machine's back.

{Oban.Plugins.Cron,
 crontab: [
   {"0 9 * * *", ObanClaude.Agent.Tick,
    args: %{
      "agent_id" => "standup",
      "arc_id" => "daily-sweep",
      "prompt" => "Summarize overnight CI failures.",
      "session" => "fresh",
      "if_offline" => "start",
      "start" => %{
        "args" => %{"model" => "sonnet"},
        "config_revision" => "routine-42:7"
      }
    }}
 ]}

Policies per tick: if_busy ("skip" default / "queue"), if_offline ("skip" default / "start" -- the crontab becomes the agent's spec, so restarts self-heal at the next beat), session ("resume" default / "fresh", the routine-friendly choice). Ticks run on their own queue (:ticks): on a queue shared with agent turns a tick serializes behind the very turn it should observe as busy, and skip-policy can never fire.

A host can serialize Tick delivery with its own configuration boundary by setting config :oban_claude, tick_admission: MyApp.TickAdmission. The module must export admit/4. It receives :claude, the agent id, the optional "start.config_revision", and a zero-arity function containing the complete status check, offline start, and prompt cast. It must either invoke and return that function's result or return another valid Oban.Worker result. Without this configuration, Tick delivery is unchanged.

Observing a fleet

• status/1 -- one atomic registry read: the state, plus the pending action/question in the gated states. Never messages the process.
• await/3 -- block until the agent settles into given states; returns the full status payload. Registry-polling.
• list/0 -- every running agent as {id, status}, off the registry.
• info/1 -- applied config_revision, turn count, accumulated cost, default session_id, all retained session_arcs, current/recent continuation, pending gates, and the current deferred_pause latch. While paused, pause_context retains the cause, reason, action, and any correlated turn identity that applied the pause (a call; in-process, so the host seeds persisted arcs after restart).
• history/1 -- the bounded event log (:max_history, default 500). A safe boundary latch records {:pause_after_turn, reason} when accepted and {:paused_after_turn, reason} when it is applied. A host boundary records {:quiesce, reason} when armed or {:quiesced, reason} when an idle agent pauses immediately.
• Telemetry: [:oban_claude, :agent, :transition] with %{agent_id, from, to}, and [:oban_claude, :agent, :turn_completed] with the arc, session, continuation decision, and typed outcome. Pass an opaque correlation_id to submit_prompt/3 or cast_prompt/3 to carry an application request identity through postponed delivery, job metadata, turn transitions, approval continuations, and completion. Turn events also expose the wrapper-owned agent_generation and agent_turn_id; when set, config_revision appears on every transition and completion. Deferred pause transitions add cause (:pause_after_turn for a correlated latch, :quiesce for a host boundary, :emergency_pause for the immediate brake), pause_reason, and pause_action (:continued, :applied, or :cleared). Gate transitions also add gate_outcome (:opened, :approved, :answered, :incomplete, or :rejected) and the relevant action_id or question.

Testing without a queue or claude

The :enqueue_fun config replaces the Oban insert. Tests capture the metadata passed to that function and return it through job_finished/3, so they drive the whole machine with no DB and no claude while preserving turn ownership:

test_pid = self()

{:ok, _} =
  ObanClaude.Agent.start_agent("t1",
    enqueue_fun: fn args, meta ->
      send(test_pid, {:enqueued, args, meta})
      {:ok, :queued}
    end
  )

:processing = ObanClaude.Agent.submit_prompt("t1", "go")
assert_receive {:enqueued, %{"prompt" => "go"}, %{"agent_id" => "t1"} = meta}

:ok =
  ObanClaude.Agent.job_finished(
    "t1",
    {:ok, ObanClaude.Testing.result("done")},
    meta
  )
{:ok, :idle} = ObanClaude.Agent.await("t1", :idle, 1_000)

Build payloads with ObanClaude.Testing (result/1, structured_result/2 for directives, error/2 for failures).

What it deliberately does not do

The machine owns lifecycle, not operations. Durable spend ledgers and budgets, persisted journals/todos, gate records that survive restarts, notifications, dashboards, and MCP surfaces for agents-driving-agents are all application concerns -- they compose cleanly on top of the telemetry and the facade (each was built and live-proven in a demo app during this layer's incubation), and they carry dependencies this library should not.

Early native session observations

Early observations require a runner implementing the wrapper's run_observed/5 contract, such as its Forcola runner. The default Port runner continues using legacy one-shot execution and retains terminal session handles. Execution ownership and terminal fencing apply with either runner.

The default worker registers an execution before validating its arguments or calling the wrapper. execution_started identifies an accepted worker attempt; it does not claim that the CLI spawned. Invalid arguments therefore complete the same accepted attempt with the existing cancellation behavior. The Agent checks the job id, process generation, logical turn, arc, attempt and Oban snooze counter, then supplies a local PID/reference observer. That observer never enters persisted job arguments or metadata. Custom query_fun/2 functions retain their existing keyword contract; they can still complete through the normal worker callbacks without producing early observations.

The owning Agent emits [:oban_claude, :agent, :execution_started] before [:oban_claude, :agent, :session_observed]. Both events and the corresponding turn_completed event carry agent_id, agent_generation, agent_turn_id, arc_id, job_id, job_attempt, job_snoozed, and execution_state: :started. Configuration revision, correlation and continuation decision metadata remain available. Session observations add session_id and source (:system_init). These events are emitted synchronously by the owning Agent, in acceptance order. A host can persist the started tuple and accept only matching later events without calling back into the Agent from a telemetry handler.

ObanClaude.Agent.info/1 exposes the registered execution tuple in continuation and sets its session_id only after that execution's accepted observation. A first accepted handle wins; duplicate or conflicting observations and old references are ignored. Retrying closes the old observer and a new attempt must advance the attempt plus snooze watermark. Late terminal callbacks are also fenced by the captured job tuple and worker process.

Observation is evidence that the CLI created or resumed a session, not that work completed. Failure, cancellation, pause and watchdog expiry retain the observed handle. An explicit typed session rejection clears the rejected arc; completion includes rejected_arc_id and rejected_session_id. A rejected fork source cannot erase an unrelated target, and only an observed new child handle can replace the target on an interrupted fork.

A failed enqueue or a watchdog timeout before any worker registered emits execution_state: :not_started with no job tuple. Legacy/custom callbacks without registered execution omit all new execution fields. Tick admission, approval gates and deliberate fresh-session requests are unchanged. Durable storage, configuration compatibility and restart recovery remain the host application's responsibility.