Architecture

Overview

The research-tools extension follows a layered architecture that mirrors Hypercontext's own design patterns while remaining completely decoupled from the core package.

When Hypercontext is installed as a Python package, the extension can be present in the same environment, but it remains dormant until the user explicitly enables it with HYPERCONTEXT_ENABLE_RESEARCH_TOOLS=1 or MetaAgent(enable_research_tools=True).

┌─────────────────────────────────────────────────────────┐
│                    Hypercontext Core                    │
│  HyperContext → EvolutionLoop → ToolRegistry → Agents   │
└──────────────────────┬──────────────────────────────────┘
                       │  (explicit registration only)
                       ▼
┌─────────────────────────────────────────────────────────┐
│               Research Tools Extension                  │
│                                                         │
│  ┌──────────────────┐  ┌──────────────────────────┐     │
│  │  Adapter Layer   │  │   Tool Output Types      │     │
│  │  (adapter.py)    │  │   (types.py)             │     │
│  │                  │  │                          │     │
│  │  register_       │  │   ToolRunReport          │     │
│  │  research_       │  │   MetricSnapshot         │     │
│  │  tools()         │  │   ExperimentProposal     │     │
│  └──────┬───────────┘  │   ExperimentObservation  │     │
│         │              │   DecisionSummary        │     │
│         │              │   ResearchSessionRecord  │     │
│         │              └──────────────────────────┘     │
│         ▼                                               │
│  ┌──────────────────┐  ┌──────────────────────────┐     │
│  │  Experiment Tool │  │   Session Tool           │     │
│  │  (experiment_    │  │   (session_tool.py)      │     │
│  │   tool.py)       │  │                          │     │
│  │                  │  │   init_session()         │     │
│  │  create_         │  │   run_next_iteration()   │     │
│  │  proposal()      │  │   finalize_session()     │     │
│  │  evaluate_       │  │   split_session()        │     │
│  │  proposal()      │  │   export_session()       │     │
│  │  run_experiment_ │  │                          │     │
│  │  loop()          │  │                          │     │
│  └──────────────────┘  └──────────────────────────┘     │
└─────────────────────────────────────────────────────────┘

Data Flow

Bounded Experiment Flow

User / Agent
    │
    ▼
create_proposal(description, hypothesis, baseline_metrics)
    │
    ├──→ ExperimentProposal (dataclass)
    │
    ▼
evaluate_proposal(proposal)
    │
    ├──→ Call evaluation_fn(proposal_dict)  ← User-provided function
    │        │
    │        ▼
    │    {metrics, duration_seconds, error?}
    │
    ├──→ Classify outcome (improvement / regression / neutral / crash)
    │
    ├──→ Make decision (keep / discard)
    │
    ├──→ Update best_metric if improved
    │
    ├──→ Append to history
    │
    ▼
ToolRunReport (structured output)
    │
    ├──→ findings: List[str]
    ├──→ metrics: List[MetricSnapshot]
    ├──→ recommendations: List[str]
    ├──→ archive_candidates: List[Dict]
    └──→ raw_output: JSON string

Research Session Flow

User / Agent
    │
    ▼
init_session(name, objective, max_iterations)
    │
    ├──→ ResearchSessionRecord (INITIALIZING phase)
    │
    ▼
run_next_iteration(description, context)  [repeated]
    │
    ├──→ Build iteration context (session state, history)
    │
    ├──→ Call iteration_fn(iteration_ctx)  ← User-provided function
    │        │
    │        ▼
    │    {observations, metrics, confidence, artifacts?, error?}
    │
    ├──→ Create ResearchIteration record
    │
    ├──→ Update session best_metric
    │
    ▼
ToolRunReport per iteration
    │
    ▼
finalize_session()
    │
    ├──→ Phase → FINALIZED
    │
    ├──→ Collect all observations, metrics, confidence
    │
    ├──→ Generate recommendations
    │
    ├──→ Identify archive candidates (high-confidence findings)
    │
    ▼
ToolRunReport (session summary)
    │
    ▼
export_session(path) → JSON file
    │
    ▼
split_session(name, objective) → New session inheriting best state

Where Hypercontext Remains in Control

Hypercontext is always the primary orchestrator:

1. ToolRegistry owns the tools — The extension registers tools on a Hypercontext ToolRegistry instance. Hypercontext agents discover and use these tools through the normal tool-use protocol.

2. No control flow inversion — The extension never calls into Hypercontext's evolution loop or agent system. It only receives calls from Hypercontext agents via the ToolRegistry.

3. State is external — The extension maintains its own state (experiment history, session records) that does not overlap with Hypercontext's archive, lineage, or evolution state.

4. Structured outputs — All tool outputs are ToolRunReport objects with structured fields (findings, metrics, recommendations, archive_candidates) that Hypercontext agents can parse and act on.

5. Explicit lifecycle — Sessions and experiments are explicitly initialized, run, and finalized. Nothing happens automatically.

6. Bootstrap is opt-in — the extension only registers when the enable flag is set or when register_research_tools() is called directly.

Session Lifecycle

                  init_session()
                       │
                       ▼
              ┌────────────────┐
              │  INITIALIZING  │
              └───────┬────────┘
                      │
                      ▼
              ┌────────────────┐
              │   ITERATING    │◄──────────────────┐
              └───────┬────────┘                   │
                      │                            │
            ┌─────────┴─────────┐                  │
            │                   │                  │
            ▼                   ▼                  │
     run_next_iteration()  finalize_session()     │
            │                   │                  │
            │                   ▼                  │
            │          ┌────────────────┐          │
            │          │  FINALIZED     │          │
            │          └────────────────┘          │
            │                                      │
            └──────────────────────────────────────┘
                      │
                      ▼
              split_session()
                      │
                      ▼
              New session with inherited state

Experiment Lifecycle

                  create_proposal()
                       │
                       ▼
              ┌────────────────┐
              │    PROPOSAL    │
              └───────┬────────┘
                      │
                      ▼
              evaluate_proposal()
                      │
            ┌─────────┼─────────┐
            │         │         │
            ▼         ▼         ▼
      IMPROVE    NEUTRAL    CRASH/TIMEOUT
            │         │         │
            ▼         ▼         ▼
          keep    keep if    discard
                   simpler    always
            │         │         │
            ▼         ▼         ▼
      Update best   discard   discard
      metric        (usually)
            │
            ▼
      Append to history
            │
            ▼
      ToolRunReport → Agent

Archive / Reflection Integration (Opt-in Only)

The extension produces archive_candidates in its ToolRunReport outputs. These are structured dicts that a Hypercontext agent could:

1. Feed into ArchiveStore for long-term storage
2. Include in EpisodicMemory for experience recall
3. Pass to ContextCompressor for context packing
4. Use with TransferScorer for cross-domain analysis

However, none of this happens automatically. The agent must explicitly read the archive_candidates from the tool output and decide what to do with them. This preserves Hypercontext's control over its own archive and memory systems.

Example of opt-in archive integration:

report = tool.evaluate_proposal(proposal)
for candidate in report.archive_candidates:
    if candidate["decision"] == "keep":
        archive_store.add(ArchiveEntry(
            gen_id=candidate["proposal_id"],
            scores={"score": candidate["metric_value"]},
            metadata={"source": "research_tools"},
        ))

Dependency Graph

research_tools/
├── types.py               (no dependencies)
├── experiment_tool.py     (depends on types.py)
├── session_tool.py        (depends on types.py)
└── adapter.py             (depends on experiment_tool.py, session_tool.py)
    └── Optional: hypercontext.agents.tools.registry (at call time only)

The native bootstrap path lives in Hypercontext core (hypercontext.bootstrap and MetaAgent(enable_research_tools=True)); it calls register_research_tools() only when the opt-in flag is enabled.

No dependency on hypercontext at import time. The adapter.py module uses TYPE_CHECKING to avoid importing Hypercontext types until the register_research_tools() function is actually called.