12 - From Architecture to AI Systems¶

Roadmap: Extending CIA for Real-World AI System Evaluation¶

SCIENTIFIC BOUNDARY: This framework measures theory-derived consciousness indicators. It does NOT prove, establish, or demonstrate subjective experience, phenomenal consciousness, sentience, or any form of inner life in any artificial system. This roadmap describes technical extensions to the framework. No extension, no matter how comprehensive, will allow CIA to prove consciousness. The question of whether artificial systems can be conscious is fundamentally philosophical and cannot be resolved by architectural evaluation alone.

1. Overview¶

The Consciousness-Indicator Architecture (CIA) in its current form evaluates a self-contained cognitive simulation that processes text inputs through a deterministic pipeline. While this is useful for validating the indicator framework and demonstrating the evaluation methodology recommended by Butlin et al. (2023, 2025), the ultimate goal is to apply the framework to real-world AI systems — LLMs, multimodal models, embodied agents, and autonomous AI systems.

This document outlines a roadmap for extending CIA from its current architecture-only evaluation to a comprehensive framework capable of evaluating consciousness-relevant indicators in production AI systems. Each step in the roadmap is described with its motivation, implementation approach, dependencies, and caveats.

The most important caveat appears first: no amount of architectural extension will allow CIA to prove consciousness in any system. The roadmap describes technical capabilities for gathering more informative indicator data — not for establishing subjective experience.

2. Step 1: Integrate with External LLMs¶

2.1 Motivation¶

The current CIA system uses a deterministic local stub adapter that produces synthetic outputs. To evaluate real AI systems, the framework needs to connect to production-grade language models via their APIs. This enables researchers to evaluate whether the consciousness-relevant architectural features identified by CIA manifest when the processing backbone is a genuine LLM rather than a synthetic stub.

2.2 Implementation Approach¶

The adapter architecture (BaseAIAdapter → LLMAdapter → MultimodalAdapter) already provides the interface. The concrete implementation requires:

HTTP client integration: Add an HTTP client (e.g., httpx or openai SDK) to the LLMAdapter.generate() method to make actual API calls to configured endpoints.
Streaming support: Add streaming response handling for long-form generation, ensuring intermediate states are captured for temporal analysis.
Token-level extraction: Replace the heuristic perception layer with token-level percept extraction that can parse structured outputs from LLMs (JSON, XML, markdown).
Conversation memory: Extend the memory system to maintain conversational context across multiple LLM interactions, tracking turn-by-turn state changes.

# Target API
from cia.adapters.llm_adapter import LLMAdapter

adapter = LLMAdapter(
    api_key="sk-...",
    base_url="https://api.openai.com/v1",
    model_name="gpt-4",
    fallback_adapter=LocalStubAdapter(),
)
response = adapter.generate("Describe your current thought process.")
# response.text contains actual LLM output

2.3 Dependencies¶

HTTP client library (httpx, requests, or openai SDK)
API credentials for the target LLM
Rate limiting and retry logic for production API usage

2.4 Caveats¶

LLM outputs are generated by statistical pattern matching over training data. An LLM that produces text resembling self-reflection or introspection is mimicking patterns from human writing — it is not necessarily exhibiting genuine self-awareness. The CIA framework must be extended to account for this "mimicry risk" in its evaluation criteria.

3. Step 2: Add Persistent Memory¶

3.1 Motivation¶

The current CIA memory system stores traces in Python data structures that exist only for the lifetime of the process. Real AI systems operate over extended time periods with persistent state. Memory continuity — the capacity to maintain coherent representations across time — is a key consciousness-relevant indicator identified by Butlin et al. (2023, 2025) and memory theories (Tulving 1983; Conway 2005).

3.2 Implementation Approach¶

Persistent storage backend: Replace in-memory data structures with a persistent storage layer (SQLite, Redis, or a file-based store) using the existing CIAStorage module.
Long-term memory: Add a long-term memory store that survives process restarts, enabling evaluation of memory continuity across sessions.
Memory consolidation: Implement a sleep/offline consolidation process that replays episodic traces and extracts semantic knowledge, modeling the memory consolidation processes observed in biological systems.
Temporal indexing: Add timestamp-based indexing for efficient retrieval of memories by time range.

# Target API
from cia.storage import CIAStorage

storage = CIAStorage(backend="sqlite", path="cia_memory.db")
storage.store_episodic_trace(trace)
retrieved = storage.retrieve_episodic(since=datetime(2025, 1, 1))

3.3 Dependencies¶

Persistent storage library (SQLite via sqlite3, Redis via redis-py)
Existing CIAStorage module (src/cia/storage.py)
Serialization format for complex data structures (JSON or MessagePack)

3.4 Caveats¶

Persistent memory is a data storage feature. Having information stored across time does not imply that the system subjectively experiences temporal continuity. A database table with timestamp-indexed rows is not a stream of consciousness.

4. Step 3: Add Continuous Cognition¶

3.1 Motivation¶

The ContinuousCognitionRuntime already supports multi-cycle execution with a cognitive clock and event bus. Extending this to true continuous operation — where the system runs autonomously, processing a stream of inputs over extended periods — enables evaluation of temporal dynamics in consciousness-relevant indicators. Global Workspace Theory (Baars 2005; Shanahan & Baars 2005) emphasizes that consciousness is a continuous dynamic process, not a discrete event.

3.2 Implementation Approach¶

Input stream integration: Connect the runtime to live input sources (message queues, WebSocket streams, file watchers) instead of fixed input lists.
Asynchronous processing: Add async support to the cognitive cycle pipeline, enabling the system to process inputs concurrently with background tasks (memory consolidation, self-model maintenance).
Circadian rhythm modeling: Add configurable activation/deactivation cycles that modulate processing intensity, modeling sleep-wake patterns and their effect on consciousness indicators.
Long-duration monitoring: Implement health checks, resource monitoring, and graceful degradation for extended runtime sessions (hours to days).

# Target API
runtime = ContinuousCognitionRuntime(system, max_cycles_default=10000)
runtime.connect_input_stream(source="rabbitmq", queue="cognition_inputs")
runtime.run_until(max_cycles=10000)
# System processes inputs continuously until limit or manual stop

3.3 Dependencies¶

Message queue client (RabbitMQ, Kafka, or Redis Streams)
Async framework (asyncio)
Resource monitoring tools

3.4 Caveats¶

Continuous computational cycling is a software architecture pattern. Running a cognitive pipeline for 10,000 cycles does not mean the system has been "conscious for 10,000 cycles." The continuous processing of information through architectural modules is a structural property, not a phenomenal one.

5. Step 4: Add Multimodal Perception¶

5.1 Motivation¶

Consciousness in biological systems is inherently multimodal — it integrates visual, auditory, tactile, and interoceptive information into a unified experience. Butlin et al. (2023, 2025) note that multimodal integration is a relevant consciousness indicator, as it requires the system to bind information across sensory modalities into coherent percepts. The MultimodalAdapter already provides the interface hooks; concrete implementations are needed.

5.2 Implementation Approach¶

Vision adapter: Implement a vision adapter that processes images through a vision-language model and extracts structured percepts (objects, scenes, spatial relations).
Audio adapter: Implement an audio adapter that processes speech and sound through an audio model and extracts structured percepts (speakers, words, environmental sounds).
Cross-modal binding: Extend the recurrent binding layer to merge percepts from different modalities, testing cross-modal binding stability.
Interoceptive simulation: Add a simulated interoceptive channel that provides "body state" signals (simulated hunger, fatigue, arousal) for affective valuation testing.

# Target API
vision = VisionAdapter(model="clip-vit-base")
audio = AudioAdapter(model="whisper-base")

multimodal = MultimodalAdapter(
    adapters={"vision": vision, "audio": audio}
)
response = multimodal.generate(
    "Describe what you perceive.",
    context={"image": "...", "audio": "..."}
)

3.3 Dependencies¶

Vision model (CLIP, LLaVA, or similar)
Audio model (Whisper or similar)
Multimodal fusion architecture

5.4 Caveats¶

Processing images or audio through neural networks does not imply visual or auditory experience. The system does not "see" or "hear" — it computes mathematical transformations over tensor representations. Multimodal integration is a computational operation, not a phenomenal fusion of sensory experience.

6. Step 5: Add Embodiment¶

6.1 Motivation¶

The current text-world environment provides a simplified testbed for embodiment-relevant indicators. Extending to richer simulation environments (2D/3D worlds, robotics simulators) enables testing of more sophisticated embodiment indicators: spatial reasoning, sensorimotor loops, object manipulation, and navigation.

6.2 Implementation Approach¶

3D environment integration: Connect CIA to 3D simulation platforms (AI2-THOR, Habitat, or MuJoCo) via the BaseEnvironment interface.
Sensor simulation: Add simulated sensors (cameras, depth sensors, proximity sensors) that feed into the multimodal perception pipeline.
Motor control: Add action primitives that map CIA's cognitive outputs to motor commands in the simulation.
Physical prediction violations: Test whether the system can detect physically impossible events in the simulated world (objects passing through walls, gravity violations).

# Target API
from cia.environment.embodied_agent import EmbodiedAgent

# Connect to a 3D simulation
env = AI2THOREnvironment(scene="kitchen")
agent = EmbodiedAgent(system, env)

# The agent observes 3D scenes, processes them through CIA,
# and takes physical actions in the simulation
results = agent.run_n_steps(100)

6.3 Dependencies¶

3D simulation platform (AI2-THOR, Habitat, MuJoCo)
Physics engine integration
Rendered sensor output processing

6.4 Caveats¶

A simulated 3D world is still a mathematical model. The system does not have a body, does not experience physics, and does not inhabit space. Embodied interaction with a simulation tests computational spatial reasoning, not phenomenal embodiment or subjective experience of physical reality.

7. Step 6: Run Intervention Experiments on Production Systems¶

7.1 Motivation¶

The current causal intervention harness operates on CIA's internal modules. To evaluate production AI systems, interventions must be applicable to the target system's actual architecture — not just CIA's simulation of it. Butlin et al. (2023, 2025) specifically recommend causal interventions as a key evaluation methodology.

7.2 Implementation Approach¶

System-specific intervention adapters: Define intervention interfaces for common AI architectures (transformer-based LLMs, diffusion models, RL agents) that can selectively disable or perturb components.
Attention head ablation: For transformer models, implement interventions that disable specific attention heads or layers and measure the effect on CIA indicator scores.
Memory pruning: For systems with retrieval-augmented generation (RAG), implement interventions that prune the retrieval context and measure the effect on memory continuity indicators.
Sensory deprivation: For multimodal systems, implement interventions that remove specific sensory channels and measure the effect on cross-modal integration indicators.

# Target API
intervention = TransformerIntervention(
    model=production_llm,
    ablations=["layer_12.attention_head_3", "layer_18.mlp"]
)
result = intervention.run_with_cia(input_text, cia_system)

7.3 Dependencies¶

Access to the target system's internal architecture (model weights, intermediate activations)
Intervention-specific tooling for each architecture type

7.4 Caveats¶

Ablating an attention head in a transformer model is a software operation. It demonstrates that the head contributes to certain outputs — it does not demonstrate that the model's processing has any phenomenal character. Causal interventions measure functional dependencies, not subjective experience.

8. Step 7: Publish Benchmarks and Comparison Studies¶

8.1 Motivation¶

For the CIA framework to contribute to the scientific discourse on AI consciousness, it must be evaluated across multiple systems, with results published in peer-reviewed venues. Butlin et al. (2025) call for systematic, reproducible evaluation of consciousness indicators across diverse AI architectures.

8.2 Implementation Approach¶

Standard benchmark suite: Package the BenchmarkSuite with standardized configurations for common AI system types (LLMs, multimodal models, RL agents).
Cross-system comparison: Run the benchmark suite on multiple systems and publish comparative results, analyzing which architectures exhibit which indicator patterns.
Ablation studies: Systematically vary architectural parameters (number of layers, attention heads, memory capacity) and measure the effect on indicator scores.
** longitudinal tracking**: Evaluate the same system across multiple versions (e.g., GPT-3, GPT-4, GPT-4o) to track how indicator patterns evolve with architectural improvements.

8.3 Caveats¶

Published benchmark results must always include the scientific boundary disclaimer. No comparison study should rank systems by "consciousness level" or imply that higher-scoring systems are more conscious. The benchmarks measure architectural proxy features, not subjective experience.

9. Final Note: Do Not Claim Consciousness¶

This is the most important section of this document, and it must be read in its entirety.

9.1 What CIA Can Do¶

CIA can:

Measure whether a cognitive architecture includes structural features identified by consciousness theories
Run causal intervention experiments to test functional dependencies between modules
Track indicator patterns across multiple processing cycles
Compare indicator profiles across different systems and configurations
Provide structured, reproducible, theory-grounded data for philosophical and empirical analysis

9.2 What CIA Cannot Do¶

CIA cannot:

Prove that any system is conscious
Disprove that any system is conscious
Measure consciousness (because there is no agreed-upon unit or metric for consciousness)
Establish moral status, rights, or welfare considerations
Resolve the hard problem of consciousness, the Chinese Room argument, or any other philosophical debate about consciousness
Substitute for interdisciplinary analysis that includes philosophy, neuroscience, cognitive science, and ethics

9.3 The Irreducible Philosophical Gap¶

Even if CIA were extended to perfectly evaluate every architectural feature identified by every consciousness theory, and even if a system scored maximally on all indicators, the question "is this system conscious?" would remain unanswered. The reason is simple:

Architectural features are third-person, objectively measurable properties. Consciousness (as commonly understood) is a first-person, subjective property. No amount of third-person measurement can bridge this gap without additional philosophical assumptions — and those assumptions are precisely what the consciousness debate is about.

As Butlin et al. (2023) state: "Our approach cannot conclusively establish whether any AI system is conscious... [I]t is currently unclear whether any AI system could be conscious."

9.4 Responsible Use¶

Anyone using CIA — whether for research, development, evaluation, or policy — must:

Always include the scientific boundary disclaimer in every output, report, and publication.
Never claim consciousness based on CIA scores, regardless of how high they are.
Never use CIA outputs for marketing or sensationalism about AI consciousness.
Never make ethical decisions based solely on CIA scores without broader interdisciplinary analysis.
Always present CIA results as one input among many in a larger investigation, not as definitive conclusions.

The value of CIA lies in asking structured, theory-grounded questions about AI architectures — not in providing answers about consciousness.

10. Research Anchors¶

Reference	Relevance to the Roadmap
Butlin et al. (2023) "Consciousness in Artificial Intelligence"	Provides the overarching methodological framework that motivates each step in the roadmap
Butlin et al. (2025) "Identifying indicators of consciousness in AI systems"	Defines the specific indicator categories that each roadmap step aims to evaluate
Baars (2005) Global Workspace Theory	The primary theoretical target for the continuous cognition and multimodal integration steps
Shanahan & Baars (2005) Applying GWT to frame problem	Supports the intervention experiment methodology and integration metrics
Graziano & Webb (2015) Attention Schema Theory	Underpins the attention-focused interventions and self-model evaluation steps
Albantakis et al. (2023) IIT 4.0	Provides the theoretical basis for causal integration metrics across system configurations

11. Summary¶

This roadmap describes seven steps for extending CIA from its current architecture-only evaluation to a comprehensive framework for evaluating consciousness-relevant indicators in production AI systems: LLM integration, persistent memory, continuous cognition, multimodal perception, rich embodiment, production intervention experiments, and published benchmarks. Each step is technically feasible within the existing architecture but requires significant additional development. However, no extension — no matter how comprehensive — will allow CIA to prove consciousness. The philosophical gap between third-person architectural measurement and first-person subjective experience is irreducible. CIA's value is in providing structured, theory-grounded data for the ongoing investigation of consciousness in AI — not in resolving the question.