codebase_map.md · 17.3 KB
MindX Codebase Comprehensive Architecture Map
Last Updated: December 2024
Purpose: Complete system comprehension and architectural overview of the MindX Augmentic Intelligence codebase
I. ARCHITECTURAL PHILOSOPHY & DESIGN PRINCIPLES
Core Design Philosophy
MindX is architected as a
hierarchical multi-agent system with clear separation of concerns:
- Core Primitives (
core/): Fundamental building blocks of agency
- Augmenting Capabilities (
learning/, monitoring/, evolution/): Enhanced system capabilities
- Expansion Agents (
agents/): Specialized autonomous agents
- Orchestration Layer (
orchestration/): System conductors and coordinators
- Tool Ecosystem (
tools/): Executable capabilities and utilities
- Infrastructure (
utils/, llm/): Support systems and interfaces
Key Architectural Patterns
- BDI (Belief-Desire-Intention) Model: Core reasoning framework
- P-O-D-A Loop: Perceive-Orient-Decide-Act cognitive cycle
- Agent Composition: Hierarchical delegation and specialization
- Safe Self-Modification: Iterative improvement with rollback mechanisms
- Constitutional Framework: Immutable governance rules and validation
II. CORE SYSTEM COMPONENTS
Core Primitives (core/)
1. bdi_agent.py (45KB, 898 lines)
- Purpose: Fundamental cognitive architecture implementing BDI reasoning
- Key Classes:
BDIAgent, BaseTool
- Core Methods:
-
run(): Main cognitive loop (max 100 cycles)
-
plan(): Goal decomposition and action planning
-
execute_current_intention(): Action execution
-
deliberate(): Strategic decision making
- Internal Actions: ANALYZE_DATA, SYNTHESIZE_INFO, UPDATE_BELIEF, EXECUTE_STRATEGIC_EVOLUTION_CAMPAIGN
- Dependencies: BeliefSystem, LLM handlers, tool registry
- Integration: Base class for all intelligent agents
2. agint.py (15KB, 284 lines)
- Purpose: Augmentic General Intelligence - strategic cognitive core
- Key Classes:
AGInt, DecisionType, AgentStatus
- Core Methods:
-
_cognitive_loop(): P-O-D-A cycle implementation
-
_perceive(),
_orient_and_decide(),
_act(): Cognitive phases
-
_delegate_task_to_bdi(): Task delegation to BDI agents
- Decision Types: BDI_DELEGATION, RESEARCH, COOLDOWN, SELF_REPAIR
- Learning: Q-learning for strategic decision improvement
- Integration: Orchestrates BDI agents and coordinates high-level strategy
3. belief_system.py (8.2KB, 210 lines)
- Purpose: Knowledge storage and management system
- Key Classes:
BeliefSystem, Belief, BeliefSource
- Core Methods:
-
add_belief(),
get_belief(),
update_belief(): Knowledge management
-
query_beliefs(): Knowledge retrieval with filtering
- Persistence: JSON file-based belief storage
- Thread Safety: Singleton pattern with threading locks
- Integration: Central knowledge hub for all agents
4. id_manager_agent.py (8.5KB, 178 lines)
- Purpose: Cryptographic identity and wallet management
- Key Features:
- Ethereum-style key pair generation
- Deterministic agent identity creation
- Secure private key storage in environment variables
- Public address management and verification
- Integration: Provides identity infrastructure for DAIO framework
III. LEARNING & EVOLUTION SYSTEMS
Learning Layer (learning/)
1. strategic_evolution_agent.py (21KB, 392 lines)
- Purpose: High-level campaign manager for strategic self-improvement
- Key Classes:
StrategicEvolutionAgent, LessonsLearned
- Core Workflow:
1. Blueprint generation via
BlueprintAgent
2. Strategic plan creation and execution
3. Coordinator backlog seeding
4. Campaign monitoring and recovery
- Action Types: REQUEST_SYSTEM_ANALYSIS, SELECT_IMPROVEMENT_TARGET, FORMULATE_SIA_TASK_GOAL
- Integration: Bridges high-level strategy with tactical execution
2. self_improve_agent.py (42KB, 524 lines)
- Purpose: Code analysis, modification, and safe self-improvement
- Key Classes:
SelfImprovementAgent
- Safety Mechanisms:
- Iteration directories for isolated changes
- Versioned backups and rollback capability
- Self-testing with timeout protection
- LLM critique and validation
-
analyze_target(): Code analysis for improvement opportunities
-
implement_improvement(): Safe code modification
-
evaluate_improvement(): Post-change validation
-
run_self_improvement_cycle(): Complete improvement workflow
- Integration: Executes tactical code changes for evolution campaigns
3. plan_management.py (20KB, 388 lines)
- Purpose: Plan creation, execution, and state management
- Key Classes:
PlanManager, Plan, Action, PlanSt
- State Management: PENDING, IN_PROGRESS, COMPLETED_SUCCESS, FAILED
- Integration: Supports strategic planning across all agent types
4. goal_management.py (16KB, 318 lines)
- Purpose: Goal hierarchies and priority management
- Key Classes: Goal creation, priority queuing, goal state tracking
- Integration: Provides goal-oriented behavior for BDI agents
Evolution Layer (evolution/)
1. blueprint_agent.py (9KB, 190 lines)
- Purpose: Strategic evolution planning and system analysis
- Key Features:
- Holistic system state analysis
- Strategic blueprint generation
- BDI todo list creation for coordinators
- LLM-driven strategic planning
- Integration: Provides strategic intelligence for evolution campaigns
IV. AGENT ECOSYSTEM
Specialized Agents (agents/)
1. memory_agent.py (11KB, 265 lines)
- Purpose: Process logging, data persistence, and workspace management
- Key Features:
- Agent workspace creation and management
- Process trace logging with metadata
- Terminal output logging
- Data directory structure management
- Integration: Provides memory and persistence services to all agents
2. automindx_agent.py (7.3KB, 136 lines)
- Purpose: Dynamic persona generation and agent specialization
- Key Features:
- Role-based persona generation
- Agent behavior customization
- Dynamic agent deployment support
- Integration: Provides behavioral templates for specialized agents
3. guardian_agent.py (5.3KB, 125 lines)
- Purpose: Security enforcement and access control
- Key Features:
- Challenge-response authentication
- Private key access brokering
- Security policy enforcement
- Integration: Security layer for sensitive operations
4. simple_coder_agent.py (13KB, 232 lines)
- Purpose: Code generation and simple programming tasks
- Integration: Provides basic coding capabilities to other agents
V. ORCHESTRATION LAYER
System Orchestrators (orchestration/)
1. mastermind_agent.py (22KB, 409 lines)
- Purpose: Top-level system orchestrator and primary user interface
- Key Classes:
MastermindAgent (singleton pattern)
- Core Methods:
-
manage_mindx_evolution(): High-level evolution management
-
manage_agent_deployment(): Agent creation and deployment
-
command_augmentic_intelligence(): Primary command interface
- BDI Actions: ASSESS_TOOL_SUITE_EFFECTIVENESS, CONCEPTUALIZE_NEW_TOOL, CREATE_AGENT
- Integration: Primary entry point and system coordinator
2. coordinator_agent.py (22KB, 439 lines)
- Purpose: Tactical task coordination and improvement backlog management
- Key Features:
- Improvement backlog processing
- Agent interaction management
- Resource coordination
- Task delegation to SelfImprovementAgent
- Integration: Bridges strategic planning with tactical execution
VI. TOOL ECOSYSTEM
Executable Capabilities (tools/)
System Analysis & Intelligence
system_analyzer_tool.py (5.2KB): System health analysis and improvement identification
base_gen_agent.py (26KB): Codebase documentation generation and analysis
audit_and_improve_tool.py (5.5KB): Code quality auditing and improvement suggestions
Infrastructure & Operations
registry_manager_tool.py (6.1KB): Tool and agent registry management
system_health_tool.py (12KB): System health monitoring and diagnostics
shell_command_tool.py (1.6KB): System command execution
cli_command_tool.py (2.4KB): Command-line interface tools
Information & Communication
web_search_tool.py (8.8KB): Web search and information gathering
note_taking_tool.py (7KB): Information storage and retrieval
summarization_tool.py (6.9KB): Content summarization capabilities
Development & Analysis
tree_agent.py (1.4KB): File system exploration
llm_tool_manager.py (5.4KB): LLM capability management
VII. LLM INTEGRATION LAYER
Language Model Infrastructure (llm/)
Core LLM Framework
llm_interface.py (1.9KB): Standard interface for all LLM providers
llm_factory.py (11KB): LLM handler creation and management
model_registry.py (5.3KB): Model capability registration and discovery
model_selector.py (6.5KB): Intelligent model selection for tasks
rate_limiter.py (2.7KB): API rate limiting and throttling
Provider Implementations
gemini_handler.py (6.3KB): Google Gemini API integration
groq_handler.py (5.8KB): Groq API integration
ollama_handler.py (16KB): Local Ollama model integration
multimodel_agent.py (30KB): Multi-provider orchestration
mock_llm_handler.py (2.2KB): Testing and development mock
VIII. MONITORING & INFRASTRUCTURE
System Monitoring (monitoring/)
performance_monitor.py (15KB): Performance metrics collection and analysis
resource_monitor.py (21KB): System resource monitoring and alerting
Core Utilities (utils/)
config.py (4.6KB): Hierarchical configuration management with environment variable support
logging_config.py (3.3KB): Structured logging configuration
logic_engine.py (29KB): Formal reasoning and constraint validation
yaml_config_loader.py (1.1KB): YAML configuration file loading
IX. OPERATIONAL SCRIPTS
System Entry Points (scripts/)
1. run_mindx.py (29KB, 478 lines)
- Purpose: Primary CLI interface and system startup
- Key Commands:
-
evolve <directive>: Initiate evolution campaigns
-
deploy <directive>: Agent deployment
-
mastermind_status: System status reporting
-
analyze_codebase <path>: Codebase analysis
-
id_create/id_list/id_deprecate: Identity management
- Integration: Main user interface to MindX system
2. dmindx.py (19KB, 288 lines)
- Purpose: Deployment and system management utilities
- Integration: System deployment and configuration management
3. audit_gemini.py (11KB, 206 lines)
- Purpose: LLM model auditing and capability assessment
- Integration: LLM provider validation and configuration
4. run_mindx_coordinator.py (12KB, 206 lines)
- Purpose: Coordinator-specific operations and management
- Integration: Tactical coordination layer management
X. DATA & CONFIGURATION ARCHITECTURE
Configuration Management (data/config/)
basegen_config.json (7.1KB): Core system configuration including autonomous loops
official_tools_registry.json (8.5KB): Tool registration and capability definitions
official_agents_registry.json (2.8KB): Agent definitions and specifications
llm_factory_config.json (1.3KB): LLM provider configurations
agint_config.json (253B): AGInt-specific settings
Model Capabilities (models/)
gemini.yaml: Gemini model capability definitions and specifications
Persistent Data Architecture (data/)
data/
├── config/ # System configuration files
├── logs/ # Runtime and process logs
├── memory/ # Agent workspaces and process traces
│ ├── action/ # Action execution logs
│ └── agent_workspaces/ # Individual agent working directories
├── mastermind_work/ # Strategic campaign data
├── id_manager_work/ # Identity management data
├── gemini/ # LLM audit reports
└── improvement_backlog.json # System improvement queue
XI. SYSTEM INTERACTION FLOWS
Primary User Interaction Flow
- User Command →
run_mindx.py CLI
- Command Parsing → MastermindAgent method selection
- Strategy Formation → AGInt P-O-D-A loop
- Tactical Planning → BDI agent goal decomposition
- Execution Coordination → CoordinatorAgent task management
- Tool Execution → Specialized tool activation
- Result Integration → BeliefSystem knowledge update
Evolution Campaign Flow
- Directive Input → MastermindAgent.manage_mindx_evolution()
- System Analysis → SystemAnalyzerTool execution
- Blueprint Generation → BlueprintAgent strategic planning
- Campaign Creation → StrategicEvolutionAgent coordination
- Backlog Population → CoordinatorAgent task queuing
- Code Modification → SelfImprovementAgent execution
- Validation & Integration → Safety checks and belief updates
Autonomous Operation Flow
- Autonomous Loop Triggers → ConfigurationDriven intervals
- System Health Assessment → Monitor integration
- Improvement Identification → Analysis tool execution
- Task Prioritization → Backlog management
- Safe Execution → Iterative improvement with rollback
- Learning Integration → Q-learning and lessons learned
XII. KEY INTEGRATION PATTERNS
Agent Communication Patterns
- Hierarchical Delegation: MastermindAgent → AGInt → BDIAgent → Tools
- Peer Coordination: CoordinatorAgent ↔ StrategicEvolutionAgent
- Service Provision: MemoryAgent, GuardianAgent, IDManagerAgent → All Agents
Data Flow Patterns
- Belief System: Central knowledge repository for all agents
- Memory Agent: Process logging and workspace management
- Configuration Hierarchy: Environment → JSON → YAML → Defaults
Safety & Governance Patterns
- Constitutional Validation: All actions validated against governance rules
- Iterative Improvement: Safe self-modification with rollback capability
- Human-in-the-Loop: Critical decisions require human approval
- Guardian Access Control: Cryptographic security for sensitive operations
XIII. DEPLOYMENT & PRODUCTION CONSIDERATIONS
Production Deployment Structure (mindx_deployment/)
- Mirror Architecture: Complete replication of development structure
- Service Layer: Backend service management
- Frontend Interface: Web UI for system interaction
- Process Management: PID-based service control
Security Architecture
- Identity Management: Cryptographic agent identities
- Access Control: Guardian-mediated security enforcement
- Configuration Security: Environment variable-based secret management
- Audit Trails: Comprehensive logging and process tracing
Scalability Considerations
- Swarm Coordination: Parallel BDI agent execution
- Resource Monitoring: Adaptive resource allocation
- Rate Limiting: LLM API usage management
- Modular Architecture: Independent agent scaling
XIV. CODEBASE METRICS & COMPLEXITY
Scale Analysis
- Total Files: ~150+ Python files across all modules
- Core Components: ~50KB+ of fundamental cognitive architecture
- Learning Systems: ~100KB+ of self-improvement and evolution code
- Tool Ecosystem: ~100KB+ of executable capabilities
- LLM Integration: ~80KB+ of multi-provider language model support
Architectural Complexity
- Agent Hierarchy: 4-layer orchestration (Mastermind → AGInt → BDI → Tools)
- Configuration Layers: 5-tier configuration system (Env → JSON → YAML → Runtime → Defaults)
- Safety Systems: Multi-layer validation and rollback mechanisms
- Integration Points: 20+ major component integration interfaces
XV. DEVELOPMENT & EVOLUTION ROADMAP ALIGNMENT
Current Capabilities (Phase I Foundation)
- ✅ Core BDI cognitive architecture
- ✅ Strategic evolution agent framework
- ✅ Safe self-improvement mechanisms
- ✅ Multi-provider LLM integration
- ✅ Identity and security infrastructure
Active Development (Phase II-III)
- 🔄 Great Ingestion implementation for repository analysis
- 🔄 FinancialMind economic engine development
- 🔄 Constitutional framework smart contract preparation
- 🔄 Advanced tool suite expansion
Future Evolution (Phase IV-VI)
- 🔮 DAIO blockchain integration
- 🔮 Sovereign AI model training (Chimaiera-1.0)
- 🔮 Physical world API integration
- 🔮 Planetary-scale autonomous operations
This comprehensive map represents the current state of the MindX Augmentic Intelligence codebase as a sophisticated, self-improving AI system with clear architectural separation, robust safety mechanisms, and a path toward digital sovereignty. The system demonstrates significant engineering complexity while maintaining modularity and extensibility for future evolution.