Project Overview
The Evolution: From RAG to Agent
Traditional RAG systems answer questions. Autonomous agents perform work. This project transforms a passive knowledge retrieval system into an active operational assistant that makes decisions and takes actions autonomously.
The Shift to Agentic AI
Moving from "tell me about compliance" to "audit this facility and handle it" - the agent autonomously orchestrates multiple tools to achieve goals
Multi-Tool Orchestration
LLM acts as the orchestrator, deciding which tools to invoke and in what sequence based on the objective
Real-World Actions
Not just information retrieval - sends emails, schedules tasks, logs maintenance, and generates reports automatically
Goal-Driven Execution
Give it a goal ("audit this facility"), and it breaks it down into steps and executes the complete workflow
Production Architecture
Designed for extensibility - easily add new tools, integrate real APIs, and scale to enterprise systems
RAG vs. Agent: The Fundamental Difference
Agent Architecture
🎯 User Goal
Natural Language Instruction
"Audit FAC-AB-001 and email results"🧠 LLM Agent Brain (GPT-4o)
LangChain Tool Calling Agent
Decides: which tools + what order🔧 Tool Orchestration (7 Tools)
1. search_aer_directives (RAG)
2. get_facility_equipment (DB)
3. check_calibration_compliance
4. send_compliance_report (Email)
5. schedule_follow_up (Calendar)
6. log_maintenance_action
7. list_facilities
⚙️ Mock Enterprise Systems
Simulated APIs
Equipment DB • Email • Calendar • Logs✅ Autonomous Actions
Complete Workflow Execution
Audit + Report + Schedule + LogCore Engineering Contributions
LangChain Agent with Function Calling
Challenge: Enable an LLM to autonomously decide which tools to use and when, without pre-programmed decision trees.
Solution: Implemented LangChain's Tool Calling Agent with GPT-4o function calling:
from langchain_openai import ChatOpenAI
from langchain.agents import AgentExecutor, create_tool_calling_agent
class AERComplianceAgent:
def __init__(self, api_key: str, model: str = "gpt-4o"):
self.llm = ChatOpenAI(
model=self.model,
temperature=0, # Deterministic for compliance work
api_key=self.api_key
)
self.agent = create_tool_calling_agent(
self.llm,
audit_tools, # 7 available tools
self.prompt
)
self.agent_executor = AgentExecutor(
agent=self.agent,
tools=audit_tools,
verbose=True,
max_iterations=15,
handle_parsing_errors=True
)Tool Design with Pydantic Schemas
Challenge: Create type-safe, self-documenting tools that the LLM can understand and invoke correctly.
Solution: Defined 7 tools with Pydantic input validation and descriptive docstrings:
from langchain.tools import tool
from pydantic import BaseModel, Field
class FacilityInput(BaseModel):
facility_id: str = Field(
description="ID of the facility to audit"
)
@tool
def check_calibration_compliance(facility_id: str) -> str:
"""
Checks equipment calibration dates against the 1-year
requirement from Directive 017. Returns detailed list
of non-compliant items with days overdue.
"""
equipment = mock_db.fetch_equipment(facility_id)
non_compliant = []
for item in equipment:
days_since = (datetime.now() - last_cal).days
if days_since > 365:
non_compliant.append({
'id': item['id'],
'days_overdue': days_since - 365,
'criticality': item.get('criticality')
})
return format_compliance_report(non_compliant)Knowledge Tools
search_aer_directives- RAG querylist_facilities- View locations
Analysis Tools
get_facility_equipment- Fetch datacheck_calibration_compliance- Audit
Action Tools
send_compliance_report- Emailschedule_follow_up- Calendarlog_maintenance_action- Logging
Mock Enterprise System Architecture
Challenge: Simulate realistic enterprise APIs (DB, Email, Calendar) without requiring actual infrastructure.
Solution: Built comprehensive mock layer with production-ready patterns:
MOCK_FACILITIES = {
"FAC-AB-001": {
"facility_id": "FAC-AB-001",
"name": "Edmonton South Terminal",
"equipment": [
{
"id": "EQ-PUMP-01",
"type": "Glycol Pump",
"last_calibration": "2023-12-07",
"criticality": "High"
}
]
}
}
def mock_api_send_email(to: str, subject: str, body: str):
"""Simulates POST /api/email/send"""
email_entry = {
"id": f"EMAIL-{len(EMAIL_OUTBOX) + 1000}",
"to": to,
"subject": subject,
"body": body,
"sent_at": datetime.now().isoformat()
}
EMAIL_OUTBOX.append(email_entry)
return {"status": "sent", "email_id": email_entry["id"]}Production-Grade UI/UX Design
Challenge: Create a professional, accessible interface that clearly shows agentic workflows in action.
Solution: Modern design system with WCAG 2.1 compliance and real-time status tracking.
Results & Impact
Example Workflow
When a user instructs: "Audit facility FAC-AB-001 for Directive 017 compliance and email results to safety@petolab.com"
Step 1: Knowledge Retrieval
- Calls
search_aer_directives - Queries RAG system for 365-day calibration requirement
- Retrieves relevant directive citations
Step 2: Data Collection
- Calls
get_facility_equipment - Fetches 4 equipment items from facility database
- Retrieves calibration dates and criticality levels
Step 3: Compliance Analysis
- Calls
check_calibration_compliance - Identifies 2 items overdue (400 and 380 days)
- Generates detailed violation report
Step 4: Report Distribution
- Calls
send_compliance_report - Emails comprehensive report to safety officer
- Includes specific equipment IDs and days overdue
Step 5: Follow-up Planning
- Calls
schedule_follow_up - Creates calendar entry for 2-week follow-up
- Generates confirmation ID
Step 6: Documentation
- Calls
log_maintenance_action(2x) - Creates maintenance logs for each violation
- Ensures full audit trail
Technology Stack
Agent Framework
RAG System
Backend & Infrastructure
Frontend & UI
Key Engineering Learnings
Function Calling is the Future
LLM function calling enables true agentic behavior. The agent makes autonomous decisions about which tools to use—this is fundamentally different from hard-coded logic or simple RAG retrieval.
Tool Descriptions Matter
Clear, detailed docstrings in tool definitions directly impact agent performance. The LLM reads these descriptions to decide which tool to invoke—ambiguous descriptions lead to wrong tool selection.
Temperature 0 for Compliance
Regulatory domains require deterministic behavior. Temperature 0 ensures consistent tool selection and prevents creative but potentially incorrect responses in compliance-critical applications.
Mock Data Accelerates Development
Building comprehensive mock systems enables rapid iteration without enterprise infrastructure. The mock layer's drop-in design makes migration to real APIs straightforward when ready.
Pydantic for Type Safety
Pydantic schemas prevent malformed tool calls and provide automatic validation. This is essential when LLMs are generating function inputs—validation catches errors before execution.
Agent Transparency Builds Trust
Real-time status tracking and verbose mode are critical for production deployment. Users need to see which tools the agent is calling to trust autonomous decision-making in compliance scenarios.
Example Use Cases
Single Facility Audit
"Audit facility FAC-AB-001 for Directive 017 compliance and email results to safety@petolab.com"
- Autonomous multi-step workflow
- 5+ tool invocations
- Complete in <30 seconds
Multi-Facility Analysis
"Check calibration compliance for all facilities and send executive summary"
- Comparative analysis across sites
- Consolidated reporting
- Prioritization by criticality
Knowledge Query + Action
"What are the calibration requirements in Directive 017? Then check if FAC-AB-001 meets them."
- RAG query for requirements
- Compliance verification
- Gap analysis and reporting
Proactive Maintenance
"For each non-compliant item, log maintenance action and schedule calibration"
- Automated work order creation
- Calendar scheduling
- Full documentation trail
Explore the Project
Production-ready autonomous agent system. Open-source and extensible architecture.
Quick Start
# Clone and setup
git clone https://github.com/morteza-mogharrab/aer-compliance-agent.git
cd aer-compliance-agent
python3 -m venv venv && source venv/bin/activate
# Install dependencies
pip install -r requirements_agent.txt
# Configure API key
export OPENAI_API_KEY='your-key'
# Run setup script
chmod +x setup_agent.sh
./setup_agent.sh
# Launch agent
python3 agent_app.py # Web interface on localhost:7860
# OR
python3 agent_core.py # Command-line mode
# OR
python3 demo_scenarios.py # Pre-configured demos