ben
/
liurenchaxin
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
liurenchaxin/src/jixia/models/financial_data_models.py

521 lines
17 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# 金融数据抽象层设计
## 概述
在"炼妖壶-稷下学宫AI辩论系统"中，我们需要构建一个统一的金融数据抽象层，以支持多种数据源（包括现有的RapidAPI永动机引擎、新增的OpenBB集成引擎，以及未来可能添加的其他数据提供商）。该抽象层将为上层AI智能体提供一致的数据接口，同时隐藏底层数据源的具体实现细节。
## 设计目标
1. **统一接口**：为所有金融数据访问提供一致的API
2. **可扩展性**：易于添加新的数据提供商
3. **容错性**：当主数据源不可用时，能够自动切换到备用数据源
4. **性能优化**：支持缓存和异步数据获取
5. **类型安全**：使用Python类型注解确保数据结构的一致性
## 核心组件
### 1. 数据模型 (Data Models)
定义标准化的金融数据结构：
```python
# src/jixia/models/financial_data_models.py
from dataclasses import dataclass
from typing import Optional, List
from datetime import datetime
@dataclass
class StockQuote:
symbol: str
price: float
change: float
change_percent: float
volume: int
timestamp: datetime
@dataclass
class HistoricalPrice:
date: datetime
open: float
high: float
low: float
close: float
volume: int
@dataclass
class CompanyProfile:
symbol: str
name: str
industry: str
sector: str
market_cap: float
pe_ratio: Optional[float]
dividend_yield: Optional[float]
@dataclass
class FinancialNews:
title: str
summary: str
url: str
timestamp: datetime
sentiment: Optional[float] # -1 (负面) to 1 (正面)
```
### 2. 抽象基类 (Abstract Base Class)
定义数据提供商的通用接口：
```python
# src/jixia/engines/data_abstraction.py
from abc import ABC, abstractmethod
from typing import List, Optional
from src.jixia.models.financial_data_models import StockQuote, HistoricalPrice, CompanyProfile, FinancialNews
class DataProvider(ABC):
"""金融数据提供商抽象基类"""
@abstractmethod
def get_quote(self, symbol: str) -> Optional[StockQuote]:
"""获取股票报价"""
pass
@abstractmethod
def get_historical_prices(self, symbol: str, days: int = 30) -> List[HistoricalPrice]:
"""获取历史价格数据"""
pass
@abstractmethod
def get_company_profile(self, symbol: str) -> Optional[CompanyProfile]:
"""获取公司概况"""
pass
@abstractmethod
def get_news(self, symbol: str, limit: int = 10) -> List[FinancialNews]:
"""获取相关新闻"""
pass
@property
@abstractmethod
def name(self) -> str:
"""数据提供商名称"""
pass
@property
@abstractmethod
def priority(self) -> int:
"""优先级（数字越小优先级越高）"""
pass
```
### 3. Provider适配器 (Provider Adapters)
为每个具体的数据源实现适配器：
#### RapidAPI永动机引擎适配器
```python
# src/jixia/engines/rapidapi_adapter.py
from typing import List, Optional
from src.jixia.engines.data_abstraction import DataProvider
from src.jixia.models.financial_data_models import StockQuote, HistoricalPrice, CompanyProfile, FinancialNews
from src.jixia.engines.perpetual_engine import JixiaPerpetualEngine
from config.settings import get_rapidapi_key
class RapidAPIDataProvider(DataProvider):
"""RapidAPI永动机引擎适配器"""
def __init__(self):
self.engine = JixiaPerpetualEngine(get_rapidapi_key())
self._name = "RapidAPI"
self._priority = 2 # 中等优先级
def get_quote(self, symbol: str) -> Optional[StockQuote]:
result = self.engine.get_immortal_data("吕洞宾", "quote", symbol)
if result.success and result.data:
# 解析RapidAPI返回的数据并转换为StockQuote
# 这里需要根据实际API返回的数据结构进行调整
return StockQuote(
symbol=symbol,
price=result.data.get("price", 0),
change=result.data.get("change", 0),
change_percent=result.data.get("change_percent", 0),
volume=result.data.get("volume", 0),
timestamp=result.data.get("timestamp")
)
return None
def get_historical_prices(self, symbol: str, days: int = 30) -> List[HistoricalPrice]:
# 实现历史价格数据获取逻辑
pass
def get_company_profile(self, symbol: str) -> Optional[CompanyProfile]:
# 实现公司概况获取逻辑
pass
def get_news(self, symbol: str, limit: int = 10) -> List[FinancialNews]:
# 实现新闻获取逻辑
pass
@property
def name(self) -> str:
return self._name
@property
def priority(self) -> int:
return self._priority
```
#### OpenBB引擎适配器
```python
# src/jixia/engines/openbb_adapter.py
from typing import List, Optional
from src.jixia.engines.data_abstraction import DataProvider
from src.jixia.models.financial_data_models import StockQuote, HistoricalPrice, CompanyProfile, FinancialNews
from src.jixia.engines.openbb_engine import OpenBBEngine
class OpenBBDataProvider(DataProvider):
"""OpenBB引擎适配器"""
def __init__(self):
self.engine = OpenBBEngine()
self._name = "OpenBB"
self._priority = 1 # 最高优先级
def get_quote(self, symbol: str) -> Optional[StockQuote]:
result = self.engine.get_immortal_data("吕洞宾", "price", symbol)
if result.success and result.data:
# 解析OpenBB返回的数据并转换为StockQuote
return StockQuote(
symbol=symbol,
price=result.data.get("close", 0),
change=0, # 需要计算
change_percent=0, # 需要计算
volume=result.data.get("volume", 0),
timestamp=result.data.get("date")
)
return None
def get_historical_prices(self, symbol: str, days: int = 30) -> List[HistoricalPrice]:
# 实现历史价格数据获取逻辑
pass
def get_company_profile(self, symbol: str) -> Optional[CompanyProfile]:
# 实现公司概况获取逻辑
pass
def get_news(self, symbol: str, limit: int = 10) -> List[FinancialNews]:
# 实现新闻获取逻辑
pass
@property
def name(self) -> str:
return self._name
@property
def priority(self) -> int:
return self._priority
```
### 4. 数据抽象层管理器 (Data Abstraction Layer Manager)
管理多个数据提供商并提供统一接口：
```python
# src/jixia/engines/data_abstraction_layer.py
from typing import List, Optional
from src.jixia.engines.data_abstraction import DataProvider
from src.jixia.models.financial_data_models import StockQuote, HistoricalPrice, CompanyProfile, FinancialNews
import asyncio
class DataAbstractionLayer:
"""金融数据抽象层管理器"""
def __init__(self):
self.providers: List[DataProvider] = []
self._initialize_providers()
def _initialize_providers(self):
"""初始化所有可用的数据提供商"""
# 根据配置和环境动态加载适配器
try:
from src.jixia.engines.rapidapi_adapter import RapidAPIDataProvider
self.providers.append(RapidAPIDataProvider())
except ImportError:
pass # RapidAPI引擎不可用
try:
from src.jixia.engines.openbb_adapter import OpenBBDataProvider
self.providers.append(OpenBBDataProvider())
except ImportError:
pass # OpenBB引擎不可用
# 按优先级排序
self.providers.sort(key=lambda p: p.priority)
def get_quote(self, symbol: str) -> Optional[StockQuote]:
"""获取股票报价（带故障转移）"""
for provider in self.providers:
try:
quote = provider.get_quote(symbol)
if quote:
return quote
except Exception as e:
print(f"警告: {provider.name} 获取报价失败: {e}")
continue
return None
async def get_quote_async(self, symbol: str) -> Optional[StockQuote]:
"""异步获取股票报价（带故障转移）"""
for provider in self.providers:
try:
# 如果提供商支持异步方法，则使用异步方法
if hasattr(provider, 'get_quote_async'):
quote = await provider.get_quote_async(symbol)
else:
# 否则在执行器中运行同步方法
quote = await asyncio.get_event_loop().run_in_executor(
None, provider.get_quote, symbol
)
if quote:
return quote
except Exception as e:
print(f"警告: {provider.name} 获取报价失败: {e}")
continue
return None
def get_historical_prices(self, symbol: str, days: int = 30) -> List[HistoricalPrice]:
"""获取历史价格数据（带故障转移）"""
for provider in self.providers:
try:
prices = provider.get_historical_prices(symbol, days)
if prices:
return prices
except Exception as e:
print(f"警告: {provider.name} 获取历史价格失败: {e}")
continue
return []
def get_company_profile(self, symbol: str) -> Optional[CompanyProfile]:
"""获取公司概况（带故障转移）"""
for provider in self.providers:
try:
profile = provider.get_company_profile(symbol)
if profile:
return profile
except Exception as e:
print(f"警告: {provider.name} 获取公司概况失败: {e}")
continue
return None
def get_news(self, symbol: str, limit: int = 10) -> List[FinancialNews]:
"""获取相关新闻（带故障转移）"""
for provider in self.providers:
try:
news = provider.get_news(symbol, limit)
if news:
return news
except Exception as e:
print(f"警告: {provider.name} 获取新闻失败: {e}")
continue
return []
```
## 八仙与数据源的智能映射
```python
# src/jixia/engines/baxian_data_mapping.py
# 设计八仙与数据源的智能映射
immortal_data_mapping = {
'吕洞宾': {
'specialty': 'technical_analysis', # 技术分析专家
'preferred_data_types': ['historical', 'price'],
'data_providers': ['OpenBB', 'RapidAPI']
},
'何仙姑': {
'specialty': 'risk_metrics', # 风险控制专家
'preferred_data_types': ['price', 'profile'],
'data_providers': ['RapidAPI', 'OpenBB']
},
'张果老': {
'specialty': 'historical_data', # 历史数据分析师
'preferred_data_types': ['historical'],
'data_providers': ['OpenBB', 'RapidAPI']
},
'韩湘子': {
'specialty': 'sector_analysis', # 新兴资产专家
'preferred_data_types': ['profile', 'news'],
'data_providers': ['RapidAPI', 'OpenBB']
},
'汉钟离': {
'specialty': 'market_movers', # 热点追踪
'preferred_data_types': ['news', 'price'],
'data_providers': ['RapidAPI', 'OpenBB']
},
'蓝采和': {
'specialty': 'value_discovery', # 潜力股发现
'preferred_data_types': ['screener', 'profile'],
'data_providers': ['OpenBB', 'RapidAPI']
},
'铁拐李': {
'specialty': 'contrarian_analysis', # 逆向思维专家
'preferred_data_types': ['profile', 'short_interest'],
'data_providers': ['RapidAPI', 'OpenBB']
},
'曹国舅': {
'specialty': 'macro_economics', # 宏观经济分析师
'preferred_data_types': ['profile', 'institutional_holdings'],
'data_providers': ['OpenBB', 'RapidAPI']
}
}
```
## 缓存策略
为了提高性能，我们将实现多级缓存策略：
```python
# src/jixia/engines/data_cache.py
import time
from typing import Any, Optional
from functools import lru_cache
class DataCache:
"""金融数据缓存"""
def __init__(self):
self._cache = {}
self._cache_times = {}
self.default_ttl = 60 # 默认缓存时间（秒）
def get(self, key: str) -> Optional[Any]:
"""获取缓存数据"""
if key in self._cache:
# 检查是否过期
if time.time() - self._cache_times[key] < self.default_ttl:
return self._cache[key]
else:
# 删除过期缓存
del self._cache[key]
del self._cache_times[key]
return None
def set(self, key: str, value: Any, ttl: Optional[int] = None):
"""设置缓存数据"""
self._cache[key] = value
self._cache_times[key] = time.time()
if ttl:
# 可以为特定数据设置不同的TTL
pass # 实际实现中需要更复杂的TTL管理机制
@lru_cache(maxsize=128)
def get_quote_cache(self, symbol: str) -> Optional[Any]:
"""LRU缓存装饰器示例"""
# 这个方法将自动缓存最近128个调用的结果
pass
```
## 数据质量监控机制
为了确保数据的准确性和可靠性，我们将实现数据质量监控：
```python
# src/jixia/engines/data_quality_monitor.py
from typing import Dict, Any
from datetime import datetime
class DataQualityMonitor:
"""数据质量监控"""
def __init__(self):
self.provider_stats = {}
def record_access(self, provider_name: str, success: bool, response_time: float, data_size: int):
"""记录数据访问统计"""
if provider_name not in self.provider_stats:
self.provider_stats[provider_name] = {
'total_requests': 0,
'successful_requests': 0,
'failed_requests': 0,
'total_response_time': 0,
'total_data_size': 0,
'last_access': None
}
stats = self.provider_stats[provider_name]
stats['total_requests'] += 1
if success:
stats['successful_requests'] += 1
else:
stats['failed_requests'] += 1
stats['total_response_time'] += response_time
stats['total_data_size'] += data_size
stats['last_access'] = datetime.now()
def get_provider_health(self, provider_name: str) -> Dict[str, Any]:
"""获取提供商健康状况"""
if provider_name not in self.provider_stats:
return {'status': 'unknown'}
stats = self.provider_stats[provider_name]
success_rate = stats['successful_requests'] / stats['total_requests'] if stats['total_requests'] > 0 else 0
avg_response_time = stats['total_response_time'] / stats['total_requests'] if stats['total_requests'] > 0 else 0
status = 'healthy' if success_rate > 0.95 and avg_response_time < 2.0 else 'degraded' if success_rate > 0.8 else 'unhealthy'
return {
'status': status,
'success_rate': success_rate,
'avg_response_time': avg_response_time,
'total_requests': stats['total_requests'],
'last_access': stats['last_access']
}
```
## 使用示例
```python
# 示例：在智能体中使用数据抽象层
from src.jixia.engines.data_abstraction_layer import DataAbstractionLayer
from src.jixia.models.financial_data_models import StockQuote
# 初始化数据抽象层
dal = DataAbstractionLayer()
# 获取股票报价
quote = dal.get_quote("AAPL")
if quote:
print(f"Apple股价: ${quote.price}")
else:
print("无法获取股价数据")
# 异步获取报价
import asyncio
async def async_example():
quote = await dal.get_quote_async("GOOGL")
if quote:
print(f"Google股价: ${quote.price}")
# asyncio.run(async_example())
```
## 总结
这个金融数据抽象层设计提供了以下优势：
1. **统一接口**：所有智能体都可以通过相同的接口访问任何数据源
2. **故障转移**：当主数据源不可用时，自动切换到备用数据源
3. **可扩展性**：可以轻松添加新的数据提供商适配器
4. **性能优化**：通过缓存机制提高数据访问速度
5. **质量监控**：实时监控各数据源的健康状况
6. **文化融合**：通过八仙与数据源的智能映射，保持项目的文化特色
这将为"炼妖壶-稷下学宫AI辩论系统"提供一个强大、可靠且可扩展的金融数据基础。
Reference in New Issue View Git Blame Copy Permalink