紧急修复：移除敏感信息和配置文件
- 修复 MongoDB URI 泄露 - 从 Git 中移除 .kiro 目录（AI助手配置） - 更新 .gitignore 忽略所有点文件（除了 .gitignore 和 .gitguardian.yaml） - 防止未来意外提交敏感配置文件
This commit is contained in:
ben
parent
a24b887e8b
commit
6b464b6e07
|
|
@ -52,4 +52,9 @@ yarn-error.log*
|
||||||
|
|
||||||
# OS
|
# OS
|
||||||
.DS_Store
|
.DS_Store
|
||||||
Thumbs.db
|
Thumbs.db
|
||||||
|
|
||||||
|
# Ignore all dotfiles/directories except specific ones
|
||||||
|
.*
|
||||||
|
!.gitignore
|
||||||
|
!.gitguardian.yaml
|
||||||
|
|
|
||||||
|
|
@ -1,26 +0,0 @@
|
||||||
{
|
|
||||||
"mcpServers": {
|
|
||||||
"MongoDB": {
|
|
||||||
"command": "npx",
|
|
||||||
"args": [
|
|
||||||
"-y",
|
|
||||||
"mongodb-mcp-server",
|
|
||||||
"--connectionString",
|
|
||||||
"mongodb+srv://ben:313131@cauldron.tx3qnoq.mongodb.net/"
|
|
||||||
],
|
|
||||||
"env": {},
|
|
||||||
"disabled": false,
|
|
||||||
"autoApprove": [
|
|
||||||
"find",
|
|
||||||
"collection-schema",
|
|
||||||
"collection-indexes",
|
|
||||||
"create-index",
|
|
||||||
"aggregate",
|
|
||||||
"delete-many",
|
|
||||||
"count",
|
|
||||||
"list-databases",
|
|
||||||
"list-collections"
|
|
||||||
]
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
@ -1,123 +0,0 @@
|
||||||
# 炼妖壶项目清理与重构需求文档
|
|
||||||
|
|
||||||
## 介绍
|
|
||||||
|
|
||||||
炼妖壶项目经过长期开发，积累了大量代码和配置文件，但项目结构变得混乱，存在安全隐患（密钥泄露）和维护困难。本规范旨在系统性地清理和重构项目，建立清晰的架构和安全的开发流程。
|
|
||||||
|
|
||||||
## 需求
|
|
||||||
|
|
||||||
### 需求1：项目结构清理
|
|
||||||
|
|
||||||
**用户故事：** 作为开发者，我希望项目有清晰的目录结构和文件组织，以便快速理解和维护代码。
|
|
||||||
|
|
||||||
#### 验收标准
|
|
||||||
|
|
||||||
1. WHEN 查看项目根目录 THEN 应该只包含必要的核心文件和目录
|
|
||||||
2. WHEN 查看任何目录 THEN 应该有清晰的README说明其用途
|
|
||||||
3. WHEN 寻找特定功能代码 THEN 应该能在预期的目录中找到
|
|
||||||
4. WHEN 删除无用文件后 THEN 项目仍能正常运行
|
|
||||||
5. IF 文件超过6个月未使用 THEN 应该被归档或删除
|
|
||||||
|
|
||||||
### 需求2：安全配置管理
|
|
||||||
|
|
||||||
**用户故事：** 作为开发者，我希望所有密钥和敏感配置都安全管理，不会意外泄露到代码库中。
|
|
||||||
|
|
||||||
#### 验收标准
|
|
||||||
|
|
||||||
1. WHEN 扫描代码库 THEN 不应该发现任何硬编码的密钥或敏感信息
|
|
||||||
2. WHEN 开发者需要密钥 THEN 应该从环境变量或Doppler获取
|
|
||||||
3. WHEN 提交代码 THEN 应该自动检查是否包含敏感信息
|
|
||||||
4. WHEN 新开发者加入 THEN 应该有清晰的密钥管理指南
|
|
||||||
5. IF 发现密钥泄露 THEN 应该有自动化的处理流程
|
|
||||||
|
|
||||||
### 需求3：核心功能保留
|
|
||||||
|
|
||||||
**用户故事：** 作为用户，我希望在项目重构后，所有核心功能（AI辩论、数据分析、Streamlit界面）仍然可用。
|
|
||||||
|
|
||||||
#### 验收标准
|
|
||||||
|
|
||||||
1. WHEN 启动Streamlit应用 THEN 应该能正常访问所有功能页面
|
|
||||||
2. WHEN 运行AI辩论 THEN 应该能正常生成辩论内容
|
|
||||||
3. WHEN 连接数据库 THEN 应该能正常读写数据
|
|
||||||
4. WHEN 调用外部API THEN 应该能正常获取响应
|
|
||||||
5. WHEN 运行测试 THEN 所有核心功能测试应该通过
|
|
||||||
|
|
||||||
### 需求4：开发体验优化
|
|
||||||
|
|
||||||
**用户故事：** 作为开发者，我希望有良好的开发体验，包括清晰的文档、简单的启动流程和有效的调试工具。
|
|
||||||
|
|
||||||
#### 验收标准
|
|
||||||
|
|
||||||
1. WHEN 新开发者克隆项目 THEN 应该能在10分钟内启动应用
|
|
||||||
2. WHEN 查看文档 THEN 应该能找到所有必要的设置和使用说明
|
|
||||||
3. WHEN 遇到问题 THEN 应该有清晰的故障排除指南
|
|
||||||
4. WHEN 添加新功能 THEN 应该有明确的开发规范可遵循
|
|
||||||
5. WHEN 部署应用 THEN 应该有自动化的部署流程
|
|
||||||
|
|
||||||
### 需求5：技术债务清理
|
|
||||||
|
|
||||||
**用户故事：** 作为维护者，我希望清理技术债务，移除过时的代码和依赖，提高代码质量。
|
|
||||||
|
|
||||||
#### 验收标准
|
|
||||||
|
|
||||||
1. WHEN 检查依赖 THEN 不应该有未使用或过时的包
|
|
||||||
2. WHEN 运行代码分析 THEN 不应该有严重的代码质量问题
|
|
||||||
3. WHEN 查看代码 THEN 应该有一致的编码风格和注释
|
|
||||||
4. WHEN 运行性能测试 THEN 应用响应时间应该在可接受范围内
|
|
||||||
5. IF 发现重复代码 THEN 应该被重构为可复用的模块
|
|
||||||
|
|
||||||
### 需求6：部署和运维简化
|
|
||||||
|
|
||||||
**用户故事：** 作为运维人员，我希望部署和监控应用变得简单可靠。
|
|
||||||
|
|
||||||
#### 验收标准
|
|
||||||
|
|
||||||
1. WHEN 部署到生产环境 THEN 应该使用一键部署脚本
|
|
||||||
2. WHEN 应用运行异常 THEN 应该有清晰的日志和监控信息
|
|
||||||
3. WHEN 需要扩展 THEN 应该支持水平扩展
|
|
||||||
4. WHEN 备份数据 THEN 应该有自动化的备份策略
|
|
||||||
5. WHEN 回滚版本 THEN 应该能快速回滚到稳定版本
|
|
||||||
|
|
||||||
## 优先级
|
|
||||||
|
|
||||||
1. **P0 (紧急)**: 安全配置管理 - 立即解决密钥泄露问题
|
|
||||||
2. **P1 (高)**: 项目结构清理 - 建立清晰的项目架构
|
|
||||||
3. **P2 (中)**: 核心功能保留 - 确保重构不影响核心功能
|
|
||||||
4. **P3 (中)**: 开发体验优化 - 改善开发流程
|
|
||||||
5. **P4 (低)**: 技术债务清理 - 长期代码质量改进
|
|
||||||
6. **P5 (低)**: 部署和运维简化 - 运维流程优化
|
|
||||||
|
|
||||||
## 迁移策略
|
|
||||||
|
|
||||||
鉴于当前项目状态混乱，采用**全新开始**的策略：
|
|
||||||
|
|
||||||
1. **文档先行** - 完善所有需求和设计文档
|
|
||||||
2. **干净迁移** - 在新目录 `/home/ben/liurenchaxin` 重新开始
|
|
||||||
3. **选择性迁移** - 只迁移核心功能代码，抛弃历史包袱
|
|
||||||
4. **安全优先** - 从一开始就建立安全的配置管理
|
|
||||||
|
|
||||||
## 核心功能清单（需要保留）
|
|
||||||
|
|
||||||
### 必须迁移的功能
|
|
||||||
- 🤖 **稷下学宫AI辩论系统** (八仙辩论)
|
|
||||||
- 📊 **Streamlit主界面**
|
|
||||||
- 🔗 **Doppler配置管理**
|
|
||||||
- 💾 **数据库连接** (PostgreSQL, MongoDB, Zilliz)
|
|
||||||
- 🔌 **外部API集成** (OpenRouter, Anthropic等)
|
|
||||||
|
|
||||||
### 可选迁移的功能
|
|
||||||
- 📈 **金融数据分析**
|
|
||||||
- 🔄 **N8N工作流**
|
|
||||||
- 📱 **MCP服务器**
|
|
||||||
- 🧪 **实验性功能**
|
|
||||||
|
|
||||||
## 成功标准
|
|
||||||
|
|
||||||
项目重构成功的标志：
|
|
||||||
- ✅ 通过GitGuardian安全扫描，无密钥泄露
|
|
||||||
- ✅ 项目目录结构清晰，符合最佳实践
|
|
||||||
- ✅ 所有核心功能正常工作
|
|
||||||
- ✅ 新开发者能在10分钟内启动项目
|
|
||||||
- ✅ 代码质量评分达到B级以上
|
|
||||||
- ✅ 部署时间缩短到5分钟以内
|
|
||||||
- ✅ 完全摆脱历史技术债务
|
|
||||||
|
|
@ -1,26 +0,0 @@
|
||||||
# Product Overview
|
|
||||||
|
|
||||||
## 炼妖壶 (Lianyaohu) - AI Debate System
|
|
||||||
|
|
||||||
A Chinese AI-powered debate platform featuring the "稷下学宫" (Jixia Academy) system that enables multi-AI agent debates with historical Chinese philosophical perspectives.
|
|
||||||
|
|
||||||
### Core Features
|
|
||||||
- **AI Debate System**: Multi-agent debates with different AI personalities representing historical figures
|
|
||||||
- **Streamlit Interface**: Web-based UI for managing and viewing debates
|
|
||||||
- **Data Analytics**: Analysis and visualization of debate patterns and outcomes
|
|
||||||
- **Multi-database Support**: PostgreSQL, MongoDB, and Zilliz vector database integration
|
|
||||||
|
|
||||||
### Target Users
|
|
||||||
- Researchers studying AI discourse and argumentation
|
|
||||||
- Educators using AI for philosophical discussions
|
|
||||||
- Developers interested in multi-agent AI systems
|
|
||||||
|
|
||||||
### Key Value Propositions
|
|
||||||
- Culturally-aware AI debates rooted in Chinese philosophical traditions
|
|
||||||
- Real-time debate generation and analysis
|
|
||||||
- Extensible architecture for adding new AI personalities and debate formats
|
|
||||||
|
|
||||||
### Current Status
|
|
||||||
- **Migration Phase**: Moving from legacy codebase to clean, secure architecture
|
|
||||||
- **Security Priority**: Eliminating hardcoded secrets and implementing proper configuration management
|
|
||||||
- **Focus**: Core functionality preservation while improving maintainability
|
|
||||||
|
|
@ -1,129 +0,0 @@
|
||||||
# Project Structure
|
|
||||||
|
|
||||||
## Directory Organization
|
|
||||||
|
|
||||||
```
|
|
||||||
liurenchaxin/
|
|
||||||
├── app/ # Application entry points
|
|
||||||
│ ├── streamlit_app.py # Main Streamlit application
|
|
||||||
│ └── components/ # Reusable UI components
|
|
||||||
├── src/ # Core business logic
|
|
||||||
│ ├── jixia/ # 稷下学宫 AI debate system
|
|
||||||
│ │ ├── agents/ # AI agent implementations
|
|
||||||
│ │ ├── debates/ # Debate logic and orchestration
|
|
||||||
│ │ └── personalities/ # Historical figure personalities
|
|
||||||
│ ├── database/ # Database connection and models
|
|
||||||
│ │ ├── postgres/ # PostgreSQL specific code
|
|
||||||
│ │ ├── mongo/ # MongoDB specific code
|
|
||||||
│ │ └── zilliz/ # Vector database code
|
|
||||||
│ └── api/ # External API integrations
|
|
||||||
│ ├── openrouter/ # OpenRouter API client
|
|
||||||
│ ├── anthropic/ # Anthropic API client
|
|
||||||
│ └── openai/ # OpenAI API client
|
|
||||||
├── config/ # Configuration management
|
|
||||||
│ ├── doppler_config.py # Doppler integration
|
|
||||||
│ ├── settings.py # Application settings
|
|
||||||
│ └── environments/ # Environment-specific configs
|
|
||||||
├── tests/ # Test suite
|
|
||||||
│ ├── unit/ # Unit tests
|
|
||||||
│ ├── integration/ # Integration tests
|
|
||||||
│ └── fixtures/ # Test data and fixtures
|
|
||||||
├── docs/ # Documentation
|
|
||||||
│ ├── api/ # API documentation
|
|
||||||
│ ├── deployment/ # Deployment guides
|
|
||||||
│ └── development/ # Development guides
|
|
||||||
├── scripts/ # Utility scripts
|
|
||||||
│ ├── setup.sh # Environment setup
|
|
||||||
│ ├── migrate.py # Data migration scripts
|
|
||||||
│ └── deploy.sh # Deployment scripts
|
|
||||||
└── .kiro/ # Kiro AI assistant configuration
|
|
||||||
├── specs/ # Feature specifications
|
|
||||||
└── steering/ # AI guidance rules
|
|
||||||
```
|
|
||||||
|
|
||||||
## File Naming Conventions
|
|
||||||
|
|
||||||
### Python Files
|
|
||||||
- **snake_case** for all Python files and modules
|
|
||||||
- **PascalCase** for class names
|
|
||||||
- **UPPER_CASE** for constants
|
|
||||||
- Descriptive names that indicate purpose
|
|
||||||
|
|
||||||
### Configuration Files
|
|
||||||
- Use `.py` for Python configuration files
|
|
||||||
- Use `.yaml` or `.json` for data configuration
|
|
||||||
- Environment-specific suffixes: `_dev.py`, `_prod.py`
|
|
||||||
|
|
||||||
### Documentation
|
|
||||||
- **README.md** in each major directory explaining its purpose
|
|
||||||
- **CHANGELOG.md** for tracking changes
|
|
||||||
- **API.md** for API documentation
|
|
||||||
|
|
||||||
## Code Organization Principles
|
|
||||||
|
|
||||||
### Separation of Concerns
|
|
||||||
- **app/**: UI and presentation layer only
|
|
||||||
- **src/**: Business logic and core functionality
|
|
||||||
- **config/**: Configuration and settings management
|
|
||||||
- **tests/**: All testing code isolated
|
|
||||||
|
|
||||||
### Module Structure
|
|
||||||
Each major module should contain:
|
|
||||||
- `__init__.py`: Module initialization and public API
|
|
||||||
- `models.py`: Data models and schemas
|
|
||||||
- `services.py`: Business logic and operations
|
|
||||||
- `utils.py`: Helper functions and utilities
|
|
||||||
- `exceptions.py`: Custom exception classes
|
|
||||||
|
|
||||||
### Import Organization
|
|
||||||
```python
|
|
||||||
# Standard library imports
|
|
||||||
import os
|
|
||||||
import sys
|
|
||||||
from typing import Dict, List
|
|
||||||
|
|
||||||
# Third-party imports
|
|
||||||
import streamlit as st
|
|
||||||
from sqlalchemy import create_engine
|
|
||||||
|
|
||||||
# Local imports
|
|
||||||
from src.jixia.agents import DebateAgent
|
|
||||||
from config.settings import get_settings
|
|
||||||
```
|
|
||||||
|
|
||||||
## Security Structure
|
|
||||||
|
|
||||||
### Configuration Security
|
|
||||||
- **No secrets in code**: All sensitive data in Doppler or environment variables
|
|
||||||
- **Environment separation**: Clear boundaries between dev/staging/prod
|
|
||||||
- **Access control**: Proper authentication for all external services
|
|
||||||
|
|
||||||
### File Security
|
|
||||||
- **`.gitignore`**: Comprehensive exclusion of sensitive files
|
|
||||||
- **`.env.example`**: Template for required environment variables
|
|
||||||
- **Pre-commit hooks**: Automated security scanning before commits
|
|
||||||
|
|
||||||
## Documentation Requirements
|
|
||||||
|
|
||||||
### Required Documentation
|
|
||||||
- **README.md**: Project overview and quick start
|
|
||||||
- **INSTALLATION.md**: Detailed setup instructions
|
|
||||||
- **API.md**: API endpoints and usage
|
|
||||||
- **CONTRIBUTING.md**: Development guidelines
|
|
||||||
|
|
||||||
### Code Documentation
|
|
||||||
- **Docstrings**: All public functions and classes must have docstrings
|
|
||||||
- **Type hints**: Use Python type hints for better code clarity
|
|
||||||
- **Comments**: Explain complex business logic and AI model interactions
|
|
||||||
|
|
||||||
## Migration Guidelines
|
|
||||||
|
|
||||||
### Legacy Code Handling
|
|
||||||
- **Selective migration**: Only migrate proven, working code
|
|
||||||
- **Clean slate approach**: Rewrite rather than copy-paste when possible
|
|
||||||
- **Documentation first**: Document before migrating
|
|
||||||
|
|
||||||
### Quality Gates
|
|
||||||
- All migrated code must pass security scans
|
|
||||||
- All migrated code must have tests
|
|
||||||
- All migrated code must follow new structure conventions
|
|
||||||
|
|
@ -1,83 +0,0 @@
|
||||||
# Technology Stack
|
|
||||||
|
|
||||||
## Core Technologies
|
|
||||||
- **Python 3.x**: Primary programming language
|
|
||||||
- **Streamlit**: Web interface framework for the main application
|
|
||||||
- **FastAPI**: API framework for backend services (if applicable)
|
|
||||||
|
|
||||||
## AI & ML Stack
|
|
||||||
- **OpenRouter**: Multi-model AI API gateway
|
|
||||||
- **Anthropic Claude**: Primary AI model for debates
|
|
||||||
- **OpenAI GPT**: Alternative AI model support
|
|
||||||
- **Vector Databases**: Zilliz for semantic search and embeddings
|
|
||||||
|
|
||||||
## Database Technologies
|
|
||||||
- **PostgreSQL**: Primary relational database
|
|
||||||
- **MongoDB**: Document database for flexible data storage
|
|
||||||
- **Zilliz**: Vector database for AI embeddings and semantic search
|
|
||||||
|
|
||||||
## Configuration & Security
|
|
||||||
- **Doppler**: Centralized secrets and configuration management
|
|
||||||
- **Environment Variables**: Local configuration override
|
|
||||||
- **GitGuardian**: Automated secret scanning and security
|
|
||||||
|
|
||||||
## Development Tools
|
|
||||||
- **Git**: Version control
|
|
||||||
- **Pre-commit Hooks**: Automated code quality and security checks
|
|
||||||
- **Virtual Environment**: Python dependency isolation
|
|
||||||
|
|
||||||
## Common Commands
|
|
||||||
|
|
||||||
### Environment Setup
|
|
||||||
```bash
|
|
||||||
# Create virtual environment
|
|
||||||
python -m venv venv
|
|
||||||
source venv/bin/activate # Linux/Mac
|
|
||||||
# or
|
|
||||||
venv\Scripts\activate # Windows
|
|
||||||
|
|
||||||
# Install dependencies
|
|
||||||
pip install -r requirements.txt
|
|
||||||
```
|
|
||||||
|
|
||||||
### Application Commands
|
|
||||||
```bash
|
|
||||||
# Start Streamlit application
|
|
||||||
streamlit run app/streamlit_app.py
|
|
||||||
|
|
||||||
# Run with specific port
|
|
||||||
streamlit run app/streamlit_app.py --server.port 8501
|
|
||||||
```
|
|
||||||
|
|
||||||
### Development Commands
|
|
||||||
```bash
|
|
||||||
# Run tests
|
|
||||||
python -m pytest tests/
|
|
||||||
|
|
||||||
# Code quality checks
|
|
||||||
pre-commit run --all-files
|
|
||||||
|
|
||||||
# Security scan
|
|
||||||
doppler secrets download --no-file --format env > .env.local
|
|
||||||
```
|
|
||||||
|
|
||||||
### Database Commands
|
|
||||||
```bash
|
|
||||||
# PostgreSQL connection test
|
|
||||||
python -c "from src.database import test_postgres_connection; test_postgres_connection()"
|
|
||||||
|
|
||||||
# MongoDB connection test
|
|
||||||
python -c "from src.database import test_mongo_connection; test_mongo_connection()"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Security Requirements
|
|
||||||
- **Zero Hardcoded Secrets**: All secrets must come from Doppler or environment variables
|
|
||||||
- **Environment Isolation**: Development, staging, and production environments must be separate
|
|
||||||
- **Automated Scanning**: All commits must pass GitGuardian security checks
|
|
||||||
- **Access Control**: Database and API access through proper authentication only
|
|
||||||
|
|
||||||
## Performance Considerations
|
|
||||||
- **Async Operations**: Use async/await for AI API calls to prevent blocking
|
|
||||||
- **Connection Pooling**: Implement database connection pooling for better performance
|
|
||||||
- **Caching**: Cache frequently accessed data and AI responses when appropriate
|
|
||||||
- **Resource Limits**: Set appropriate timeouts and rate limits for external API calls
|
|
||||||
|
|
@ -325,7 +325,7 @@ services:
|
||||||
ports:
|
ports:
|
||||||
- "{self.config.mcp_server_port}:{self.config.mcp_server_port}"
|
- "{self.config.mcp_server_port}:{self.config.mcp_server_port}"
|
||||||
environment:
|
environment:
|
||||||
MONGODB_URL: mongodb://admin:password@mongodb:27017/{self.config.default_database}?authSource=admin
|
MONGODB_URL: [REDACTED - 从Doppler获取MONGODB_URL]
|
||||||
MCP_SERVER_PORT: {self.config.mcp_server_port}
|
MCP_SERVER_PORT: {self.config.mcp_server_port}
|
||||||
LOG_LEVEL: {self.config.log_level}
|
LOG_LEVEL: {self.config.log_level}
|
||||||
depends_on:
|
depends_on:
|
||||||
|
|
@ -588,4 +588,4 @@ print(response.messages[-1]["content"])
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
# 创建完整设置
|
# 创建完整设置
|
||||||
create_complete_setup()
|
create_complete_setup()
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue