cbazza/AFC-Demo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
AFC-Demo/docs/data-model-documentation.md
T

1154 lines
32 KiB
Markdown
Raw Permalink Normal View History

local-docker-db branch openend
2025-12-03 12:59:30 +01:00
# Datenmodell-Dokumentation: Risk Intelligence Platform
## 📋 Inhaltsverzeichnis
1. [Übersicht](#übersicht)
2. [Entity-Relationship-Diagramm](#entity-relationship-diagramm)
3. [Schema-Architektur](#schema-architektur)
4. [Production Models (PUBLIC Schema)](#production-models-public-schema)
5. [Backend Models (BACKEND Schema)](#backend-models-backend-schema)
6. [ETL-Modell (Data Pool)](#etl-modell-data-pool)
7. [Beziehungen & Constraints](#beziehungen--constraints)
8. [Indizes & Performance](#indizes--performance)
9. [JSONB-Spalten](#jsonb-spalten)
10. [Business-Logik](#business-logik)
---
## Übersicht
Die Risk Intelligence Platform nutzt ein **Multi-Schema PostgreSQL-Design** mit klarer Trennung zwischen:
- **PUBLIC Schema** - Production Tables (Companies, Transactions, Users)
- **BACKEND Schema** - Source Data vom KI-Backend (Transactions, Outputs)
- **ETL Layer** - Backend Data Pool als Zwischenspeicher
### Datenbankverbindungen
```php
'pgsql_second' => [
'search_path' => 'public', // Default Connection
],
'backend' => [
'search_path' => 'backend', // Backend Connection
],
```
**Wichtig:** Beide Connections nutzen **dieselbe Datenbank**, aber unterschiedliche Schemas.
---
## Entity-Relationship-Diagramm
```
┌─────────────────────────────────────────────────────────────────────────┐
│ PUBLIC SCHEMA (Production) │
└─────────────────────────────────────────────────────────────────────────┘
┌──────────────────┐
│ users │
├──────────────────┤
│ id │ PK
│ name │
│ email │ UNIQUE
│ password │
│ email_verified_at│
│ two_factor_secret│
│ two_factor_codes │
│ remember_token │
│ created_at │
│ updated_at │
└──────────────────┘
┌──────────────────┐
│ companies │
├──────────────────┤
│ id │ PK
│ name │ UNIQUE
│ legal_name │
│ ticker │
│ sector │
│ country │ (ISO 3166-1 alpha-2)
│ headquarters │
│ kyc_risk_level │ (low|high|critical)
│ summary │
│ created_at │
│ updated_at │
└────────┬─────────┘
│
│ 1:N (One Company has Many Transactions)
│
┌────────▼─────────┐
│ transactions │
├──────────────────┤
│ id │ PK
│ company_id │ FK → companies.id (CASCADE DELETE)
│ reference │ UNIQUE (e.g., "MIGRATED-1")
│ │
│ ─── Core ─────── │
│ amount │ DECIMAL(16,2)
│ currency │ (3 chars, default: EUR)
│ counterparty │
│ counterparty_country │
│ channel │
│ executed_at │ DATETIME
│ │
│ ─── Risk ─────── │
│ risk_score │ TINYINT (0-255)
│ status │ (true_positive|false_positive|cleared)
│ requires_review │ BOOLEAN
│ flagged_by │
│ flagged_reason │
│ signals │ JSON
│ │
│ ─── Entity ───── │
│ entity │ Corporate entity name
│ counterparty_kyc_risk_level │
│ │
│ ─── Risk Breakdown ─── │
│ transaction_risk_score │
│ sanctions_risk_score │
│ country_risk_score │
│ pep_adverse_risk_score │
│ corruption_risk_score │
│ │
│ ─── External Data (JSONB) ─── │
│ registry_data │
│ genesis_context │
│ govdata_data │
│ bundesanzeiger_data │
│ insolvency_data │
│ rss_alerts │
│ sanctions_data │
│ pep_data │
│ gleif_data │
│ eu_sanctions_data│
│ handelsregister_data │
│ │
│ ─── 102 JSONB Output Columns ─── │
│ corporate_summary│ JSONB
│ corporate_history│ JSONB
│ tranx_score │ JSONB
│ ... (99 weitere) │
│ │
│ created_at │
│ updated_at │
└──────────────────┘
┌─────────────────────────────────────────────────────────────────────────┐
│ ETL LAYER (PUBLIC Schema) │
└─────────────────────────────────────────────────────────────────────────┘
┌──────────────────────────┐
│ backend_data_pool │
├──────────────────────────┤
│ id │ PK
│ │
│ ─── From backend.transactions ─── │
│ transaction_id │ INDEX
│ corporate_entity │
│ corporate_counterparty │
│ tx_date │
│ tx_amount │
│ tx_currency │
│ tx_purpose │
│ tx_country_outgoing │
│ tx_country_incoming │
│ source_file │
│ raw_payload │
│ status │ INDEX
│ created_at │
│ last_modified_at │
│ │
│ ─── From backend.transaction_outputs ─── │
│ prompt_id │
│ output_key │ INDEX
│ content │ TEXT (JSON)
│ run_id │
│ │
│ ─── Metadata ───── │
│ synced_at │ INDEX
│ updated_at │
│ │
│ UNIQUE(transaction_id, prompt_id) │
└──────────────────────────┘
┌─────────────────────────────────────────────────────────────────────────┐
│ BACKEND SCHEMA (Source Data) │
└─────────────────────────────────────────────────────────────────────────┘
┌──────────────────────┐
│ backend.transactions │
├──────────────────────┤
│ id │ PK
│ corporate_entity │
│ corporate_counterparty│
│ tx_date │
│ tx_amount │
│ tx_currency │
│ tx_purpose │
│ tx_country_outgoing │
│ tx_country_incoming │
│ source_file │
│ raw_payload │
│ status │ (pending|processing|done|failed)
│ risk_score │ INTEGER (-100 to 100)
│ created_at │
│ last_modified_at │
└──────────┬───────────┘
│
│ 1:N (One Transaction has Many Outputs)
│
┌──────────▼──────────────────┐
│ backend.transaction_outputs │
├─────────────────────────────┤
│ id │ PK
│ transaction_id │ FK → backend.transactions.id
│ prompt_id │
│ output_key │ VARCHAR (e.g., "corporate_summary")
│ content │ JSONB (LLM-Generated)
│ run_id │
│ created_at │
│ │
│ UNIQUE(transaction_id, prompt_id) │
└─────────────────────────────┘
```
---
## Schema-Architektur
### 1. PUBLIC Schema (Production)
**Zweck:** Produktive Applikationsdaten
**Models:**
- `Company` - Unternehmensprofile mit KYC Risk Level
- `Transaction` - Transaktionsanalysen mit 139 Spalten
- `User` - Benutzeraccounts
- `BackendDataPool` - ETL Zwischenspeicher
**Besonderheit:** Alle produktiven Daten, optimiert für Frontend-Zugriff
### 2. BACKEND Schema (Source)
**Zweck:** KI-Backend Source Data (Read-Only aus Frontend-Sicht)
**Models:**
- `Backend\Transaction` - Rohdaten vom KI-System
- `Backend\TransactionOutput` - LLM-generierte Analysen
**Besonderheit:** Wird nur gelesen, nie geschrieben (außer vom Backend-System)
---
## Production Models (PUBLIC Schema)
### Company Model
**Tabelle:** `public.companies`
**Model:** `App\Models\Company`
#### Struktur
```sql
CREATE TABLE public.companies (
id SERIAL PRIMARY KEY,
name VARCHAR(255) UNIQUE NOT NULL,
legal_name VARCHAR(255),
ticker VARCHAR(50),
sector VARCHAR(100),
country VARCHAR(2) DEFAULT 'DE', -- ISO 3166-1 alpha-2
headquarters TEXT,
kyc_risk_level VARCHAR(20) DEFAULT 'medium',
summary TEXT,
created_at TIMESTAMP,
updated_at TIMESTAMP
);
```
#### Felder-Erklärung
| Feld | Typ | Beschreibung | Beispiel |
|------|-----|--------------|----------|
| `id` | SERIAL | Primary Key | `1` |
| `name` | VARCHAR(255) | Firmenname (Unique) | `"Mercedes-Benz Group AG"` |
| `legal_name` | VARCHAR(255) | Offizieller rechtlicher Name | `"Mercedes-Benz Group Aktiengesellschaft"` |
| `ticker` | VARCHAR(50) | Börsen-Ticker | `"MBG"` |
| `sector` | VARCHAR(100) | Branche | `"Automotive"` |
| `country` | VARCHAR(2) | Land (ISO 3166-1 alpha-2) | `"DE"` |
| `headquarters` | TEXT | Hauptsitz | `"Stuttgart, Germany"` |
| `kyc_risk_level` | VARCHAR(20) | KYC Risiko-Level | `"low"`, `"high"`, `"critical"` |
| `summary` | TEXT | Zusammenfassung | `"German automobile manufacturer..."` |
#### KYC Risk Levels
| Level | Beschreibung | Bedeutung |
|-------|--------------|-----------|
| `low` | Geringes Risiko | Unauffälliges Unternehmen, Standard-KYC |
| `high` | Hohes Risiko | Erhöhte Due Diligence erforderlich |
| `critical` | Kritisches Risiko | Sanctions/PEP/AML Flags, intensive Prüfung |
#### Relationships
```php
class Company extends Model
{
// 1:N - Eine Company hat viele Transactions
public function transactions(): HasMany
{
return $this->hasMany(Transaction::class);
}
}
```
#### Constraints & Indizes
```sql
-- Unique Constraint auf Name
ALTER TABLE companies ADD CONSTRAINT companies_name_unique UNIQUE (name);
-- Index auf country für geografische Queries
CREATE INDEX idx_companies_country ON companies(country);
-- Index auf kyc_risk_level für Risiko-Filtering
CREATE INDEX idx_companies_kyc_risk_level ON companies(kyc_risk_level);
```
---
### Transaction Model
**Tabelle:** `public.transactions`
**Model:** `App\Models\Transaction`
#### Struktur (139 Spalten)
Die Transaction-Tabelle ist in mehrere logische Gruppen unterteilt:
##### Core Fields (31 Spalten)
```sql
-- Identity & Relationships
id SERIAL PRIMARY KEY
company_id INTEGER REFERENCES companies(id) ON DELETE CASCADE
reference VARCHAR(255) UNIQUE
-- Transaction Details
amount DECIMAL(16,2)
currency VARCHAR(3) DEFAULT 'EUR'
counterparty VARCHAR(255)
counterparty_country VARCHAR(2)
channel VARCHAR(50)
executed_at TIMESTAMP
-- Risk Assessment
risk_score TINYINT(0-255)
status VARCHAR(32) -- true_positive | false_positive | cleared
requires_review BOOLEAN DEFAULT true
flagged_by VARCHAR(255)
flagged_reason TEXT
signals JSON
-- Entity Information
entity VARCHAR(255)
counterparty_kyc_risk_level VARCHAR(20)
-- Risk Breakdown
transaction_risk_score INTEGER
sanctions_risk_score INTEGER
country_risk_score INTEGER
pep_adverse_risk_score INTEGER
corruption_risk_score INTEGER
-- Timestamps
created_at TIMESTAMP
updated_at TIMESTAMP
```
##### External Data Sources (11 JSONB Spalten)
```sql
-- Registry & Official Data
registry_data JSONB
registry_company_number TEXT
registry_source TEXT
registry_match_score DOUBLE
registry_last_refreshed_at TIMESTAMP
-- Government & Public Data
genesis_context JSONB
genesis_last_refreshed_at TIMESTAMP
govdata_data JSONB
govdata_last_refreshed_at TIMESTAMP
bundesanzeiger_data JSONB
bundesanzeiger_last_refreshed_at TIMESTAMP
handelsregister_data JSONB
handelsregister_last_refreshed_at TIMESTAMP
handelsregister_status TEXT
handelsregister_entity_id BIGINT
-- Compliance Data
insolvency_data JSONB
insolvency_last_refreshed_at TIMESTAMP
sanctions_data JSONB
sanctions_last_refreshed_at TIMESTAMP
eu_sanctions_data JSONB
eu_sanctions_last_refreshed_at TIMESTAMP
pep_data JSONB
pep_last_refreshed_at TIMESTAMP
-- GLEIF (Legal Entity Identifier)
gleif_lei TEXT
gleif_data JSON
gleif_last_refreshed_at TIMESTAMP
-- Alerts
rss_alerts JSONB
rss_last_refreshed_at TIMESTAMP
```
##### LLM-Generated Output Fields (102 JSONB Spalten)
<details>
<summary>📋 Vollständige Liste der 102 JSONB Output-Spalten (klicken zum Ausklappen)</summary>
**Corporate Information (57 Spalten):**
```sql
corporate_summary JSONB
corporate_history JSONB
corporate_purpose JSONB
corporate_sector JSONB
corporate_nace JSONB
corporate_products JSONB
corporate_markets JSONB
corporate_supply JSONB
corporate_name JSONB
corporate_form JSONB
corporate_forum JSONB
corporate_HQ JSONB
corporate_locations JSONB
corporate_holding JSONB
corporate_shareholders JSONB
corporate_board JSONB
corporate_supervisory JSONB
corporate_powerofattorney JSONB
corporate_taxID JSONB
corporate_LEI JSONB
corporate_UBO JSONB
corporate_employeecount JSONB
corporate_turnover JSONB
corporate_EBIT JSONB
corporate_netprofits JSONB
corporate_balancesheet JSONB
corporate_auditfindings JSONB
corporate_insolvency JSONB
corporate_liquidation JSONB
corporate_adhoc JSONB
corporate_pressrelease JSONB
corporate_votes JSONB
corporate_directordealings JSONB
corporate_brands JSONB
corporate_website JSONB
corporate_domain JSONB
corporate_IBAN JSONB
corporate_solvency JSONB
corporate_rating JSONB
corporate_ESG JSONB
corporate_license JSONB
corporate_warnings JSONB
corporate_eusanctions JSONB
corporate_ofacsanctions JSONB
corporate_uksanctions JSONB
corporate_pepexposure JSONB
corporate_adversemediascanning JSONB
corporate_corruptionexposure JSONB
corporate_AMLexposure JSONB
corporate_CTYexposure JSONB
corporate_adverse JSONB
corporate_pep JSONB
corporate_adverse2 JSONB
corporate_exportcontrol JSONB
corporate_mediamatch JSONB
corporate_haven JSONB
corporate_haven2 JSONB
```
**Transaction Analysis (30 Spalten):**
```sql
tranx_TX_AMOUNT JSONB
tranx_historical JSONB
tranx_TX_PURPOSE JSONB
tranx_counterpartyassessment JSONB
tranx_context JSONB
tranx_report JSONB
tranx_PURPOSEbase JSONB
tranx_performanceperiod JSONB
tranx_seasonality JSONB
tranx_resolution JSONB
trans_IBANvalidation JSONB
tranx_businesslogic JSONB
tranx_plausibilty JSONB
tranx_outliers JSONB
tranx_patterns JSONB
tranx_revenueimpact JSONB
tranx_profitimpact JSONB
tranx_marketimpact JSONB
tranx_duplicate JSONB
tranx_pattern2 JSONB
tranx_contracttenor JSONB
tranx_mismatch JSONB
tranx_fakepurposecheck JSONB
tranx_structuring JSONB
tranx_newbankaccount JSONB
tranx_weekday JSONB
tranx_holdingobfuscation JSONB
tranx_score JSONB
tranx_reasoning JSONB
tranx_recommendation JSONB
```
**Risk & Compliance (10 Spalten):**
```sql
sanctions_circumvention JSONB
corruption_sector JSONB
corruption_country JSONB
corruption_relationship JSONB
country_risk JSONB
corporate_insiders JSONB
corporate_courtcases JSONB
corporate_manda JSONB
corporate_pepdetail JSONB
tranx_sourcerepository JSONB
```
**Source Validation (5 Spalten):**
```sql
source_domaincheck JSONB
source_check JSONB
source_approveuniqueID JSONB
source_randomplausibility JSONB
```
</details>
#### Status-Werte
```php
class Transaction extends Model
{
const STATUS_TRUE_POSITIVE = 'true_positive'; // Kritisches Risiko
const STATUS_FALSE_POSITIVE = 'false_positive'; // Hohes Risiko
const STATUS_CLEARED = 'cleared'; // Geringes Risiko
}
```
| Status | Label (DE) | Bedeutung |
|--------|-----------|-----------|
| `true_positive` | Kritisches Risiko | Bestätigtes Risiko, Aktion erforderlich |
| `false_positive` | Hohes Risiko | Potenzielles Risiko, Review erforderlich |
| `cleared` | Geringes Risiko | Geprüft und freigegeben |
#### Relationships
```php
class Transaction extends Model
{
// N:1 - Viele Transactions gehören zu einer Company
public function company(): BelongsTo
{
return $this->belongsTo(Company::class);
}
}
```
#### Dynamic JSONB Casting
```php
// app/Models/Transaction.php
protected function casts(): array
{
// Automatisches Casting aller JSONB-Spalten als Array
$allColumns = Schema::getColumnListing('transactions');
$standardColumns = ['id', 'company_id', 'reference', ...];
$jsonbColumns = array_diff($allColumns, $standardColumns);
foreach ($jsonbColumns as $column) {
$casts[$column] = 'array';
}
return $casts;
}
```
#### Business Logic Methods
```php
// Status Label
public function statusLabel(): string
{
return match($this->status) {
self::STATUS_TRUE_POSITIVE => 'Kritisches Risiko',
self::STATUS_FALSE_POSITIVE => 'Hohes Risiko',
self::STATUS_CLEARED => 'Geringes Risiko',
default => 'Unbekanntes Risiko',
};
}
```
---
### User Model
**Tabelle:** `public.users`
**Model:** `App\Models\User`
#### Struktur
```sql
CREATE TABLE public.users (
id SERIAL PRIMARY KEY,
name VARCHAR(255) NOT NULL,
email VARCHAR(255) UNIQUE NOT NULL,
email_verified_at TIMESTAMP,
password VARCHAR(255) NOT NULL,
-- Two-Factor Authentication (Laravel Fortify)
two_factor_secret TEXT,
two_factor_recovery_codes TEXT,
two_factor_confirmed_at TIMESTAMP,
remember_token VARCHAR(100),
created_at TIMESTAMP,
updated_at TIMESTAMP
);
```
#### Features
- ✅ Email/Password Authentication
- ✅ Email Verification
- ✅ Two-Factor Authentication (2FA)
- ✅ Password Reset
- ✅ Remember Me
---
## Backend Models (BACKEND Schema)
### Backend\Transaction Model
**Tabelle:** `backend.transactions`
**Model:** `App\Models\Backend\Transaction`
**Connection:** `backend`
#### Struktur
```sql
CREATE TABLE backend.transactions (
id SERIAL PRIMARY KEY,
corporate_entity TEXT,
corporate_counterparty TEXT,
tx_date DATE,
tx_amount DECIMAL(15,2),
tx_currency VARCHAR(3),
tx_purpose TEXT,
tx_country_outgoing VARCHAR(2),
tx_country_incoming VARCHAR(2),
source_file TEXT,
raw_payload TEXT,
status VARCHAR(50), -- pending|processing|done|failed
risk_score INTEGER, -- -100 to 100 (Backend Scale)
created_at TIMESTAMP,
last_modified_at TIMESTAMP
);
```
#### Status-Werte
| Status | Bedeutung |
|--------|-----------|
| `pending` | Wartet auf Verarbeitung |
| `processing` | KI-Analyse läuft |
| `done` | Verarbeitung abgeschlossen |
| `failed` | Fehler bei Verarbeitung |
#### Relationships
```php
class Transaction extends Model
{
protected $connection = 'backend';
// 1:N - Eine Transaction hat viele Outputs
public function outputs(): HasMany
{
return $this->hasMany(TransactionOutput::class, 'transaction_id');
}
}
```
#### Helper Methods
```php
// Hole spezifischen Output-Typ
public function getOutput(string $key): ?array
{
return $this->outputs()
->where('output_key', $key)
->first()
?->content;
}
// Alle Outputs als Key-Value Array
public function getOutputsArray(): array
{
return $this->outputs()
->get()
->pluck('content', 'output_key')
->toArray();
}
// Convenience Methods
public function getCompanyInfo(): ?array;
public function getRiskAssessment(): ?array;
public function getSanctionsCheck(): ?array;
public function getPepCheck(): ?array;
```
---
### Backend\TransactionOutput Model
**Tabelle:** `backend.transaction_outputs`
**Model:** `App\Models\Backend\TransactionOutput`
**Connection:** `backend`
#### Struktur
```sql
CREATE TABLE backend.transaction_outputs (
id SERIAL PRIMARY KEY,
transaction_id INTEGER REFERENCES transactions(id),
prompt_id INTEGER,
output_key VARCHAR(255), -- z.B. "corporate_summary", "tranx_score"
content JSONB, -- LLM-Generated Output
run_id INTEGER,
created_at TIMESTAMP,
UNIQUE(transaction_id, prompt_id)
);
```
#### Output Keys (Konstanten)
```php
class TransactionOutput extends Model
{
const KEY_COMPANY_INFO = 'company_info';
const KEY_RISK_ASSESSMENT = 'risk_assessment';
const KEY_SANCTIONS = 'sanctions';
const KEY_PEP = 'pep';
const KEY_REGISTRY = 'registry';
const KEY_GLEIF = 'gleif';
// ... weitere 96 Keys
}
```
#### Relationships
```php
// N:1 - Viele Outputs gehören zu einer Transaction
public function transaction(): BelongsTo
{
return $this->belongsTo(Transaction::class, 'transaction_id');
}
```
---
## ETL-Modell (Data Pool)
### BackendDataPool Model
**Tabelle:** `public.backend_data_pool`
**Model:** `App\Models\BackendDataPool` (via DB Facade)
#### Zweck
Denormalisierte Zwischentabelle für ETL-Pipeline:
```sql
SELECT * FROM backend.transactions t
LEFT JOIN backend.transaction_outputs tout ON t.id = tout.transaction_id
WHERE t.status = 'done'
```
#### Struktur
```sql
CREATE TABLE public.backend_data_pool (
id SERIAL PRIMARY KEY,
-- From backend.transactions
transaction_id INTEGER INDEX,
corporate_entity TEXT,
corporate_counterparty TEXT,
tx_date TEXT,
tx_amount DOUBLE,
tx_currency TEXT,
tx_purpose TEXT,
tx_country_outgoing TEXT,
tx_country_incoming TEXT,
source_file TEXT,
raw_payload TEXT,
status TEXT INDEX,
created_at TEXT,
last_modified_at TEXT,
-- From backend.transaction_outputs
prompt_id INTEGER,
output_key TEXT INDEX,
content TEXT, -- JSON als Text gespeichert
run_id INTEGER,
-- Metadata
synced_at TIMESTAMP DEFAULT NOW(),
updated_at TIMESTAMP,
UNIQUE(transaction_id, prompt_id)
);
```
#### Indizes
```sql
CREATE INDEX idx_backend_data_pool_transaction_id ON backend_data_pool(transaction_id);
CREATE INDEX idx_backend_data_pool_output_key ON backend_data_pool(output_key);
CREATE INDEX idx_backend_data_pool_transaction_output ON backend_data_pool(transaction_id, output_key);
CREATE INDEX idx_backend_data_pool_transaction_prompt ON backend_data_pool(transaction_id, prompt_id);
CREATE INDEX idx_backend_data_pool_synced_at ON backend_data_pool(synced_at);
CREATE INDEX idx_backend_data_pool_status ON backend_data_pool(status);
```
---
## Beziehungen & Constraints
### Foreign Key Constraints
```sql
-- Companies → Transactions (1:N)
ALTER TABLE transactions
ADD CONSTRAINT fk_transactions_company_id
FOREIGN KEY (company_id)
REFERENCES companies(id)
ON DELETE CASCADE;
-- Backend: Transactions → Outputs (1:N)
ALTER TABLE backend.transaction_outputs
ADD CONSTRAINT fk_outputs_transaction_id
FOREIGN KEY (transaction_id)
REFERENCES backend.transactions(id)
ON DELETE CASCADE;
```
### Unique Constraints
```sql
-- Company Name muss eindeutig sein
ALTER TABLE companies
ADD CONSTRAINT companies_name_unique
UNIQUE (name);
-- Transaction Reference muss eindeutig sein
ALTER TABLE transactions
ADD CONSTRAINT transactions_reference_unique
UNIQUE (reference);
-- User Email muss eindeutig sein
ALTER TABLE users
ADD CONSTRAINT users_email_unique
UNIQUE (email);
-- Backend: Ein Output pro (transaction_id, prompt_id) Kombination
ALTER TABLE backend.transaction_outputs
ADD CONSTRAINT transaction_outputs_unique
UNIQUE (transaction_id, prompt_id);
-- Data Pool: Ein Record pro (transaction_id, prompt_id) Kombination
ALTER TABLE backend_data_pool
ADD CONSTRAINT backend_data_pool_unique
UNIQUE (transaction_id, prompt_id);
```
---
## Indizes & Performance
### Performance-kritische Indizes
```sql
-- Companies
CREATE INDEX idx_companies_country ON companies(country);
CREATE INDEX idx_companies_kyc_risk_level ON companies(kyc_risk_level);
CREATE INDEX idx_companies_sector ON companies(sector);
-- Transactions
CREATE INDEX idx_transactions_company_id ON transactions(company_id);
CREATE INDEX idx_transactions_status ON transactions(status);
CREATE INDEX idx_transactions_executed_at ON transactions(executed_at);
CREATE INDEX idx_transactions_risk_score ON transactions(risk_score);
CREATE INDEX idx_transactions_requires_review ON transactions(requires_review);
CREATE INDEX idx_transactions_counterparty_country ON transactions(counterparty_country);
-- Backend Transactions
CREATE INDEX idx_backend_transactions_status ON backend.transactions(status);
CREATE INDEX idx_backend_transactions_last_modified ON backend.transactions(last_modified_at);
-- Backend Outputs
CREATE INDEX idx_backend_outputs_transaction_id ON backend.transaction_outputs(transaction_id);
CREATE INDEX idx_backend_outputs_output_key ON backend.transaction_outputs(output_key);
-- Data Pool (siehe oben)
```
### Query-Optimierungen
#### Beispiel: Companies mit High-Risk Transactions
```sql
-- Ohne Index: Full Table Scan
SELECT c.*, COUNT(t.id) as high_risk_count
FROM companies c
JOIN transactions t ON c.id = t.company_id
WHERE t.status = 'true_positive'
GROUP BY c.id;
-- Mit Indizes: Index Scan
-- Index auf t.status ermöglicht schnelles Filtern
-- Index auf t.company_id ermöglicht schnelles Join
```
---
## JSONB-Spalten
### Vorteile von JSONB
1. **Flexibilität:** Dynamische Struktur ohne Schema-Änderungen
2. **Performance:** Binäres Format, schneller als JSON
3. **Indexierung:** GIN-Indizes für schnelle Queries
4. **Query-Support:** Volle PostgreSQL JSONB-Funktionen
### Typische JSONB-Struktur
#### corporate_summary
```json
{
"answer": "Mercedes-Benz Group AG ist ein deutscher Automobilhersteller...",
"confidence": 0.95,
"sources": [
"https://www.mercedes-benz-group.com",
"Wikipedia"
],
"generated_at": "2025-11-24T10:30:00Z"
}
```
#### tranx_score
```json
{
"score": 45,
"level": "medium",
"breakdown": {
"transaction": 20,
"sanctions": 0,
"country": 15,
"pep": 0,
"corruption": 10
},
"reasoning": "Moderate risk due to transaction amount and country...",
"generated_at": "2025-11-24T10:30:00Z"
}
```
### JSONB Queries
```sql
-- Zugriff auf JSONB-Felder
SELECT
reference,
corporate_summary->>'answer' as summary,
tranx_score->'score' as risk_score
FROM transactions
WHERE tranx_score->>'level' = 'high';
-- JSONB Array-Elemente
SELECT
reference,
jsonb_array_elements(corporate_shareholders->'shareholders') as shareholder
FROM transactions;
-- JSONB Aggregation
SELECT
AVG((tranx_score->>'score')::int) as avg_risk_score
FROM transactions;
```
---
## Business-Logik
### Risk Score Transformation
Backend nutzt **-100 bis 100**, Frontend **0 bis 255**:
```php
// TransformDataPoolToProduction Job
private function normalizeRiskScore(int $backendScore): int
{
return (int) round((($backendScore + 100) / 200) * 255);
}
// Beispiele:
// Backend: -100 → Frontend: 0 (Sehr niedrig)
// Backend: 0 → Frontend: 127 (Medium)
// Backend: 100 → Frontend: 255 (Sehr hoch)
```
### Risk Level Mapping
```php
private function calculateRiskLevel(int $normalizedScore): string
{
return match(true) {
$normalizedScore <= 153 => 'low', // 0-153
$normalizedScore <= 229 => 'high', // 154-229
default => 'critical', // 230-255
};
}
```
### Transaction Status Mapping
```php
private function mapStatus(string $riskLevel): string
{
return match($riskLevel) {
'low' => Transaction::STATUS_CLEARED,
'high' => Transaction::STATUS_FALSE_POSITIVE,
'critical' => Transaction::STATUS_TRUE_POSITIVE,
};
}
```
### Company KYC Risk Calculation
```php
// KycRiskCalculator Service
public function calculateCompanyRisk(Collection $transactions): string
{
$criticalCount = $transactions->where('status', 'true_positive')->count();
$highCount = $transactions->where('status', 'false_positive')->count();
$total = $transactions->count();
// Worst-Case-Prinzip
if ($criticalCount > 0) {
return 'critical';
}
// >30% high → critical
if (($highCount / $total) > 0.3) {
return 'critical';
}
// >10% high → high
if (($highCount / $total) > 0.1) {
return 'high';
}
return 'low';
}
```
---
## Daten-Lifecycle
### ETL-Pipeline
```
1. Backend Processing
↓
backend.transactions (status='done')
+ backend.transaction_outputs
2. Sync to Data Pool (Every 6h)
↓
public.backend_data_pool
(Denormalized JOIN)
3. Transform to Production (30min after Sync)
↓
public.companies (updateOrCreate by name)
+ public.transactions (updateOrCreate by reference)
4. Frontend Display
↓
Livewire Components query Production Tables
```
### Data Retention
- **Backend Schema:** Unbegrenzt (Source of Truth)
- **Data Pool:** Unbegrenzt (für Re-Processing)
- **Production Tables:** Unbegrenzt
- **User Sessions:** 120 Minuten (configurable)
- **Cache:** Abhängig von Cache-Driver
---
## Zusammenfassung
### Datenbank-Übersicht
| Schema | Tables | Zweck | Zugriff |
|--------|--------|-------|---------|
| `public` | 4 | Production Data | Read/Write (Frontend) |
| `backend` | 2 | Source Data | Read-Only (Frontend) |
### Model-Übersicht
| Model | Table | Schema | Relationships |
|-------|-------|--------|---------------|
| `Company` | companies | public | hasMany(Transaction) |
| `Transaction` | transactions | public | belongsTo(Company) |
| `User` | users | public | - |
| `Backend\Transaction` | transactions | backend | hasMany(TransactionOutput) |
| `Backend\TransactionOutput` | transaction_outputs | backend | belongsTo(Transaction) |
| - | backend_data_pool | public | ETL Layer (DB Facade) |
### Datenmenge (Beispiel)
- **Companies:** ~70 unique
- **Transactions:** 74
- **Backend Transactions:** 74
- **Backend Outputs:** 7,471 (74 × ~101 outputs)
- **Data Pool Records:** 7,471
---
**Erstellt:** 2025-11-24
**Version:** 1.0
**Autor:** Risk Intelligence Platform Team
Reference in New Issue Copy Permalink