Files
AFC-Demo/docs/data-model-documentation.md
T
cbazza 2f7bd923df
Build and Push Docker Image only on Conventional Commits / build (push) Has been cancelled
local-docker-db branch openend
2025-12-03 12:59:30 +01:00

1154 lines
32 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Datenmodell-Dokumentation: Risk Intelligence Platform
## 📋 Inhaltsverzeichnis
1. [Übersicht](#übersicht)
2. [Entity-Relationship-Diagramm](#entity-relationship-diagramm)
3. [Schema-Architektur](#schema-architektur)
4. [Production Models (PUBLIC Schema)](#production-models-public-schema)
5. [Backend Models (BACKEND Schema)](#backend-models-backend-schema)
6. [ETL-Modell (Data Pool)](#etl-modell-data-pool)
7. [Beziehungen & Constraints](#beziehungen--constraints)
8. [Indizes & Performance](#indizes--performance)
9. [JSONB-Spalten](#jsonb-spalten)
10. [Business-Logik](#business-logik)
---
## Übersicht
Die Risk Intelligence Platform nutzt ein **Multi-Schema PostgreSQL-Design** mit klarer Trennung zwischen:
- **PUBLIC Schema** - Production Tables (Companies, Transactions, Users)
- **BACKEND Schema** - Source Data vom KI-Backend (Transactions, Outputs)
- **ETL Layer** - Backend Data Pool als Zwischenspeicher
### Datenbankverbindungen
```php
'pgsql_second' => [
'search_path' => 'public', // Default Connection
],
'backend' => [
'search_path' => 'backend', // Backend Connection
],
```
**Wichtig:** Beide Connections nutzen **dieselbe Datenbank**, aber unterschiedliche Schemas.
---
## Entity-Relationship-Diagramm
```
┌─────────────────────────────────────────────────────────────────────────┐
│ PUBLIC SCHEMA (Production) │
└─────────────────────────────────────────────────────────────────────────┘
┌──────────────────┐
│ users │
├──────────────────┤
│ id │ PK
│ name │
│ email │ UNIQUE
│ password │
│ email_verified_at│
│ two_factor_secret│
│ two_factor_codes │
│ remember_token │
│ created_at │
│ updated_at │
└──────────────────┘
┌──────────────────┐
│ companies │
├──────────────────┤
│ id │ PK
│ name │ UNIQUE
│ legal_name │
│ ticker │
│ sector │
│ country │ (ISO 3166-1 alpha-2)
│ headquarters │
│ kyc_risk_level │ (low|high|critical)
│ summary │
│ created_at │
│ updated_at │
└────────┬─────────┘
│ 1:N (One Company has Many Transactions)
┌────────▼─────────┐
│ transactions │
├──────────────────┤
│ id │ PK
│ company_id │ FK → companies.id (CASCADE DELETE)
│ reference │ UNIQUE (e.g., "MIGRATED-1")
│ │
│ ─── Core ─────── │
│ amount │ DECIMAL(16,2)
│ currency │ (3 chars, default: EUR)
│ counterparty │
│ counterparty_country │
│ channel │
│ executed_at │ DATETIME
│ │
│ ─── Risk ─────── │
│ risk_score │ TINYINT (0-255)
│ status │ (true_positive|false_positive|cleared)
│ requires_review │ BOOLEAN
│ flagged_by │
│ flagged_reason │
│ signals │ JSON
│ │
│ ─── Entity ───── │
│ entity │ Corporate entity name
│ counterparty_kyc_risk_level │
│ │
│ ─── Risk Breakdown ─── │
│ transaction_risk_score │
│ sanctions_risk_score │
│ country_risk_score │
│ pep_adverse_risk_score │
│ corruption_risk_score │
│ │
│ ─── External Data (JSONB) ─── │
│ registry_data │
│ genesis_context │
│ govdata_data │
│ bundesanzeiger_data │
│ insolvency_data │
│ rss_alerts │
│ sanctions_data │
│ pep_data │
│ gleif_data │
│ eu_sanctions_data│
│ handelsregister_data │
│ │
│ ─── 102 JSONB Output Columns ─── │
│ corporate_summary│ JSONB
│ corporate_history│ JSONB
│ tranx_score │ JSONB
│ ... (99 weitere) │
│ │
│ created_at │
│ updated_at │
└──────────────────┘
┌─────────────────────────────────────────────────────────────────────────┐
│ ETL LAYER (PUBLIC Schema) │
└─────────────────────────────────────────────────────────────────────────┘
┌──────────────────────────┐
│ backend_data_pool │
├──────────────────────────┤
│ id │ PK
│ │
│ ─── From backend.transactions ─── │
│ transaction_id │ INDEX
│ corporate_entity │
│ corporate_counterparty │
│ tx_date │
│ tx_amount │
│ tx_currency │
│ tx_purpose │
│ tx_country_outgoing │
│ tx_country_incoming │
│ source_file │
│ raw_payload │
│ status │ INDEX
│ created_at │
│ last_modified_at │
│ │
│ ─── From backend.transaction_outputs ─── │
│ prompt_id │
│ output_key │ INDEX
│ content │ TEXT (JSON)
│ run_id │
│ │
│ ─── Metadata ───── │
│ synced_at │ INDEX
│ updated_at │
│ │
│ UNIQUE(transaction_id, prompt_id) │
└──────────────────────────┘
┌─────────────────────────────────────────────────────────────────────────┐
│ BACKEND SCHEMA (Source Data) │
└─────────────────────────────────────────────────────────────────────────┘
┌──────────────────────┐
│ backend.transactions │
├──────────────────────┤
│ id │ PK
│ corporate_entity │
│ corporate_counterparty│
│ tx_date │
│ tx_amount │
│ tx_currency │
│ tx_purpose │
│ tx_country_outgoing │
│ tx_country_incoming │
│ source_file │
│ raw_payload │
│ status │ (pending|processing|done|failed)
│ risk_score │ INTEGER (-100 to 100)
│ created_at │
│ last_modified_at │
└──────────┬───────────┘
│ 1:N (One Transaction has Many Outputs)
┌──────────▼──────────────────┐
│ backend.transaction_outputs │
├─────────────────────────────┤
│ id │ PK
│ transaction_id │ FK → backend.transactions.id
│ prompt_id │
│ output_key │ VARCHAR (e.g., "corporate_summary")
│ content │ JSONB (LLM-Generated)
│ run_id │
│ created_at │
│ │
│ UNIQUE(transaction_id, prompt_id) │
└─────────────────────────────┘
```
---
## Schema-Architektur
### 1. PUBLIC Schema (Production)
**Zweck:** Produktive Applikationsdaten
**Models:**
- `Company` - Unternehmensprofile mit KYC Risk Level
- `Transaction` - Transaktionsanalysen mit 139 Spalten
- `User` - Benutzeraccounts
- `BackendDataPool` - ETL Zwischenspeicher
**Besonderheit:** Alle produktiven Daten, optimiert für Frontend-Zugriff
### 2. BACKEND Schema (Source)
**Zweck:** KI-Backend Source Data (Read-Only aus Frontend-Sicht)
**Models:**
- `Backend\Transaction` - Rohdaten vom KI-System
- `Backend\TransactionOutput` - LLM-generierte Analysen
**Besonderheit:** Wird nur gelesen, nie geschrieben (außer vom Backend-System)
---
## Production Models (PUBLIC Schema)
### Company Model
**Tabelle:** `public.companies`
**Model:** `App\Models\Company`
#### Struktur
```sql
CREATE TABLE public.companies (
id SERIAL PRIMARY KEY,
name VARCHAR(255) UNIQUE NOT NULL,
legal_name VARCHAR(255),
ticker VARCHAR(50),
sector VARCHAR(100),
country VARCHAR(2) DEFAULT 'DE', -- ISO 3166-1 alpha-2
headquarters TEXT,
kyc_risk_level VARCHAR(20) DEFAULT 'medium',
summary TEXT,
created_at TIMESTAMP,
updated_at TIMESTAMP
);
```
#### Felder-Erklärung
| Feld | Typ | Beschreibung | Beispiel |
|------|-----|--------------|----------|
| `id` | SERIAL | Primary Key | `1` |
| `name` | VARCHAR(255) | Firmenname (Unique) | `"Mercedes-Benz Group AG"` |
| `legal_name` | VARCHAR(255) | Offizieller rechtlicher Name | `"Mercedes-Benz Group Aktiengesellschaft"` |
| `ticker` | VARCHAR(50) | Börsen-Ticker | `"MBG"` |
| `sector` | VARCHAR(100) | Branche | `"Automotive"` |
| `country` | VARCHAR(2) | Land (ISO 3166-1 alpha-2) | `"DE"` |
| `headquarters` | TEXT | Hauptsitz | `"Stuttgart, Germany"` |
| `kyc_risk_level` | VARCHAR(20) | KYC Risiko-Level | `"low"`, `"high"`, `"critical"` |
| `summary` | TEXT | Zusammenfassung | `"German automobile manufacturer..."` |
#### KYC Risk Levels
| Level | Beschreibung | Bedeutung |
|-------|--------------|-----------|
| `low` | Geringes Risiko | Unauffälliges Unternehmen, Standard-KYC |
| `high` | Hohes Risiko | Erhöhte Due Diligence erforderlich |
| `critical` | Kritisches Risiko | Sanctions/PEP/AML Flags, intensive Prüfung |
#### Relationships
```php
class Company extends Model
{
// 1:N - Eine Company hat viele Transactions
public function transactions(): HasMany
{
return $this->hasMany(Transaction::class);
}
}
```
#### Constraints & Indizes
```sql
-- Unique Constraint auf Name
ALTER TABLE companies ADD CONSTRAINT companies_name_unique UNIQUE (name);
-- Index auf country für geografische Queries
CREATE INDEX idx_companies_country ON companies(country);
-- Index auf kyc_risk_level für Risiko-Filtering
CREATE INDEX idx_companies_kyc_risk_level ON companies(kyc_risk_level);
```
---
### Transaction Model
**Tabelle:** `public.transactions`
**Model:** `App\Models\Transaction`
#### Struktur (139 Spalten)
Die Transaction-Tabelle ist in mehrere logische Gruppen unterteilt:
##### Core Fields (31 Spalten)
```sql
-- Identity & Relationships
id SERIAL PRIMARY KEY
company_id INTEGER REFERENCES companies(id) ON DELETE CASCADE
reference VARCHAR(255) UNIQUE
-- Transaction Details
amount DECIMAL(16,2)
currency VARCHAR(3) DEFAULT 'EUR'
counterparty VARCHAR(255)
counterparty_country VARCHAR(2)
channel VARCHAR(50)
executed_at TIMESTAMP
-- Risk Assessment
risk_score TINYINT(0-255)
status VARCHAR(32) -- true_positive | false_positive | cleared
requires_review BOOLEAN DEFAULT true
flagged_by VARCHAR(255)
flagged_reason TEXT
signals JSON
-- Entity Information
entity VARCHAR(255)
counterparty_kyc_risk_level VARCHAR(20)
-- Risk Breakdown
transaction_risk_score INTEGER
sanctions_risk_score INTEGER
country_risk_score INTEGER
pep_adverse_risk_score INTEGER
corruption_risk_score INTEGER
-- Timestamps
created_at TIMESTAMP
updated_at TIMESTAMP
```
##### External Data Sources (11 JSONB Spalten)
```sql
-- Registry & Official Data
registry_data JSONB
registry_company_number TEXT
registry_source TEXT
registry_match_score DOUBLE
registry_last_refreshed_at TIMESTAMP
-- Government & Public Data
genesis_context JSONB
genesis_last_refreshed_at TIMESTAMP
govdata_data JSONB
govdata_last_refreshed_at TIMESTAMP
bundesanzeiger_data JSONB
bundesanzeiger_last_refreshed_at TIMESTAMP
handelsregister_data JSONB
handelsregister_last_refreshed_at TIMESTAMP
handelsregister_status TEXT
handelsregister_entity_id BIGINT
-- Compliance Data
insolvency_data JSONB
insolvency_last_refreshed_at TIMESTAMP
sanctions_data JSONB
sanctions_last_refreshed_at TIMESTAMP
eu_sanctions_data JSONB
eu_sanctions_last_refreshed_at TIMESTAMP
pep_data JSONB
pep_last_refreshed_at TIMESTAMP
-- GLEIF (Legal Entity Identifier)
gleif_lei TEXT
gleif_data JSON
gleif_last_refreshed_at TIMESTAMP
-- Alerts
rss_alerts JSONB
rss_last_refreshed_at TIMESTAMP
```
##### LLM-Generated Output Fields (102 JSONB Spalten)
<details>
<summary>📋 Vollständige Liste der 102 JSONB Output-Spalten (klicken zum Ausklappen)</summary>
**Corporate Information (57 Spalten):**
```sql
corporate_summary JSONB
corporate_history JSONB
corporate_purpose JSONB
corporate_sector JSONB
corporate_nace JSONB
corporate_products JSONB
corporate_markets JSONB
corporate_supply JSONB
corporate_name JSONB
corporate_form JSONB
corporate_forum JSONB
corporate_HQ JSONB
corporate_locations JSONB
corporate_holding JSONB
corporate_shareholders JSONB
corporate_board JSONB
corporate_supervisory JSONB
corporate_powerofattorney JSONB
corporate_taxID JSONB
corporate_LEI JSONB
corporate_UBO JSONB
corporate_employeecount JSONB
corporate_turnover JSONB
corporate_EBIT JSONB
corporate_netprofits JSONB
corporate_balancesheet JSONB
corporate_auditfindings JSONB
corporate_insolvency JSONB
corporate_liquidation JSONB
corporate_adhoc JSONB
corporate_pressrelease JSONB
corporate_votes JSONB
corporate_directordealings JSONB
corporate_brands JSONB
corporate_website JSONB
corporate_domain JSONB
corporate_IBAN JSONB
corporate_solvency JSONB
corporate_rating JSONB
corporate_ESG JSONB
corporate_license JSONB
corporate_warnings JSONB
corporate_eusanctions JSONB
corporate_ofacsanctions JSONB
corporate_uksanctions JSONB
corporate_pepexposure JSONB
corporate_adversemediascanning JSONB
corporate_corruptionexposure JSONB
corporate_AMLexposure JSONB
corporate_CTYexposure JSONB
corporate_adverse JSONB
corporate_pep JSONB
corporate_adverse2 JSONB
corporate_exportcontrol JSONB
corporate_mediamatch JSONB
corporate_haven JSONB
corporate_haven2 JSONB
```
**Transaction Analysis (30 Spalten):**
```sql
tranx_TX_AMOUNT JSONB
tranx_historical JSONB
tranx_TX_PURPOSE JSONB
tranx_counterpartyassessment JSONB
tranx_context JSONB
tranx_report JSONB
tranx_PURPOSEbase JSONB
tranx_performanceperiod JSONB
tranx_seasonality JSONB
tranx_resolution JSONB
trans_IBANvalidation JSONB
tranx_businesslogic JSONB
tranx_plausibilty JSONB
tranx_outliers JSONB
tranx_patterns JSONB
tranx_revenueimpact JSONB
tranx_profitimpact JSONB
tranx_marketimpact JSONB
tranx_duplicate JSONB
tranx_pattern2 JSONB
tranx_contracttenor JSONB
tranx_mismatch JSONB
tranx_fakepurposecheck JSONB
tranx_structuring JSONB
tranx_newbankaccount JSONB
tranx_weekday JSONB
tranx_holdingobfuscation JSONB
tranx_score JSONB
tranx_reasoning JSONB
tranx_recommendation JSONB
```
**Risk & Compliance (10 Spalten):**
```sql
sanctions_circumvention JSONB
corruption_sector JSONB
corruption_country JSONB
corruption_relationship JSONB
country_risk JSONB
corporate_insiders JSONB
corporate_courtcases JSONB
corporate_manda JSONB
corporate_pepdetail JSONB
tranx_sourcerepository JSONB
```
**Source Validation (5 Spalten):**
```sql
source_domaincheck JSONB
source_check JSONB
source_approveuniqueID JSONB
source_randomplausibility JSONB
```
</details>
#### Status-Werte
```php
class Transaction extends Model
{
const STATUS_TRUE_POSITIVE = 'true_positive'; // Kritisches Risiko
const STATUS_FALSE_POSITIVE = 'false_positive'; // Hohes Risiko
const STATUS_CLEARED = 'cleared'; // Geringes Risiko
}
```
| Status | Label (DE) | Bedeutung |
|--------|-----------|-----------|
| `true_positive` | Kritisches Risiko | Bestätigtes Risiko, Aktion erforderlich |
| `false_positive` | Hohes Risiko | Potenzielles Risiko, Review erforderlich |
| `cleared` | Geringes Risiko | Geprüft und freigegeben |
#### Relationships
```php
class Transaction extends Model
{
// N:1 - Viele Transactions gehören zu einer Company
public function company(): BelongsTo
{
return $this->belongsTo(Company::class);
}
}
```
#### Dynamic JSONB Casting
```php
// app/Models/Transaction.php
protected function casts(): array
{
// Automatisches Casting aller JSONB-Spalten als Array
$allColumns = Schema::getColumnListing('transactions');
$standardColumns = ['id', 'company_id', 'reference', ...];
$jsonbColumns = array_diff($allColumns, $standardColumns);
foreach ($jsonbColumns as $column) {
$casts[$column] = 'array';
}
return $casts;
}
```
#### Business Logic Methods
```php
// Status Label
public function statusLabel(): string
{
return match($this->status) {
self::STATUS_TRUE_POSITIVE => 'Kritisches Risiko',
self::STATUS_FALSE_POSITIVE => 'Hohes Risiko',
self::STATUS_CLEARED => 'Geringes Risiko',
default => 'Unbekanntes Risiko',
};
}
```
---
### User Model
**Tabelle:** `public.users`
**Model:** `App\Models\User`
#### Struktur
```sql
CREATE TABLE public.users (
id SERIAL PRIMARY KEY,
name VARCHAR(255) NOT NULL,
email VARCHAR(255) UNIQUE NOT NULL,
email_verified_at TIMESTAMP,
password VARCHAR(255) NOT NULL,
-- Two-Factor Authentication (Laravel Fortify)
two_factor_secret TEXT,
two_factor_recovery_codes TEXT,
two_factor_confirmed_at TIMESTAMP,
remember_token VARCHAR(100),
created_at TIMESTAMP,
updated_at TIMESTAMP
);
```
#### Features
- ✅ Email/Password Authentication
- ✅ Email Verification
- ✅ Two-Factor Authentication (2FA)
- ✅ Password Reset
- ✅ Remember Me
---
## Backend Models (BACKEND Schema)
### Backend\Transaction Model
**Tabelle:** `backend.transactions`
**Model:** `App\Models\Backend\Transaction`
**Connection:** `backend`
#### Struktur
```sql
CREATE TABLE backend.transactions (
id SERIAL PRIMARY KEY,
corporate_entity TEXT,
corporate_counterparty TEXT,
tx_date DATE,
tx_amount DECIMAL(15,2),
tx_currency VARCHAR(3),
tx_purpose TEXT,
tx_country_outgoing VARCHAR(2),
tx_country_incoming VARCHAR(2),
source_file TEXT,
raw_payload TEXT,
status VARCHAR(50), -- pending|processing|done|failed
risk_score INTEGER, -- -100 to 100 (Backend Scale)
created_at TIMESTAMP,
last_modified_at TIMESTAMP
);
```
#### Status-Werte
| Status | Bedeutung |
|--------|-----------|
| `pending` | Wartet auf Verarbeitung |
| `processing` | KI-Analyse läuft |
| `done` | Verarbeitung abgeschlossen |
| `failed` | Fehler bei Verarbeitung |
#### Relationships
```php
class Transaction extends Model
{
protected $connection = 'backend';
// 1:N - Eine Transaction hat viele Outputs
public function outputs(): HasMany
{
return $this->hasMany(TransactionOutput::class, 'transaction_id');
}
}
```
#### Helper Methods
```php
// Hole spezifischen Output-Typ
public function getOutput(string $key): ?array
{
return $this->outputs()
->where('output_key', $key)
->first()
?->content;
}
// Alle Outputs als Key-Value Array
public function getOutputsArray(): array
{
return $this->outputs()
->get()
->pluck('content', 'output_key')
->toArray();
}
// Convenience Methods
public function getCompanyInfo(): ?array;
public function getRiskAssessment(): ?array;
public function getSanctionsCheck(): ?array;
public function getPepCheck(): ?array;
```
---
### Backend\TransactionOutput Model
**Tabelle:** `backend.transaction_outputs`
**Model:** `App\Models\Backend\TransactionOutput`
**Connection:** `backend`
#### Struktur
```sql
CREATE TABLE backend.transaction_outputs (
id SERIAL PRIMARY KEY,
transaction_id INTEGER REFERENCES transactions(id),
prompt_id INTEGER,
output_key VARCHAR(255), -- z.B. "corporate_summary", "tranx_score"
content JSONB, -- LLM-Generated Output
run_id INTEGER,
created_at TIMESTAMP,
UNIQUE(transaction_id, prompt_id)
);
```
#### Output Keys (Konstanten)
```php
class TransactionOutput extends Model
{
const KEY_COMPANY_INFO = 'company_info';
const KEY_RISK_ASSESSMENT = 'risk_assessment';
const KEY_SANCTIONS = 'sanctions';
const KEY_PEP = 'pep';
const KEY_REGISTRY = 'registry';
const KEY_GLEIF = 'gleif';
// ... weitere 96 Keys
}
```
#### Relationships
```php
// N:1 - Viele Outputs gehören zu einer Transaction
public function transaction(): BelongsTo
{
return $this->belongsTo(Transaction::class, 'transaction_id');
}
```
---
## ETL-Modell (Data Pool)
### BackendDataPool Model
**Tabelle:** `public.backend_data_pool`
**Model:** `App\Models\BackendDataPool` (via DB Facade)
#### Zweck
Denormalisierte Zwischentabelle für ETL-Pipeline:
```sql
SELECT * FROM backend.transactions t
LEFT JOIN backend.transaction_outputs tout ON t.id = tout.transaction_id
WHERE t.status = 'done'
```
#### Struktur
```sql
CREATE TABLE public.backend_data_pool (
id SERIAL PRIMARY KEY,
-- From backend.transactions
transaction_id INTEGER INDEX,
corporate_entity TEXT,
corporate_counterparty TEXT,
tx_date TEXT,
tx_amount DOUBLE,
tx_currency TEXT,
tx_purpose TEXT,
tx_country_outgoing TEXT,
tx_country_incoming TEXT,
source_file TEXT,
raw_payload TEXT,
status TEXT INDEX,
created_at TEXT,
last_modified_at TEXT,
-- From backend.transaction_outputs
prompt_id INTEGER,
output_key TEXT INDEX,
content TEXT, -- JSON als Text gespeichert
run_id INTEGER,
-- Metadata
synced_at TIMESTAMP DEFAULT NOW(),
updated_at TIMESTAMP,
UNIQUE(transaction_id, prompt_id)
);
```
#### Indizes
```sql
CREATE INDEX idx_backend_data_pool_transaction_id ON backend_data_pool(transaction_id);
CREATE INDEX idx_backend_data_pool_output_key ON backend_data_pool(output_key);
CREATE INDEX idx_backend_data_pool_transaction_output ON backend_data_pool(transaction_id, output_key);
CREATE INDEX idx_backend_data_pool_transaction_prompt ON backend_data_pool(transaction_id, prompt_id);
CREATE INDEX idx_backend_data_pool_synced_at ON backend_data_pool(synced_at);
CREATE INDEX idx_backend_data_pool_status ON backend_data_pool(status);
```
---
## Beziehungen & Constraints
### Foreign Key Constraints
```sql
-- Companies → Transactions (1:N)
ALTER TABLE transactions
ADD CONSTRAINT fk_transactions_company_id
FOREIGN KEY (company_id)
REFERENCES companies(id)
ON DELETE CASCADE;
-- Backend: Transactions → Outputs (1:N)
ALTER TABLE backend.transaction_outputs
ADD CONSTRAINT fk_outputs_transaction_id
FOREIGN KEY (transaction_id)
REFERENCES backend.transactions(id)
ON DELETE CASCADE;
```
### Unique Constraints
```sql
-- Company Name muss eindeutig sein
ALTER TABLE companies
ADD CONSTRAINT companies_name_unique
UNIQUE (name);
-- Transaction Reference muss eindeutig sein
ALTER TABLE transactions
ADD CONSTRAINT transactions_reference_unique
UNIQUE (reference);
-- User Email muss eindeutig sein
ALTER TABLE users
ADD CONSTRAINT users_email_unique
UNIQUE (email);
-- Backend: Ein Output pro (transaction_id, prompt_id) Kombination
ALTER TABLE backend.transaction_outputs
ADD CONSTRAINT transaction_outputs_unique
UNIQUE (transaction_id, prompt_id);
-- Data Pool: Ein Record pro (transaction_id, prompt_id) Kombination
ALTER TABLE backend_data_pool
ADD CONSTRAINT backend_data_pool_unique
UNIQUE (transaction_id, prompt_id);
```
---
## Indizes & Performance
### Performance-kritische Indizes
```sql
-- Companies
CREATE INDEX idx_companies_country ON companies(country);
CREATE INDEX idx_companies_kyc_risk_level ON companies(kyc_risk_level);
CREATE INDEX idx_companies_sector ON companies(sector);
-- Transactions
CREATE INDEX idx_transactions_company_id ON transactions(company_id);
CREATE INDEX idx_transactions_status ON transactions(status);
CREATE INDEX idx_transactions_executed_at ON transactions(executed_at);
CREATE INDEX idx_transactions_risk_score ON transactions(risk_score);
CREATE INDEX idx_transactions_requires_review ON transactions(requires_review);
CREATE INDEX idx_transactions_counterparty_country ON transactions(counterparty_country);
-- Backend Transactions
CREATE INDEX idx_backend_transactions_status ON backend.transactions(status);
CREATE INDEX idx_backend_transactions_last_modified ON backend.transactions(last_modified_at);
-- Backend Outputs
CREATE INDEX idx_backend_outputs_transaction_id ON backend.transaction_outputs(transaction_id);
CREATE INDEX idx_backend_outputs_output_key ON backend.transaction_outputs(output_key);
-- Data Pool (siehe oben)
```
### Query-Optimierungen
#### Beispiel: Companies mit High-Risk Transactions
```sql
-- Ohne Index: Full Table Scan
SELECT c.*, COUNT(t.id) as high_risk_count
FROM companies c
JOIN transactions t ON c.id = t.company_id
WHERE t.status = 'true_positive'
GROUP BY c.id;
-- Mit Indizes: Index Scan
-- Index auf t.status ermöglicht schnelles Filtern
-- Index auf t.company_id ermöglicht schnelles Join
```
---
## JSONB-Spalten
### Vorteile von JSONB
1. **Flexibilität:** Dynamische Struktur ohne Schema-Änderungen
2. **Performance:** Binäres Format, schneller als JSON
3. **Indexierung:** GIN-Indizes für schnelle Queries
4. **Query-Support:** Volle PostgreSQL JSONB-Funktionen
### Typische JSONB-Struktur
#### corporate_summary
```json
{
"answer": "Mercedes-Benz Group AG ist ein deutscher Automobilhersteller...",
"confidence": 0.95,
"sources": [
"https://www.mercedes-benz-group.com",
"Wikipedia"
],
"generated_at": "2025-11-24T10:30:00Z"
}
```
#### tranx_score
```json
{
"score": 45,
"level": "medium",
"breakdown": {
"transaction": 20,
"sanctions": 0,
"country": 15,
"pep": 0,
"corruption": 10
},
"reasoning": "Moderate risk due to transaction amount and country...",
"generated_at": "2025-11-24T10:30:00Z"
}
```
### JSONB Queries
```sql
-- Zugriff auf JSONB-Felder
SELECT
reference,
corporate_summary->>'answer' as summary,
tranx_score->'score' as risk_score
FROM transactions
WHERE tranx_score->>'level' = 'high';
-- JSONB Array-Elemente
SELECT
reference,
jsonb_array_elements(corporate_shareholders->'shareholders') as shareholder
FROM transactions;
-- JSONB Aggregation
SELECT
AVG((tranx_score->>'score')::int) as avg_risk_score
FROM transactions;
```
---
## Business-Logik
### Risk Score Transformation
Backend nutzt **-100 bis 100**, Frontend **0 bis 255**:
```php
// TransformDataPoolToProduction Job
private function normalizeRiskScore(int $backendScore): int
{
return (int) round((($backendScore + 100) / 200) * 255);
}
// Beispiele:
// Backend: -100 → Frontend: 0 (Sehr niedrig)
// Backend: 0 → Frontend: 127 (Medium)
// Backend: 100 → Frontend: 255 (Sehr hoch)
```
### Risk Level Mapping
```php
private function calculateRiskLevel(int $normalizedScore): string
{
return match(true) {
$normalizedScore <= 153 => 'low', // 0-153
$normalizedScore <= 229 => 'high', // 154-229
default => 'critical', // 230-255
};
}
```
### Transaction Status Mapping
```php
private function mapStatus(string $riskLevel): string
{
return match($riskLevel) {
'low' => Transaction::STATUS_CLEARED,
'high' => Transaction::STATUS_FALSE_POSITIVE,
'critical' => Transaction::STATUS_TRUE_POSITIVE,
};
}
```
### Company KYC Risk Calculation
```php
// KycRiskCalculator Service
public function calculateCompanyRisk(Collection $transactions): string
{
$criticalCount = $transactions->where('status', 'true_positive')->count();
$highCount = $transactions->where('status', 'false_positive')->count();
$total = $transactions->count();
// Worst-Case-Prinzip
if ($criticalCount > 0) {
return 'critical';
}
// >30% high → critical
if (($highCount / $total) > 0.3) {
return 'critical';
}
// >10% high → high
if (($highCount / $total) > 0.1) {
return 'high';
}
return 'low';
}
```
---
## Daten-Lifecycle
### ETL-Pipeline
```
1. Backend Processing
backend.transactions (status='done')
+ backend.transaction_outputs
2. Sync to Data Pool (Every 6h)
public.backend_data_pool
(Denormalized JOIN)
3. Transform to Production (30min after Sync)
public.companies (updateOrCreate by name)
+ public.transactions (updateOrCreate by reference)
4. Frontend Display
Livewire Components query Production Tables
```
### Data Retention
- **Backend Schema:** Unbegrenzt (Source of Truth)
- **Data Pool:** Unbegrenzt (für Re-Processing)
- **Production Tables:** Unbegrenzt
- **User Sessions:** 120 Minuten (configurable)
- **Cache:** Abhängig von Cache-Driver
---
## Zusammenfassung
### Datenbank-Übersicht
| Schema | Tables | Zweck | Zugriff |
|--------|--------|-------|---------|
| `public` | 4 | Production Data | Read/Write (Frontend) |
| `backend` | 2 | Source Data | Read-Only (Frontend) |
### Model-Übersicht
| Model | Table | Schema | Relationships |
|-------|-------|--------|---------------|
| `Company` | companies | public | hasMany(Transaction) |
| `Transaction` | transactions | public | belongsTo(Company) |
| `User` | users | public | - |
| `Backend\Transaction` | transactions | backend | hasMany(TransactionOutput) |
| `Backend\TransactionOutput` | transaction_outputs | backend | belongsTo(Transaction) |
| - | backend_data_pool | public | ETL Layer (DB Facade) |
### Datenmenge (Beispiel)
- **Companies:** ~70 unique
- **Transactions:** 74
- **Backend Transactions:** 74
- **Backend Outputs:** 7,471 (74 × ~101 outputs)
- **Data Pool Records:** 7,471
---
**Erstellt:** 2025-11-24
**Version:** 1.0
**Autor:** Risk Intelligence Platform Team