1415 lines
43 KiB
Markdown
1415 lines
43 KiB
Markdown
# ETL-Pipeline Migration: CSV → KI Workflow → Frontend
|
|
|
|
## System-Architektur (Klargestellt)
|
|
|
|
### Aktueller Zustand (Nicht optimal)
|
|
```
|
|
┌─────────────────────────────────────────────────────┐
|
|
│ Laravel Frontend │
|
|
│ │
|
|
│ CSV Upload → Manual Processing │
|
|
│ ↓ │
|
|
│ public.companies (11 Felder) │
|
|
│ ↓ │
|
|
│ public.transactions (49 Felder, komplex) │
|
|
│ - Core-Felder (14) │
|
|
│ - Enrichment-Felder (35) │
|
|
│ │
|
|
└─────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
**Probleme**:
|
|
- ❌ Companies und Transactions vermischt
|
|
- ❌ Enrichment-Logik direkt in Laravel
|
|
- ❌ Keine klare Trennung: Raw Data vs Processed Data
|
|
- ❌ Skalierbarkeit begrenzt
|
|
|
|
---
|
|
|
|
### Ziel-Zustand (Backend-Pipeline)
|
|
```
|
|
┌──────────────────────────────────────────────────────────────┐
|
|
│ CSV Upload │
|
|
│ ↓ │
|
|
│ backend.transactions (14 Felder) │
|
|
│ - corporate_entity │
|
|
│ - corporate_counterparty │
|
|
│ - tx_date, tx_amount, tx_currency │
|
|
│ - raw_payload (Original-CSV) │
|
|
│ - status: 'pending' │
|
|
└─────────────────────────┬────────────────────────────────────┘
|
|
│
|
|
↓
|
|
┌──────────────────────────────────────────────────────────────┐
|
|
│ KI Workflow (Python/MCP) │
|
|
│ │
|
|
│ Für jede Transaction: │
|
|
│ 1. Analyse: Risk Assessment │
|
|
│ 2. Enrichment: External APIs │
|
|
│ 3. Output: Strukturierte Ergebnisse │
|
|
│ │
|
|
│ Nutzt: backend.prompt_templates │
|
|
│ backend.prompt_runs │
|
|
└─────────────────────────┬────────────────────────────────────┘
|
|
│
|
|
↓
|
|
┌──────────────────────────────────────────────────────────────┐
|
|
│ backend.transaction_outputs (5 Felder) │
|
|
│ │
|
|
│ - transaction_id (FK → backend.transactions.id) │
|
|
│ - prompt_id (FK → backend.prompt_templates.id) │
|
|
│ - output_key (z.B. 'risk_assessment', 'sanctions_check') │
|
|
│ - content (JSON mit Ergebnissen) │
|
|
│ - run_id (Batch-Tracking) │
|
|
│ │
|
|
│ Beispiel-Outputs: │
|
|
│ • 'risk_assessment' → {score: 85, level: 'high'} │
|
|
│ • 'sanctions_check' → {found: true, matches: [...]} │
|
|
│ • 'pep_check' → {is_pep: false} │
|
|
│ • 'company_info' → {name, sector, country} │
|
|
└─────────────────────────┬────────────────────────────────────┘
|
|
│
|
|
↓
|
|
┌──────────────────────────────────────────────────────────────┐
|
|
│ Laravel Frontend │
|
|
│ │
|
|
│ Liest von: │
|
|
│ - backend.transactions (Raw Data) │
|
|
│ - backend.transaction_outputs (Processed Results) │
|
|
│ │
|
|
│ Zeigt an: │
|
|
│ - Transaction Details │
|
|
│ - Risk Score & Assessment │
|
|
│ - Enrichment Results (Sanctions, PEP, etc.) │
|
|
│ - Review Queue │
|
|
└──────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## Mapping: Alt → Neu
|
|
|
|
### 1. Companies Table → ENTFÄLLT
|
|
|
|
**Warum?**
|
|
- Companies sind **keine separate Entität** mehr
|
|
- Firmeninformationen kommen aus:
|
|
1. `backend.transactions.corporate_entity` (Name aus CSV)
|
|
2. `backend.transaction_outputs` mit `output_key='company_info'` (KI-Enrichment)
|
|
|
|
**Migration**:
|
|
```sql
|
|
-- Bestehende Companies werden zu Transaction Outputs
|
|
INSERT INTO backend.transaction_outputs (
|
|
transaction_id,
|
|
prompt_id,
|
|
output_key,
|
|
content
|
|
)
|
|
SELECT
|
|
bt.id as transaction_id,
|
|
1 as prompt_id, -- Default prompt für Company Info
|
|
'company_info' as output_key,
|
|
jsonb_build_object(
|
|
'name', c.name,
|
|
'legal_name', c.legal_name,
|
|
'ticker', c.ticker,
|
|
'sector', c.sector,
|
|
'country', c.country,
|
|
'headquarters', c.headquarters,
|
|
'kyc_risk_level', c.kyc_risk_level,
|
|
'summary', c.summary
|
|
) as content
|
|
FROM public.companies c
|
|
JOIN public.transactions t ON t.company_id = c.id
|
|
JOIN backend.transactions bt ON bt.corporate_entity = c.name;
|
|
```
|
|
|
|
### 2. Transactions Table → transaction_outputs
|
|
|
|
**public.transactions (49 Felder)**:
|
|
```
|
|
Core-Felder (14):
|
|
- company_id, reference, amount, currency
|
|
- counterparty, executed_at, status
|
|
- risk_score, requires_review
|
|
→ Gespeichert in backend.transactions
|
|
|
|
Enrichment-Felder (35):
|
|
- registry_data, sanctions_data, pep_data, ...
|
|
→ Jedes wird zu einem Output in transaction_outputs
|
|
```
|
|
|
|
**Mapping-Strategie**:
|
|
|
|
| public.transactions Feld | Ziel | output_key |
|
|
|--------------------------|------|------------|
|
|
| amount, currency, counterparty | backend.transactions | - |
|
|
| risk_score, requires_review | transaction_outputs | 'risk_assessment' |
|
|
| registry_data | transaction_outputs | 'registry' |
|
|
| sanctions_data | transaction_outputs | 'sanctions' |
|
|
| pep_data | transaction_outputs | 'pep' |
|
|
| gleif_data | transaction_outputs | 'gleif' |
|
|
| ... | ... | ... |
|
|
|
|
---
|
|
|
|
## Phase 1: Backend Models & Relationships (Tag 1-2)
|
|
|
|
### 1.1 Database Config
|
|
|
|
**config/database.php**:
|
|
```php
|
|
'backend' => [
|
|
'driver' => 'pgsql',
|
|
'host' => env('DB_HOST', '127.0.0.1'),
|
|
'port' => env('DB_PORT', '5432'),
|
|
'database' => env('DB_DATABASE', 'forge'),
|
|
'username' => env('DB_USERNAME', 'forge'),
|
|
'password' => env('DB_PASSWORD', ''),
|
|
'charset' => 'utf8',
|
|
'prefix' => '',
|
|
'search_path' => 'backend',
|
|
'sslmode' => 'prefer',
|
|
],
|
|
```
|
|
|
|
### 1.2 Backend\Transaction Model
|
|
|
|
**app/Models/Backend/Transaction.php**:
|
|
```php
|
|
<?php
|
|
|
|
namespace App\Models\Backend;
|
|
|
|
use Illuminate\Database\Eloquent\Factories\HasFactory;
|
|
use Illuminate\Database\Eloquent\Model;
|
|
use Illuminate\Database\Eloquent\Relations\HasMany;
|
|
|
|
class Transaction extends Model
|
|
{
|
|
use HasFactory;
|
|
|
|
protected $connection = 'backend';
|
|
protected $table = 'transactions';
|
|
|
|
public const UPDATED_AT = 'last_modified_at';
|
|
|
|
public const STATUS_PENDING = 'pending';
|
|
public const STATUS_PROCESSING = 'processing';
|
|
public const STATUS_COMPLETED = 'completed';
|
|
public const STATUS_FAILED = 'failed';
|
|
|
|
protected $fillable = [
|
|
'corporate_entity',
|
|
'corporate_counterparty',
|
|
'tx_date',
|
|
'tx_amount',
|
|
'tx_currency',
|
|
'tx_purpose',
|
|
'tx_country_outgoing',
|
|
'tx_country_incoming',
|
|
'source_file',
|
|
'raw_payload',
|
|
'status',
|
|
];
|
|
|
|
protected $casts = [
|
|
'tx_amount' => 'decimal:2',
|
|
// tx_date ist noch text - später zu timestamp migrieren
|
|
];
|
|
|
|
/**
|
|
* KI-generierte Outputs für diese Transaktion
|
|
*/
|
|
public function outputs(): HasMany
|
|
{
|
|
return $this->hasMany(TransactionOutput::class, 'transaction_id');
|
|
}
|
|
|
|
/**
|
|
* Hole spezifischen Output-Typ
|
|
*/
|
|
public function getOutput(string $key): ?array
|
|
{
|
|
return $this->outputs()
|
|
->where('output_key', $key)
|
|
->first()
|
|
?->content;
|
|
}
|
|
|
|
/**
|
|
* Hole alle Outputs als Key-Value Array
|
|
*/
|
|
public function getOutputsArray(): array
|
|
{
|
|
return $this->outputs()
|
|
->get()
|
|
->pluck('content', 'output_key')
|
|
->toArray();
|
|
}
|
|
|
|
/**
|
|
* Company Info aus Outputs
|
|
*/
|
|
public function getCompanyInfo(): ?array
|
|
{
|
|
return $this->getOutput('company_info');
|
|
}
|
|
|
|
/**
|
|
* Risk Assessment aus Outputs
|
|
*/
|
|
public function getRiskAssessment(): ?array
|
|
{
|
|
return $this->getOutput('risk_assessment');
|
|
}
|
|
|
|
/**
|
|
* Helper: Hat diese Transaction einen bestimmten Output?
|
|
*/
|
|
public function hasOutput(string $key): bool
|
|
{
|
|
return $this->outputs()->where('output_key', $key)->exists();
|
|
}
|
|
|
|
/**
|
|
* Ist KI-Verarbeitung abgeschlossen?
|
|
*/
|
|
public function isProcessed(): bool
|
|
{
|
|
return $this->status === self::STATUS_COMPLETED;
|
|
}
|
|
|
|
/**
|
|
* Benötigt Review?
|
|
*/
|
|
public function requiresReview(): bool
|
|
{
|
|
$risk = $this->getRiskAssessment();
|
|
|
|
if (!$risk) {
|
|
return true; // Keine Risk-Assessment → Review
|
|
}
|
|
|
|
return ($risk['score'] ?? 0) >= 70 || ($risk['requires_review'] ?? false);
|
|
}
|
|
}
|
|
```
|
|
|
|
### 1.3 Backend\TransactionOutput Model
|
|
|
|
**app/Models/Backend/TransactionOutput.php**:
|
|
```php
|
|
<?php
|
|
|
|
namespace App\Models\Backend;
|
|
|
|
use Illuminate\Database\Eloquent\Model;
|
|
use Illuminate\Database\Eloquent\Relations\BelongsTo;
|
|
|
|
class TransactionOutput extends Model
|
|
{
|
|
protected $connection = 'backend';
|
|
protected $table = 'transaction_outputs';
|
|
|
|
public $timestamps = false; // Diese Tabelle hat keine timestamps
|
|
|
|
protected $fillable = [
|
|
'transaction_id',
|
|
'prompt_id',
|
|
'output_key',
|
|
'content',
|
|
'run_id',
|
|
];
|
|
|
|
protected $casts = [
|
|
'content' => 'array', // Automatisch JSON encode/decode
|
|
];
|
|
|
|
/**
|
|
* Transaction zu der dieser Output gehört
|
|
*/
|
|
public function transaction(): BelongsTo
|
|
{
|
|
return $this->belongsTo(Transaction::class, 'transaction_id');
|
|
}
|
|
|
|
/**
|
|
* Prompt Template das diesen Output erzeugt hat
|
|
*/
|
|
public function promptTemplate(): BelongsTo
|
|
{
|
|
return $this->belongsTo(PromptTemplate::class, 'prompt_id');
|
|
}
|
|
|
|
/**
|
|
* Output-Key Konstanten für Type Safety
|
|
*/
|
|
public const KEY_COMPANY_INFO = 'company_info';
|
|
public const KEY_RISK_ASSESSMENT = 'risk_assessment';
|
|
public const KEY_SANCTIONS = 'sanctions';
|
|
public const KEY_PEP = 'pep';
|
|
public const KEY_REGISTRY = 'registry';
|
|
public const KEY_GLEIF = 'gleif';
|
|
public const KEY_INSOLVENCY = 'insolvency';
|
|
public const KEY_BUNDESANZEIGER = 'bundesanzeiger';
|
|
public const KEY_RSS = 'rss';
|
|
public const KEY_EU_SANCTIONS = 'eu_sanctions';
|
|
public const KEY_HANDELSREGISTER = 'handelsregister';
|
|
|
|
/**
|
|
* Alle verfügbaren Output-Keys
|
|
*/
|
|
public static function availableKeys(): array
|
|
{
|
|
return [
|
|
self::KEY_COMPANY_INFO,
|
|
self::KEY_RISK_ASSESSMENT,
|
|
self::KEY_SANCTIONS,
|
|
self::KEY_PEP,
|
|
self::KEY_REGISTRY,
|
|
self::KEY_GLEIF,
|
|
self::KEY_INSOLVENCY,
|
|
self::KEY_BUNDESANZEIGER,
|
|
self::KEY_RSS,
|
|
self::KEY_EU_SANCTIONS,
|
|
self::KEY_HANDELSREGISTER,
|
|
];
|
|
}
|
|
}
|
|
```
|
|
|
|
### 1.4 Backend\PromptTemplate Model (optional)
|
|
|
|
**app/Models/Backend/PromptTemplate.php**:
|
|
```php
|
|
<?php
|
|
|
|
namespace App\Models\Backend;
|
|
|
|
use Illuminate\Database\Eloquent\Model;
|
|
use Illuminate\Database\Eloquent\Relations\HasMany;
|
|
|
|
class PromptTemplate extends Model
|
|
{
|
|
protected $connection = 'backend';
|
|
protected $table = 'prompt_templates';
|
|
|
|
protected $fillable = [
|
|
'name',
|
|
'template',
|
|
'output_key',
|
|
'active',
|
|
];
|
|
|
|
public function outputs(): HasMany
|
|
{
|
|
return $this->hasMany(TransactionOutput::class, 'prompt_id');
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Phase 2: CSV Upload Service (Tag 3-4)
|
|
|
|
### 2.1 CSV Upload Controller
|
|
|
|
**app/Http/Controllers/TransactionUploadController.php**:
|
|
```php
|
|
<?php
|
|
|
|
namespace App\Http\Controllers;
|
|
|
|
use App\Services\TransactionUploadService;
|
|
use Illuminate\Http\Request;
|
|
|
|
class TransactionUploadController extends Controller
|
|
{
|
|
public function __construct(
|
|
private TransactionUploadService $uploadService
|
|
) {}
|
|
|
|
public function create()
|
|
{
|
|
return view('transactions.upload');
|
|
}
|
|
|
|
public function store(Request $request)
|
|
{
|
|
$validated = $request->validate([
|
|
'csv_file' => 'required|file|mimes:csv,txt|max:10240', // 10MB
|
|
]);
|
|
|
|
try {
|
|
$result = $this->uploadService->process($validated['csv_file']);
|
|
|
|
return redirect()
|
|
->route('transactions.index')
|
|
->with('success', "Imported {$result['count']} transactions. Processing started.");
|
|
} catch (\Exception $e) {
|
|
return back()
|
|
->withErrors(['csv_file' => $e->getMessage()])
|
|
->withInput();
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### 2.2 Upload Service
|
|
|
|
**app/Services/TransactionUploadService.php**:
|
|
```php
|
|
<?php
|
|
|
|
namespace App\Services;
|
|
|
|
use App\Jobs\ProcessTransactionWithAI;
|
|
use App\Models\Backend\Transaction;
|
|
use Illuminate\Http\UploadedFile;
|
|
use Illuminate\Support\Facades\DB;
|
|
use Illuminate\Support\Facades\Storage;
|
|
|
|
class TransactionUploadService
|
|
{
|
|
/**
|
|
* Verarbeite CSV Upload
|
|
*/
|
|
public function process(UploadedFile $file): array
|
|
{
|
|
// 1. Speichere Original-Datei
|
|
$fileName = $this->storeFile($file);
|
|
|
|
// 2. Parse CSV
|
|
$rows = $this->parseCsv($file);
|
|
|
|
// 3. Validiere Daten
|
|
$this->validate($rows);
|
|
|
|
// 4. Importiere zu backend.transactions
|
|
$transactions = $this->import($rows, $fileName);
|
|
|
|
// 5. Starte KI-Verarbeitung (async)
|
|
$this->queueAiProcessing($transactions);
|
|
|
|
return [
|
|
'count' => count($transactions),
|
|
'file' => $fileName,
|
|
];
|
|
}
|
|
|
|
/**
|
|
* Speichere Original-CSV für Audit
|
|
*/
|
|
private function storeFile(UploadedFile $file): string
|
|
{
|
|
return Storage::disk('local')->putFileAs(
|
|
'uploads/transactions',
|
|
$file,
|
|
date('Y-m-d_His') . '_' . $file->getClientOriginalName()
|
|
);
|
|
}
|
|
|
|
/**
|
|
* Parse CSV-Datei
|
|
*/
|
|
private function parseCsv(UploadedFile $file): array
|
|
{
|
|
$handle = fopen($file->getRealPath(), 'r');
|
|
$headers = fgetcsv($handle); // Erste Zeile = Header
|
|
|
|
$rows = [];
|
|
while (($data = fgetcsv($handle)) !== false) {
|
|
$rows[] = array_combine($headers, $data);
|
|
}
|
|
|
|
fclose($handle);
|
|
|
|
return $rows;
|
|
}
|
|
|
|
/**
|
|
* Validiere CSV-Daten
|
|
*/
|
|
private function validate(array $rows): void
|
|
{
|
|
if (empty($rows)) {
|
|
throw new \InvalidArgumentException('CSV file is empty');
|
|
}
|
|
|
|
$requiredColumns = [
|
|
'corporate_entity',
|
|
'corporate_counterparty',
|
|
'tx_date',
|
|
'tx_amount',
|
|
];
|
|
|
|
$headers = array_keys($rows[0]);
|
|
$missing = array_diff($requiredColumns, $headers);
|
|
|
|
if (!empty($missing)) {
|
|
throw new \InvalidArgumentException(
|
|
'Missing required columns: ' . implode(', ', $missing)
|
|
);
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Importiere Transaktionen
|
|
*/
|
|
private function import(array $rows, string $fileName): array
|
|
{
|
|
$transactions = [];
|
|
|
|
DB::connection('backend')->transaction(function () use ($rows, $fileName, &$transactions) {
|
|
foreach ($rows as $row) {
|
|
$transactions[] = Transaction::create([
|
|
'corporate_entity' => $row['corporate_entity'],
|
|
'corporate_counterparty' => $row['corporate_counterparty'],
|
|
'tx_date' => $row['tx_date'],
|
|
'tx_amount' => (float) $row['tx_amount'],
|
|
'tx_currency' => $row['tx_currency'] ?? 'EUR',
|
|
'tx_purpose' => $row['tx_purpose'] ?? null,
|
|
'tx_country_outgoing' => $row['tx_country_outgoing'] ?? null,
|
|
'tx_country_incoming' => $row['tx_country_incoming'] ?? null,
|
|
'source_file' => $fileName,
|
|
'raw_payload' => json_encode($row),
|
|
'status' => Transaction::STATUS_PENDING,
|
|
]);
|
|
}
|
|
});
|
|
|
|
return $transactions;
|
|
}
|
|
|
|
/**
|
|
* Starte KI-Verarbeitung für alle Transaktionen
|
|
*/
|
|
private function queueAiProcessing(array $transactions): void
|
|
{
|
|
foreach ($transactions as $transaction) {
|
|
ProcessTransactionWithAI::dispatch($transaction);
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Phase 3: KI Workflow Integration (Tag 5-7)
|
|
|
|
### 3.1 AI Processing Job
|
|
|
|
**app/Jobs/ProcessTransactionWithAI.php**:
|
|
```php
|
|
<?php
|
|
|
|
namespace App\Jobs;
|
|
|
|
use App\Models\Backend\Transaction;
|
|
use App\Models\Backend\TransactionOutput;
|
|
use App\Services\AiWorkflowService;
|
|
use Illuminate\Bus\Queueable;
|
|
use Illuminate\Contracts\Queue\ShouldQueue;
|
|
use Illuminate\Foundation\Bus\Dispatchable;
|
|
use Illuminate\Queue\InteractsWithQueue;
|
|
use Illuminate\Queue\SerializesModels;
|
|
|
|
class ProcessTransactionWithAI implements ShouldQueue
|
|
{
|
|
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
|
|
|
|
public function __construct(
|
|
private Transaction $transaction
|
|
) {}
|
|
|
|
public function handle(AiWorkflowService $aiService): void
|
|
{
|
|
// Update Status
|
|
$this->transaction->update(['status' => Transaction::STATUS_PROCESSING]);
|
|
|
|
try {
|
|
// 1. Company Info Enrichment
|
|
$companyInfo = $aiService->enrichCompanyInfo($this->transaction);
|
|
$this->storeOutput('company_info', $companyInfo);
|
|
|
|
// 2. Risk Assessment
|
|
$riskAssessment = $aiService->assessRisk($this->transaction);
|
|
$this->storeOutput('risk_assessment', $riskAssessment);
|
|
|
|
// 3. Sanctions Check
|
|
$sanctionsCheck = $aiService->checkSanctions($this->transaction);
|
|
$this->storeOutput('sanctions', $sanctionsCheck);
|
|
|
|
// 4. PEP Check
|
|
$pepCheck = $aiService->checkPep($this->transaction);
|
|
$this->storeOutput('pep', $pepCheck);
|
|
|
|
// 5. Weitere Checks (conditional)
|
|
if ($riskAssessment['score'] >= 50) {
|
|
$registryData = $aiService->checkRegistry($this->transaction);
|
|
$this->storeOutput('registry', $registryData);
|
|
}
|
|
|
|
// Fertig
|
|
$this->transaction->update(['status' => Transaction::STATUS_COMPLETED]);
|
|
|
|
} catch (\Exception $e) {
|
|
$this->transaction->update([
|
|
'status' => Transaction::STATUS_FAILED,
|
|
]);
|
|
|
|
// Store Error Output
|
|
$this->storeOutput('error', [
|
|
'message' => $e->getMessage(),
|
|
'trace' => $e->getTraceAsString(),
|
|
]);
|
|
|
|
throw $e;
|
|
}
|
|
}
|
|
|
|
private function storeOutput(string $key, array $data): void
|
|
{
|
|
TransactionOutput::create([
|
|
'transaction_id' => $this->transaction->id,
|
|
'prompt_id' => 1, // TODO: Map zu echtem Prompt Template
|
|
'output_key' => $key,
|
|
'content' => $data,
|
|
'run_id' => null, // TODO: Track Batch-Runs
|
|
]);
|
|
}
|
|
}
|
|
```
|
|
|
|
### 3.2 AI Workflow Service (Stub)
|
|
|
|
**app/Services/AiWorkflowService.php**:
|
|
```php
|
|
<?php
|
|
|
|
namespace App\Services;
|
|
|
|
use App\Models\Backend\Transaction;
|
|
|
|
class AiWorkflowService
|
|
{
|
|
/**
|
|
* Enriche Company-Informationen
|
|
*/
|
|
public function enrichCompanyInfo(Transaction $transaction): array
|
|
{
|
|
// TODO: Integration mit MCP/KI-Backend
|
|
// Für jetzt: Dummy-Implementierung
|
|
|
|
return [
|
|
'name' => $transaction->corporate_entity,
|
|
'legal_name' => null,
|
|
'country' => $transaction->tx_country_outgoing,
|
|
'sector' => 'Unknown',
|
|
'kyc_risk_level' => 'medium',
|
|
'enriched_at' => now()->toIso8601String(),
|
|
];
|
|
}
|
|
|
|
/**
|
|
* Risk Assessment
|
|
*/
|
|
public function assessRisk(Transaction $transaction): array
|
|
{
|
|
// TODO: Echte Risk-Logik via KI
|
|
|
|
$amount = (float) $transaction->tx_amount;
|
|
$score = 0;
|
|
|
|
// Einfache Heuristik
|
|
if ($amount > 100000) {
|
|
$score += 30;
|
|
}
|
|
if ($transaction->tx_country_incoming !== 'DE') {
|
|
$score += 20;
|
|
}
|
|
|
|
return [
|
|
'score' => $score,
|
|
'level' => $score >= 70 ? 'high' : ($score >= 40 ? 'medium' : 'low'),
|
|
'requires_review' => $score >= 70,
|
|
'factors' => [
|
|
'high_amount' => $amount > 100000,
|
|
'foreign_country' => $transaction->tx_country_incoming !== 'DE',
|
|
],
|
|
'assessed_at' => now()->toIso8601String(),
|
|
];
|
|
}
|
|
|
|
/**
|
|
* Sanctions Check
|
|
*/
|
|
public function checkSanctions(Transaction $transaction): array
|
|
{
|
|
// TODO: API zu Sanktionslisten
|
|
|
|
return [
|
|
'checked' => true,
|
|
'found' => false,
|
|
'matches' => [],
|
|
'sources' => ['EU', 'OFAC'],
|
|
'checked_at' => now()->toIso8601String(),
|
|
];
|
|
}
|
|
|
|
/**
|
|
* PEP Check
|
|
*/
|
|
public function checkPep(Transaction $transaction): array
|
|
{
|
|
// TODO: PEP Database Check
|
|
|
|
return [
|
|
'is_pep' => false,
|
|
'confidence' => 0.0,
|
|
'matches' => [],
|
|
'checked_at' => now()->toIso8601String(),
|
|
];
|
|
}
|
|
|
|
/**
|
|
* Registry Check
|
|
*/
|
|
public function checkRegistry(Transaction $transaction): array
|
|
{
|
|
// TODO: Handelsregister API
|
|
|
|
return [
|
|
'found' => false,
|
|
'company_number' => null,
|
|
'match_score' => 0.0,
|
|
'data' => null,
|
|
'checked_at' => now()->toIso8601String(),
|
|
];
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Phase 4: Frontend Anpassung (Tag 8-10)
|
|
|
|
### 4.1 Transaction Repository
|
|
|
|
**app/Repositories/TransactionRepository.php**:
|
|
```php
|
|
<?php
|
|
|
|
namespace App\Repositories;
|
|
|
|
use App\Models\Backend\Transaction;
|
|
use Illuminate\Database\Eloquent\Collection;
|
|
|
|
class TransactionRepository
|
|
{
|
|
/**
|
|
* Alle Transaktionen mit Outputs
|
|
*/
|
|
public function allWithOutputs(): Collection
|
|
{
|
|
return Transaction::with('outputs')
|
|
->orderBy('created_at', 'desc')
|
|
->get();
|
|
}
|
|
|
|
/**
|
|
* Transaktionen die Review benötigen
|
|
*/
|
|
public function requiresReview(): Collection
|
|
{
|
|
return Transaction::with('outputs')
|
|
->where('status', Transaction::STATUS_COMPLETED)
|
|
->get()
|
|
->filter(fn($t) => $t->requiresReview());
|
|
}
|
|
|
|
/**
|
|
* High-Risk Transaktionen
|
|
*/
|
|
public function highRisk(int $threshold = 70): Collection
|
|
{
|
|
return $this->allWithOutputs()
|
|
->filter(function ($transaction) use ($threshold) {
|
|
$risk = $transaction->getRiskAssessment();
|
|
return ($risk['score'] ?? 0) >= $threshold;
|
|
});
|
|
}
|
|
|
|
/**
|
|
* Transaktionen nach Firma
|
|
*/
|
|
public function byCompany(string $companyName): Collection
|
|
{
|
|
return Transaction::with('outputs')
|
|
->where('corporate_entity', $companyName)
|
|
->orderBy('tx_date', 'desc')
|
|
->get();
|
|
}
|
|
|
|
/**
|
|
* Statistiken
|
|
*/
|
|
public function stats(): array
|
|
{
|
|
$total = Transaction::count();
|
|
$pending = Transaction::where('status', Transaction::STATUS_PENDING)->count();
|
|
$processing = Transaction::where('status', Transaction::STATUS_PROCESSING)->count();
|
|
$completed = Transaction::where('status', Transaction::STATUS_COMPLETED)->count();
|
|
$failed = Transaction::where('status', Transaction::STATUS_FAILED)->count();
|
|
|
|
return compact('total', 'pending', 'processing', 'completed', 'failed');
|
|
}
|
|
}
|
|
```
|
|
|
|
### 4.2 Transaction Controller
|
|
|
|
**app/Http/Controllers/TransactionController.php**:
|
|
```php
|
|
<?php
|
|
|
|
namespace App\Http\Controllers;
|
|
|
|
use App\Repositories\TransactionRepository;
|
|
|
|
class TransactionController extends Controller
|
|
{
|
|
public function __construct(
|
|
private TransactionRepository $transactions
|
|
) {}
|
|
|
|
public function index()
|
|
{
|
|
$transactions = $this->transactions->allWithOutputs();
|
|
$stats = $this->transactions->stats();
|
|
|
|
return view('transactions.index', compact('transactions', 'stats'));
|
|
}
|
|
|
|
public function show(int $id)
|
|
{
|
|
$transaction = $this->transactions->find($id);
|
|
|
|
if (!$transaction) {
|
|
abort(404);
|
|
}
|
|
|
|
return view('transactions.show', compact('transaction'));
|
|
}
|
|
|
|
public function review()
|
|
{
|
|
$transactions = $this->transactions->requiresReview();
|
|
|
|
return view('transactions.review', compact('transactions'));
|
|
}
|
|
}
|
|
```
|
|
|
|
### 4.3 Volt Component für Transaction List
|
|
|
|
**resources/views/pages/transactions/index.blade.php**:
|
|
```php
|
|
<?php
|
|
|
|
use App\Repositories\TransactionRepository;
|
|
use function Livewire\Volt\{state, computed};
|
|
|
|
$transactionRepo = app(TransactionRepository::class);
|
|
|
|
$transactions = computed(fn() => $transactionRepo->allWithOutputs());
|
|
$stats = computed(fn() => $transactionRepo->stats());
|
|
|
|
?>
|
|
|
|
<div>
|
|
<flux:heading size="lg">Transactions</flux:heading>
|
|
|
|
{{-- Stats Cards --}}
|
|
<div class="grid grid-cols-4 gap-4 mt-6">
|
|
<flux:card>
|
|
<div class="text-sm text-gray-600">Total</div>
|
|
<div class="text-2xl font-bold">{{ $this->stats['total'] }}</div>
|
|
</flux:card>
|
|
|
|
<flux:card>
|
|
<div class="text-sm text-gray-600">Pending</div>
|
|
<div class="text-2xl font-bold text-yellow-600">{{ $this->stats['pending'] }}</div>
|
|
</flux:card>
|
|
|
|
<flux:card>
|
|
<div class="text-sm text-gray-600">Processing</div>
|
|
<div class="text-2xl font-bold text-blue-600">{{ $this->stats['processing'] }}</div>
|
|
</flux:card>
|
|
|
|
<flux:card>
|
|
<div class="text-sm text-gray-600">Completed</div>
|
|
<div class="text-2xl font-bold text-green-600">{{ $this->stats['completed'] }}</div>
|
|
</flux:card>
|
|
</div>
|
|
|
|
{{-- Transaction List --}}
|
|
<flux:card class="mt-6">
|
|
<flux:table>
|
|
<flux:thead>
|
|
<flux:tr>
|
|
<flux:th>Date</flux:th>
|
|
<flux:th>Company</flux:th>
|
|
<flux:th>Counterparty</flux:th>
|
|
<flux:th>Amount</flux:th>
|
|
<flux:th>Risk Score</flux:th>
|
|
<flux:th>Status</flux:th>
|
|
<flux:th>Actions</flux:th>
|
|
</flux:tr>
|
|
</flux:thead>
|
|
<flux:tbody>
|
|
@foreach($this->transactions as $transaction)
|
|
@php
|
|
$risk = $transaction->getRiskAssessment();
|
|
$riskScore = $risk['score'] ?? 0;
|
|
$riskColor = $riskScore >= 70 ? 'red' : ($riskScore >= 40 ? 'yellow' : 'green');
|
|
@endphp
|
|
<flux:tr>
|
|
<flux:td>{{ $transaction->tx_date }}</flux:td>
|
|
<flux:td>{{ $transaction->corporate_entity }}</flux:td>
|
|
<flux:td>{{ $transaction->corporate_counterparty }}</flux:td>
|
|
<flux:td>{{ number_format($transaction->tx_amount, 2) }} {{ $transaction->tx_currency }}</flux:td>
|
|
<flux:td>
|
|
<flux:badge :color="$riskColor">
|
|
{{ $riskScore }}
|
|
</flux:badge>
|
|
</flux:td>
|
|
<flux:td>
|
|
<flux:badge :variant="$transaction->isProcessed() ? 'success' : 'warning'">
|
|
{{ $transaction->status }}
|
|
</flux:badge>
|
|
</flux:td>
|
|
<flux:td>
|
|
<flux:button size="sm" href="{{ route('transactions.show', $transaction) }}">
|
|
View
|
|
</flux:button>
|
|
</flux:td>
|
|
</flux:tr>
|
|
@endforeach
|
|
</flux:tbody>
|
|
</flux:table>
|
|
</flux:card>
|
|
</div>
|
|
```
|
|
|
|
### 4.4 Transaction Detail View
|
|
|
|
**resources/views/pages/transactions/show.blade.php**:
|
|
```php
|
|
<?php
|
|
|
|
use App\Models\Backend\Transaction;
|
|
|
|
$transaction = Transaction::with('outputs')->findOrFail($id);
|
|
$companyInfo = $transaction->getCompanyInfo();
|
|
$riskAssessment = $transaction->getRiskAssessment();
|
|
$sanctionsCheck = $transaction->getOutput('sanctions');
|
|
$pepCheck = $transaction->getOutput('pep');
|
|
|
|
?>
|
|
|
|
<div>
|
|
<flux:heading size="lg">Transaction Details</flux:heading>
|
|
|
|
{{-- Transaction Info --}}
|
|
<flux:card class="mt-6">
|
|
<flux:heading size="md">Transaction Information</flux:heading>
|
|
|
|
<dl class="grid grid-cols-2 gap-4 mt-4">
|
|
<div>
|
|
<dt class="text-sm text-gray-600">Company</dt>
|
|
<dd class="font-medium">{{ $transaction->corporate_entity }}</dd>
|
|
</div>
|
|
<div>
|
|
<dt class="text-sm text-gray-600">Counterparty</dt>
|
|
<dd class="font-medium">{{ $transaction->corporate_counterparty }}</dd>
|
|
</div>
|
|
<div>
|
|
<dt class="text-sm text-gray-600">Amount</dt>
|
|
<dd class="font-medium">{{ number_format($transaction->tx_amount, 2) }} {{ $transaction->tx_currency }}</dd>
|
|
</div>
|
|
<div>
|
|
<dt class="text-sm text-gray-600">Date</dt>
|
|
<dd class="font-medium">{{ $transaction->tx_date }}</dd>
|
|
</div>
|
|
<div>
|
|
<dt class="text-sm text-gray-600">Status</dt>
|
|
<dd>
|
|
<flux:badge :variant="$transaction->isProcessed() ? 'success' : 'warning'">
|
|
{{ $transaction->status }}
|
|
</flux:badge>
|
|
</dd>
|
|
</div>
|
|
</dl>
|
|
</flux:card>
|
|
|
|
{{-- Company Info --}}
|
|
@if($companyInfo)
|
|
<flux:card class="mt-6">
|
|
<flux:heading size="md">Company Information</flux:heading>
|
|
|
|
<dl class="grid grid-cols-2 gap-4 mt-4">
|
|
<div>
|
|
<dt class="text-sm text-gray-600">Name</dt>
|
|
<dd class="font-medium">{{ $companyInfo['name'] }}</dd>
|
|
</div>
|
|
<div>
|
|
<dt class="text-sm text-gray-600">Country</dt>
|
|
<dd class="font-medium">{{ $companyInfo['country'] }}</dd>
|
|
</div>
|
|
<div>
|
|
<dt class="text-sm text-gray-600">Sector</dt>
|
|
<dd class="font-medium">{{ $companyInfo['sector'] ?? 'Unknown' }}</dd>
|
|
</div>
|
|
<div>
|
|
<dt class="text-sm text-gray-600">KYC Risk Level</dt>
|
|
<dd>
|
|
<flux:badge>{{ $companyInfo['kyc_risk_level'] ?? 'medium' }}</flux:badge>
|
|
</dd>
|
|
</div>
|
|
</dl>
|
|
</flux:card>
|
|
@endif
|
|
|
|
{{-- Risk Assessment --}}
|
|
@if($riskAssessment)
|
|
<flux:card class="mt-6">
|
|
<flux:heading size="md">Risk Assessment</flux:heading>
|
|
|
|
<div class="mt-4">
|
|
<div class="flex items-center justify-between">
|
|
<span class="text-sm text-gray-600">Risk Score</span>
|
|
<flux:badge :color="$riskAssessment['score'] >= 70 ? 'red' : ($riskAssessment['score'] >= 40 ? 'yellow' : 'green')">
|
|
{{ $riskAssessment['score'] }} / 100
|
|
</flux:badge>
|
|
</div>
|
|
<div class="mt-2">
|
|
<div class="text-sm text-gray-600">Risk Level</div>
|
|
<div class="font-medium">{{ ucfirst($riskAssessment['level']) }}</div>
|
|
</div>
|
|
<div class="mt-2">
|
|
<div class="text-sm text-gray-600">Requires Review</div>
|
|
<div class="font-medium">{{ $riskAssessment['requires_review'] ? 'Yes' : 'No' }}</div>
|
|
</div>
|
|
</div>
|
|
</flux:card>
|
|
@endif
|
|
|
|
{{-- Sanctions Check --}}
|
|
@if($sanctionsCheck)
|
|
<flux:card class="mt-6">
|
|
<flux:heading size="md">Sanctions Check</flux:heading>
|
|
|
|
<div class="mt-4">
|
|
<flux:badge :color="$sanctionsCheck['found'] ? 'red' : 'green'">
|
|
{{ $sanctionsCheck['found'] ? 'Matches Found' : 'Clear' }}
|
|
</flux:badge>
|
|
</div>
|
|
</flux:card>
|
|
@endif
|
|
|
|
{{-- All Outputs (Debug) --}}
|
|
<flux:card class="mt-6">
|
|
<flux:heading size="md">All AI Outputs</flux:heading>
|
|
|
|
<div class="mt-4 space-y-2">
|
|
@foreach($transaction->outputs as $output)
|
|
<details class="border rounded p-2">
|
|
<summary class="cursor-pointer font-medium">{{ $output->output_key }}</summary>
|
|
<pre class="mt-2 text-xs bg-gray-50 p-2 rounded overflow-auto">{{ json_encode($output->content, JSON_PRETTY_PRINT) }}</pre>
|
|
</details>
|
|
@endforeach
|
|
</div>
|
|
</flux:card>
|
|
</div>
|
|
```
|
|
|
|
---
|
|
|
|
## Phase 5: Datenmigration (Tag 11-15)
|
|
|
|
### 5.1 Migration: public.companies → transaction_outputs
|
|
|
|
**database/migrations/2025_11_12_migrate_companies_to_outputs.php**:
|
|
```php
|
|
<?php
|
|
|
|
use Illuminate\Database\Migrations\Migration;
|
|
use Illuminate\Support\Facades\DB;
|
|
|
|
return new class extends Migration
|
|
{
|
|
public function up(): void
|
|
{
|
|
DB::statement("
|
|
INSERT INTO backend.transaction_outputs (
|
|
transaction_id,
|
|
prompt_id,
|
|
output_key,
|
|
content
|
|
)
|
|
SELECT DISTINCT ON (t.id)
|
|
t.id as transaction_id,
|
|
1 as prompt_id,
|
|
'company_info' as output_key,
|
|
jsonb_build_object(
|
|
'name', c.name,
|
|
'legal_name', c.legal_name,
|
|
'ticker', c.ticker,
|
|
'sector', c.sector,
|
|
'country', c.country,
|
|
'headquarters', c.headquarters,
|
|
'kyc_risk_level', c.kyc_risk_level,
|
|
'summary', c.summary
|
|
) as content
|
|
FROM public.companies c
|
|
CROSS JOIN LATERAL (
|
|
SELECT id
|
|
FROM backend.transactions bt
|
|
WHERE bt.corporate_entity = c.name
|
|
LIMIT 1
|
|
) t
|
|
WHERE NOT EXISTS (
|
|
SELECT 1 FROM backend.transaction_outputs o
|
|
WHERE o.transaction_id = t.id
|
|
AND o.output_key = 'company_info'
|
|
)
|
|
");
|
|
}
|
|
|
|
public function down(): void
|
|
{
|
|
DB::statement("
|
|
DELETE FROM backend.transaction_outputs
|
|
WHERE output_key = 'company_info'
|
|
");
|
|
}
|
|
};
|
|
```
|
|
|
|
### 5.2 Migration: public.transactions → backend.transactions + outputs
|
|
|
|
**database/migrations/2025_11_12_migrate_transactions_to_backend.php**:
|
|
```php
|
|
<?php
|
|
|
|
use Illuminate\Database\Migrations\Migration;
|
|
use Illuminate\Support\Facades\DB;
|
|
|
|
return new class extends Migration
|
|
{
|
|
public function up(): void
|
|
{
|
|
// 1. Core Transaction Data
|
|
DB::statement("
|
|
INSERT INTO backend.transactions (
|
|
corporate_entity,
|
|
corporate_counterparty,
|
|
tx_date,
|
|
tx_amount,
|
|
tx_currency,
|
|
tx_purpose,
|
|
tx_country_incoming,
|
|
status,
|
|
created_at,
|
|
last_modified_at
|
|
)
|
|
SELECT
|
|
c.name as corporate_entity,
|
|
t.counterparty as corporate_counterparty,
|
|
t.executed_at::text as tx_date,
|
|
t.amount as tx_amount,
|
|
t.currency as tx_currency,
|
|
t.flagged_reason as tx_purpose,
|
|
t.counterparty_country as tx_country_incoming,
|
|
'completed' as status,
|
|
t.created_at::text,
|
|
t.updated_at::text
|
|
FROM public.transactions t
|
|
JOIN public.companies c ON t.company_id = c.id
|
|
WHERE NOT EXISTS (
|
|
SELECT 1 FROM backend.transactions bt
|
|
WHERE bt.corporate_entity = c.name
|
|
AND bt.tx_date = t.executed_at::text
|
|
AND bt.tx_amount = t.amount
|
|
)
|
|
");
|
|
|
|
// 2. Risk Assessment Output
|
|
DB::statement("
|
|
INSERT INTO backend.transaction_outputs (
|
|
transaction_id,
|
|
prompt_id,
|
|
output_key,
|
|
content
|
|
)
|
|
SELECT
|
|
bt.id,
|
|
1,
|
|
'risk_assessment',
|
|
jsonb_build_object(
|
|
'score', t.risk_score,
|
|
'level', CASE
|
|
WHEN t.risk_score >= 70 THEN 'high'
|
|
WHEN t.risk_score >= 40 THEN 'medium'
|
|
ELSE 'low'
|
|
END,
|
|
'requires_review', t.requires_review,
|
|
'flagged_by', t.flagged_by,
|
|
'flagged_reason', t.flagged_reason,
|
|
'signals', t.signals
|
|
)
|
|
FROM public.transactions t
|
|
JOIN public.companies c ON t.company_id = c.id
|
|
JOIN backend.transactions bt ON (
|
|
bt.corporate_entity = c.name
|
|
AND bt.tx_date = t.executed_at::text
|
|
AND bt.tx_amount = t.amount
|
|
)
|
|
");
|
|
|
|
// 3. Enrichment Outputs (Loop through all sources)
|
|
$sources = [
|
|
'registry' => ['registry_data', 'registry_last_refreshed_at'],
|
|
'sanctions' => ['sanctions_data', 'sanctions_last_refreshed_at'],
|
|
'pep' => ['pep_data', 'pep_last_refreshed_at'],
|
|
'gleif' => ['gleif_data', 'gleif_last_refreshed_at'],
|
|
// ... weitere
|
|
];
|
|
|
|
foreach ($sources as $key => [$dataCol, $refreshCol]) {
|
|
DB::statement("
|
|
INSERT INTO backend.transaction_outputs (
|
|
transaction_id,
|
|
prompt_id,
|
|
output_key,
|
|
content
|
|
)
|
|
SELECT
|
|
bt.id,
|
|
1,
|
|
'{$key}',
|
|
t.{$dataCol}
|
|
FROM public.transactions t
|
|
JOIN public.companies c ON t.company_id = c.id
|
|
JOIN backend.transactions bt ON (
|
|
bt.corporate_entity = c.name
|
|
AND bt.tx_date = t.executed_at::text
|
|
AND bt.tx_amount = t.amount
|
|
)
|
|
WHERE t.{$dataCol} IS NOT NULL
|
|
");
|
|
}
|
|
}
|
|
|
|
public function down(): void
|
|
{
|
|
// Rollback: Lösche migrierte Daten
|
|
DB::statement("
|
|
DELETE FROM backend.transactions
|
|
WHERE status = 'completed'
|
|
AND EXISTS (
|
|
SELECT 1 FROM backend.transaction_outputs
|
|
WHERE transaction_id = backend.transactions.id
|
|
)
|
|
");
|
|
}
|
|
};
|
|
```
|
|
|
|
---
|
|
|
|
## Phase 6: Testing & Rollout
|
|
|
|
### 6.1 Feature Flag basierter Rollout
|
|
|
|
**.env**:
|
|
```env
|
|
# Phase 1: Backend verfügbar, aber inaktiv
|
|
FEATURE_USE_BACKEND_TRANSACTIONS=false
|
|
|
|
# Phase 2: Neue Uploads gehen zu Backend
|
|
FEATURE_BACKEND_CSV_UPLOAD=true
|
|
|
|
# Phase 3: Frontend liest von Backend
|
|
FEATURE_USE_BACKEND_TRANSACTIONS=true
|
|
|
|
# Phase 4: Public deprecated
|
|
FEATURE_DEPRECATE_PUBLIC_SCHEMA=true
|
|
```
|
|
|
|
### 6.2 Monitoring
|
|
|
|
**app/Console/Commands/MonitorTransactionProcessing.php**:
|
|
```php
|
|
<?php
|
|
|
|
namespace App\Console\Commands;
|
|
|
|
use App\Models\Backend\Transaction;
|
|
use Illuminate\Console\Command;
|
|
|
|
class MonitorTransactionProcessing extends Command
|
|
{
|
|
protected $signature = 'transactions:monitor';
|
|
|
|
public function handle(): void
|
|
{
|
|
$stats = [
|
|
'pending' => Transaction::where('status', Transaction::STATUS_PENDING)->count(),
|
|
'processing' => Transaction::where('status', Transaction::STATUS_PROCESSING)->count(),
|
|
'completed' => Transaction::where('status', Transaction::STATUS_COMPLETED)->count(),
|
|
'failed' => Transaction::where('status', Transaction::STATUS_FAILED)->count(),
|
|
];
|
|
|
|
$this->table(
|
|
['Status', 'Count'],
|
|
collect($stats)->map(fn($count, $status) => [$status, $count])->values()
|
|
);
|
|
|
|
// Alert bei vielen Failed
|
|
if ($stats['failed'] > 10) {
|
|
$this->error("⚠️ Warning: {$stats['failed']} failed transactions!");
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Zeitplan
|
|
|
|
| Phase | Beschreibung | Dauer |
|
|
|-------|--------------|-------|
|
|
| 1 | Backend Models & DB Config | 1-2 Tage |
|
|
| 2 | CSV Upload Service | 2-3 Tage |
|
|
| 3 | KI Workflow Integration | 3-5 Tage |
|
|
| 4 | Frontend Anpassung | 3-4 Tage |
|
|
| 5 | Datenmigration | 3-5 Tage |
|
|
| 6 | Testing & Rollout | 2-3 Tage |
|
|
|
|
**Total**: 14-22 Tage (3-4 Wochen)
|
|
|
|
---
|
|
|
|
## Nächste Schritte
|
|
|
|
### Sofort starten:
|
|
1. ✅ Database Config erweitern
|
|
2. ✅ Backend Models erstellen
|
|
3. ✅ Ersten Upload-Test durchführen
|
|
|
|
Soll ich mit der **Implementierung von Phase 1** beginnen?
|