Place-Order-Trading-Bot/PNL_TRACKER_GUIDE.md

💰 P&L Tracker Complete Guide

Status: ✅ FULLY INTEGRATED Date: 2026-01-21 Cells Added: 6 new cells (Positions 85-90) Version: V1.9

---

✅ What Was Implemented

I've added complete P&L tracking with automatic MT5 history import to your trading bot!

6 New Cells:

Cell 85: Markdown - Section Header

💰 P&L TRACKING & PERFORMANCE ANALYTICS (V1.9)
- Automatic MT5 History Import
- Real P&L Calculation
- Win Rate Analysis
- Performance Metrics

Cell 86: Code - Setup P&L Tracker

pnl_tracker = MT5PnLTracker(db_path="trading_bot.db")
pnl_tracker.connect_db()

Cell 87: Code - Initial MT5 History Sync

sync_results = pnl_tracker.sync_and_update(days_back=30)
# Imports last 30 days of MT5 trading history

Cell 88: Code - Add to Scheduler

# Automatic hourly sync
scheduler.add_job(scheduled_pnl_sync, ...)

Cell 89: Markdown - Usage Instructions

Cell 90: Code - Live Dashboard Display

dashboard = pnl_tracker.generate_dashboard()
# Shows All-Time, Month, Week, Today performance

---

🚀 How to Start

Step 1: Open Notebook

jupyter notebook TradingBot_V1.6_Adaptive_Complete_CORRECTED.ipynb

Step 2: Restart Kernel (CRITICAL!)

Menu: Kernel → Restart & Clear Output
Confirm: Yes

Why critical?

• Loads new mt5_pnl_tracker.py module
• Clears old cached imports
• Fresh start with new P&L system

Step 3: Run All Cells

Menu: Cell → Run All

What happens:

1. All existing cells run normally (Cells 1-84)
2. Cell 86 initializes P&L tracker
3. Cell 87 imports MT5 history (last 30 days)
4. Cell 88 adds hourly auto-sync to scheduler
5. Cell 90 displays live P&L dashboard!

Step 4: Verify Installation

After Run All, scroll to Cell 87:

Expected Output:

📥 Importing MT5 history...
   This will import last 30 days of trades from MT5
   Please wait...

================================================================================
✅ SYNC SUCCESSFUL!
================================================================================

📥 Import Results:
   New Deals:          184
   Matched Positions:  92

📊 Current Performance:
   Total Trades:       92
   Win Rate:           78.3%
   Net P&L:            $1,245.67
================================================================================

Cell 88 Output:

🔄 Adding P&L sync to scheduler...
✅ P&L sync scheduled (every 1 hour)
   Syncs last 7 days from MT5

📋 Active Scheduler Jobs:
   • adaptive_trading_check: interval[0:01:00]
   • threshold_optimization: cron[day='*' hour='0']
   • enhanced_trailing_stop: interval[0:01:00]
   • position_monitor: interval[0:05:00]
   • pnl_sync: interval[1:00:00]

✅ Scheduler updated successfully!

Cell 90 Output (Dashboard):

================================================================================
💰 MT5 P&L TRACKER - LIVE PERFORMANCE DASHBOARD
================================================================================

Generated: 2026-01-21 14:30:15

================================================================================
📊 ALL TIME PERFORMANCE
================================================================================

Total Trades:        92
Winning Trades:      72 (78.3%)
Losing Trades:       20

Net Profit:          $1,245.67
Total Profit:        $1,580.00
Total Loss:          $334.33
Profit Factor:       4.72

Average Win:         $21.94
Average Loss:        $16.72
Largest Win:         $45.50
Largest Loss:        $28.00

Max Drawdown:        $-112.50
Avg Duration:        4.2 hours
Total Pips:          1,850.5

================================================================================
📅 THIS MONTH
================================================================================

Trades:      24 (79.2% WR)
Net Profit:  $456.00
Profit/Loss: +$580.00 / -$124.00

================================================================================
📅 THIS WEEK
================================================================================

Trades:      8 (87.5% WR)
Net Profit:  $168.50
Profit/Loss: +$195.00 / -$26.50

================================================================================
📅 TODAY
================================================================================

Trades:      2 (100.0% WR)
Net Profit:  $42.50
Profit/Loss: +$42.50 / -$0.00

================================================================================

================================================================================
📜 RECENT TRADES (Last 10)
================================================================================

 position_id symbol  type      entry_time       exit_time  net_profit  pips  duration_hours     status
    12345678 XAUUSD  LONG  2026-01-21 10:00  2026-01-21 14:00       22.50  15.0             4.0   ✅ WIN
    12345677 XAUUSD  LONG  2026-01-20 23:45  2026-01-21 03:30       18.75  12.5             3.8   ✅ WIN
    12345676 XAUUSD SHORT  2026-01-20 18:30  2026-01-20 22:15      -14.20  -9.5             3.8  ❌ LOSS
    ...

================================================================================
✅ Dashboard refresh complete!
Last updated: 2026-01-21 14:30:15
================================================================================

---

🎯 What Now Runs Automatically

1. Hourly MT5 History Sync

Every hour:

1. Connects to MT5
2. Imports deals from last 7 days
3. Matches Entry + Exit deals to create complete positions
4. Calculates P&L for each position
5. Updates database

Job name: pnl_sync Frequency: Every 1 hour Data synced: Last 7 days

2. Real-Time P&L Calculation

For each closed position:

1. Finds Entry deal (IN)
2. Finds Exit deal (OUT)
3. Calculates:
   - Gross Profit = Exit.profit
   - Commission = Entry.commission + Exit.commission
   - Swap = Entry.swap + Exit.swap
   - Net P&L = Gross + Commission + Swap
   - Pips = (Exit.price - Entry.price) / pip_value
4. Stores in matched_positions table

3. Performance Metrics

Automatically calculates:

• Win Rate = (Winning Trades / Total Trades) × 100
• Profit Factor = |Total Profit / Total Loss|
• Max Drawdown = Largest cumulative loss from peak
• Average Win = Mean profit of winning trades
• Average Loss = Mean loss of losing trades

4. Multi-Period Analysis

Four timeframes:

1. All Time - Complete trading history
2. This Month - Last 30 days
3. This Week - Last 7 days
4. Today - Current day only

---

📊 How to Use P&L Tracker

View Live Dashboard

Run Cell 90 anytime to refresh:

# Cell 90 already has this code
dashboard = pnl_tracker.generate_dashboard()
print(dashboard)

Shows:

• All-time performance summary
• Monthly/Weekly/Daily breakdown
• Recent 10 trades
• Win/Loss status

Manual Sync (if needed)

Force immediate sync:

# In a new cell or Cell 87
sync_results = pnl_tracker.sync_and_update(days_back=30)

if sync_results['success']:
    print(f"✅ Synced: {sync_results['summary']['new_deals']} new deals")
    print(f"📊 Matched: {sync_results['summary']['matched_positions']} positions")
else:
    print(f"❌ Error: {sync_results.get('error')}")

Get Specific Period Metrics

Custom analysis:

# In a new cell
all_time = pnl_tracker.calculate_pnl_metrics('all')
print(f"All-time Win Rate: {all_time['win_rate']:.1f}%")
print(f"Profit Factor: {all_time['profit_factor']:.2f}")

month = pnl_tracker.calculate_pnl_metrics('month')
print(f"This month: {month['total_trades']} trades, ${month['net_profit']:.2f}")

week = pnl_tracker.calculate_pnl_metrics('week')
today = pnl_tracker.calculate_pnl_metrics('today')

View Recent Trades

Get last N trades:

# In a new cell
recent = pnl_tracker.get_recent_trades(limit=20)
print(recent.to_string(index=False))

Export to DataFrame

For further analysis:

# In a new cell
import pandas as pd

# Get all matched positions
query = "SELECT * FROM matched_positions ORDER BY exit_time DESC"
df = pd.read_sql_query(query, pnl_tracker.conn)

# Analyze by session (if you track session data)
print(df.groupby('symbol')['net_profit'].agg(['count', 'sum', 'mean']))

# Analyze by hour
df['hour'] = pd.to_datetime(df['entry_time']).dt.hour
hourly = df.groupby('hour')['net_profit'].sum()
print(hourly.sort_values(ascending=False))

---

🎯 Integration with Existing Features

1. Dynamic Threshold Optimizer

P&L data enhances threshold optimization:

# The threshold optimizer can now use REAL Win Rate from MT5!

# Get real Win Rate
metrics = pnl_tracker.calculate_pnl_metrics('month')
real_win_rate = metrics['win_rate']

# Compare with bot's internal tracking
print(f"Real MT5 Win Rate: {real_win_rate:.1f}%")
print(f"Bot Internal WR: {threshold_optimizer.current_win_rate:.1f}%")

# Use real data for optimization
if real_win_rate < 60:
    print("⚠️ Real WR below target → Increase threshold")
elif real_win_rate > 70:
    print("✅ Real WR excellent → Consider lower threshold for more trades")

2. Enhanced Signal Scoring

Validate enhanced scoring impact:

# Compare P&L before/after enhanced scoring implementation

# Get trades from last 30 days
recent_metrics = pnl_tracker.calculate_pnl_metrics('month')

# If you added enhanced scoring recently, you can:
# 1. Compare Win Rate before/after
# 2. Check if Profit Factor improved
# 3. Analyze if drawdown reduced

print(f"Recent Performance:")
print(f"   Win Rate: {recent_metrics['win_rate']:.1f}%")
print(f"   Profit Factor: {recent_metrics['profit_factor']:.2f}")
print(f"   Max Drawdown: ${recent_metrics['max_drawdown']:.2f}")

3. Enhanced Trailing Stop

Measure trailing stop effectiveness:

# Check average profit per winning trade
metrics = pnl_tracker.calculate_pnl_metrics('all')

print(f"Average Win: ${metrics['avg_win']:.2f}")
print(f"Average Loss: ${metrics['avg_loss']:.2f}")
print(f"Win/Loss Ratio: {abs(metrics['avg_win'] / metrics['avg_loss']):.2f}")

# If Win/Loss ratio improved after enhanced trailing stop:
# → Trailing stop is working! (Locking profits, cutting losses faster)

---

📈 Advanced Features

1. Database Structure

Three main tables:

mt5_deals - Raw MT5 deals

deal_id, ticket, order, time, type, entry, magic,
position_id, volume, price, commission, swap, profit,
symbol, comment, imported_at

matched_positions - Complete trades (Entry + Exit)

position_id, symbol, type, volume,
entry_price, exit_price, entry_time, exit_time,
profit, commission, swap, net_profit, pips,
is_win, duration_hours, magic, matched_at

pnl_summary - Aggregated metrics

period_type, period_start, period_end,
total_trades, win_rate, net_profit, profit_factor,
max_drawdown, calculated_at

2. Direct SQL Queries

Custom analysis:

import sqlite3
import pandas as pd

conn = sqlite3.connect("trading_bot.db")

# Get best trading hours
query = """
    SELECT
        strftime('%H', entry_time) as hour,
        COUNT(*) as trades,
        SUM(CASE WHEN is_win = 1 THEN 1 ELSE 0 END) as wins,
        ROUND(AVG(CASE WHEN is_win = 1 THEN 1.0 ELSE 0 END) * 100, 1) as win_rate,
        ROUND(SUM(net_profit), 2) as total_profit
    FROM matched_positions
    GROUP BY hour
    ORDER BY total_profit DESC
    LIMIT 10
"""

best_hours = pd.read_sql_query(query, conn)
print(best_hours)

conn.close()

3. Session-Based Analysis

If you add session tracking to matched_positions:

# After adding session column to matched_positions
# (requires modifying mt5_pnl_tracker.py to detect session from entry_time)

query = """
    SELECT
        session,
        COUNT(*) as trades,
        ROUND(AVG(CASE WHEN is_win = 1 THEN 1.0 ELSE 0 END) * 100, 1) as win_rate,
        ROUND(SUM(net_profit), 2) as total_profit
    FROM matched_positions
    GROUP BY session
    ORDER BY total_profit DESC
"""

session_analysis = pd.read_sql_query(query, pnl_tracker.conn)
print(session_analysis)

---

⚠️ Important Notes

1. First Sync May Take Time

Initial sync imports 30 days:

• 100+ deals → Takes 5-10 seconds
• 500+ deals → Takes 20-30 seconds
• 1000+ deals → Takes 40-60 seconds

Normal! Be patient during first run.

2. MT5 Must Be Running

P&L sync requires:

• ✅ MT5 Terminal running
• ✅ Account logged in
• ✅ Trading history available

If sync fails:

❌ SYNC FAILED
Error: MT5 initialization failed

Solution:

1. Check MT5 is running
2. Verify account connected
3. Re-run Cell 87

3. Hourly Sync Is Automatic

After initial setup:

• Bot automatically syncs every hour
• No manual intervention needed
• Runs in background via scheduler

To check sync status:

# Check last sync time
import sqlite3
conn = sqlite3.connect("trading_bot.db")
cursor = conn.cursor()

cursor.execute("SELECT MAX(imported_at) FROM mt5_deals")
last_import = cursor.fetchone()[0]
print(f"Last MT5 import: {last_import}")

conn.close()

4. Magic Number Filter (Optional)

If you want to track only bot trades:

# Cell 86 - Modify initialization
pnl_tracker = MT5PnLTracker(
    db_path="trading_bot.db",
    magic_number=123456  # Your bot's magic number
)

Benefits:

• Excludes manual trades
• Tracks only bot performance
• More accurate bot metrics

To find your magic number:

# In MT5 or in your bot config
# Usually set in execute_trade_v2_adaptive()
# Check Cell 25 or trading config

---

🆘 Troubleshooting

Problem 1: "ModuleNotFoundError: mt5_pnl_tracker"

Error:

ModuleNotFoundError: No module named 'mt5_pnl_tracker'

Solution:

# Add to Cell 1 (after imports)
import sys
sys.path.append('/path/to/Place-Order-Trading-Bot')

Or verify file exists:

ls Place-Order-Trading-Bot/mt5_pnl_tracker.py

Problem 2: "No deals found"

Message:

✅ SYNC SUCCESSFUL!
📥 Import Results:
   New Deals:          0
   Matched Positions:  0

Possible reasons:

1. No trading history in MT5

   • Check MT5 → Account History
   • Verify date range

2. Magic number filter too restrictive

   • Set magic_number=None to include all trades

3. History already imported

   • Normal on subsequent syncs
   • Only new deals are imported

Solution:

# Try longer period
sync_results = pnl_tracker.sync_and_update(days_back=60)

Problem 3: "MT5 initialization failed"

Error:

❌ SYNC FAILED
Error: MT5 initialization failed

Solutions:

1. Check MT5 is running:

   import MetaTrader5 as mt5
   if not mt5.initialize():
       print(f"Error: {mt5.last_error()}")
   else:
       print("✅ MT5 connected")
       mt5.shutdown()
   

2. Restart MT5 Terminal

3. Check account login:

   • MT5 → File → Login to Trade Account

4. Verify MT5 Python package:

   python -m pip install --upgrade MetaTrader5
   

Problem 4: Positions Not Matching

Message:

✅ SYNC SUCCESSFUL!
📥 Import Results:
   New Deals:          50
   Matched Positions:  0

Reason: Only open positions (no closed trades yet)

Solution: Wait for trades to close, then re-sync

Problem 5: Scheduler Conflict

Error:

ConflictingIdError: Job identifier (pnl_sync) conflicts with...

Solution:

# Cell 88 - Remove old job first
scheduler.remove_job('pnl_sync')
# Then re-run Cell 88

---

✅ Verification Checklist

After setup, verify:

• Kernel restarted
• All cells ran without errors
• Cell 86 output: "✅ P&L Tracker initialized"
• Cell 87 output: "✅ SYNC SUCCESSFUL"
• Cell 88 shows pnl_sync in scheduler jobs
• Cell 90 displays dashboard with metrics
• Recent trades shown (if trades exist)
• Scheduler shows 5 jobs total (including pnl_sync)

---

🎯 Success Metrics

When fully working, you'll see:

1. Real-Time P&L Dashboard

   • Updates every time you run Cell 90
   • Shows accurate Win Rate from MT5
   • Tracks total profit/loss

2. Hourly Auto-Sync

   • New trades automatically imported
   • No manual intervention needed
   • Always up-to-date metrics

3. Comprehensive Performance Data

   • All-time statistics
   • Period-based analysis
   • Trade-by-trade breakdown

4. Integration with Optimizations

   • Dynamic Thresholds use real Win Rate
   • Enhanced Scoring validation
   • Trailing Stop effectiveness measurement

---

🎉 You Now Have

✅ Automatic MT5 History Import - Never manually track trades again ✅ Real P&L Calculation - Accurate profit/loss from closed trades ✅ Multi-Period Analysis - Today, Week, Month, All-Time ✅ Win Rate Tracking - Real Win Rate from MT5, not estimates ✅ Performance Metrics - Profit Factor, Drawdown, Win/Loss Ratio ✅ Live Dashboard - Always up-to-date performance view ✅ Hourly Auto-Sync - Runs in background automatically

= Complete P&L tracking system integrated with your trading bot!

---

📚 Additional Resources

• MT5 Python Documentation: https://www.mql5.com/en/docs/integration/python_metatrader5
• SQLite Documentation: https://www.sqlite.org/docs.html
• Pandas Documentation: https://pandas.pydata.org/docs/

---

🎯 Next Steps

1. Run the notebook - Restart kernel → Run All
2. Verify P&L sync - Check Cell 87 output
3. View dashboard - Scroll to Cell 90
4. Monitor hourly sync - Check logs every hour
5. Analyze performance - Use dashboard to track Win Rate, P&L
6. Optimize based on data - Adjust thresholds based on real metrics

---

Questions? See existing documentation:

• OPTIMIZATION_INTEGRATION_GUIDE.md - For other optimizations
• CONFIGURATION_GUIDE.md - For config settings
• QUICK_START_OPTIMIZATIONS.md - For Option E setup

---

🎯 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4.5 noreply@anthropic.com