Skip to content

Python Analyzer - Known Limitations

Document Date: 2025-11-21 Phase: Phase 6 Testing Status: Documented for future enhancement

Overview

During Phase 6 testing with PHASE6-django-flask-security.py, we identified conservative detection patterns that intentionally minimize false positives but miss some valid vulnerabilities. These are known limitations, not bugs.

Limitation #1: Missing @login_required Detection (TOO NARROW)

Current Behavior

CHECK #26: Django Missing @login_required decorator (lines 1469-1497 in python-analyzer.ts)

Detection Logic: 

if ((trimmed.includes('def ') || trimmed.includes('class ')) &&
    (trimmed.toLowerCase().includes('view') ||
     trimmed.toLowerCase().includes('delete') ||
     trimmed.toLowerCase().includes('update'))) {
  // Only flags if ALSO contains: 'delete' OR 'remove' OR 'update'
  if (trimmed.includes('delete') || trimmed.includes('remove') || trimmed.includes('update')) {
    // Flag as missing @login_required
  }
}

What's DETECTED ✅

  • def delete_user(request): - Contains 'delete'
  • def update_profile(request): - Contains 'update'
  • def remove_account(request): - Contains 'remove'

What's MISSED ❌

  • def admin_dashboard(request): - No required keywords
  • def user_settings(request): - No required keywords
  • def view_invoices(request): - Has 'view' but not 'delete/update/remove'

Impact

Severity: Medium False Negatives: High (misses many sensitive views) False Positives: Low (very conservative)

Recommendation

Expand keyword list to include: - 'admin' (admin_dashboard, admin_panel) - 'dashboard' (user_dashboard, control_dashboard) - 'settings' (user_settings, account_settings) - 'invoice' (view_invoices, manage_invoices) - 'payment' (process_payment, view_payments) - 'transfer' (transfer_funds, money_transfer) - 'export' (export_data, download_report)

Alternative: Use heuristics like: - Any function starting with def view_* - Any function with 'admin' in name - Any function that queries User/Account models

Limitation #2: Flask CSRF Detection (APP-LEVEL ONLY)

Current Behavior

CHECK #29: Flask Missing CSRF Protection (lines 1555-1580 in python-analyzer.ts)

Detection Logic: 

// Fires ONCE when seeing: from flask import Flask OR Flask(__name__)
if (trimmed.includes('from flask import Flask') || trimmed.includes('Flask(__name__)')) {
  // Checks entire file for CSRFProtect import
  const hasCSRF = lines.some(line =>
    line.includes('CSRFProtect') || line.includes('flask_wtf')
  );
  if (!hasCSRF) {
    // Flag as missing CSRF protection
  }
}

What's DETECTED ✅

  • Flask app without CSRFProtect import (app-level check)

What's MISSED ❌

  • Individual routes without CSRF validation
  • Routes with @app.route() but no CSRF check
  • POST/PUT/DELETE endpoints without CSRF tokens

Impact

Severity: Low False Negatives: Medium (misses route-level issues) False Positives: None

Recommendation

Add route-level CSRF detection: 

// Check individual POST/PUT/DELETE routes
if (trimmed.includes('@app.route(') &&
    (trimmed.includes('POST') || trimmed.includes('PUT') || trimmed.includes('DELETE'))) {
  // Look for CSRF validation in function
  // Check for @csrf.exempt decorator
}

Limitation #3: Variable Tracking Scope Issues

Current Behavior

CHECK #25: Django ORM raw() SQL injection with data flow analysis (lines 747-775, 1419-1467)

Detection Logic: - First pass: Track variables with SQL + string formatting - Second pass: Flag .raw(variable) if variable is tracked - Uses simple Map<string, number> for tracking

What's DETECTED ✅

  • Inline formatting: User.objects.raw(f"SELECT...")
  • Variable usage: User.objects.raw(query) where query is tracked

What's MISSED ❌

  • Variable name collisions (same name used twice) 
    query = "SELECT..." % var  # Line 139
query = f"SELECT..."       # Line 155 (overwrites)
User.objects.raw(query)    # References line 155, should be 139

Impact

Severity: Low False Negatives: None (still detects the issue) False Positives: None User Experience: Line reference may be wrong with name collisions

Recommendation

Implement scope-aware tracking: - Track function boundaries - Use Map<string, number[]> for multiple assignments - Find "closest" assignment to usage (same function scope)

Complexity: High (requires scope analysis) Priority: Low (detection still works, just UX issue)

Testing Evidence

Test File

test-files/PHASE6-django-flask-security.py

Results

  • Total checks: 11 Django/Flask framework-specific checks
  • Detected: 11 vulnerabilities (various lines)
  • False positives: 0 (all comments properly skipped)
  • False negatives: 3+ (missing @login_required detections)

User Validation

Testing performed with user feedback on 2025-11-21 Images: #1, #2, #3 show missing @login_required detections All documented and accepted as known limitations

Priority for Future Work

High Priority

  • Limitation #1: Expand @login_required keyword detection Effort: Low (add keywords to array) Impact: High (catches more missing auth checks)

Medium Priority

  • Limitation #2: Add route-level Flask CSRF checks Effort: Medium (new detection pattern) Impact: Medium (more granular CSRF detection)

Low Priority

  • Limitation #3: Scope-aware variable tracking Effort: High (requires scope analysis) Impact: Low (UX improvement only)
  • docs/technical/ANALYZER_COVERAGE.md - Full analyzer capabilities
  • docs/technical/WEBTOOL_GITHUB_SEPARATION.md - Shared module safety
  • docs/technical/ui-fixes/MONACO_DISPLAY_BUG_PHASE6_TESTING.md - UI issues

Changelog

2025-11-21: Initial documentation Phase 6 Testing: Limitations identified and documented Status: Accepted, deferred to future enhancement