GitHub Integration Testing Guide¶

Date: October 28, 2025 Status: Ready for Testing Version: 20251028.09:28

Prerequisites Check¶

✅ Verified¶

GitHub App credentials configured in .env.local
GITHUB_APP_ID configured
GITHUB_APP_PRIVATE_KEY configured
GITHUB_APP_WEBHOOK_SECRET configured

⚠️ Current Status¶

ENABLE_GITHUB_INTEGRATION=false (disabled for safety)
GitHub App installed on test repository
Webhook URL configured in GitHub App settings
Test repository identified

Step 1: Enable GitHub Integration (Local Testing)¶

Option A: Local Development with ngrok/cloudflare tunnel¶

Why: GitHub webhooks need a public URL to send events

Install ngrok (if not already):

brew install ngrok
# or download from https://ngrok.com/download

Start dev server:

npm run dev
# Server runs on http://localhost:3000

Create tunnel (in new terminal):
```
ngrok http 3000
```

You'll get a public URL like: https://abc123.ngrok-free.app

Update GitHub App webhook URL:
Go to: https://github.com/settings/apps/codeslick
Webhook URL: https://abc123.ngrok-free.app/api/github/webhook
Click "Save changes"
Enable integration in .env.local:
```
ENABLE_GITHUB_INTEGRATION=true
```
Restart dev server (Ctrl+C, then npm run dev)

Option B: Production Testing (Vercel)¶

Skip local testing and deploy to Vercel:

Deploy to Vercel:
```
vercel --prod
```
Update GitHub App webhook URL:
Webhook URL: https://your-domain.vercel.app/api/github/webhook
Set environment variable in Vercel:
Go to Vercel dashboard → Project → Settings → Environment Variables
Add: ENABLE_GITHUB_INTEGRATION=true
Redeploy

Step 2: Install GitHub App on Test Repository¶

Go to GitHub App installation page:
```
https://github.com/apps/codeslick/installations/new
```
(Replace 'codeslick' with your actual app name)
Select repository:
Choose a test repository
Recommendation: Create a new test repo (e.g., codeslick-test-pr)
Or use: vikthor/codeslick2 (this repo)
Grant permissions:
✅ Contents: Read
✅ Pull requests: Read & Write
✅ Checks: Write
✅ Webhooks: Active
Complete installation

Step 3: Create Test PR with Vulnerable Code¶

Create Test File¶

Create new branch:

cd /path/to/test/repo
git checkout -b test/pr-analysis

Create vulnerable test file (test-vulnerable.js):

// SQL Injection vulnerability
function getUserData(userId) {
  const query = "SELECT * FROM users WHERE id = " + userId;
  return db.query(query);
}

// XSS vulnerability
function displayMessage(msg) {
  document.getElementById('output').innerHTML = msg;
}

// Hardcoded credentials
const API_KEY = "sk_live_1234567890abcdef";
const PASSWORD = "admin123";

// Command injection
function pingHost(host) {
  const cmd = "ping " + host;
  exec(cmd);
}

// eval usage
function calculate(expr) {
  return eval(expr);
}

Commit and push:

git add test-vulnerable.js
git commit -m "Add test file with security vulnerabilities"
git push origin test/pr-analysis

Create Pull Request:
Go to GitHub repository
Click "Compare & pull request"
Title: "Test PR: Security analysis"
Description: "Testing CodeSlick PR analysis with vulnerable code"
Click "Create pull request"

Step 4: Monitor Webhook & Analysis¶

Monitor Server Logs¶

Watch your dev server console (or Vercel logs) for:

[GitHub Webhook] Received event: pull_request
[GitHub Webhook] Action: opened
[GitHub Webhook] PR #X in repo owner/repo
[PR Analyzer] Starting analysis for PR #X
[PR Analyzer] Found 5 files to analyze
[PR Analyzer] Analyzing: test-vulnerable.js
[PR Analyzer] Found 10 security issues
[Comment Formatter] Formatting PR comment
[GitHub API] Posting comment to PR #X
✅ Analysis complete - comment posted

Check GitHub PR¶

Go to your PR:
```
https://github.com/owner/repo/pull/X
```
Verify comment appears:
Should see CodeSlick bot comment
Within 10-30 seconds of PR creation

Step 5: Verify PR Comment Format¶

Expected Comment Structure¶

## 🔒 CodeSlick Security Analysis

**Analysis Summary**
- **Files Analyzed**: 1
- **Total Issues**: 10
- **Critical**: 3 🔴
- **High**: 4 🟠
- **Medium**: 2 🟡
- **Low**: 1 🔵

---

### 🔴 Critical Issues (3)

#### SQL Injection in `test-vulnerable.js`
**Line 3**: Direct string concatenation in SQL query
**Severity**: Critical (CVSS 9.8)
**OWASP**: A03:2021 - Injection
**CWE**: CWE-89

**Vulnerable Code**:
```javascript
const query = "SELECT * FROM users WHERE id = " + userId;

Recommendation: Use parameterized queries or prepared statements

🟠 High Issues (4)¶

[... more issues ...]

📊 OWASP Top 10 2021 Mapping¶

A03:2021 - Injection: 3 issues
A02:2021 - Cryptographic Failures: 1 issue
A04:2021 - Insecure Design: 2 issues
A08:2021 - Software and Data Integrity Failures: 1 issue

🤖 Powered by CodeSlick

### Verify These Elements

- [x] **Summary table** with file/issue counts
- [x] **Severity icons** (🔴 🟠 🟡 🔵)
- [x] **OWASP mapping** for each issue
- [x] **CWE references** for each issue
- [x] **CVSS scores** for each issue
- [x] **Code snippets** showing vulnerable lines
- [x] **Recommendations** for fixing each issue
- [x] **OWASP Top 10 summary** at bottom
- [x] **Professional formatting** with proper markdown

---

## Step 6: Test Different Scenarios

### Scenario A: Clean Code (No Issues)

Create PR with clean code:
```javascript
// test-clean.js
function getUserData(userId) {
  // Using parameterized query (safe)
  const query = "SELECT * FROM users WHERE id = ?";
  return db.query(query, [userId]);
}

Expected: Comment should say "✅ No security issues found"

Scenario B: Multiple Files¶

Create PR with 3 files: - file1.js - 5 issues - file2.js - 3 issues - file3.js - 0 issues

Expected: Comment shows breakdown per file

Scenario C: PR Update¶

Push more commits to existing PR:

git add new-file.js
git commit -m "Add another vulnerable file"
git push

Expected: New comment or updated comment with re-analysis

Scenario D: Different Languages¶

Test with Python file:

# test-vulnerable.py
def get_user(user_id):
    # SQL injection
    query = f"SELECT * FROM users WHERE id = {user_id}"
    return db.execute(query)

Expected: Python-specific vulnerability detection

Troubleshooting¶

❌ Webhook not received¶

Check: 1. Ngrok/tunnel is running 2. Webhook URL is correct in GitHub App settings 3. Dev server is running 4. Check ngrok dashboard: http://127.0.0.1:4040

Debug:

# Check webhook deliveries in GitHub
# Go to: https://github.com/settings/apps/codeslick
# Click "Advanced" → "Recent Deliveries"
# Check response status and payload

❌ Comment not posted¶

Check server logs for:

[PR Analyzer] Error: ...
[GitHub API] Error posting comment: ...

Common issues: - GitHub App not installed on repository - Missing permissions (Pull requests: Write) - Invalid authentication token - Rate limiting

Debug endpoint:

# Test webhook endpoint manually
curl -X POST http://localhost:3000/api/github/webhook \
  -H "Content-Type: application/json" \
  -H "X-GitHub-Event: pull_request" \
  -d '{"action":"opened","pull_request":{"number":1}}'

❌ Analysis fails¶

Check: - File size limits (>1MB files skipped) - Unsupported languages - Syntax errors in code

Server logs should show:

[PR Analyzer] Skipping large file: huge-file.js (2.5MB)
[PR Analyzer] Unsupported language: .txt

Success Criteria¶

✅ Integration Working If:¶

Webhook received within 1 second of PR creation
Analysis completes within 10-30 seconds
Comment appears on PR automatically
Comment format matches expected structure
OWASP mapping correctly identifies vulnerability types
CVSS scores are accurate
Code snippets show exact vulnerable lines
Recommendations are helpful and specific

📊 Performance Targets¶

Webhook processing: <100ms
Single file analysis: 2-5 seconds
PR with 5 files: <30 seconds total
Comment posting: <1 second

Monitoring Commands¶

Watch Logs (Local)¶

# Terminal 1: Dev server
npm run dev

# Terminal 2: Ngrok tunnel
ngrok http 3000

# Terminal 3: Tail logs
tail -f /path/to/logs  # Or watch console output

Check Database (Optional)¶

# If tracking PR analyses in database
# Check analysis_records table

GitHub Webhook Deliveries¶

https://github.com/settings/apps/codeslick/advanced

Check "Recent Deliveries" for webhook status

Quick Test Script¶

#!/bin/bash
# quick-github-test.sh

echo "🚀 Starting GitHub Integration Test"

# 1. Start dev server
echo "1. Starting dev server..."
npm run dev &
DEV_PID=$!
sleep 5

# 2. Start ngrok
echo "2. Starting ngrok tunnel..."
ngrok http 3000 > /dev/null &
NGROK_PID=$!
sleep 3

# 3. Get ngrok URL
NGROK_URL=$(curl -s http://localhost:4040/api/tunnels | jq -r '.tunnels[0].public_url')
echo "   Ngrok URL: $NGROK_URL"

# 4. Update GitHub webhook
echo "3. Update GitHub App webhook to: $NGROK_URL/api/github/webhook"
echo "   Go to: https://github.com/settings/apps/codeslick"

# 5. Create test PR
echo "4. Create test PR with vulnerable code..."
echo "   Follow manual steps above"

echo ""
echo "✅ Setup complete!"
echo "   Dev server: http://localhost:3000"
echo "   Ngrok URL: $NGROK_URL"
echo "   Webhook: $NGROK_URL/api/github/webhook"
echo ""
echo "Now create a PR to test!"

Next Steps After Successful Test¶

Document findings in test results

Disable integration if testing on local:

# .env.local
ENABLE_GITHUB_INTEGRATION=false

Update webhook URL back to production (if applicable)
Review performance metrics
Plan production rollout

Status: Ready for Testing Requirements: ngrok or Vercel deployment + test repository Estimated Time: 15-30 minutes for full test