Phase 7 Week 2: API & Automation

Status: In Progress Timeline: 5 days Prerequisites: ✅ Week 1 complete (95 tests passing)

---

Week 2 Overview

Goal

Build the API layer and webhook automation that allows users to trigger auto-fix PRs and handles GitHub events automatically.

What We're Building

Day 1: API endpoint (/api/teams/[teamId]/create-fix-pr) - User-triggered fix PR creation - Authentication, authorization, quota checks - Integration with Week 1 job queue

Day 2: Webhook integration - Detect fix opportunities in PR analysis - Post comments with "Create Fix PR" button - Handle PR merge events

Day 3: Error handling & edge cases - GitHub API rate limits - Fix validation failures - PR conflicts

Day 4: Quota management - Define quotas per plan - Track usage in database - Enforce limits

Day 5: Monitoring & logging - PostHog telemetry - Error alerting - Admin dashboard

---

Day 1: API Endpoint

Objective

Create REST API endpoint that allows users to trigger auto-fix PR creation from the dashboard.

Component: /api/teams/[teamId]/create-fix-pr/route.ts

Input:

POST /api/teams/:teamId/create-fix-pr
{
  "prUrl": "https://github.com/owner/repo/pull/123"
}

Output:

{
  "success": true,
  "jobId": "codeslick-fix-owner-repo-123-abc456",
  "message": "Fix PR creation job queued"
}

Implementation Steps

1. Authentication (NextAuth session required)
2. Authorization (user must be team owner/admin/member)
3. Parse PR URL (extract owner, repo, prNumber)
4. Fetch PR data (via GitHub API to get files/vulnerabilities)
5. Check quota (based on team's plan)
6. Enqueue job (using FixPRQueue from Week 1)
7. Return job ID (for tracking)

Error Handling

• 401: Not authenticated
• 403: Not authorized (not team member)
• 400: Invalid PR URL
• 404: PR not found
• 429: Quota exceeded
• 500: Server error

Tests

1. ✅ Successful fix PR creation
2. ✅ Unauthorized user (not logged in)
3. ✅ Forbidden user (not team member)
4. ✅ Invalid PR URL format
5. ✅ PR not found on GitHub
6. ✅ Quota exceeded (Free plan)
7. ✅ Quota limit (Team plan at limit)
8. ✅ PR already has fix PR
9. ✅ No fixable vulnerabilities
10. ✅ GitHub API error

Target: 10+ tests, all passing

Acceptance Criteria

• Endpoint responds with 200 + jobId for valid request
• Endpoint rejects unauthenticated requests (401)
• Endpoint rejects unauthorized requests (403)
• Endpoint validates PR URL format
• Endpoint checks quota before enqueueing
• Job is successfully enqueued to FixPRQueue
• All tests passing

---

Day 2: Webhook Integration

Objective

Automatically detect fix opportunities when PRs are analyzed and post helpful comments.

Component: Update /api/github/webhook/route.ts

Webhook Events to Handle: 1. pull_request.opened - Analyze PR, post auto-fix CTA if fixable issues found 2. pull_request.synchronize - Re-analyze on new commits 3. pull_request.closed - Update fix_prs table if PR merged

Implementation Steps

1. Enhance PR analysis (detect fixable vulnerabilities)
2. Post comment with CTA (if fixable issues found)
3. Track fix PR lifecycle (update database on merge)
4. Handle re-analysis (when original PR updated)

Comment Format

## 🔒 CodeSlick Security Analysis

Found **X vulnerabilities** (Y fixable automatically)

### Fixable Issues
- SQL Injection in `auth.js:42`
- XSS in `UserInput.tsx:18`
- Hardcoded secret in `config.py:7`

[🔧 Create Auto-Fix PR](https://codeslick.dev/teams/abc123/create-fix?pr=...)

Tests

1. ✅ PR opened → comment posted with CTA
2. ✅ PR with no fixable issues → no CTA
3. ✅ PR synchronized → re-analyze
4. ✅ PR merged → update fix_prs table
5. ✅ Fix PR merged → post success comment
6. ✅ Multiple PRs → handle concurrently
7. ✅ Invalid installation → graceful error
8. ✅ GitHub API timeout → retry
9. ✅ Webhook signature validation
10. ✅ Duplicate webhook events (idempotency)

Target: 10+ tests, all passing

Acceptance Criteria

• Webhook posts comment when fixable issues found
• Comment includes link to create fix PR
• PR merge events update fix_prs table
• Re-analysis triggers on PR updates
• All tests passing

---

Day 3: Error Handling & Edge Cases

Objective

Handle all the ways things can go wrong gracefully.

Error Scenarios

1. GitHub API rate limits
2. Detect 403 rate limit response
3. Exponential backoff (1s, 2s, 4s, 8s, 16s)

4. Queue job for retry later

5. Fix validation failures

6. Fixed code has syntax errors
7. Fixed code breaks tests

8. Rollback and notify user

9. PR conflicts

10. Base branch deleted
11. PR already merged
12. Base branch has conflicts

13. Skip and notify

14. GitHub permissions

15. Installation lacks write permissions
16. Branch protection rules
17. User-friendly error messages

Implementation

Create src/lib/github/error-handler.ts:

export class GitHubAPIError extends Error {
  constructor(
    public statusCode: number,
    public isRateLimit: boolean,
    public retryAfter?: number
  ) {
    super();
  }
}

export function handleGitHubError(error: any): GitHubAPIError {
  // Parse error, determine retry strategy
}

Tests

1. ✅ Rate limit → exponential backoff
2. ✅ Invalid syntax after fix → rollback
3. ✅ Branch deleted → skip job
4. ✅ PR merged → skip job
5. ✅ Insufficient permissions → user-friendly error
6. ✅ Network timeout → retry
7. ✅ Invalid credentials → alert admin
8. ✅ Concurrent PR updates → handle race condition

Target: 8+ tests, all passing

---

Day 4: Quota Management

Objective

Define and enforce quotas for auto-fix PR feature.

Quota Definitions

Plan	Auto-Fix PRs/Month	Cost
Free	0	€0
Team	10	€99/month
Enterprise	Unlimited	€299/month

Database Schema Addition

ALTER TABLE teams ADD COLUMN auto_fix_prs_used INTEGER DEFAULT 0;
ALTER TABLE teams ADD COLUMN auto_fix_prs_reset_at TIMESTAMP DEFAULT NOW();

Implementation

Update src/lib/usage/quota-checker.ts:

export async function checkAutoFixQuota(teamId: string): Promise<QuotaCheckResult> {
  const team = await db.select().from(teams).where(eq(teams.id, teamId));

  // Reset counter if month passed
  const now = new Date();
  const resetAt = team.autoFixPrsResetAt;
  if (now.getTime() - resetAt.getTime() > 30 * 24 * 60 * 60 * 1000) {
    await db.update(teams)
      .set({ autoFixPrsUsed: 0, autoFixPrsResetAt: now })
      .where(eq(teams.id, teamId));
  }

  // Check quota
  const limit = getAutoFixLimit(team.plan);
  const remaining = limit === Infinity ? Infinity : limit - team.autoFixPrsUsed;

  return {
    allowed: remaining > 0,
    limit,
    used: team.autoFixPrsUsed,
    remaining
  };
}

Tests

1. ✅ Free plan → quota denied
2. ✅ Team plan → 10 per month
3. ✅ Enterprise plan → unlimited
4. ✅ Counter resets monthly
5. ✅ Concurrent requests → no race condition
6. ✅ Quota exceeded → 429 error

Target: 6+ tests, all passing

---

Day 5: Monitoring & Logging

Objective

Add comprehensive monitoring and logging for production readiness.

PostHog Events

posthog.capture('fix_pr_requested', {
  teamId,
  prUrl,
  fixableCount
});

posthog.capture('fix_pr_created', {
  teamId,
  prNumber,
  filesFixed,
  vulnerabilitiesFixed,
  duration
});

posthog.capture('fix_pr_merged', {
  teamId,
  prNumber,
  timeToMerge
});

posthog.capture('fix_pr_failed', {
  teamId,
  prNumber,
  errorType,
  errorMessage
});

Logging

logger.info('Fix PR creation started', {
  teamId,
  prUrl,
  jobId
});

logger.error('Fix PR creation failed', {
  teamId,
  prUrl,
  error,
  stack
});

Admin Dashboard

Add section to /dashboard/analytics: - Total fix PRs created - Success rate (% merged) - Average time to merge - Top teams using feature

Error Alerting

Email notification on: - 3+ consecutive job failures - GitHub API quota exhausted - Critical errors (500s)

Tests

1. ✅ PostHog events captured correctly
2. ✅ Logs written to Vercel
3. ✅ Admin dashboard displays metrics
4. ✅ Error alerts sent

Target: 4+ tests, all passing

---

Week 2 Deliverables

By end of Week 2, we will have:

1. ✅ API Endpoint (/api/teams/[teamId]/create-fix-pr)
2. User-triggered fix PR creation

3. 10+ tests passing

4. ✅ Webhook Integration (/api/github/webhook)

5. Automatic comment posting
6. PR lifecycle tracking

7. 10+ tests passing

8. ✅ Error Handling (src/lib/github/error-handler.ts)

9. Rate limit handling
10. Validation failures

11. 8+ tests passing

12. ✅ Quota Management (src/lib/usage/quota-checker.ts)

13. Per-plan limits
14. Monthly reset

15. 6+ tests passing

16. ✅ Monitoring (PostHog + Vercel logs)

17. Event tracking
18. Admin dashboard
19. 4+ tests passing

Total New Tests: 38+ (Week 1: 95, Week 2: 38 = 133 total)

---

Success Criteria

• API endpoint works end-to-end (manual test)
• Webhook posts comments automatically
• Quotas enforced correctly
• All tests passing (133+)
• No breaking changes to existing features
• Ready for Week 3 (UI development)

---

Next: Week 3 (UI & Polish)

After Week 2, we'll build: - Team dashboard UI - PR comment UI improvements - Settings & preferences - Documentation & launch

Estimated: 5 days