Phase 4: GitHub PR Integration + Team Collaboration¶
Date: 2025-10-21 Status: 🚀 READY TO START Strategic Goal: Build B2B competitive moat through GitHub automation and team features
Strategic Rationale¶
Why Phase 4 is Critical¶
From CLAUDE.md Strategic Pivot: - ✅ Phase 1-3 Complete: 79+ security checks (competitive with free tools) - ❌ No revenue yet: Individual developers won't pay for static analysis - ✅ Market Opportunity: Teams WILL pay for automated PR reviews - ✅ Competitive Moat: GitHub integration + team learning is defensible
What We're Building: - NOT: More security checks (already have 79+, won't create revenue) - YES: GitHub PR automation (teams pay €99-299/month for this) - YES: Team dashboard (multi-user accounts, analytics, ROI tracking) - YES: Auto-fix PR creation (killer feature for Month 5-6)
Target Outcome: - Launch Team tier (€99/month) by end of Phase 4 - First paying customer within 2 weeks of launch - Foundation for Enterprise tier (€299/month)
Phase 4 Architecture¶
4.1 GitHub App Integration (Week 1-2)¶
Goal: Automatic PR analysis when developers create pull requests
Architecture Design¶
GitHub PR Created
↓
GitHub Webhook → /api/github/webhook → Verify Signature
↓
Queue Analysis Job (Vercel Serverless Function)
↓
Run CodeSlick Analysis (79+ checks)
↓
Post Results as PR Comment (inline + summary)
↓
Update PR Status Check (pass/fail based on severity)
Implementation Structure¶
src/
├── app/api/github/
│ ├── webhook/route.ts # Receives GitHub webhooks
│ ├── install/route.ts # GitHub App installation flow
│ ├── oauth/route.ts # OAuth callback handler
│ └── pr-comment/route.ts # Post results to PR
├── lib/github/
│ ├── github-client.ts # Octokit wrapper
│ ├── webhook-handler.ts # Process webhook events
│ ├── pr-analyzer.ts # Run analysis on PR diff
│ ├── comment-formatter.ts # Format results as markdown
│ └── status-check.ts # Update PR status checks
├── lib/queue/
│ ├── job-queue.ts # Vercel KV-based queue
│ └── analysis-worker.ts # Background analysis job
└── __tests__/github/
├── webhook-handler.test.ts
├── pr-analyzer.test.ts
└── fixtures/
├── pr-payload.json
└── webhook-signature.json
Step 1: GitHub App Setup (Day 1)¶
Tasks:
1. Create GitHub App in GitHub Developer Settings
- App name: "CodeSlick Security Scanner"
- Webhook URL: https://codeslick.vercel.app/api/github/webhook
- Permissions: Read repository content, write PR comments, write status checks
- Events: Pull request (opened, synchronize)
-
Store credentials in environment variables:
-
Create installation flow:
// src/app/api/github/install/route.ts export async function GET(req: Request) { const { searchParams } = new URL(req.url); const code = searchParams.get('code'); const installationId = searchParams.get('installation_id'); // Exchange code for access token const octokit = new Octokit({ auth: code }); // Store installation in database await db.installations.create({ installationId, accountType: 'organization', // or 'user' installedAt: new Date() }); return redirect('/dashboard?github_installed=true'); }
Acceptance Criteria: - ✅ GitHub App created and configured - ✅ Installation flow works (OAuth callback) - ✅ Webhook endpoint receives events
Estimated Time: 1 day Risk: Low (standard GitHub App setup)
Step 2: Webhook Handler (Day 2-3)¶
Implementation:
// src/lib/github/webhook-handler.ts
import { Webhooks } from '@octokit/webhooks';
import { verifyWebhookSignature } from './verify-signature';
export class GitHubWebhookHandler {
private webhooks: Webhooks;
constructor() {
this.webhooks = new Webhooks({
secret: process.env.GITHUB_APP_WEBHOOK_SECRET!
});
// Register handlers
this.webhooks.on('pull_request.opened', this.handlePROpened.bind(this));
this.webhooks.on('pull_request.synchronize', this.handlePRSync.bind(this));
}
async handlePROpened(event: PullRequestOpenedEvent) {
const { repository, pull_request } = event.payload;
// Queue analysis job (don't block webhook response)
await this.queueAnalysisJob({
repoOwner: repository.owner.login,
repoName: repository.name,
prNumber: pull_request.number,
headSha: pull_request.head.sha,
installationId: event.payload.installation.id
});
return { status: 'queued' };
}
async handlePRSync(event: PullRequestSynchronizeEvent) {
// Same as opened (new commits pushed to PR)
return this.handlePROpened(event);
}
private async queueAnalysisJob(job: AnalysisJob) {
// Use Vercel KV for job queue
await kv.lpush('analysis_queue', JSON.stringify(job));
// Trigger background worker (separate Vercel function)
await fetch('/api/github/analyze-worker', { method: 'POST' });
}
}
// src/app/api/github/webhook/route.ts
import { GitHubWebhookHandler } from '@/lib/github/webhook-handler';
export async function POST(req: Request) {
const signature = req.headers.get('x-hub-signature-256');
const body = await req.text();
// Verify webhook signature (security critical)
if (!verifyWebhookSignature(body, signature)) {
return Response.json({ error: 'Invalid signature' }, { status: 401 });
}
const handler = new GitHubWebhookHandler();
await handler.processWebhook(body);
return Response.json({ status: 'ok' });
}
Acceptance Criteria: - ✅ Webhook signature verification works - ✅ PR events trigger analysis jobs - ✅ Jobs queued in Vercel KV - ✅ Response time < 500ms (webhook timeout is 10s)
Testing:
// __tests__/github/webhook-handler.test.ts
describe('GitHubWebhookHandler', () => {
it('should queue analysis job on PR opened', async () => {
const payload = loadFixture('pr-opened.json');
const handler = new GitHubWebhookHandler();
await handler.processWebhook(payload);
const queue = await kv.lrange('analysis_queue', 0, -1);
expect(queue).toHaveLength(1);
expect(JSON.parse(queue[0])).toMatchObject({
repoName: 'test-repo',
prNumber: 123
});
});
it('should reject invalid webhook signatures', async () => {
const response = await POST(new Request('http://localhost', {
method: 'POST',
headers: { 'x-hub-signature-256': 'invalid' },
body: JSON.stringify({})
}));
expect(response.status).toBe(401);
});
});
Estimated Time: 2 days Risk: Medium (webhook signature verification is critical)
Step 3: PR Analysis Worker (Day 4-5)¶
Implementation:
// src/lib/github/pr-analyzer.ts
import { Octokit } from '@octokit/rest';
import { analyzeCode } from '@/lib/analyzers';
export class PRAnalyzer {
private octokit: Octokit;
async analyzePR(job: AnalysisJob) {
// 1. Get PR diff
const { data: files } = await this.octokit.pulls.listFiles({
owner: job.repoOwner,
repo: job.repoName,
pull_number: job.prNumber
});
// 2. Filter to supported languages (JS/TS/PY/Java)
const supportedFiles = files.filter(file =>
/\.(js|ts|py|java)$/.test(file.filename)
);
// 3. Analyze each file
const results = await Promise.all(
supportedFiles.map(file => this.analyzeFile(file))
);
// 4. Aggregate results
const aggregated = this.aggregateResults(results);
// 5. Post PR comment
await this.postPRComment(job, aggregated);
// 6. Update status check
await this.updateStatusCheck(job, aggregated);
return aggregated;
}
private async analyzeFile(file: PRFile) {
// Get file content
const content = await this.getFileContent(file.blob_url);
// Detect language
const language = this.detectLanguage(file.filename);
// Run CodeSlick analysis (existing functionality)
const result = await analyzeCode({
code: content,
language,
filename: file.filename
});
// Map line numbers to PR diff positions
return {
filename: file.filename,
vulnerabilities: result.security.vulnerabilities.map(vuln => ({
...vuln,
diffPosition: this.mapLineToDiffPosition(vuln.line, file.patch)
}))
};
}
private async postPRComment(job: AnalysisJob, results: AggregatedResults) {
const comment = this.formatComment(results);
await this.octokit.issues.createComment({
owner: job.repoOwner,
repo: job.repoName,
issue_number: job.prNumber,
body: comment
});
// Also post inline comments for CRITICAL vulnerabilities
for (const vuln of results.criticalVulnerabilities) {
await this.octokit.pulls.createReviewComment({
owner: job.repoOwner,
repo: job.repoName,
pull_number: job.prNumber,
body: this.formatInlineComment(vuln),
commit_id: job.headSha,
path: vuln.filename,
position: vuln.diffPosition
});
}
}
private formatComment(results: AggregatedResults): string {
return `## 🛡️ CodeSlick Security Analysis
### Summary
- **Files Analyzed**: ${results.filesAnalyzed}
- **Vulnerabilities Found**: ${results.totalVulnerabilities}
- 🔴 Critical: ${results.criticalCount}
- 🟠 High: ${results.highCount}
- 🟡 Medium: ${results.mediumCount}
- 🔵 Low: ${results.lowCount}
### OWASP Top 10 Coverage
- **Score**: ${results.owaspScore}/10
- **Compliance**: ${results.compliancePercentage}%
${results.criticalVulnerabilities.length > 0 ? `
### ⚠️ Critical Vulnerabilities
${results.criticalVulnerabilities.map(v => `
#### ${v.title} (${v.filename}:${v.line})
- **Severity**: ${v.cvssScore} (CRITICAL)
- **CWE**: ${v.cwe}
- **Fix**: ${v.recommendation}
`).join('\n')}
` : ''}
---
*Analyzed by [CodeSlick](https://codeslick.vercel.app) • [View Full Report](${results.reportUrl})*
`;
}
private async updateStatusCheck(job: AnalysisJob, results: AggregatedResults) {
// Fail if critical vulnerabilities found
const state = results.criticalCount > 0 ? 'failure' : 'success';
await this.octokit.repos.createCommitStatus({
owner: job.repoOwner,
repo: job.repoName,
sha: job.headSha,
state,
context: 'CodeSlick Security',
description: `${results.totalVulnerabilities} vulnerabilities found`,
target_url: results.reportUrl
});
}
}
Acceptance Criteria: - ✅ Analyzes all JS/TS/PY/Java files in PR - ✅ Posts summary comment with results - ✅ Posts inline comments for critical vulnerabilities - ✅ Updates PR status check (pass/fail) - ✅ Analysis completes in < 30 seconds for typical PR (5-10 files)
Estimated Time: 2 days Risk: Medium (GitHub API rate limits, diff mapping complexity)
Step 4: Job Queue System (Day 6)¶
Why Needed: Webhook responses must be < 10s, but analysis takes 20-30s
Implementation:
// src/lib/queue/job-queue.ts
import { kv } from '@vercel/kv';
export class JobQueue {
async enqueue(job: AnalysisJob) {
await kv.lpush('analysis_queue', JSON.stringify(job));
// Track job status
await kv.set(`job:${job.id}:status`, 'queued', { ex: 3600 });
}
async dequeue(): Promise<AnalysisJob | null> {
const job = await kv.rpop('analysis_queue');
if (!job) return null;
const parsed = JSON.parse(job);
await kv.set(`job:${parsed.id}:status`, 'processing', { ex: 3600 });
return parsed;
}
async markComplete(jobId: string, result: any) {
await kv.set(`job:${jobId}:status`, 'complete', { ex: 3600 });
await kv.set(`job:${jobId}:result`, JSON.stringify(result), { ex: 86400 });
}
async markFailed(jobId: string, error: string) {
await kv.set(`job:${jobId}:status`, 'failed', { ex: 3600 });
await kv.set(`job:${jobId}:error`, error, { ex: 3600 });
}
}
// src/app/api/github/analyze-worker/route.ts
import { JobQueue } from '@/lib/queue/job-queue';
import { PRAnalyzer } from '@/lib/github/pr-analyzer';
export async function POST(req: Request) {
const queue = new JobQueue();
const analyzer = new PRAnalyzer();
// Process one job (Vercel will spawn multiple instances if needed)
const job = await queue.dequeue();
if (!job) {
return Response.json({ status: 'no_jobs' });
}
try {
const result = await analyzer.analyzePR(job);
await queue.markComplete(job.id, result);
return Response.json({ status: 'complete', jobId: job.id });
} catch (error) {
await queue.markFailed(job.id, error.message);
return Response.json({ status: 'failed', error: error.message }, { status: 500 });
}
}
// Vercel Cron: Trigger worker every minute
export const config = {
maxDuration: 60, // 1 minute
schedule: '* * * * *' // Every minute
};
Acceptance Criteria: - ✅ Webhook response < 500ms (job queued immediately) - ✅ Analysis completes within 1 minute - ✅ Failed jobs tracked and retried - ✅ Queue depth monitored (alert if > 100 jobs)
Estimated Time: 1 day Risk: Low (Vercel KV is reliable)
4.2 Team Dashboard (Week 3)¶
Goal: Multi-user accounts, team analytics, ROI tracking
Database Schema¶
// Using Vercel Postgres (or Supabase for simplicity)
// Teams table
interface Team {
id: string;
name: string;
plan: 'free' | 'team' | 'enterprise';
stripeCustomerId: string;
createdAt: Date;
updatedAt: Date;
}
// Team members
interface TeamMember {
id: string;
teamId: string;
userId: string;
role: 'owner' | 'admin' | 'member';
invitedAt: Date;
joinedAt: Date;
}
// GitHub installations (linked to teams)
interface Installation {
id: string;
teamId: string;
installationId: number; // GitHub App installation ID
accountType: 'organization' | 'user';
repositories: string[]; // Allowed repos
installedAt: Date;
}
// Analysis history
interface AnalysisRecord {
id: string;
teamId: string;
prUrl: string;
repoName: string;
analyzedAt: Date;
filesAnalyzed: number;
vulnerabilitiesFound: number;
criticalCount: number;
highCount: number;
mediumCount: number;
lowCount: number;
fixesApplied: number; // For ROI tracking
}
Team Dashboard UI¶
// src/app/dashboard/team/page.tsx
export default function TeamDashboardPage() {
const { team, members, analytics } = useTeamData();
return (
<div className="container mx-auto px-4 py-8">
{/* Team Overview */}
<section className="mb-8">
<h1 className="text-3xl font-bold mb-4">{team.name}</h1>
<div className="grid grid-cols-4 gap-4">
<MetricCard
label="PRs Analyzed This Month"
value={analytics.prsAnalyzed}
trend="+12% vs last month"
/>
<MetricCard
label="Vulnerabilities Prevented"
value={analytics.vulnerabilitiesPrevented}
trend="+8% vs last month"
/>
<MetricCard
label="Time Saved"
value={`${analytics.hoursSaved}h`}
description="vs manual code review"
/>
<MetricCard
label="ROI"
value={`${analytics.roi}x`}
description="Based on prevented incidents"
/>
</div>
</section>
{/* Recent Analyses */}
<section className="mb-8">
<h2 className="text-2xl font-bold mb-4">Recent PR Analyses</h2>
<AnalysisHistoryTable analyses={analytics.recentAnalyses} />
</section>
{/* Team Members */}
<section className="mb-8">
<h2 className="text-2xl font-bold mb-4">Team Members ({members.length})</h2>
<TeamMembersTable members={members} />
<button className="mt-4 px-4 py-2 bg-indigo-600 text-white rounded">
Invite Team Member
</button>
</section>
{/* GitHub Repositories */}
<section>
<h2 className="text-2xl font-bold mb-4">Connected Repositories</h2>
<RepositoriesTable installations={team.installations} />
<button className="mt-4 px-4 py-2 bg-green-600 text-white rounded">
Install on More Repos
</button>
</section>
</div>
);
}
Key Features: 1. Team Analytics: - PRs analyzed (total, this month, this week) - Vulnerabilities found by severity - Top vulnerable repositories - Most common vulnerability types - Time saved vs manual review (estimate based on avg 15 min/PR)
- ROI Tracking:
- Cost of prevented security incidents (industry avg $4M per breach)
- Time saved (manual review vs automated)
-
Developer productivity improvement
-
Team Management:
- Invite members via email
- Role-based permissions (owner/admin/member)
- Repository access control
Acceptance Criteria: - ✅ Team owners can invite members - ✅ Dashboard shows last 30 days analytics - ✅ ROI calculation based on industry benchmarks - ✅ Export analytics as PDF report
Estimated Time: 5 days Risk: Medium (multi-tenancy complexity)
4.3 Subscription & Billing (Week 4)¶
Goal: Launch Team tier (€99/month) with Stripe integration
Pricing Tiers¶
// src/lib/pricing/plans.ts
export const PLANS = {
free: {
name: 'Free',
price: 0,
currency: 'EUR',
limits: {
prsPerMonth: 20,
repositories: 1,
teamMembers: 1,
apiSecurity: true,
dependencyScanning: true,
prComments: true,
teamDashboard: false,
prioritySupport: false
}
},
team: {
name: 'Team',
price: 99,
currency: 'EUR',
stripePriceId: 'price_xxx',
limits: {
prsPerMonth: -1, // Unlimited
repositories: 5,
teamMembers: 10,
apiSecurity: true,
dependencyScanning: true,
prComments: true,
teamDashboard: true,
prioritySupport: true,
customRules: false
}
},
enterprise: {
name: 'Enterprise',
price: 299,
currency: 'EUR',
stripePriceId: 'price_yyy',
limits: {
prsPerMonth: -1,
repositories: -1, // Unlimited
teamMembers: -1,
apiSecurity: true,
dependencyScanning: true,
prComments: true,
teamDashboard: true,
prioritySupport: true,
customRules: true,
sso: true,
onPremise: true
}
}
};
Stripe Integration¶
// src/app/api/stripe/checkout/route.ts
import Stripe from 'stripe';
const stripe = new Stripe(process.env.STRIPE_SECRET_KEY!);
export async function POST(req: Request) {
const { plan, teamId } = await req.json();
const session = await stripe.checkout.sessions.create({
mode: 'subscription',
payment_method_types: ['card'],
line_items: [{
price: PLANS[plan].stripePriceId,
quantity: 1
}],
success_url: `${process.env.NEXT_PUBLIC_URL}/dashboard/team?checkout=success`,
cancel_url: `${process.env.NEXT_PUBLIC_URL}/pricing`,
metadata: {
teamId,
plan
}
});
return Response.json({ url: session.url });
}
// src/app/api/stripe/webhook/route.ts
export async function POST(req: Request) {
const signature = req.headers.get('stripe-signature');
const body = await req.text();
const event = stripe.webhooks.constructEvent(
body,
signature,
process.env.STRIPE_WEBHOOK_SECRET!
);
switch (event.type) {
case 'checkout.session.completed':
await handleCheckoutCompleted(event.data.object);
break;
case 'customer.subscription.updated':
await handleSubscriptionUpdated(event.data.object);
break;
case 'customer.subscription.deleted':
await handleSubscriptionCancelled(event.data.object);
break;
}
return Response.json({ received: true });
}
async function handleCheckoutCompleted(session: Stripe.Checkout.Session) {
const { teamId, plan } = session.metadata;
await db.teams.update({
where: { id: teamId },
data: {
plan,
stripeCustomerId: session.customer as string,
stripeSubscriptionId: session.subscription as string
}
});
// Send welcome email
await sendEmail({
to: session.customer_email,
subject: 'Welcome to CodeSlick Team!',
template: 'welcome-team'
});
}
Usage Tracking:
// src/lib/usage/tracker.ts
export async function trackPRAnalysis(teamId: string) {
const team = await db.teams.findUnique({ where: { id: teamId } });
if (team.plan === 'free') {
const count = await db.analysisRecords.count({
where: {
teamId,
analyzedAt: { gte: startOfMonth(new Date()) }
}
});
if (count >= PLANS.free.limits.prsPerMonth) {
throw new Error('Monthly PR limit reached. Upgrade to Team plan for unlimited analyses.');
}
}
// Record analysis
await db.analysisRecords.create({
data: { teamId, /* ... */ }
});
}
Acceptance Criteria: - ✅ Stripe Checkout flow works - ✅ Subscriptions sync with database - ✅ Usage limits enforced (free tier) - ✅ Upgrade/downgrade flow works - ✅ Invoice emails sent automatically
Estimated Time: 3 days Risk: Low (Stripe is well-documented)
Testing Strategy¶
Unit Tests¶
- ✅ GitHub webhook signature verification
- ✅ PR diff analysis
- ✅ Comment formatting (markdown)
- ✅ Usage limit enforcement
- ✅ Stripe webhook processing
Integration Tests¶
- ✅ End-to-end PR analysis flow
- ✅ Team member invitation flow
- ✅ Subscription upgrade/downgrade
- ✅ Multi-repository installation
Manual Testing Checklist¶
- Install GitHub App on test repository
- Create PR with vulnerable code
- Verify PR comment posted
- Verify inline comments on critical issues
- Verify status check updated
- Invite team member via dashboard
- Upgrade from free to team tier
- Verify usage limits enforced
Launch Strategy¶
Beta Launch (Week 5)¶
Goal: Get 10 beta teams using CodeSlick on real repositories
Outreach: 1. Personal network (founders, CTOs you know) 2. DevSecOps communities (Reddit, Discord, Slack) 3. GitHub Marketplace listing 4. ProductHunt launch (soft launch)
Pricing: - Free tier: Always free (20 PRs/month) - Team tier: 50% off first 3 months (€49.50 instead of €99) - Enterprise: Custom pricing (sales call required)
Success Metrics: - 10 beta teams signed up - 100+ PRs analyzed - 5+ paying teams (Team tier) - NPS score > 40
Public Launch (Week 6)¶
Marketing: - ProductHunt main launch - Twitter/X announcement thread - Dev.to article: "How we built automated PR security reviews" - LinkedIn post showcasing ROI data from beta teams
Launch Checklist: - [ ] GitHub App approved by GitHub - [ ] Stripe live mode enabled - [ ] Privacy policy & terms of service published - [ ] GDPR compliance verified - [ ] Load testing completed (100 concurrent PRs) - [ ] Error monitoring (Sentry) configured - [ ] Customer support email set up
Risk Mitigation¶
Risk 1: GitHub API Rate Limits¶
Impact: Analysis fails for high-volume teams Mitigation: - Cache file contents (24h TTL) - Use conditional requests (If-None-Match) - Implement exponential backoff - Monitor rate limit headers Probability: Medium (30%)
Risk 2: Slow Analysis Time¶
Impact: PRs wait > 1 minute for results Mitigation: - Analyze only changed files (not entire repo) - Parallel file analysis - Skip node_modules, build artifacts - Cache dependency scan results Probability: Low (15%)
Risk 3: Low Conversion Rate (Free → Paid)¶
Impact: No revenue despite usage Mitigation: - Show value early (ROI metrics in free tier) - Usage limit nudges ("You've analyzed 18/20 PRs this month") - Email drip campaign highlighting premium features - Offer discount code for early adopters Probability: High (50%)
Risk 4: GitHub App Security Vulnerability¶
Impact: Reputational damage, loss of trust Mitigation: - Strict webhook signature verification - Principle of least privilege (minimal GitHub permissions) - Regular security audits - Bug bounty program (after public launch) Probability: Low (10%)
Success Metrics¶
Phase 4 Success Criteria¶
Technical: - [ ] GitHub App approved and listed - [ ] Webhook → Analysis → Comment flow works - [ ] Analysis completes in < 60 seconds - [ ] 99% uptime (Vercel status page)
Business: - [ ] 10 beta teams onboarded - [ ] 5 paying teams (€495/month MRR) - [ ] Average 50 PRs analyzed per team per month - [ ] NPS score > 40 (beta users)
Product: - [ ] Team dashboard shows actionable insights - [ ] ROI calculator shows 10x+ return - [ ] Onboarding flow < 5 minutes - [ ] First PR analysis < 2 minutes from GitHub App install
Timeline¶
| Week | Focus | Deliverables |
|---|---|---|
| Week 1 | GitHub App Setup | App created, webhook handler, signature verification |
| Week 2 | PR Analysis | File analysis, comment posting, status checks |
| Week 3 | Team Dashboard | Multi-user accounts, analytics, ROI tracking |
| Week 4 | Billing | Stripe integration, usage limits, upgrade flow |
| Week 5 | Beta Launch | 10 beta teams, iterate based on feedback |
| Week 6 | Public Launch | ProductHunt, marketing push, scale monitoring |
Total Duration: 6 weeks Team Size: 1 developer (you) + 1 designer (optional, for dashboard UI) Budget: €500 (Stripe fees, GitHub App submission, marketing)
Next Steps¶
- Review & Approve this Phase 4 plan
- Set up GitHub App (Day 1 - can start immediately)
- Daily check-ins to track progress
- Ship beta by Week 5 (mid-December 2025)
Status: 📋 Plan Complete - Ready for Implementation Next Action: Create GitHub App and start webhook handler Strategic Outcome: First paying customers by end of 2025