Skip to main content

Factory Safety & SOP Assistant - Frontend Specification

Date: December 8, 2025
Owner: Gary Sheng (Frontend)
Status: Draft for Team Discussion

Overview​

A policy-aware video analysis dashboard that helps factory supervisors identify, review, and act on safety/SOP violations detected from egocentric worker footage.

Core User: Factory supervisor or safety engineer who needs to:

  1. Upload and analyze worker footage
  2. Review flagged violations efficiently
  3. Track follow-up conversations with workers
  4. Spot patterns across workers/shifts/locations

Page Architecture​

1. Onboarding Flow​

Purpose: Get new users to immediate value while capturing context for personalization.

Flow:

Welcome → Factory Setup → First Upload → First Result → Dashboard

Screen 1: Welcome

  • Brief value prop (2-3 sentences max)
  • "Get started" CTA

Screen 2: Factory Context (Chat-style interface)

  • Conversational prompts:
    • "What's your factory name?"
    • "What type of manufacturing? (dropdown or free text)"
    • "How many workers on your floor?"
    • "What are your top 3 safety concerns right now?"
  • This feels lighter than a form. User types naturally.

Screen 2b: Policy Upload (Critical step)

Primary CTA: Upload your safety policies

  • Drag-drop zone for PDF, DOCX, or TXT
  • Or paste text directly
  • Strong recommendation messaging: "For best results, upload your existing safety manual or SOP document. The system works best when it knows YOUR specific rules."

Policy Writing Guidance Panel:

┌─────────────────────────────────────────────────────┐
│  💡 What makes a good policy for this system?      │
│                                                     │
│  The AI looks for specific, observable actions.    │
│  Write rules that describe what CAN or CANNOT      │
│  be seen in video footage.                         │
│                                                     │
│  ✓ GOOD: "Workers must wear cut-resistant gloves   │
│    when within 3 feet of cutting equipment"        │
│                                                     │
│  ✓ GOOD: "No reaching into conveyor while belt    │
│    is moving"                                      │
│                                                     │
│  ✓ GOOD: "Hard hats required in zones marked      │
│    with yellow floor tape"                         │
│                                                     │
│  ✗ LESS USEFUL: "Workers should prioritize safety" │
│    (too abstract - can't be seen in video)         │
│                                                     │
│  ✗ LESS USEFUL: "Complete safety training annually"│
│    (compliance action, not observable behavior)    │
└─────────────────────────────────────────────────────┘

If user skips or provides no policies → Default Policy Suggestions:

┌─────────────────────────────────────────────────────┐
│  No policies uploaded. Here are common factory     │
│  safety rules we can start with:                   │
│                                                     │
│  ☑️ PPE: Gloves required near cutting/grinding     │
│  ☑️ PPE: Safety glasses in all production areas   │
│  ☑️ PPE: Hard hat in overhead crane zones         │
│  ☑️ Equipment: No reaching into moving machinery  │
│  ☑️ Equipment: Machine guards must remain in place│
│  ☑️ Proximity: Maintain 3ft from forklift paths   │
│  ☐ Chemical: Gloves + goggles when handling       │
│  ☐ Electrical: Lockout/tagout before maintenance  │
│                                                     │
│  [Select All]  [Continue with Selected]            │
│                                                     │
│  ⚠️ These are generic defaults. For best accuracy,│
│  upload your actual facility policies.             │
└─────────────────────────────────────────────────────┘

Discussion Points:

  • Should we parse uploaded PDFs to extract rules automatically, or require structured input?
  • Can the AI suggest additional rules based on factory type selected earlier?
  • Do we allow editing default rules, or just on/off toggles?

Screen 2c: Employee Setup (Required before first upload)

┌─────────────────────────────────────────────────────┐
│  Add Your Workers                                   │
│                                                     │
│  Before uploading footage, register the workers    │
│  who might appear in videos.                       │
│                                                     │
│  ┌─────────────────────────────────────────────┐   │
│  │ Name: [_________________________]           │   │
│  │ Employee ID (optional): [___________]       │   │
│  │ Default Shift: [Day ▼]                      │   │
│  │ Station/Area: [Cutting ▼]                   │   │
│  │                          [+ Add Worker]     │   │
│  └─────────────────────────────────────────────┘   │
│                                                     │
│  Added:                                            │
│  • John D. — Day Shift — Cutting        [Remove]   │
│  • Sarah M. — Day Shift — Assembly      [Remove]   │
│  • Mike R. — Night Shift — Packaging    [Remove]   │
│                                                     │
│  [Add at least 1 worker to continue]               │
│                                                     │
│  [Continue →]                                      │
└─────────────────────────────────────────────────────┘
  • Minimum 1 worker required to proceed
  • Can always add more workers later from Settings or Employee page
  • Shift options: Day, Swing, Night (or custom)
  • Station/Area dropdown populated from factory type or free text

Screen 3: Upload Your First Clip

  • Drag-and-drop zone
  • Prompt: "Upload a short clip (under 5 mins recommended) to see the system in action"
  • Skip option if they want to explore first

Screen 4: First Result

  • Show analysis results inline
  • If violations found: "We found X potential issues. Let's review them."
  • If clean: "No violations detected. Here's what we checked for."
  • CTA: "Go to Dashboard"

Discussion Point: How minimal can onboarding be? Could we skip factory setup entirely for the hackathon demo?

2. Video Upload & Analysis Page​

URL: /upload or /analyze

Layout:

┌─────────────────────────────────────────────────────┐
│  [Upload Zone - drag/drop or click to browse]       │
│                                                     │
│  Supported: MP4, MOV • Max 60 min • Max X GB        │
└─────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────┐
│  Video Details (Required)                           │
│                                                     │
│  Worker *                [John D. ▼]  [+ New]      │
│  Shift *                 [Day ▼]                   │
│  Date recorded           [Dec 7, 2025]  (auto)     │
│  Station/Area            [Cutting Station ▼]       │
│  Notes (optional)        [_____________________]   │
│                                                     │
│  * Required fields                                 │
│                                     [Start Upload] │
└─────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────┐
│  Upload History / Active Analyses                   │
│                                                     │
│  ┌───────────────────────────────────────────────┐  │
│  │ video_001.mp4         [████████░░] 78%        │  │
│  │ Analyzing frame 2,340 of 3,000                │  │
│  │ Found: 2 potential violations so far          │  │
│  └───────────────────────────────────────────────┘  │
│                                                     │
│  ┌───────────────────────────────────────────────┐  │
│  │ video_002.mp4         ✓ Complete              │  │
│  │ 5 violations found • 3 high severity          │  │
│  │ [View Results]  [Replay Full Video]           │  │
│  └───────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────┘

Required Metadata:

  • Worker: Dropdown of registered employees. [+ New] button opens quick-add modal. This is the person wearing the camera (egocentric footage = first-person POV from the worker).
  • Shift: Day / Swing / Night (defaults to worker's usual shift)
  • Date recorded: Auto-populates from file metadata if available, editable.
  • Station/Area: Optional but helpful for pattern detection.

Upload button stays disabled until required fields are filled.

Features:

Upload Handling

  • Large file upload with chunked streaming (for 60-min videos)
  • Progress indicator showing upload % then analysis %
  • Graceful handling if user closes tab (resume capability?)

Live Analysis Progress

  • Antivirus-scan style: "Checking frame 1,234..."
  • Running count of violations found during analysis
  • Optional: Live preview of detected objects/hands (if backend can stream this)

Completed Analysis Row

  • Summary: violation count, severity breakdown
  • Quick actions: View Results, Replay Video, Download Report

Expanded Results View (in-page accordion or modal)

  • Timeline visualization showing where violations occurred
  • Click timestamp to jump to that moment
  • Tab 1: Full video with violation markers on scrubber
  • Tab 2: Clips view (Tesla-style grid of violation moments)

Discussion Points:

  • Do we stream the video file to backend, or upload to a bucket first? (Impacts progress UX)
  • Can backend emit incremental results via WebSocket/SSE, or do we poll?
  • Do we need worker metadata on upload (who filmed this, which shift)?

3. Infraction Review Page (Inbox Zero)​

URL: /review

Purpose: Supervisor's primary workspace. Goal is to clear the queue.

Layout:

┌─────────────────────────────────────────────────────┐
│  Inbox (47 unreviewed)                    [Filters] │
│                                                     │
│  Filter: [All Rules ▼] [All Workers ▼] [All Sev ▼] │
│  Sort: [Newest First ▼]                             │
└─────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────┐
│ ⚠️ HIGH  No gloves near cutting station            │
│ Worker: John D. • Dec 7, 2025 • 2:34 PM            │
│                                                     │
│ [▶ Video Clip]                    [3s clip loads]  │
│                                                     │
│ Rule: "Must wear gloves within 3ft of cutting tools"│
│ Why flagged: "Worker's hands detected near blade   │
│ without gloves visible in frame"                   │
│                                                     │
│ [✓ Valid Violation]  [✗ False Positive]  [Skip]    │
└─────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────┐
│ ⚠️ MED  Reached toward moving conveyor             │
│ Worker: Sarah M. • Dec 7, 2025 • 11:15 AM          │
│ ...                                                 │
└─────────────────────────────────────────────────────┘

Features:

Inbox List

  • Card per violation with severity badge
  • Worker name, timestamp, rule triggered
  • Inline video clip preview (auto-play on hover or expand)
  • AI-generated explanation of why it was flagged

Actions Per Item

  • Valid Violation: Confirms the flag. Creates follow-up task.
  • False Positive: Dismisses flag. Feeds back to improve model.
  • Skip/Later: Moves to end of queue

Filters

  • By rule category (PPE, equipment, proximity, etc.)
  • By worker
  • By severity (High / Medium / Low)
  • By date range
  • By factory/line (if multi-location)
  • By video source

Bulk Actions

  • Select multiple, mark all as reviewed
  • Export selected to PDF report

Discussion Points:

  • What data does backend need when marking false positive? Just the clip ID, or more context?
  • Should severity auto-calculate, or can supervisor adjust it?
  • Do we need a "needs more context" option for ambiguous clips?

4. Task / Follow-Up Page​

URL: /tasks

Purpose: Ensure supervisors actually have conversations with workers.

Layout:

┌─────────────────────────────────────────────────────┐
│  Follow-Up Tasks (12 pending)                       │
│                                                     │
│  ┌───────────────────────────────────────────────┐  │
│  │ 📋 Talk to John D. about glove compliance     │  │
│  │    3 violations this week • Last: Dec 7       │  │
│  │    [Mark Complete] [View Violations] [Snooze] │  │
│  └───────────────────────────────────────────────┘  │
│                                                     │
│  ┌───────────────────────────────────────────────┐  │
│  │ 📋 Talk to Sarah M. about conveyor safety     │  │
│  │    1 violation • Dec 7                        │  │
│  │    [Mark Complete] [View Violations] [Snooze] │  │
│  └───────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────┘

Features:

  • Auto-generated from confirmed violations
  • Groups multiple violations per worker into single task
  • "Mark Complete" with optional notes field
  • Snooze with date picker
  • Links to relevant violation clips for reference during conversation

Discussion Point: Should this integrate with calendar/Slack for reminders? (Probably not for hackathon, but note as future feature)

5. Employee Directory / Analytics​

URL: /employees and /employees/[id]

List View:

┌─────────────────────────────────────────────────────┐
│  Employees                  [Search]   [+ Add New]  │
│                                                     │
│  Name          ID      Shift    Violations (30d)   │
│  ─────────────────────────────────────────────────  │
│  John D.       E-001   Day      7           ↑      │
│  Sarah M.      E-002   Day      2           ↓      │
│  Mike R.       E-003   Night    0           —      │
│  ...                                                │
│                                                     │
│  [Edit] [Archive] buttons on hover per row          │
└─────────────────────────────────────────────────────┘

Add/Edit Employee Modal:

┌─────────────────────────────────────────────────────┐
│  Add New Employee                           [×]     │
│                                                     │
│  Full Name *           [_____________________]      │
│  Employee ID           [_____________________]      │
│  Default Shift *       [Day ▼]                     │
│  Primary Station/Area  [Cutting ▼]                 │
│  Start Date            [Dec 1, 2025]               │
│  Email (optional)      [_____________________]      │
│  Phone (optional)      [_____________________]      │
│                                                     │
│  * Required                                        │
│                        [Cancel]  [Save Employee]   │
└─────────────────────────────────────────────────────┘

Employee Entity Fields:

  • Name (required)
  • Employee ID (optional, for internal reference)
  • Default Shift (required)
  • Primary Station/Area
  • Start Date
  • Contact info (email, phone) - optional, for follow-up tasks
  • Status: Active / Archived (don't delete, just hide from dropdowns)

Employee Detail View:

┌─────────────────────────────────────────────────────┐
│  John D.                                            │
│  Day Shift • Cutting Station • Hired: Jan 2024     │
│                                                     │
│  ┌─────────────────────────────────────────────┐   │
│  │ Violations Over Time (30/60/90d toggle)     │   │
│  │ [Chart: bar or line graph]                  │   │
│  └─────────────────────────────────────────────┘   │
│                                                     │
│  Violation Breakdown by Type:                      │
│  • PPE (gloves): 5                                 │
│  • Equipment proximity: 2                          │
│                                                     │
│  Recent Violations:                                │
│  • Dec 7 - No gloves near cutting station [View]   │
│  • Dec 5 - No gloves near cutting station [View]   │
│  • ...                                             │
│                                                     │
│  Completed Follow-Ups:                             │
│  • Dec 6 - Discussed glove policy (Supervisor: AB) │
└─────────────────────────────────────────────────────┘

BONUS: Smart Analytics Over Time (Rostam suggestion):

Pattern Detection Panel:
┌─────────────────────────────────────────────────────┐
│  🔍 Detected Patterns                              │
│                                                     │
│  • John D. typically has glove violations between  │
│    2-3 PM (post-lunch)                             │
│                                                     │
│  • Night shift has 2.3x more conveyor proximity    │
│    violations than day shift                       │
│                                                     │
│  • Station 4 (cutting) accounts for 60% of all    │
│    PPE violations                                  │
└─────────────────────────────────────────────────────┘

Discussion Points:

  • How much analytics is realistic for hackathon scope?
  • Pattern detection requires significant data. Mock it for demo?
  • Privacy considerations: should employees be identifiable, or anonymized to worker IDs?

Global Components​

  • Sidebar or top nav with: Upload, Review (with badge count), Tasks (with badge count), Employees, Settings
  • User profile / logout in corner

Notification System​

  • Toast notifications for: "Analysis complete", "New violations detected"
  • Badge counts on nav items

Settings Page​

  • Rule management (view/edit safety rules)
  • Factory configuration
  • User management (if multi-user)
  • Export/import data

Technical Considerations​

State Management​

  • Upload progress: local state + backend polling or WebSocket
  • Review queue: server state, refetch on action
  • Tasks: server state

Real-Time Updates​

  • Option A: Polling (simpler, fine for hackathon)
  • Option B: WebSocket/SSE for live analysis progress

Video Handling​

  • Use native HTML5 video player or library like Video.js
  • Need to handle scrubbing to specific timestamps
  • Clip extraction: does frontend request clips, or does backend pre-generate them?

File Upload​

  • Chunked upload for large files (tus.io protocol or similar)
  • Resume capability for dropped connections
  • Client-side validation before upload (file type, size)

Hackathon Scope Recommendations​

Must Have (MVP)​

  1. Onboarding flow (factory context, policy upload/guidance, employee setup)
  2. Basic upload page with progress indicator + required metadata (worker, shift)
  3. Results view showing violations with timestamps
  4. Simple video playback with timestamp jumping
  5. Violation cards with Valid/False Positive actions
  6. Basic employee list

Nice to Have​

  1. Live analysis progress streaming
  2. Follow-up task management page
  3. Analytics charts
  4. Pattern detection

Cut for Hackathon​

  1. Multi-user auth
  2. Calendar integrations
  3. Advanced filtering
  4. Export/reporting

Open Questions for Team​

  1. Backend API Shape: What endpoints exist? What's the response format for violations?
  2. Video Storage: Where do uploaded videos live? Can frontend request direct URLs?
  3. Clip Generation: Does backend return clip URLs, or timestamps for frontend to clip?
  4. Real-Time: Can backend stream analysis progress, or do we poll?
  5. Worker Identity: Are workers identified in the video metadata, or detected/matched by the system?
  6. Rule Format: What does the "rulebook" look like that we display to users?

Next Steps​

  1. Team reviews this doc
  2. Align on MVP scope
  3. Gary scaffolds frontend with mock data
  4. Backend team provides API contract
  5. Integration

This is a living document. Update as decisions are made.