Skip to main content

Norns iOS App

The Norns iOS App is a native iOS client for the Norns AI agent platform, providing chat, voice interaction, location services, and push notifications.

Overview​

  • Platform: iOS 15.0+
  • Architecture: SwiftUI + MVVM pattern
  • Bundle ID: dev.ravenhelm.norns
  • URL Scheme: dev.ravenhelm.norns://
  • API Gateway: All traffic routes through Bifrost API gateway
  • Authentication: Zitadel OIDC with PKCE flow
  • Project Location: /Users/nate/Projects/NornsApp/

Table of Contents​

Architecture​

Design Pattern​

The app follows the MVVM (Model-View-ViewModel) pattern with SwiftUI:

┌─────────────┐
│   SwiftUI   │  (Views)
│    Views    │
└──────┬──────┘
       │
┌──────▼──────┐
│  ViewModels │  (ChatViewModel, VoiceViewModel, etc.)
└──────┬──────┘
       │
┌──────▼──────┐
│  Services   │  (AuthService, APIClient, VoiceService, etc.)
└──────┬──────┘
       │
┌──────▼──────┐
│   Bifrost   │  (API Gateway)
│  iOS Channel│
└─────────────┘

Traffic Flow​

All iOS app traffic is routed through the Bifrost API gateway at https://bifrost-api.ravenhelm.dev. The gateway provides:

  • Request routing to appropriate backend services
  • Authentication verification
  • Rate limiting and security
  • iOS-specific endpoints under /api/ios/*

Key Components​

NornsApp/
├── Configuration/
│   └── AppConfiguration.swift       # Central config (URLs, OIDC, endpoints)
├── Core/
│   ├── Authentication/
│   │   ├── AuthService.swift        # OIDC authentication with PKCE
│   │   └── KeychainManager.swift    # Secure token storage
│   ├── Networking/
│   │   ├── APIClient.swift          # REST API client
│   │   └── SSEClient.swift          # Server-Sent Events for chat streaming
│   └── Services/
│       ├── VoiceService.swift       # LiveKit integration
│       ├── LocationService.swift    # CoreLocation + background updates
│       ├── NotificationService.swift # APNs management
│       └── AudioSessionManager.swift # Audio session handling
├── Features/
│   ├── Auth/                        # Login/logout views
│   ├── Chat/                        # Chat interface + streaming
│   ├── Voice/                       # Voice controls
│   └── Settings/                    # User preferences
└── Models/                          # Data structures

Authentication​

OIDC Configuration​

The app uses Zitadel for authentication via OpenID Connect (OIDC) with PKCE flow:

  • Issuer: https://auth.ravenhelm.dev
  • Client ID: 354141675762810915
  • Redirect URI: dev.ravenhelm.norns://oauth/callback
  • Scopes: openid, profile, email, offline_access
  • Flow: Authorization Code with PKCE (no client secret)

Implementation​

The AuthService singleton manages the complete auth lifecycle:

// Key methods:
authorize()              // Initiate OIDC login flow
getValidAccessToken()    // Returns valid token (refreshes if needed)
handleRedirectURL()      // Process OAuth callback
getUserInfo()            // Fetch user profile from OIDC userinfo endpoint
logout()                 // Clear auth state and keychain

Token Management​

  • Storage: Tokens stored securely in iOS Keychain via KeychainManager
  • Automatic Refresh: AuthService.getValidAccessToken() automatically refreshes expired tokens
  • API Integration: APIClient automatically adds Authorization: Bearer <token> headers
  • Token Retry: 401 responses trigger automatic token refresh and request retry

User Profile​

User information is retrieved from Zitadel's userinfo endpoint:

struct UserProfile: Codable {
    let id: String                  // Subject (sub)
    let name: String?
    let preferredUsername: String?
    let email: String?
    let emailVerified: Bool?
    let picture: String?
}

Core Services​

APIClient​

REST API client with automatic authentication and error handling.

Features:

  • Automatic JWT token injection
  • Automatic token refresh on 401
  • JSON encoding/decoding with snake_case conversion
  • Type-safe request methods (GET, POST, PUT, DELETE)
  • Streaming support for SSE

Example Usage:

// GET request
let tokenResponse: VoiceTokenResponse = try await APIClient.shared.get(
    AppConfiguration.Endpoints.voiceToken
)

// POST request
try await APIClient.shared.post(
    AppConfiguration.Endpoints.location,
    body: locationUpdate
)

SSEClient​

Server-Sent Events client for real-time chat streaming.

Event Types:

  • token(String) - Streaming text token from AI response
  • error(String) - Error message from server
  • interrupt(String) - Replace current response with new content
  • done - Stream completed
  • unknown(String) - Unknown event type (ignored)

Event Format:

data: {"event": "token", "token": "Hello"}
data: {"event": "token", "token": " world"}
data: [DONE]

Example Usage:

let stream = sseClient.stream(
    endpoint: AppConfiguration.Endpoints.chatStream,
    body: conversationRequest
)

for try await event in stream {
    switch event {
    case .token(let token):
        // Append token to message
    case .done:
        // Stream complete
    case .error(let error):
        // Handle error
    }
}

VoiceService​

LiveKit integration for real-time voice communication with the AI agent.

Connection States:

  • disconnected - Not connected
  • connecting - Establishing connection
  • connected - Active voice session
  • reconnecting - Attempting to reconnect
  • failed(String) - Connection failed with error

Key Methods:

connect()                // Fetch token from Bifrost, connect to LiveKit
disconnect()             // End voice session
toggleMute()             // Mute/unmute microphone
setSpeakerEnabled(Bool)  // Switch between speaker/earpiece

LiveKit Room Lifecycle:

  1. Request token from /api/ios/voice/token (via Bifrost)
  2. Connect to LiveKit server with token and URL
  3. Enable local microphone track
  4. Monitor remote participant (AI agent) for audio
  5. Track speaking state and audio levels

Audio Session:

  • Managed by AudioSessionManager
  • Configured for voice chat (.voiceChat mode)
  • Supports speaker/earpiece switching
  • Automatic deactivation on disconnect

LocationService​

CoreLocation integration with background updates and server synchronization.

Features:

  • Background location updates (requires "Always" authorization)
  • Significant location change monitoring
  • Minimum update distance: 100 meters
  • Automatic server sync via /api/ios/location

Authorization States:

  • notDetermined - User hasn't chosen
  • authorizedWhenInUse - Foreground only
  • authorizedAlways - Background + foreground
  • denied - User denied access
  • restricted - System restrictions

Background Configuration:

locationManager.allowsBackgroundLocationUpdates = true
locationManager.pausesLocationUpdatesAutomatically = true
locationManager.desiredAccuracy = kCLLocationAccuracyHundredMeters
locationManager.distanceFilter = 100 // meters

Location Update Payload:

struct LocationUpdate {
    let latitude: Double
    let longitude: Double
    let accuracy: Double
    let timestamp: Date
}

NotificationService​

Apple Push Notification Service (APNs) integration with actionable notifications.

Notification Categories:

  1. Task Reminders (taskReminders)

    • Actions: Complete, Snooze

  2. Calendar Alerts (calendarAlerts)

    • Actions: Join Meeting, Dismiss

  3. Security (security)

    • Actions: Approve, Deny (authentication required)

Device Registration Flow:

  1. Request notification authorization
  2. iOS provides device token via AppDelegate
  3. Register token with backend: POST /api/ios/devices
  4. Backend stores token for push notifications

Device Registration Payload:

struct DeviceRegistration {
    let deviceToken: String      // APNs token (hex)
    let deviceName: String        // e.g., "Nate's iPhone"
    let deviceModel: String       // e.g., "iPhone14,2"
    let osVersion: String         // e.g., "16.4"
    let appVersion: String        // e.g., "1.0.0"
    let environment: String       // "development" or "production"
}

Features​

Chat​

Real-time chat with the Norns AI agent using Server-Sent Events (SSE) for streaming responses.

Features:

  • Streaming AI responses (token-by-token)
  • Conversation history included in each request
  • Session management (UUID per chat session)
  • Error handling and retry logic
  • Message persistence (in-memory)

Request Payload:

struct ConversationRequest {
    let message: String                       // Current user message
    let sessionId: String                     // UUID for this chat session
    let userId: String                        // From OIDC user profile
    let conversationHistory: [HistoryItem]    // Previous messages (role + content)
}

Voice​

Real-time voice communication with the AI agent via LiveKit.

Features:

  • WebRTC-based audio streaming
  • Mute/unmute controls
  • Speaker/earpiece switching
  • Agent speaking detection
  • Automatic audio session management
  • Connection state monitoring

Voice Token Response:

struct VoiceTokenResponse {
    let token: String    // JWT token for LiveKit
    let url: String      // LiveKit server URL (wss://)
}

Settings​

User preferences and profile management.

Features:

  • User profile display (from OIDC)
  • Notification preferences
  • Account management (logout)
  • App version and build info

Notification Preferences:

struct NotificationPreferences {
    let taskReminders: Bool
    let calendarAlerts: Bool
    let smartHomeEvents: Bool
    let security: Bool
    let general: Bool
}

API Endpoints​

All endpoints are prefixed with the Bifrost base URL: https://bifrost-api.ravenhelm.dev

Chat​

POST /api/ios/chat/stream​

Stream chat response from Norns AI agent.

Authentication: Bearer token (JWT from OIDC)

Request Body:

{
  "message": "What's the weather today?",
  "session_id": "uuid-v4",
  "user_id": "zitadel-user-id",
  "conversation_history": [
    {"role": "user", "content": "Hello"},
    {"role": "assistant", "content": "Hi! How can I help?"}
  ]
}

Response: Server-Sent Events (text/event-stream)

data: {"event": "token", "token": "The"}
data: {"event": "token", "token": " weather"}
data: {"event": "token", "token": " is"}
data: [DONE]

Voice​

GET /api/ios/voice/token​

Request LiveKit access token for voice session.

Authentication: Bearer token (JWT from OIDC)

Response:

{
  "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
  "url": "wss://livekit.ravenhelm.dev"
}

Device Management​

POST /api/ios/devices​

Register device for push notifications.

Authentication: Bearer token (JWT from OIDC)

Request Body:

{
  "device_token": "hex-encoded-apns-token",
  "device_name": "Nate's iPhone",
  "device_model": "iPhone14,2",
  "os_version": "16.4",
  "app_version": "1.0.0",
  "environment": "production"
}

Response: 201 Created

DELETE /api/ios/devices/{token}​

Unregister device (e.g., on logout).

Authentication: Bearer token (JWT from OIDC)

Response: 204 No Content

Location​

POST /api/ios/location​

Send location update to backend.

Authentication: Bearer token (JWT from OIDC)

Request Body:

{
  "latitude": 37.7749,
  "longitude": -122.4194,
  "accuracy": 65.0,
  "timestamp": "2025-01-05T12:34:56Z"
}

Response: 200 OK

Notifications​

GET /api/ios/notifications/preferences​

Fetch user notification preferences.

Authentication: Bearer token (JWT from OIDC)

Response:

{
  "task_reminders": true,
  "calendar_alerts": true,
  "smart_home_events": false,
  "security": true,
  "general": true
}

PUT /api/ios/notifications/preferences​

Update notification preferences.

Authentication: Bearer token (JWT from OIDC)

Request Body:

{
  "task_reminders": false,
  "calendar_alerts": true,
  "smart_home_events": true,
  "security": true,
  "general": false
}

Response: 200 OK

Configuration​

App Configuration​

enum AppConfiguration {
    // OIDC
    static let oidcIssuer = "https://auth.ravenhelm.dev"
    static let clientID = "354141675762810915"
    static let redirectURI = "dev.ravenhelm.norns://oauth/callback"
    static let scopes = ["openid", "profile", "email", "offline_access"]

    // API
    static let apiBaseURL = "https://bifrost-api.ravenhelm.dev"
}

Required Info.plist Keys​

<!-- OIDC Redirect Scheme -->
<key>CFBundleURLTypes</key>
<array>
    <dict>
        <key>CFBundleURLSchemes</key>
        <array>
            <string>dev.ravenhelm.norns</string>
        </array>
    </dict>
</array>

<!-- Privacy Descriptions -->
<key>NSMicrophoneUsageDescription</key>
<string>Norns needs microphone access for voice conversations</string>

<key>NSLocationAlwaysAndWhenInUseUsageDescription</key>
<string>Norns uses your location to provide location-aware assistance</string>

<key>NSLocationWhenInUseUsageDescription</key>
<string>Norns uses your location to provide location-aware assistance</string>

<!-- Background Modes -->
<key>UIBackgroundModes</key>
<array>
    <string>audio</string>
    <string>location</string>
    <string>remote-notification</string>
</array>

Required Capabilities​

  • Push Notifications: For APNs
  • Background Modes: Audio, Location updates, Remote notifications
  • Sign in with Apple: Optional (not currently used)

Zitadel OIDC Configuration​

Client Configuration (on Zitadel):

  • Client ID: 354141675762810915
  • Application Type: User Agent (PKCE)
  • Grant Types: Authorization Code, Refresh Token
  • Redirect URIs: dev.ravenhelm.norns://oauth/callback
  • Post Logout Redirect URIs: dev.ravenhelm.norns://
  • Token Endpoint Auth Method: None (PKCE)

Dependencies​

Swift Package Manager​

The app uses the following dependencies via SPM:

  1. AppAuth-iOS (latest)

  2. LiveKit Swift SDK (latest)

Adding Dependencies​

In Xcode:

  1. File → Add Packages...
  2. Enter package URL
  3. Select version/branch
  4. Add to NornsApp target

Development​

Project Structure​

/Users/nate/Projects/NornsApp/
├── NornsApp/                    # Main app target
│   ├── NornsApp.swift          # App entry point
│   ├── Configuration/           # App config
│   ├── Core/                    # Core services
│   ├── Features/                # Feature modules
│   └── Resources/               # Assets, Info.plist
├── NornsApp.xcodeproj          # Xcode project
└── Package.resolved            # SPM lock file

Building the App​

Prerequisites:

  • Xcode 14.0+
  • iOS 15.0+ device or simulator
  • Valid Apple Developer account (for device testing)

Build Steps:

  1. Open /Users/nate/Projects/NornsApp/NornsApp.xcodeproj in Xcode
  2. Select target device/simulator
  3. Product → Build (⌘B)
  4. Product → Run (⌘R)

Testing Authentication​

To test OIDC flow:

  1. Run app on simulator
  2. Tap "Sign In"
  3. Safari opens with Zitadel login
  4. Sign in with test account
  5. Redirected back to app via dev.ravenhelm.norns:// scheme

Testing Voice​

Requirements:

  • Physical device (simulator doesn't support microphone well)
  • LiveKit server running and accessible
  • Bifrost configured to return valid LiveKit tokens

Testing Notifications​

Requirements:

  • Physical device (push notifications don't work on simulator)
  • Valid APNs certificate/key configured
  • Device registered with backend

Debugging​

View Logs:

# iOS device logs
xcrun simctl spawn booted log stream --predicate 'processImagePath contains "NornsApp"'

# Or use Console.app on macOS

Network Debugging:

  • Use Charles Proxy or Proxyman to inspect API traffic
  • HTTPS traffic requires SSL proxy configuration

Common Issues:

  • 401 Unauthorized: Token expired, check refresh logic
  • OIDC redirect fails: Verify URL scheme in Info.plist
  • LiveKit connection fails: Check token validity and server URL
  • Location not updating: Check authorization status and Info.plist permissions

Deployment​

App Store Prerequisites​

  1. Bundle ID: dev.ravenhelm.norns (registered in Apple Developer Portal)
  2. Provisioning Profile: App Store distribution profile
  3. Certificates: Distribution certificate
  4. Capabilities: Push Notifications enabled
  5. App Icon: 1024x1024px (required)

Build for Release​

  1. Set scheme to "Release"
  2. Select "Any iOS Device (arm64)"
  3. Product → Archive
  4. Organizer → Distribute App
  5. Choose distribution method (App Store, Ad Hoc, Enterprise)
  6. Sign with distribution certificate
  7. Upload to App Store Connect

TestFlight​

For beta testing:

  1. Upload build to App Store Connect
  2. Add to TestFlight
  3. Add internal/external testers
  4. Distribute via email

Version Numbering​

  • Version: Semantic versioning (e.g., 1.0.0)
  • Build: Incrementing number (e.g., 1, 2, 3...)

Update in Xcode:

  • Target → General → Identity → Version
  • Target → General → Identity → Build

Environment Configuration​

For different environments (dev, staging, prod):

Option 1: Build configurations

  • Duplicate "Release" config
  • Set preprocessor macros: ENVIRONMENT=STAGING
  • Use in code: #if ENVIRONMENT == STAGING

Option 2: Multiple schemes

  • Create scheme per environment
  • Different Info.plist per scheme
  • Different bundle IDs (e.g., dev.ravenhelm.norns.dev)

APNs Certificate​

Production:

  1. Create APNs certificate in Apple Developer Portal
  2. Download .p12 file
  3. Upload to backend push notification service
  4. Configure Bifrost to use production APNs endpoint

Development:

  • Use sandbox APNs endpoint
  • Different certificate than production

Troubleshooting​

Authentication Issues​

Problem: Login redirects to app but fails

Solution:

  • Check URL scheme matches in Info.plist and Zitadel config
  • Verify handleRedirectURL() is called in SceneDelegate
  • Check Zitadel client config allows PKCE
  • Ensure redirect URI exactly matches (including trailing slash)

Voice Connection Fails​

Problem: Cannot connect to LiveKit

Solution:

  • Verify LiveKit server is accessible
  • Check token validity (decode JWT)
  • Ensure microphone permission granted
  • Check network connectivity (try different network)
  • Review LiveKit server logs

Push Notifications Not Received​

Problem: Device registered but no notifications

Solution:

  • Verify APNs certificate is valid
  • Check device token format (hex string)
  • Ensure backend is sending to correct APNs endpoint (sandbox vs production)
  • Check notification payload format
  • Review device logs for APNs errors

Location Updates Not Working​

Problem: Location not updating in background

Solution:

  • Verify "Always" authorization granted
  • Check allowsBackgroundLocationUpdates = true
  • Ensure Background Modes → Location updates enabled
  • Check Info.plist has required privacy descriptions
  • Review battery settings (Low Power Mode disables background location)

SSE Stream Disconnects​

Problem: Chat stream cuts off mid-response

Solution:

  • Increase timeout in URLSessionConfiguration
  • Check network stability
  • Verify Bifrost isn't timing out the connection
  • Review backend logs for errors
  • Test with cellular data vs WiFi

Security Considerations​

Token Storage​

  • Access tokens stored in iOS Keychain (encrypted)
  • Keychain items protected by device passcode
  • Tokens deleted on logout
  • No tokens stored in UserDefaults or files

API Communication​

  • All API traffic over HTTPS (TLS 1.2+)
  • Certificate pinning recommended for production
  • Bearer token in Authorization header
  • Automatic token refresh (no long-lived tokens)

Permissions​

  • Request minimum permissions needed
  • Location: Only when feature is enabled
  • Microphone: Only during voice sessions
  • Notifications: Only when user opts in

Data Privacy​

  • No user data logged to console in production builds
  • Location data sent only when user authorizes
  • Chat history stored in-memory only (not persisted)
  • No analytics or tracking by default

Future Enhancements​

Planned Features​

  1. Offline Support

    • Cache recent conversations
    • Queue location updates when offline
    • Sync when connection restored

  2. Rich Message Types

    • Images and attachments
    • Interactive cards
    • Action buttons

  3. Voice Improvements

    • Push-to-talk mode
    • Voice activity detection
    • Audio quality settings

  4. Widgets

    • iOS 14+ home screen widgets
    • Quick actions
    • Glanceable information

  5. Watch App

    • Voice-only interface
    • Notifications on wrist
    • Quick commands

  6. Shortcuts Integration

    • Siri shortcuts
    • Automation triggers
    • Custom intents