Give Your Coding Agent Eyes: Screenshot-Driven SwiftUI Iteration

July 1, 2026 |

Coding agents are much better at SwiftUI when they can see the thing they are changing.

That sounds obvious, but a lot of agent-assisted UI work still happens blind. You ask for a nicer card, a more playful animation, or a denser toolbar. The agent edits the Swift. The app builds. Then you open the screen and discover the layout is clipped, the glow is too much, the locked state looks broken, or the long name case was never considered.

The fix is not to make the agent more autonomous. The fix is to make the feedback loop more concrete.

For UI work, I want a screenshot-driven loop:

Give the target UI a deterministic preview or debug harness.
Seed every important state with fake data.
Launch directly to the screen with a debug route or deep link.
Capture screenshots from the simulator.
Inspect those screenshots and make one targeted adjustment.
Keep the useful harness, script, or snapshot coverage after the visual work is done.

That is the narrow follow-up to the broader Xcode 27 agent workflow. WWDC26 shows Xcode agents rendering previews, producing artifacts, reacting to build and test failures, using images and annotations, and helping with UI prototypes. Apple’s prototyping guidance also points at named previews, realistic sample content, edge cases, and fast visual iteration. I add temporary tuning controls when subjective constants need quick exploration.

Outside Xcode’s native agent UI, be more explicit. A command-line agent may be able to build, launch a simulator, and capture a screenshot when you grant the tools, but it does not automatically have Xcode’s preview canvas, inline annotations, Device Hub UI, or artifact sidebar. Treat screenshots as deliberate inputs. For a physical device, private data, or a screen that depends on real account state, have the developer capture the screenshot and attach it. For repeatable simulator work, use a local script.

Start with a small visual target

Do not begin with “make the whole app feel more premium.” That is where agents go wandering.

Pick one surface. In a word game, that might be a flair card shown after a player earns a streak reward. The UI is visual enough to benefit from agent iteration, but bounded enough to inspect in a screenshot.

Here is a deliberately small model:

import Foundation

struct FlairCardModel: Equatable, Identifiable {
    enum State: String, CaseIterable, Identifiable {
        case earned
        case locked
        case empty

        var id: Self { self }
    }

    let id: UUID
    var state: State
    var title: String
    var subtitle: String
    var streakCount: Int
    var progress: Double
}

The important choice is that FlairCardModel can represent the states you actually need to inspect. If the only preview data is the happy path, the agent will optimize for the happy path and quietly break everything else.

Make preview fixtures boring and reusable

Create fake, deterministic data that can be used by previews, debug routes, screenshots, and tests. Put it somewhere that cannot accidentally read production data.

import Foundation

#if DEBUG
enum FlairPreviewFixtures {
    static let earned = FlairCardModel(
        id: UUID(uuidString: "8ACD0D31-C53A-4FB6-9E9D-59A821BEE001")!,
        state: .earned,
        title: "Seven Day Streak",
        subtitle: "You kept your practice rhythm alive.",
        streakCount: 7,
        progress: 1
    )

    static let locked = FlairCardModel(
        id: UUID(uuidString: "8ACD0D31-C53A-4FB6-9E9D-59A821BEE002")!,
        state: .locked,
        title: "Perfect Week",
        subtitle: "Finish every daily puzzle this week to unlock it.",
        streakCount: 3,
        progress: 0.42
    )

    static let empty = FlairCardModel(
        id: UUID(uuidString: "8ACD0D31-C53A-4FB6-9E9D-59A821BEE003")!,
        state: .empty,
        title: "No Flair Yet",
        subtitle: "Start a daily puzzle to earn your first reward.",
        streakCount: 0,
        progress: 0
    )

    static let longText = FlairCardModel(
        id: UUID(uuidString: "8ACD0D31-C53A-4FB6-9E9D-59A821BEE004")!,
        state: .earned,
        title: "Extraordinarily Persistent Vocabulary Builder",
        subtitle: "A deliberately long subtitle that should wrap without pushing the action area off screen.",
        streakCount: 128,
        progress: 1
    )

    static func model(for state: FlairCardModel.State) -> FlairCardModel {
        switch state {
        case .earned:
            earned
        case .locked:
            locked
        case .empty:
            empty
        }
    }
}
#endif

There are two details worth copying.

First, the UUIDs are stable. That keeps screenshots, snapshots, and preview behavior from changing because the data changed.

Second, the fixtures describe product states, not random examples. earned, locked, empty, and longText are names a developer can ask an agent to inspect.

Give every meaningful state a named preview

Xcode previews are not just for looking at a nice default screen. For agent work, previews are state labels.

import SwiftUI

#if DEBUG
#Preview("Flair - earned") {
    FlairCardView(model: FlairPreviewFixtures.earned)
        .padding()
}

#Preview("Flair - locked") {
    FlairCardView(model: FlairPreviewFixtures.locked)
        .padding()
}

#Preview("Flair - empty") {
    FlairCardView(model: FlairPreviewFixtures.empty)
        .padding()
}

#Preview("Flair - long text, dark, accessibility") {
    FlairCardView(model: FlairPreviewFixtures.longText)
        .padding()
        .frame(width: 340)
        .environment(\.dynamicTypeSize, .accessibility3)
        .preferredColorScheme(.dark)
}
#endif

Those names become part of the prompt vocabulary:

Use the Flair previews as the source of truth. Fix the locked and long-text
states without changing the earned state. Keep the card readable at
accessibility3 Dynamic Type and in dark mode.

This is much better than “make the card better.” The agent gets a target, a non-goal, and a validation matrix.

For a serious visual component, I would usually include:

Happy path.
Locked or disabled state.
Empty state.
Long text.
Compact width.
Large Dynamic Type.
Dark mode.
One localization or pseudo-localized string case if text length is risky.

Previews do not prove the feature works in the app. They make the visual state cheap to render and cheap to discuss.

Add a debug route for simulator screenshots

Previews are fast, but screenshots from a running app catch a different class of problems: navigation bars, safe areas, tab bars, device scale, launch state, and presentation context.

For that, add a debug-only route that lands directly on the target surface.

import SwiftUI

#if DEBUG
enum DebugRoute: Hashable {
    case flair(FlairCardModel.State)
}

@MainActor
final class DebugRouteStore: ObservableObject {
    @Published var route: DebugRoute?

    func open(_ url: URL) {
        guard url.scheme == "wordie",
              url.host == "debug",
              url.path == "/flair"
        else {
            return
        }

        let components = URLComponents(url: url, resolvingAgainstBaseURL: false)
        let stateValue = components?
            .queryItems?
            .first { $0.name == "state" }?
            .value

        guard let stateValue,
              let state = FlairCardModel.State(rawValue: stateValue)
        else {
            route = .flair(.earned)
            return
        }

        route = .flair(state)
    }
}

struct DebugRouteHost<Content: View>: View {
    @StateObject private var debugRoutes = DebugRouteStore()

    let content: Content

    var body: some View {
        ZStack {
            content

            if let route = debugRoutes.route {
                debugView(for: route)
            }
        }
        .onOpenURL { url in
            debugRoutes.open(url)
        }
    }

    @ViewBuilder
    private func debugView(for route: DebugRoute) -> some View {
        switch route {
        case .flair(let state):
            NavigationStack {
                FlairCardView(model: FlairPreviewFixtures.model(for: state))
                    .padding()
                    .navigationTitle("Flair Debug")
            }
        }
    }
}
#endif

The production app should not expose this route. Keep it behind #if DEBUG, a debug build setting, or an internal scheme. The route is not a public deep link; it is a repeatable way to put the simulator in the state you want the agent and the developer to inspect.

Also register the debug URL scheme in the app target’s Info.plist for the Debug or internal build. Without a CFBundleURLTypes entry for wordie, simctl openurl will not deliver wordie://debug/flair?... to the app. If you do not want a custom URL scheme, use launch arguments or an internal debug menu instead.

Depending on your app architecture, you might not wrap the root view this way. You might route into an existing navigation model, inject a fake store, or show a hidden debug screen. The rule is simpler than the implementation: get to the visual target without tapping through onboarding, auth, networking, or randomness.

Capture the same screenshot every time

Once a debug route exists, a simulator screenshot loop can be small.

This example assumes the simulator is already booted. It defaults to booted for convenience, but set DEVICE to a simulator UDID when repeatability matters. booted is ambiguous if more than one simulator is running.

#!/usr/bin/env bash
set -euo pipefail

PROJECT="iOS-App/Wordie.xcodeproj"
SCHEME="Wordie (iOS)"
CONFIGURATION="Debug"
DESTINATION="generic/platform=iOS Simulator"
DERIVED_DATA="${TMPDIR:-/tmp}/wordie-visual-loop-derived-data"
DEVICE="${DEVICE:-booted}"
APP_BUNDLE_ID="com.example.wordie"
OUTPUT_DIR="${TMPDIR:-/tmp}/wordie-visual-loop"
SCREENSHOT_DELAY="${SCREENSHOT_DELAY:-1}"

xcodebuild \
  -project "$PROJECT" \
  -scheme "$SCHEME" \
  -configuration "$CONFIGURATION" \
  -destination "$DESTINATION" \
  -derivedDataPath "$DERIVED_DATA" \
  build

APP_PATH="$DERIVED_DATA/Build/Products/$CONFIGURATION-iphonesimulator/Wordie.app"

xcrun simctl bootstatus "$DEVICE" -b
xcrun simctl install "$DEVICE" "$APP_PATH"
xcrun simctl launch --terminate-running-process "$DEVICE" "$APP_BUNDLE_ID"
xcrun simctl openurl "$DEVICE" "wordie://debug/flair?state=locked"

mkdir -p "$OUTPUT_DIR"
sleep "$SCREENSHOT_DELAY"
xcrun simctl io "$DEVICE" screenshot "$OUTPUT_DIR/flair-locked.png"

In a real repo, the script needs your actual project path, scheme, bundle identifier, app name, and route. I would keep those near the top, not buried inside the commands.

The important part is the loop:

Build.
Install.
Launch.
Open the exact debug route.
Capture the exact screenshot.

Now the agent can make a layout change, run the script, inspect the screenshot if it has a local image-viewing tool, and make a smaller follow-up change. If the agent cannot inspect local images, the developer can attach the screenshot back into the conversation.

For physical devices, real user data, or private business state, I prefer a human-captured screenshot. The goal is not to give an agent unlimited visual access. The goal is to make the visual evidence explicit, safe, and repeatable.

Use a temporary tuning surface for feel

Some UI work is not about correctness. It is about feel.

Spacing, shadow, blur, glow, material opacity, animation duration, and spring response are hard to pick from code alone. This is where a temporary debug-only tuning surface is useful.

struct FlairCardStyle: Equatable {
    var cornerRadius: Double = 24
    var shadowRadius: Double = 18
    var glowOpacity: Double = 0.28
    var verticalPadding: Double = 18
    var animationDuration: Double = 0.42
}

#if DEBUG
struct FlairTuningHarness: View {
    @State private var cornerRadius = 24.0
    @State private var shadowRadius = 18.0
    @State private var glowOpacity = 0.28
    @State private var verticalPadding = 18.0
    @State private var animationDuration = 0.42

    var body: some View {
        VStack(spacing: 24) {
            FlairCardView(
                model: FlairPreviewFixtures.earned,
                style: FlairCardStyle(
                    cornerRadius: cornerRadius,
                    shadowRadius: shadowRadius,
                    glowOpacity: glowOpacity,
                    verticalPadding: verticalPadding,
                    animationDuration: animationDuration
                )
            )
            .padding()

            Form {
                SliderRow(
                    title: "Corner radius",
                    value: $cornerRadius,
                    range: 8...40
                )

                SliderRow(
                    title: "Shadow radius",
                    value: $shadowRadius,
                    range: 0...40
                )

                SliderRow(
                    title: "Glow opacity",
                    value: $glowOpacity,
                    range: 0...0.8
                )

                SliderRow(
                    title: "Vertical padding",
                    value: $verticalPadding,
                    range: 8...36
                )

                SliderRow(
                    title: "Animation duration",
                    value: $animationDuration,
                    range: 0.1...1.2
                )
            }
        }
    }
}

private struct SliderRow: View {
    let title: String
    @Binding var value: Double
    let range: ClosedRange<Double>

    var body: some View {
        LabeledContent(title) {
            Slider(value: $value, in: range)
                .frame(maxWidth: 180)
        }
    }
}

#endif

That surface is scaffolding. Use it to find values quickly, then collapse the chosen values back into normal production constants:

extension FlairCardStyle {
    static let production = FlairCardStyle(
        cornerRadius: 22,
        shadowRadius: 14,
        glowOpacity: 0.22,
        verticalPadding: 16,
        animationDuration: 0.36
    )
}

Do not ship a permanent bag of tuning sliders unless the product actually needs user-facing customization. The point is to give the agent and developer a fast way to search the design space, then turn the result back into boring code.

Keep the prompt tied to screenshots

Once you have screenshots, use them precisely.

Weak prompt:

The flair card still looks off. Improve it.

Better prompt:

Use the attached locked-state screenshot. The title and progress text are too
close to the trailing edge at compact width. Keep the earned-state screenshot
visually unchanged. Adjust only the FlairCardView layout and the production
style constants. Do not change the model, debug route, or preview fixtures.
Run the visual-loop screenshot script again after the change.

That prompt names the artifact, the problem, the protected state, the allowed files, the non-goals, and the validation step.

Agents usually do better with constraints that describe what must not move. For visual work, I like to include:

Which screenshot is wrong.
Which screenshot should remain unchanged.
Which files are in scope.
Which states must still be inspected.
Whether the change is layout, copy, color, animation, or data.
Whether a temporary tuning surface may be used.

The screenshot is evidence. The prompt is the change contract.

Leave behind one durable check

Not every visual state deserves a pixel snapshot. Snapshot tests can be brittle across OS versions, simulator devices, fonts, rendering changes, and antialiasing differences. If you snapshot everything, future UI work becomes archaeology.

But a few narrow snapshots are useful when the visual contract matters.

With Point-Free’s SnapshotTesting library, a small test might look like this:

import SnapshotTesting
import SwiftUI
import XCTest
@testable import Wordie

final class FlairCardSnapshotTests: XCTestCase {
    @MainActor
    func testLockedFlairCompactDark() {
        let view = FlairCardView(model: FlairPreviewFixtures.locked)
            .padding()
            .frame(width: 340, height: 260)
            .environment(\.dynamicTypeSize, .large)
            .preferredColorScheme(.dark)

        assertSnapshot(
            of: UIHostingController(rootView: view),
            as: .image(on: .iPhoneSe)
        )
    }
}

Keep this kind of test for stable, high-value visual contracts:

A locked paywall state that must not hide the price.
A compact widget-like card that often clips.
A receipt, ticket, or share image.
A critical onboarding screen.
A component with a history of visual regressions.

Do not use snapshots as a substitute for accessibility checks. A screenshot can look correct while VoiceOver order is wrong, touch targets are too small, contrast is weak, or Dynamic Type clips text in another size. Pair visual screenshots with semantic tests and manual accessibility inspection where the UI matters.

The workflow I would use

For a flair-heavy SwiftUI task, I would give the agent this setup work before asking it to polish pixels:

Before changing the Flair UI, create a visual-loop harness:

- Add DEBUG-only preview fixtures for earned, locked, empty, long text,
  compact width, dark mode, and large Dynamic Type.
- Add named SwiftUI previews for each state.
- Add a DEBUG-only deep link or debug route that opens the Flair surface
  directly with a selected fixture state.
- Add a local screenshot script that builds, installs, launches, opens the
  debug route, and captures screenshots into /tmp.
- If animation or visual constants need tuning, add a temporary DEBUG-only
  tuning surface with sliders for spacing, glow, shadow, material opacity,
  and timing.
- Use fake data only. Do not expose real user data in screenshots.

Then I would ask for the actual UI change:

Use the visual loop to tune the locked Flair state. The goal is a premium
reward card that still reads clearly at compact width and large Dynamic Type.
Do not change navigation or production data flow. After each adjustment, run
the screenshot script and inspect the locked, earned, and long-text states.
When the constants are stable, remove any temporary values that should not
ship and leave the screenshot script in place.

That gives the agent a working surface and a definition of done. It also leaves the project better than it started. The next UI change does not begin from scratch.

Where Xcode 27 fits

Apple’s Xcode 27 agent story is directly pointed at this kind of loop. In Xcode, the agent can work with project context, render previews, show artifacts, use images and annotations, react to build failures, and help create prototypes. Device Hub adds a stronger simulator and device surface for screenshots, rotation, Dynamic Type, app data, diagnostics, and devicectl-style automation.

This is based on WWDC26 and Xcode 27 beta-era material. Exact labels, Device Hub affordances, agent tools, and automation behavior may change between seeds.

That does not mean every agent environment has the same powers.

For Xcode-native agents, lean on previews, artifacts, inline annotations, and Device Hub. For a command-line agent, make the visual evidence explicit: scripts, screenshots, local image inspection, and developer-attached images. For real devices or private data, keep the human in the capture loop.

The safest mental model is:

The agent can edit and validate code.
The screenshot shows what changed.
The developer decides whether the UI is good.
Tests and scripts preserve the useful parts of the loop.

A final checklist

Before asking an agent to do visual SwiftUI work, set up this checklist:

Is the target surface isolated enough to preview?
Are the important states named?
Is the data fake, stable, and realistic?
Can the simulator open the target screen without manual navigation?
Can a script capture the screenshot again tomorrow?
Is there a temporary tuning surface for subjective constants?
Are large Dynamic Type and dark mode part of the matrix?
Is at least one durable check left behind?
Are screenshots free of private user data?
Is the agent constrained to the files and behavior that should change?

The core idea is simple: give the agent eyes, but do not hand it the steering wheel.

Screenshot-driven UI work is useful because it narrows the argument. The question stops being “does this code look right?” and becomes “does this exact state look right after this exact change?” That is the loop SwiftUI agents need.