Hover → Move
Point near the screen and the cursor follows.
In development · macOS + iPhone
Your Mac screen. Now touchable.
MagicGlass turns your Mac into a touch-capable computer using an iPhone you already own.
Place your iPhone beside the display, complete a quick calibration, and start interacting naturally. Point at the screen to move the cursor. Touch it to click.
No touchscreen. No special hardware. No cloud processing.
Point to move
Touch to click
Continuity Camera
No extra hardware
Runs entirely on-device
Point. Hover. Touch.
Your finger becomes the pointer.
Hold it near the display and the cursor follows your movement. Touch the screen only when you want to click.
That distinction is what makes MagicGlass feel natural: you don't need to drag your finger across the glass just to move around macOS.
Touch → Click
Make contact with the display to select.
Your iPhone watches from the side, leaving the space directly in front of your Mac completely unobstructed.
Private by design
Camera frames are processed locally on your Mac.
- No account.
- No cloud processing.
- No video uploads.
Your camera feed never needs to leave your computer.
macOS + iPhone first
MagicGlass currently targets macOS with iPhone through Continuity Camera.
Windows and Android support are planned.
Setup
From camera to touchscreen in about 30 seconds.
No proprietary mount. No calibration sheet. No cables.
Just stand your iPhone somewhere beside the display where it can see the screen.
01
Connect the camera
Your iPhone appears automatically through Continuity Camera.
Place it off to either side of the Mac with the display visible in frame. It does not need to sit directly in front of you.
02
Find the screen
MagicGlass displays four coded markers, one at each corner.
The camera identifies those markers and determines the exact position and perspective of your display—even when the phone is viewing it from an angle.
03
Calibrate your hand
Touch nine points across the screen. Then hover your finger above the display for a few seconds.
Those two passes teach MagicGlass both where you're pointing and the difference between hovering and actually touching the glass.
04
Start touching
That's it.
- Point to move the cursor.
- Touch to click.
- Touch and move to drag.
Your display becomes the interaction surface.
Gestures
You don't have to touch the screen until you mean to.
Most touchscreen interfaces require constant contact. MagicGlass doesn't.
Hover your finger near the display and the pointer tracks underneath it. Make contact only when you want to perform an action.
A lightweight indicator can remain visible above your applications so you always know what MagicGlass thinks your hand is doing without interfering with clicks.
Hover
Point without touching
Move your finger near the display and the cursor follows.
One finger
Click and drag
Tap the screen to click. Touch and move to drag.
Two fingers
Right-click
Touch using two fingers to trigger a secondary click.
How MagicGlass works
A touchscreen made from computer vision.
Turning an ordinary display into a reliable touch surface requires solving several different problems at once. MagicGlass handles each one independently.
01 · Locating the display
Four markers. One exact screen plane.
MagicGlass places a unique marker in each corner of the display. Each marker identifies which corner it belongs to, so the system can determine screen orientation automatically—even if the camera is rotated or viewing the display from an unusual angle.
Those four known points are used to calculate a homography, transforming the camera's perspective view of the screen back into a precise rectangle.
This is intentionally more deterministic than simply searching the camera image for a large rectangle. A generic rectangle detector can easily lock onto another monitor, a bright window, a reflection, or something in the background. MagicGlass knows exactly which rectangle is yours.
02 · Tracking your hand
21 landmarks, every frame.
MagicGlass detects 21 landmarks across the hand. The full landmark set can be visualized during calibration and diagnostics, while the tracking model primarily uses the index and middle finger chains.
Importantly, the system does not require your entire hand to remain visible. At the edges of the camera frame, your wrist or palm may disappear while your fingertip is still clearly visible.
MagicGlass keeps tracking the part that actually matters.
03 · Detecting contact
Measuring height relative to the screen.
Knowing where your finger appears in the camera image isn't enough. MagicGlass also needs to determine whether your finger is touching the display or hovering several centimeters in front of it.
The screen homography provides an important clue. A homography is exact for points lying directly on the screen plane. When points along your finger are projected through that same transformation, their rectified spacing changes as your finger moves away from the glass. That distortion becomes a depth signal.
Instead of asking: “How far apart are these pixels?”
MagicGlass can estimate: “How far is this finger from the screen plane?”
That makes contact detection far more consistent across different areas of the display and different camera angles.
04 · Learning your interaction
Nine touch targets—and one critical hover pass.
During calibration, MagicGlass records examples of your finger touching different areas of the display. But touch samples alone aren't enough.
Without examples of hovering, a finger held motionless above an icon can look almost identical to a finger resting against the glass. That's why calibration ends with a dedicated hover phase.
Touch samples provide the positive examples. Hover samples provide the negative examples. Together, they let the system learn the distinction between pointing and pressing.
Validation by screen position
Model validation is grouped by calibration target rather than individual camera frames. That matters because hundreds of consecutive frames from one tap are extremely similar—testing against another frame from the same tap would make accuracy appear better than it really is.
Instead, MagicGlass evaluates whether the model can generalize to screen locations it did not train on.
Hardware research
Tested on the hardware—not assumed from the documentation.
MagicGlass relies heavily on Apple's camera and sensor stack. Several important behaviors were discovered by probing the hardware directly rather than assuming the documented behavior matched what applications actually receive. Those findings shaped the design.
The built-in camera sensor can expose more than apps normally see
On tested Mac hardware, the camera exposes a square 1552 × 1552 sensor mode. The familiar 1920 × 1080 image applications commonly receive represents a widescreen view rather than the full usable sensor geometry.
The selected activeFormat also has to be applied at the correct point in the capture-session lifecycle. Otherwise, a session retaining the .high preset can silently return to the cropped mode.
For MagicGlass, extra field of view matters. More visible screen means more freedom in where the camera can be placed.
Camera effects can interfere with tracking
Different camera effects cause different problems.
- Center Stage changes the camera crop.
- Portrait Mode can blur the display MagicGlass is trying to analyze.
- Edge Light uses the display itself to illuminate the subject, potentially washing out the calibration markers being detected.
Where macOS permits it, MagicGlass controls these settings directly. Otherwise, it detects incompatible conditions and prevents calibration until they are corrected.
Continuity Camera has physical requirements
A Continuity Camera iPhone must remain available to macOS throughout the session.
Its rear camera needs a clear view, the device must remain stationary, and certain state changes—such as picking up or unlocking the phone—can cause the camera to disappear from the available device list.
MagicGlass monitors that connection and handles it as part of the interaction system.
Two Mac sensors matter more than expected
Display angle
On supported MacBooks, lid-angle information provides another useful signal. It can contribute to the interaction model and, more importantly, reveal when the physical position of the display has changed enough that calibration may no longer be valid.
Ambient light
Reliable hand tracking depends on the camera being able to see your finger clearly. Ambient-light data allows MagicGlass to identify environments that may be too dark for dependable tracking before poor lighting turns into confusing input.
Coming soon
MagicGlass is currently in development.
The core macOS + iPhone system is working, but MagicGlass is not publicly available yet.
We're continuing to refine tracking, calibration, gesture recognition, hardware compatibility, and the overall interaction experience before release. Windows and Android support are also planned.
When MagicGlass is ready, you'll find the release here.