Apple’s Magic Keyboard may turn out to be a key component in creating social augmented reality experiences. The latest version shows the attention they’re paying to this way of using the tablet.
The device helps to turn your iPad Pro into something that works a lot like a laptop. It’s not that you couldn’t before, but the keyboards were often clunky, the iPad didn’t support split windows, and the software wasn’t there yet to support running, for example, XCode on your tablet.
Blurring the Lines Between iOS and MacOS
Behind the scenes, Apple has been shifting its development frameworks to more easily allow you to publish to both Mac (OS) and tablets (iPad). Now, you can “configure your app for Mac can take just a click in a checkbox, although you may need more steps, depending on the features and frameworks that your app uses.”
Further, “iPadOS features additional technologies that take advantage of the unique functionality of iPad using the iOS SDK. With iPadOS, your iPad apps can now deliver multiwindow experiences, add full drawing experiences for Apple Pencil, and contribute fonts for systemwide use.”
In other words, developers now have tools to make it easier to create desktop-ready applications for iPad, or to port iPad applications to the desktop.
Magic Keyboard therefore isn’t just a keyboard, it’s a purpose-built device that supports the evolving Apple strategy of creating user expectations that the iPad can work like a laptop.
iPad and LiDAR Support
So now combine your new “looks like a laptop” iPad set-up with back-facing LiDAR support on the iPad Pro.
But that isn’t really the point of a LiDAR scan, which is primarily used to map physical objects in a space. This “map” then lets you blend physical and digital objects, making it look like there’s a dragon on your dining room table, for example.
I was thinking about this scanner as something you use while standing up. You rotate around, take a scan of your room, and then you can play Lego in AR or check out the new couch from IKEA.
But then I saw this image from Heckler Design:
Which got me thinking about the fact that you now have a stationary iPad with a pretty good view of the space behind it.
Scanners and Cameras and Glasses, Oh My!
So now, whether using a stand like that or your Magic Keyboard, you’re sitting with your iPad and it has the opportunity to take a pretty good 3D scan of the space behind it.
And the forward facing cameras have a clear view of you and your face. Apple’s Vision framework already does a pretty good job of mapping facial features and creating the kind of mesh used by Snapchat to create filters.
And so you have all the components you need for an augmented reality social experience. All you need are glasses.
But this wouldn’t be LIKE Facetime for VR, this would BE Facetime for Augmented Reality.
This wouldn’t be ‘LIKE’ Facetime for VR/AR, this would ‘BE’ Facetime for Augmented Reality.
And like Facetime, it might be limited (at first) to other users with LiDAR-equipped devices.
Simply sit at your iPad with your Magic Keyboard, receive an incoming call, and instead of the user appearing as a video window they can appear as an avatar in the space behind your iPad: their position facilitated by that LiDAR scanner which has figured out where to place them in the room.
The Path to Social Augmented Reality
I personally think that the first version of Apple Glasses will focus on being glasses first. Apple will have a win if they can disrupt the eye glasses industry.
Their glasses will make a fashion statement, be beautiful and do beautiful things. There’s a lot you can do with wearables other than create rich 3D scenes.
But I DO think their glasses may have a ‘light’ version of AR: subtle notifications or relatively easily-rendered content that gets ‘placed’ in the world around you.
Extend Your Screen
And so while FacetimeAR might not happen with round one of Apple Glasses, the LiDAR/iPAD/Apple Glasses set-up could still deliver value.
The Hololens, for example, will already allow you to extend the screen on your PC.
Apple could do the same: instead of just a multi-windowed iPad, you can now have a multi-windowed iPads where some of the windows float above or behind your screen.
The LiDAR scanner on the iPad Pro is actually a bit of a game-changer for AR, even if the ‘scans’ are primarily used to capture physical objects and spaces for uses in other applications.
But the fact that the Magic Keyboard suggests that Apple is committed to you using your iPad like a laptop could lead to the LiDAR scanner on the back being used, eventually, to support social augmented reality.