Apple augmented reality by tutorials

How to Get Started With Apple’s ARKit Augmented Reality Platform

By most accounts, Apple’s new smartphone, the “iPhone 8,” is due out soon—in September 2017. But I feel confident in saying the company’s most groundbreaking release for the year actually happened earlier this summer, at Apple’s Worldwide Developer Conference (WWDC) in San Jose, California.

At that event , Apple unveiled “ ARKit ,” a new, free set of software tools that anyone can use to build impressive new apps and experiences on the iPhone and iPad. The “AR” in “ARKit” refers to “augmented reality,” a technology that places interactive virtual objects and effects into your view of the real world, blending the virtual world with physical reality.

Today, this is usually accomplished by apps that use a smartphone’s (or other device’s) camera to show you a view of the real world in front of you, then layering virtual objects on top of it. For example, some of the most popularly used AR features right now are the “lenses” or face filters in Snapchat—the effects that put animal ears on your head or warp your face in weird ways. Another one is Pokémon Go, the gaming app that made the pocket monsters look as though they were bouncing around your desk or the street. What these apps have in common is that the virtual objects inside of them appear to be inhabiting or changing real-world scenes—“augmenting” our reality.

The Best Video Player for iPhone

A long time ago, Apple made it difficult for third-party developers to make a good media player for

But those apps were custom-built by pro software developers. What makes Apple’s ARKit version of augmented reality so exciting is that it’s not an app—instead, it’s a free way for anybody to build new AR apps that will be able to run on many of Apple’s iOS devices from the last few years. And the proof is in the pudding: Apple’s ARKit has already only been publicly available for about two months, but already, independent software developers have created some incredible-looking experiences for the iPhone—everything from virtual portals you can enter to putting virtual zombies in your living room and more. Even Lord of the Rings director Peter Jackson has gotten in on the fun with an ARKit short film .

Mmm, coffee
You can order subscriptions as small as two bags per month and as large as 24, which is ludicrous, but nice to know that it’s an option.

I’ve been interested in augmented reality for years, so when Apple announced ARKit, I was thrilled. If you feel the same or are just interested in experimenting with it yourself, here’s how you can get started.

Check to make sure you have a compatible iOS device

ARKit runs only Apple iOS devices containing an A9 processor (or newer). That limits it to the following devices, according to RedmondPie :

• iPhone 6s and 6s Plus
• iPhone 7 and 7 Plus
• iPhone SE
• iPad Pro (9.7, 10.5 or 12.9)
• iPad (2017)

Presumably, the new iPhone 8 lineup will also support ARKit. You’ll also want to make sure the camera on your iOS device is working, as ARKit relies upon it. And you need a cable for connecting your device to your Mac computer.

Download iOS 11 on your iPhone or iPad

Once you have one of these compatible iOS devices handy, you’ll need to download iOS 11 on it. That’s the latest version of Apple’s mobile operating system, and it’s only available as a beta right now, which means it potentially contains bugs and could mess up some things on your device. If this is a major concern, buy or use another device that’s not your primary one. Or, wait for the official release of iOS 11, which is also expected in September 2017.

If you don’t want to wait, you can download the iOS 11 beta and get started. To do this, you’ll need to enroll in Apple’s Beta Software Program . This is free, but requires anywhere between 20 minutes to a couple hours to set up. Head over here and follow the instructions .

Check to make sure you have a compatible Mac computer

You’ll also need a computer for coding your ARKit app. Specifically, a Mac with the free operating system mac OS Sierra 10.12.4 (or newer version) installed. That means the following machines are eligible:

• MacBook (Late 2009 or newer)
• MacBook Pro (Mid 2010 or newer)
• MacBook Air (Late 2010 or newer)
• Mac mini (Mid 2010 or newer)
• iMac (Late 2009 or newer)
• Mac Pro (Mid 2010 or newer)

iPhone: Get a One-Handed Keyboard Without Waiting for iOS 11

Later this year, Apple will publicly release iOS 11, which includes a one-handed mode for the…

Источник

More to Explore with ARKit 5

ARKit 5 brings Location Anchors to London and more cities across the United States, allowing you to create AR experiences for specific places, like the London Eye, Times Square, and even your own neighborhood. ARKit 5 also features improvements to Motion Tracking and support for Face Tracking in the Ultra Wide camera on iPad Pro (5th generation). And with a new App Clip Code anchor, you can pin virtual content from your App Clip or ARKit app to a printed or digital App Clip Code.

Expanded Face Tracking support

Support for Face Tracking extends to the front-facing camera on any device with the A12 Bionic chip and later, including iPhone SE, so even more users can delight in AR experiences using the front-facing camera. Face Tracking is now also supported by the Ultra Wide camera in the latest iPad Pro (5th generation). Track up to three faces at once using the TrueDepth camera to power front-facing camera experiences, such as Memoji and Snapchat.

Location Anchors

Place AR experiences at specific places, such as cities and famous landmarks. Location Anchors lets you anchor your AR creations at a certain latitude, longitude, and altitude. Users can move around virtual objects to see them from different perspectives, exactly as real objects are seen through a camera lens.

Requires iPhone XS, iPhone XS Max, iPhone XR, or later. Available in select cities.

Discover more ARKit features

Depth API

The advanced scene understanding capabilities built into the LiDAR Scanner allow this API to use per-pixel depth information about the surrounding environment. When combined with the 3D mesh data generated by Scene Geometry, this depth information makes virtual object occlusion even more realistic by enabling instant placement of virtual objects and blending them seamlessly with their physical surroundings. This can drive new capabilities within your apps, like taking more precise measurements and applying effects to a user’s environment.

Instant AR

The LiDAR Scanner enables incredibly quick plane detection, allowing for the instant placement of AR objects in the real world without scanning. Instant AR placement is automatically enabled on iPhone 12 Pro, iPhone 12 Pro Max, and iPad Pro for all apps built with ARKit, without any code changes.

Motion Capture

Capture the motion of a person in real time with a single camera. By understanding body position and movement as a series of joints and bones, you can use motion and poses as an input to the AR experience — placing people at the center of AR. Height estimation improves on iPhone 12, iPhone 12 Pro, and iPad Pro in all apps built with ARKit, without any code changes.

Simultaneous front and back camera

You can simultaneously use face and world tracking on the front and back cameras, opening up new possibilities. For example, users can interact with AR content in the back camera view using just their face.

Scene Geometry

Create a topological map of your space with labels identifying floors, walls, ceilings, windows, doors, and seats. This deep understanding of the real world unlocks object occlusion and real-world physics for virtual objects, and also gives you more information to power your AR workflows.

People Occlusion

AR content realistically passes behind and in front of people in the real world, making AR experiences more immersive while also enabling green screen-style effects in almost any environment. Depth estimation improves on iPhone 12, iPhone 12 Pro, and iPad Pro in all apps built with ARKit, without any code changes.

Additional improvements

Detect up to 100 images at a time and get an automatic estimate of the physical size of the object in the image. 3D object detection is more robust, as objects are better recognized in complex environments. And now, machine learning is used to detect planes in the environment even faster.

The Depth API is specific to devices equipped with the LiDAR Scanner (iPad Pro 11-inch (2nd generation), iPad Pro 12.9-inch (4th generation), iPhone 12 Pro, iPhone 12 Pro Max).

Resources and documentation

Download the latest version of Xcode, and use these resources to create AR experiences.

SDKs and betas

Download SDKs and beta operating systems for all Apple platforms.

Documentation

Browse the latest documentation including API reference, articles, and sample code.

Session videos

See the latest in Apple technologies presented at WWDC and other events.

Forums

Ask questions and discuss development topics with Apple engineers and other developers.

Источник

Apple Augmented Reality by Tutorials: Complete Book Now Available!

Start building next-generation AR apps today using RealityKit, Reality Composer, ARKit and more with our latest book, Apple Augmented Reality by Tutorials — available today!

We’re excited to announce that Apple Augmented Reality by Tutorials, First Edition, is 100% complete – and available today!

This book is the easiest and fastest way to get hands-on experience using Apple frameworks and technologies like Reality Composer, RealityKit, and ARKit — all the available rendering technologies Apple has to offer, as well as a collection of fun projects for creating various real-world AR experiences.

After reading this book, you’ll have a deep understanding of the technologies and frameworks used to create powerful, immersive AR experiences for the Apple platform.

Who is This Book For?

This book is for beginner to intermediate iOS developers who already know the basics of Swift development and are looking to build immersive AR experiences for the Apple platform.

Our previous AR book, ARKit by Tutorials, was a collection of ARKit-specific projects for creating various real-world AR experiences. This book is the successor to that one: Apple Augmented Reality by Tutorials provides the readers with a single, unified path to all of Apple’s available AR technologies that includes ARKit, RealityKit and Reality Composer, and SceneKit and SpriteKit as well!

This book includes a collection of fresh new projects for creating various real-world AR experiences.

What’s Inside Apple Augmented Reality by Tutorials

The full first edition of Apple Augmented Reality by Tutorials contains seventeen complete chapters, so you can get started building your next-generation killer AR app!

Section I: Reality Composer

Welcome to the first section of the book. In this section, you’ll dip your toes into the shallow end of augmented reality as you learn about AR Quick Look, Reality Composer, Reality and USDZ Files. You’ll create an interactive virtual drum kit that you’ll be able to share with your friends on iMessage.

Build interactive models that you can share through Messages!

This section contains the following chapters:

1. AR Quick Start: Get up to speed quickly with your first project in AR!
2. AR Quick Look: Discover how to integrate AR Quick Look into your apps to give them some cool AR superpowers.
3. Reality Composer & Reality Files: Find out how to leverage the power of Reality Composer to create interactive AR-based experiences.
4. USDZ Files & USD Python Tools: Learn about USD and the USDZ file format, and find out how to convert your virtual content into this universal format using Python-based command-line tools.
5. Reality Converter & PBR Materials: Discover how PBR materials can add a level of realism to your AR objects, and how you can use Reality Converter to convert, view and customize USDZ content.

Section II: Reality Composer & Image Tracking

In this section, you’ll dive a little deeper into augmented reality and learn about image anchors and tracking. You’ll also create an interactive AR Business Card that you can show off to your friends, making them green with envy.

Learn how to add custom actions and interactions to your AR scenes!

This section contains the following chapters:

1. Image Anchors: Learn how to use image anchors to scale and place virtual objects within the real world.
2. Behaviors, Triggers & Actions: Spice up your reality apps with animations and sound effects using behaviors, triggers and actions.

Section III: Reality Composer & Object Tracking

In this section, you’ll add another dimension with Object tracking. Using object anchors, you’ll augment a physical toy truck with interactive buttons that will reveal some fun facts about its real-world counterpart.

Master 3D real-world object tracking for increased precision and accuracy!

This section contains the following chapters:

1. Object Anchors: Learn how to use object anchors to anchor AR content to physical objects within the real world, and take a tour of ARKit Scanner — an iOS app that lets you scan physical objects.

Section IV: RealityKit & Face Tracking

In this section, you’ll learn about RealityKit and face tracking. You’ll create a SnapChat-like face filter app with SwiftUI that lets you mockup your face with funny props. You’ll also create an animated mask that you can control with your eyes, brows and mouth.

Create realistic and interactive face-tracking features for your app!

1. RealityKit: Find out how to set up and use RealityKit to build a face-based augmented reality app.
2. Face Anchors: Discover how you can use face anchors and face occlusion to track a user’s face and apply a virtual mask.
3. Facial Blend Shapes:
4. : Build a fully interactive augmented reality face mask that reacts to your facial expressions using blend shapes.

Section V: ARKit & SpriteKit

In this section, you’ll get a full introduction to ARKit and find out what makes it so powerful. In doing so, you’ll create a fun 2D SpriteKit game where you get to save tiny Emoji’s before they fall to their death.

Handle spawn points, physics and raytracing with ease!

This section contains the following chapters:

1. ARKit: Get a complete introduction to ARKit, Apple’s framework for creating fully interactive augmented reality, and learn about the different types of rendering options available with ARKit.
2. ARKit & SpriteKit: Discover how to combine ARKit and SpriteKit to create exciting augmented reality 2D games.
3. Raycasting & Physics: Learn about raycasting, 2D hit-testing and the SpriteKit physics engine as you add more features and functionality to your game.

Section VI: ARKit & SceneKit

In this section, you’ll continue learning about ARKit. You’ll also learn about SceneKit, Apple’s framework for creating 3D content, as you build a miniature interactive virtual airport that allows customers to access basic departure and arrivals information.

Create billboards and overlays as you build up your app using SceneKit!

This section contains the following chapters:

1. ARKit & SceneKit: Find out how to use the power of SceneKit and ARKit together to create powerful 3D augmented reality scenes.
2. Focus Nodes & Billboards: Discover how to detect, manage and use plane anchors and focus nodes. Learn how to enable environmental mapping and add lights and shadows.

Section VII: ECS & Collaborative Experiences (Bonus Section)

In this section, you’ll create a multiplayer AR shared experience using RealityKit. In this experience, two players can play a basic Naughts & Crosses game on separate devices.

This section contains the following chapter:

1. ECS & Collaborative Experiences: Build a collaborative AR experience and learn how to create and manage a multi-peer connection.

About the Author

Chris Language is the author of this book. Chris is a seasoned coder with 20+ years of experience, and the author of 3D Apple Games by Tutorials. He has fond memories of his childhood and his Commodore 64; more recently he started adding more good memories of life with all his Apple devices. By day, he fights for survival in the corporate jungle of Johannesburg, South Africa. By night he fights demons, dragons and zombies! For relaxation, he codes. You can find him on Twitter @ChrisLanguage Forever Coder, Artist, Musician, Gamer and Dreamer.

Where to Go From Here?

Apple Augmented Reality by Tutorials, First Edition, is 100% complete and available today. Like every one of our books, Augmented Apple Reality by Tutorials has been automatically added to the collections of all Ultimate Pro subscribers!

Want your own copy of Apple Augmented Reality by Tutorials? There are two ways you can get this book:

• Buy it individually: If you want to build up a custom library without a subscription, you can choose to buy the book individually. It’s on sale for just $29.99 during our Black Friday event — that’s 50% off!
• Get it as part an Ultimate Pro book and video subscription: Enjoy access to over 4,000+ videos and every single one of our 40+ professional mobile development books, including Apple Augmented Reality by Tutorials! And during our Black Friday event, you can get a full year’s Ultimate Pro subscription for just $299/year — you’ll save over 50% off our normal monthly price!

We hope you enjoy this complete first edition of Apple Augmented Reality by Tutorials. From the entire book team, thank you for your support!

Источник