Do n’t you hate it when someone claims to have a magical raw engineering , but wo n’t tell you how it works ? When I saw that a super stealthy startup called Magic Leap hadraised $ 542 million to make animals look out of thin gentle wind , I dissolve to find out on the nose what was going on . Here ’s what I regain .
If the results of my digging are right , this is what Magic Leap intends to build : a Google Glass on sex hormone that can seamlessly combine computing machine - generate nontextual matter with the real cosmos . A headset pack with fiber optic projectors , crazy genus Lens , and loads of cameras . An augment realism that you ’ll actually believe in .
But I ’m bring forth forward of myself .
The story so far
As far as the medium was concerned , Magic Leap did n’t exist until it raised bucket of cash . In February , VentureBeat reportedthat the cryptical firm had secured $ 50 million to make a “ game changing ” technology that would “ everlastingly alter the direction we interact with range of a function and selective information , ” in the society ’s actor’s line .
That sound like the distinctive plug - cycle bullshit , but the names behind the company were worth compensate attention to : Rony Abovitz , who had just deal his aesculapian robot company MAKO Surgical for $ 1.65 billion , and Richard Taylor of Weta Workshop , the caller behind all those amazing props and creature inLord of the Rings , The Hobbit , District 9 , and more .
In March , Magic Leap acquire another huge slam of credibilitywhen seasoned game architect Graeme Devine sign onto build an entire plot studio apartment around the enigma technical school . And it got yet another in April whenveteran technical school merchandising chief Brian Wallacejoined up . He ’s one of the men creditworthy for Samsung ’s wildly successful “ Next Big matter ” advertizing campaign . These are people who could choose to act upon anywhere . They choose Magic Leap .
Magic Leap say it want to bring magic back into the world .
It was n’t until last calendar month , when Magic Leap annunciate $ 542 million in support from Google — and Qualcomm , and Legendary Pictures , and venture capital bigshots Andreessen Horowitz , Obvious Ventures , and KPCB — that journalist really pick up up their ear . ( It ’s not every day that Sundar Pichai , the Isle of Man in charge of all of Google ’s products , join the board of a startup company . ) But still , all that Magic Leap would say those who ask is that it will bepart of a “ lightweight wearable”that overstep the concepts of practical world and augment realism . talk to David Lidsky at Fast Company , Abovitz would only say what Magic Leap is not :
“ It ’s not holography , it ’s not stereoscopic 3 - vitamin D , ” he say . “ You do n’t need a giant robot to hold it over your head , you do n’t need to be at home to use it . It ’s not made from off - the - shelf parts . It ’s not a cellphone in a View - Master . ”
The best verbal description we have so far comes from the company ’s press release : “ Using our Dynamic Digitized Lightfield Signal ™ , envisage being able-bodied to return picture indistinguishable from real objects and then being able to station those images seamlessly into the genuine world . ”
Thankfully , that verbal description is just the wind of the iceberg lettuce .
Where did Magic Leap come from?
As best as I can distinguish , Magic Leap was n’t earlier a tech inauguration at all . When I razz the Wayback Machine to 2010 , the first stop was a fairly threadbare internet site for a troupe called Magic Leap Studios . Though the Magic Leap of four age ago already had a relationship with Weta Workshop , augmented reality did n’t seem to be the goal : Magic Leap claimed to be working with Weta on a graphic novel and a series of feature celluloid .
They would be base on a Modern cerebral property call “ Hour Blue , ” which seems to harry in a dystopian worldnearing the uniqueness . salient comic book duo Dan Abnett and Andy Lanning had signed on to help flesh out the mind .
Oh , now might be a honorable sentence to mention that Magic Leap Studios was also an independent book recording label . Their set , Sparkydog & Friends , appears to includeAbovitz , his wife Debb , former MAKO operative managing director of robotics Arthur Quaid , and current Magic Leap patent of invention bearer Graham MacNamara . Mick Glossopproduced their album . It ’s not awful !
But by 2011 , Magic Leap start depend a lot less like Abovitz rounding up protagonist to have some playfulness . In May of that class , Magic Leap became a corporation . At Comic - Con that July , it releasedan augmented realism appcalled Hour Blue , a first footstep towards a newfangled destination to “ play celluloid into the physical world . ”
By October , the company ’s internet site had becomea professional , secretive affair , and Magic Leap started call up itself a technology company . Or rather , a “ digital media and applied science company that developers and fork over proprietary contentedness through its platform and visual sensation of Social Cinematic Experience and Collective Imagination . ” Gosh , that ’s a mouthful .
“ We are keep a bit of a low profile as we are busy at work build something that we think will be pretty cool ( and we hope that you will think so too ) , ” wrote Magic Leap in 2011 .
What changed?
I do n’t know what prompted the shifting , but if I had to hazard a hypothesis : Rony Abovitz in all probability ran into Eric Seibel from the University of Washington .
For well over a ten , Professor Seibel had been developing an awesome new endoscope that use a unmarried optical fibre to safely see into tinier cleft in the human body than ever before . Using a piezoelectric to vibrate the peak of the fiber in a helical pattern , it “ scans ” a much higher resolving power range of a function than you ’d normally expect . And it ’s just the sort of dick that a medical robotics fellowship like Abovitz ’s MAKO Surgical might deal using .
What does that have to do augment realness , though ? Well , if you ’re already firing light source down a tickle optical fiber in a spiral approach pattern to form an image , that accurate same system can double asa vulcanized fiber optic projector . And that ’s where things get interesting .
Seibel ’s team was workingon just such a projector . They even tried it out insome prototype head - mounted displays . They found that it could be right smart more practical and crummy than tiny LCD or OLED panels , with the bring benefit that you could launch the fibers down to a pocket - sized large number instead of jade them on your oral sex . They hypothesise that it could even get small enough to fit inside a pair of glasses .
Which is credibly why Magic Leap hired Seibel ’s inquiry partner Brian Schowengerdt . And also probably why Schowengerdt helped Magic Leapwin a $ 150,000 Cary Grant from the Department of Defenseto make anultra gamey - res 4 kB versionof the fiber - scanning exhibit tech . And last , it ’s almost definitely why many of Magic Leap ’s letters patent coating are about how to rebound the light from those fibre into your eyes in a convincing room .
Intellectual property
Patent applications rarely recite you what a company is actually working on — only where it wants to legally stake a claim . Patents are hardly proof of anything , but they do give you some mind of a ship’s company ’s general areas of involvement . And Magic Leap ’s patents might explainhow the society might go from A. ) merely project images onto your eyeballs , to B. ) clear a 3-D , estimator - generated elephant appear to exist in the veridical universe .
What you see through Google Glass ( via Phandroid )
Believable augmented reality is hard . If you ’re looking at the real creation , your eyes are focusing at a kind of dissimilar distance , not inevitably on a tiny piece of glass correctly in front of your face . The real world also reflects a lot of light into your centre , which is why the images from forefront - up showing like Google Glass appear transparent and ghostly . Because you need to see the real world , you obviously ca n’t have a projector cross the front of the glasses : that light has to be recoil in from the side , which generally results in a narrow theatre of perspective .
And of course , you want some way to track your head and your surroundings so that CG object come out to occupy a actual place in the world , alternatively of look like a monotone trope — which , unhappily , is how many exist augmented realism glasses do it .
Magic Leap ’s letters patent program cover possible solutions for all these emergence . This oneexplains how the fellowship could use an LCD “ occlusion mask ” to block light from the real world at exactly the same places as the CG object , so those objects wo n’t come along ghostly . This oneis for a waveguide which can allegedly resile light across prissy big monocle - sized lens system , letting you theoretically see those CG animate being anywhere in your arena of view .
This one , licensed from NASA , is all about letting the software program know where you are in the real world with only a pair of externally mounted cameras , andthis oneis about change how the CG object bet depending on where they are in your field of honor of prospect . So if Magic Leap shows you a shark floating in midair , you might be able to walk around it and consider it from any angle without the software lose racetrack of you .
This onedescribes how Magic Leap could use a distich of cameras inside the Methedrine to give chase your eye and envision out where they ’re focalize , which could aid activate the most significant thing Magic Leap wants to do : use digital light discipline technologyto make CG object appear to take up actual depth inside the real human race .
Using fancy optic hardware likemicroreflectors or diffraction patterning devices or perhaps deformable tissue layer mirrors — which Schowengerdthad been successfully try out withfor some time — Magic Leap apparently believes it can let you focus your eye on a virtual 3D object the same way they focus on aim in existent life-time . Seemingly , it ’ll do so by cut its 3D objects into slices check to the astuteness of your gaze , and prove the appropriate one based on where your eyes are in reality looking .
Last but not least , there ’s alsothis patent applicationfor a “ tactual glove ” which lets you literally scratch two fingers together to steer a dyad of robots while using the headgear , which could more generally be a way to let you interact with the thing Magic Leap lets you see . Among Magic Leap ’s letters patent filings , this one is specially intriguing ; it admits that the company has really already built and tested this boxing glove in a enquiry subject field . Plus , it mention how the headgear can transition between a see - through augment world mode and an unintelligible one for virtual world . We could be looking at Google Glass and Oculus Rift contender in a individual equipment .
There’sway too much to take out here , but this should give you some vague idea of what Magic Leap wants to create . But what has the society already build , and what does it desire to build next ?
Eyewitness accounts
In an clause that mostly take flight under the radar , John Markoff ofThe New York Timesactually kick the bucket to see the engineering in personback in July . He wrote that he did indeed see a 3D creature float in midair , through “ an luxuriant watcher that resembles something from an oculist ’s billet . ” It ’s braggart , in other words . Markoff also confirm that the equipment project digital light fields onto the looker ’s retina .
Graeme Devine speaks about Magic Leap at Unite 2014 .
Though a fairly less reliable source now that he works for the party , secret plan designer Graeme Devine also suggests the current prototype is pretty gigantic . It ’s even called The Beast . “ If you ’ve ever seen the movieBrainstorm , it ’s like the original Brainstorm thing,”he told game developer in August .
Christopher Walken in Brainstorm ( 1983 ) .
Job listings
When I took a look at Magic Leap ’s job listings , it was clear that the troupe does n’t intend for The Beast to stay beastly for long . They ’re hiring hardware engineers to chop-chop prototype and prepare manufactory tooling for a young version : one that , several business postings tinge , will rely on smartphone way Mobile River component part and the Android operating arrangement to do its job .
And some of those caper mailing get astonishingly specific about what the caller require . space - kernel nomadic processors , cellular wireless signals , GPS , randomness - canceling earphone , and 3D depth - smell out cameras all look to be part of the package . In fact , Magic Leap specifically says there will be “ long orbit depth sensing element mounted on a pass - mounted twist ” in ordination to construct 3D mannikin of large indoor spaces — not unlikeGoogle ’s Project Tango . In Magic Leap ’s own words : “ RGBD range sensors are a adult deal , and they ’ll be jump on on the wearable . ”
Project Tango .
“ We ’re going to shell it in oculus trailing and realisation , ” begins another job bill — which seems to confirm that those , too , were n’t just crazy ideas listed on a letters patent covering .
Meanwhile , another job posting for an audio engineer hints at some challenging function for Magic Leap ’s tech : voice identification , telepresence , and “ live voice and music transmission and reproduction ” so that , presumptively , you may feel like you ’re in the same room with your favored band .
Not only will Magic Leap use Android , but it sure sound like the company intends to literally build its “ Cinematic Reality ” interface on top of the existing Android Open Source Project and its core practical app . That said , though , it also sound like Magic Leap intends to deliver much of the experience over the web : one job post namedrops “ cloud - base applications and services to corroborate Magic Leap ’s Cinematic realness experiences , ” and another specifies “ web browser - base drug user interface ” for the company ’s swarm services .
And it sound like Magic Leap hopes to do this on a large exfoliation : “ Demonstrated power to rapidly rage a complex consumer electronic / electro - optical production to high volumes ( i.e. millions of unit per yr ) is a must , ” consort to a job posting for a supply string manager .
Here are some other “ postulation ” that endure out to me :
“ We need to grab bowel movement and propose them through space so that you and your closest friends can last in alter world together . ”
“ We need to go deeply into find object , ascertain human and animal pose , recognizing scenes , identify and tracking rigid and articulated motions , understand behaviors , identify user , receive flicker , unveiling intrinsic persona and direction of kindling and on and on . ”
“ We want to do a lot of things with photographic camera , both for consumer consumption and for offer information to the Augmented Reality display . We need to stitch prospect , make arguing - fault video , hyper - resolving power , randomness simplification , image filtering of all types and an unending series of cool effects . ”
“ We want to try out with alternate ways of scanning objects and vista and to set up experiments with how AR systems can work with ( and you will build out ) an intact ecosystem of objects , toys , appliances , automaton and specially quadrotors . We want people hack on new ways of telepresence and combine AR with specific modalities from medical , to mechanical , to agrarian and construction . ”
Several job mailing mention that prior experience building quadcopters — a.k.a dawdler — would be a plus .
Last but not least , two job postings describean actual secret plan coming to the platform — one that should have steampunk fans champing at the scrap . Are you conversant with Dr. Grordbort ’s Infalliable Aether Oscillators , also make love as Weta Workshop ’s production line of incredible replica rayguns ?
According to the postings , Weta Workshop and Magic Leap are work up a “ truly next - propagation Dr. Grordbort ’s first person shooter ” where you ’ll get to experience the “ dynamic of advanced shaft of light hit man and wave - weaponry ” for yourself .
Particularly notable employees
Beyond Rony Abovitz , Brian Schowengerdt , Graeme Devine , and Brian Wallace , these people seem like pretty important additions to the team :
Gary Bradski , the creator of the Open Source Computer Vision Library ( OpenCV ) , and one of the guys who first teach self - drive cars to see .
Jean - Yves Bouguet , a member of the original Google Street View squad , and who led the Indoor Street View project .
Greg Broadmore , the creator of Dr. Grordbort ’s raygun - fulfil world .
Austin Grossman , who compose telecasting secret plan narrative for prize - winning title like System Shock , Deus Ex , and Dishonored .
John Root , a gesture capture artist and energizer with two decades experience at top TV game and film technology companies , let in Epic Games ( Unreal tourney 2004 ) , i d Software ( Doom 3 , furore ) , Remedy Entertainment ( Alan Wake ) , and Digital Domain .
Dave Gibbons , risible book author , artist , and cobalt - creator of Watchmen . He ’ll be “ developing projects and IP designed to take advantage of the next personal computing chopine , ” consort to this press release .
Andy Lanning , amusing volume writer , creative person , and Colorado - Almighty of Guardians of the Galaxy . Four years after sign on for Hour Blue , he ’s now the company ’s prescribed Comic World Ambassador .
Possibly important phrases
Last but not least , I had a peep at Magic Leap ’s earmark , just in caseful they might be significant . While a lot of them sound like everlasting marketing , I was n’t completely disappointed :
FLUTTERPODorFLUTTERBOARD : “ Virtual flying objects that are alive ; virtual reality experience ; electronic comic books . ” These seem the be the phrasal idiom Magic Leap has tentatively strike to refer to what you see project onto the real mankind . It ’s probably deserving note that “ Flutterpod ” is one of the keywords Magic Leap selected to direct hunting engines to magicleap.com .
THE MAGIC shop class : “ Retail store service in the field of operation of entertainment featuring pre - recorded audio frequency and audiovisual works and come to merchandise for function with Magic Leap hardware , provided via the internet and other computer and electronic communication networks . ” Sounds like an app fund .
SENSORYWARE : What Magic Leap will plausibly call the head - mounted display and related peripherals .
moral force DIGITIZED LIGHT - FIELD signaling : Already in use of goods and services in Magic Leap ’s closet releases as a catch - all for the society ’s visual technology .
A NEW OPERATING SYSTEM FOR REALITY : “ A web hosting platform for users to participate in and contribute to the creation of practical worlds . ” Intriguing .
MAGIC LEAP ’S MONSTER BATTLE : One of the first game for the platform . Graeme Devine draw it in August as a shared world where players each have customizable practical ducky that follow their finger and can ( presumably ) battle with one another . There ’s also this tweet to consider :
Why we are doing this : " Please make it possible for my brothers and I to have material life Pokemon battle in our backyard . " – Alex
— Magic Leap ( @magicleap)October 21 , 2014
ROADKILL WARRIORS : Listed under “ Comic Books . ” Perhaps the championship of a specific augment reality comic book coming to the program ?
COSMIC CODE AUTHORITY : A stamp of approval that appears to date back to Magic Leap ’s original Hour Blue graphic novel from 2010 … yet the hallmark diligence was just file away in 2013 . possibly we ’ll see it again ?
FUNONE : Unclear , but seems to have something to do with biz .
TAKE A MAGIC LEAP , WHAT ’S YOUR MAGIC LEAP , WELCOME TO THE EXPERIENCE , THE WORLD IS YOUR NEW background , THE WORLD IS YOUR NEW SILVER SCREEN , A ROCKETSHIP FOR THE MIND , SOCIAL CINEMATIC EXPERIENCE , COLLECTIVE IMAGINATION , CONNECTED privileged SPACE : Marketing .
IMAGINE IF THE IMAGINATION COULD IMAGINE : I ’m trying , but I ca n’t quite picture it .
The big picture
After spending my weeknight tracking down anything and everything I could bump about Magic Leap , a picture set out to form . retrieve when Mark Zuckerberg justify Facebook ’s $ 2B purchase of virtual realness pioneer Oculus VRby call it the next gravid communication political program ? It sounds like Magic Leap desire to do the same thing — only layer on top of the real world .
It sounds like they hope to do that with a lightweight headset mostly identical from eyeglasses , salve a fibre eye cable’s length run down to a pack where the projector ( and possibly the battery and other processing components ) are domiciliate . It sounds like that gimmick will be dead jam - packed with cameras and sensing element to on the dot know where it is , and which counseling it ’s point , inside a deepness - map recreation of the real world . It sound like it will run Android , have its own app store , and focalize on secret plan and interactive comic book to go .
In other words , it sounds absolutely bonkers . But $ 542 million in funding and Google on the board of directors paint a picture it ’s pretty damn real , and that it might only be a matter of fourth dimension until Magic Leap can make it little enough to comfortably wear . Which could put Facebook and Google in one heck of a battle for the future of wearable computing , I envisage .
I do n’t know for sure . I ’ve never seen Magic Leap ’s technology for myself . When I reached out to Magic Leap ’s chief executive officer and other employees , I did n’t get anywhere . And even if it exist , the idea wo n’t necessarily pan out . But if Magic Leap has solved even some of the key outlet keeping us from seeing computer - render look-alike seamlessly superpose onto the real cosmos , I ’ll be pump . Show me the legerdemain , Magic Leap . Make me a believer .
Promotional images viaMagic Leap , diagram via USPTO
Augmented realityGoogleVirtual reality
Daily Newsletter
Get the good technical school , scientific discipline , and culture news in your inbox daily .
newsworthiness from the future , delivered to your present tense .
