00:00SPEAKERWell, have you guys talked about large language models on TechPod before?
00:03SPEAKERTells you how much I listen to the podcast.
00:05SPEAKERIt's fine.
00:06SPEAKERI don't expect friends to listen to, but like, I don't expect my partner, I don't expect
00:09SPEAKERmy friends to listen to the podcast.
00:11SPEAKERIt's totally acceptable in the modern context because everybody makes podcasts.
00:15SPEAKERThat's right.
00:16SPEAKERWe have talked about LLMs.
00:18SPEAKERI am generally pretty bearish on generative stuff, but I think it's pretty cool for like
00:25SPEAKERspeaking spoken language and making it parsable English and stuff like that.
00:30SPEAKERGenerally, like, I don't know, call me old fashioned, teaching computers to lie seems
00:33SPEAKERlike a mistake.
00:35SPEAKERWell, this is teaching computers, something else.
00:37SPEAKERSo Google has a project called NotebookLM and it's a service where you can upload documents,
00:44SPEAKERso research papers, your notes, you know, a whole corpus that you upload.
00:50SPEAKERIt could be voice memos, it could be whatever it is, and the idea is that it helps you gather
00:53SPEAKERinformation and then you can then prompt it and this becomes the corpus that it pulls
00:59SPEAKERinformation from and you can find patterns.
01:02SPEAKERSo people use it to summarize legal documents.
01:05SPEAKERThey use it to, you know, look for patterns in research.
01:09SPEAKERBut what are the features?
01:10SPEAKERSo people have been using it.
01:11SPEAKERIt's a novel application that may be useful in the future, like practical usage.
01:16SPEAKERAnd it still has information like contextually of the greater world.
01:19SPEAKERBut the whole idea is that it's not supposed to pull in facts from the greater world necessarily.
01:23SPEAKERBut it's pulling from your notes, right?
01:25SPEAKERExactly.
01:26SPEAKERLike you upload and you say, this is the universe of information that you should pull from.
01:30SPEAKERHelp me find relevant XYZ structure in a certain way.
01:35SPEAKERAnd it's not like rewriting papers, but really it's just to gather your thoughts, collating,
01:38SPEAKERright?
01:39SPEAKERIt's collating information for you.
01:41SPEAKERSo is the idea that it doesn't lie then because it's only pulling from stuff that you've put
01:46SPEAKERin your known good data?
01:48SPEAKEROr is it, is the line just a feature of the LLM?
01:51SPEAKERI mean, hallucinations, I think definitely are a part of it and they're continually improving.
01:55SPEAKERBut with the smaller set of data than the greater world data, and with the expectation
01:59SPEAKERthat you're not going to be asking it to make up things necessarily, it's theoretically,
02:03SPEAKERit's hallucinating less or the way you use it, you're not going to generate hallucinations
02:08SPEAKERin the way you converse with it.
02:09SPEAKERYou're not conversing with it with like a chatbot.
02:11SPEAKERSo anyway, that's just like the basic premise.
02:13SPEAKERThe interesting part though, is one feature they have is it will let you use that corpus
02:19SPEAKERof data and generate a 10 minute podcast.
02:21SPEAKEROh, I've seen these.
02:23SPEAKERYes.
02:24SPEAKERSo it will, in about like a 10 minutes time and generate a conversation between two podcasts
02:31SPEAKERhosts, a fictional podcast called Deep Dives, where they talk about in their episode, just
02:37SPEAKERwhat information you've uploaded.
02:40SPEAKERI find this personally insulting Norm, the idea that you could distill the depth and
02:46SPEAKERbreadth of knowledge that a typical podcast host represents.
02:48SPEAKERI mean, it's not Joe Rogan up in here.
02:52SPEAKERWe're asking serious questions of serious people.
02:54SPEAKERHow much data do you have to have to make this thing work?
02:57SPEAKERNot much.
02:58SPEAKERSo the people have obviously pushed it to their extremes.
03:00SPEAKEROne extreme is someone uploaded like a document with nothing but the word fart and poop and
03:08SPEAKERthey generated a coherent and existentially interesting podcast about a document of nothing
03:18SPEAKERbut fart and poop.
03:19SPEAKERIs this where the podcast about the podcast hosts that were informed by their producer,
03:24SPEAKERthey were AI constructs and where people came from?
03:26SPEAKERYep.
03:27SPEAKERThat's absolutely it.
03:28SPEAKERYeah.
03:29SPEAKERYou know how dismissive I am of this idea.
03:32SPEAKERNorm made a podcast, had the thing make a podcast with me in it and he sent it to us,
03:38SPEAKERme and Kishore and I didn't even listen to it.
03:42SPEAKERI have no idea.
03:44SPEAKERI have no idea what the result of this thing even was.
03:48SPEAKERIt sounds conversationally accurate.
03:50SPEAKERI mean, if you, if you gave the audio file to someone, you know, a lay person and didn't
03:56SPEAKERtell you what it was generated by, it's just sounds like two people talking there and they
04:01SPEAKERpull in, they, they, they, there's definitely a structure to it, but they, they pull in
04:06SPEAKERall the conversational ticks and there's definitely a personality.
04:10SPEAKERThey invent the new different voices.
04:12SPEAKERIt's like a dad doing story time to a child is different voice, a man and a woman.
04:16SPEAKERAnd they just, it's a two same hosts.
04:18SPEAKERAnd so I think the way they've structured it and the way they've programmed it, it's
04:21SPEAKERalways these two same hosts, same voices.
04:24SPEAKERI see.
04:25SPEAKERAnd so they have like a, you know, a theme, like they have a format that they go through.
04:29SPEAKERIt reminds me of the, of the people that got Alexa and the Google homes to talk to each
04:35SPEAKERother for awhile.
04:36SPEAKERCause it's a little non-sequit, the one that I, so I listened to existential horror one
04:40SPEAKERand the setup for that was, Hey, these, these these robot robot podcast hosts find out that
04:46SPEAKERthey're not real people, even though they thought they were.
04:49SPEAKERAnd it was, it was like a lot of the generative stuff.
04:53SPEAKERIt was funny and creepy and novel.
04:55SPEAKERAnd I don't think it would have held my attention for one minute longer than it did, which was
04:58SPEAKERabout two minutes into the podcast.
05:00SPEAKERProbably.
05:01SPEAKEROh, I listened to the whole thing does get stale.
05:04SPEAKERHaving listened to a bunch of it, like you realize what the, the, the patterns are in
05:09SPEAKERthe way they talk.
05:10SPEAKERSo, but it's an interesting way to distill information.
05:13SPEAKERLike I think just yesterday, someone uploaded all of the unsealed January 6th.
05:19SPEAKERIndictment documents.
05:20SPEAKEROkay.
05:21SPEAKERAnd then I listened to that and it got me the top line summary and an interesting digestible
05:26SPEAKERformat.
05:27SPEAKERIt was saying that the election was a hoax, Trump won and, uh, freedom fighters on January
05:33SPEAKER6th.
05:34SPEAKERThat's the, that's the top line.
05:35SPEAKERIt all depends on the training data will.
05:46SPEAKERWelcome to brand will made a tech bot I'm will, and I'm here today without Brad, who's
06:09SPEAKERactually back home in North Carolina, helping his folks out, uh, deal with the aftermath
06:13SPEAKERof hurricane.
06:14SPEAKERUh, but I have the extreme pleasure.
06:19SPEAKERI think, I think that you two are the first two people I ever recorded a podcast with
06:23SPEAKERway back in the day, way back in like 2006 or 2005, back in the maximum PC, no BS podcast
06:30SPEAKERdays.
06:31SPEAKERUh, w today, today with us to talk about a project or not, sorry, it's not project Orion
06:35SPEAKERmetas Orion are Norman Chan and Jeremy Williams, maybe more, more well known for talking about
06:42SPEAKERVR than, than the ancient days of the no BS podcast.
06:45SPEAKERWelcome.
06:46SPEAKERWelcome to the show guys.
06:47SPEAKERThank you.
06:48SPEAKERMy good friends.
06:49SPEAKERI love seeing you guys.
06:50SPEAKERYou're giving it and giving you my podcast fix.
06:51SPEAKERWell, I haven't podcasted in years, but you, you know, if you want to have a podcast, you
06:55SPEAKERshould call me.
06:56SPEAKERI have time right now.
06:57SPEAKERSo I actually don't tell your wife, I said that she would probably be very upset with
07:02SPEAKERme.
07:03SPEAKERUm, but yeah, it's great to have you guys on the show and, and, uh, for folks who don't
07:06SPEAKERknow Norman, Norman, Jeremy did, uh, this is only a test for years and years and years
07:11SPEAKERafter I left tested with Kishore Hari friend, friend of the show, who's been on to talk
07:16SPEAKERabout the science multiple times.
07:17SPEAKERThis is Jeremy's first appearance on the show.
07:19SPEAKERWe realized immediately before this and I felt like a heel, but, uh, game frame's been
07:24SPEAKERin the background the whole time.
07:25SPEAKERSo it's all good.
07:26SPEAKERI mean, it's not a video show, so people don't know this, but the game frame is in like over
07:29SPEAKERmy, I think it's my right shoulder.
07:31SPEAKERYeah.
07:32SPEAKERMy right shoulder.
07:33SPEAKERAnd every time I do a video call with somebody, I get what is that thing?
07:37SPEAKERYeah.
07:38SPEAKERI'm like, Oh, it's this thing.
07:39SPEAKERI, it's a Kickstarter friend of mine made years and years ago.
07:41SPEAKERThe pictures of the grid that we printed on one of our old maker bots popped up on my
07:47SPEAKERGoogle photos the other day, Jeremy, and I was like, Oh, I remember doing that.
07:50SPEAKERThat was fun.
07:51SPEAKERUh, but anyway, we're here to talk today to talk about the future, not the past or a future
07:56SPEAKERmeta's future Zuckerberg's future.
07:58SPEAKERUh, the, uh, project, I keep wanting to call it project Orion.
08:03SPEAKERIt's just Orion.
08:04SPEAKERIt's there.
08:05SPEAKERIt's it's an AR glass.
08:06SPEAKERLike he did a thing on the, on the connect keynote, which is there like a virtual re
08:10SPEAKERaugmented reality, uh, kind of, kind of event every year where he stood up and he was talking
08:16SPEAKERabout the Ray-Ban glasses, which for people who don't know, they're glasses with cameras
08:21SPEAKERand they connect to your phone and they do a little bit of image recognition.
08:23SPEAKERThey can play music and stuff like that.
08:25SPEAKERRight there.
08:26SPEAKERThey, you seem excited about them, Norm.
08:28SPEAKERI've never used them.
08:29SPEAKERI'm wearing them.
08:30SPEAKERI'm wearing, I've been wearing them for the past year.
08:32SPEAKERAnd to give you a little preview of my favorite things, it is, I, and I, I've told many people
08:36SPEAKERthis, it is the single, the piece of technology that I have used more frequently in terms
08:43SPEAKERof it not sitting in a drawer, it being on my person than any other piece of technology,
08:49SPEAKERbecause it also functions as my normal prescription glasses.
08:53SPEAKERWhat components of the glasses do you use so frequently?
08:57SPEAKERIt's 100% the audio that I think the pictures, the AI, the multimodal AI, the fact that you
09:02SPEAKERcan ask me, I met AI things is novel and the pictures are okay when I reviewed them, it's
09:07SPEAKERlike, it's against the wide angle camera on your iPhone, the ultra wide angle camera on
09:11SPEAKERyour iPhone, but it's off center.
09:12SPEAKERSo you're kind of reframing it, you know, it's fixed focus.
09:15SPEAKERSo it's not like pictures I would ever print out, but yeah, they're okay.
09:19SPEAKERThere are pictures in the moment of kids, it's videos where I may be chasing, you know,
09:23SPEAKERnot having bone in my hand, I could be on a roller coaster or whatever, but it's the
09:26SPEAKERaudio that are built into the arms of these glasses, which are totally fine.
09:32SPEAKERThey're open air audio, so it's not like closed ear headphones, you're not getting noise cancellation.
09:38SPEAKERBut when I'm walking the dog, when I'm walking back from taking the kid to school, I'm listening
09:43SPEAKERto podcasts, I'm listening to music.
09:45SPEAKERAnd one of the transformer things is, you know how if you're browsing a social media
09:48SPEAKERfeed, and most of the times by default, like a video is going to have audio off, right?
09:53SPEAKERIf you're on Reddit, and you're scrolling through a feed of a movie trailer plays, it's
09:58SPEAKERby default muted, and that's because most of time people don't want to be blasting music
10:02SPEAKERout of their phones.
10:03SPEAKERIf you have AirPods on or headphones on great, you can turn them on.
10:07SPEAKERBut this allows me to have audio on as I'm browsing through a feed without something
10:12SPEAKERin my ear.
10:13SPEAKERSo my experience of using a feed with video is so different than what it was before that
10:21SPEAKERI can just without having anything in my ear, I can discreetly listen to the audio or video
10:27SPEAKERright, I was just gonna say I have myself a pair of the same glasses, and I have a very
10:32SPEAKERdifferent experience.
10:33SPEAKERI don't use them for audio at all.
10:34SPEAKERI find AirPods to be far more clear and, you know, controllable in terms of how much I
10:41SPEAKERwant to dial in the audio.
10:43SPEAKERBut what I do use it for is, is taking candid photos on particular events, like if I go
10:49SPEAKERto like an event, like we took my daughter to an escape room with all of her friends,
10:54SPEAKERand I wore the glasses the whole time, and even she didn't know I was taking pictures
10:59SPEAKERthe entire day.
11:00SPEAKERSo the whole day, it's just filled with people like looking completely natural, and maybe
11:05SPEAKEReven talking to me, and I'm just like reaching up, they just think I'm adjusting my glasses,
11:09SPEAKERbut it's a quick little tap and you take a photo.
11:12SPEAKERAnd at the end of the day, like we're looking through the photos and like, as you say, they're
11:16SPEAKERnot the best.
11:17SPEAKERBut as they say, like the best cameras, the one you have on you, and this one happens
11:21SPEAKERto take really good candid photos.
11:22SPEAKERSo it's handy for that.
11:24SPEAKERAlso the AI stuff, I have not been able to get my kids to try ChatGPT even minimally.
11:31SPEAKERBut when I showed my daughter that you can put the glasses on and say, what am I looking
11:35SPEAKERat?
11:36SPEAKERShe just was nonstop.
11:37SPEAKERShe was running around the house, looking at things, taking pictures, and asking what
11:42SPEAKERshe was looking at and just getting fun details back from, from the meta AI.
11:46SPEAKERSo that was, that's a fun component of it.
11:48SPEAKERThere's a little bit of latency there.
11:50SPEAKERIt's to be clear, like, is your pressure asking the question, it's sending the image to the
11:53SPEAKERphone, you know, it's snapping the picture.
11:55SPEAKERIt's not continuous because there's battery drain, but I think you're absolutely right
11:57SPEAKERJeremy about not just the camera that's on you, but the fact that it's so discreet that
12:03SPEAKERpeople are more natural in those pictures, which is something that you don't honestly
12:08SPEAKERhave these days.
12:09SPEAKERAnd there's that balance between, you know, privacy and how much, how candid a scene should
12:14SPEAKERbe.
12:15SPEAKERI guess that's worth talking about.
12:16SPEAKERDo you find that people know that you're wearing glasses?
12:19SPEAKERHas anyone walked up to you and said, Hey, are those, those Ray-Ban meta glasses or?
12:23SPEAKERNever.
12:24SPEAKERNo.
12:25SPEAKERAnd I think, I think the glass, the black ones are probably the most secretive, discreet,
12:30SPEAKERlike good for spy glasses.
12:32SPEAKERThe clear ones are probably the opposite of that.
12:35SPEAKERSo I don't know if, if you know, if you'll have the same experience with those glasses,
12:39SPEAKERbut yeah, no, nobody knows.
12:40SPEAKERI'm kind of surprised because there are two, there's clearly two, you know, lenses right
12:45SPEAKERon the side of them.
12:46SPEAKERAnd if you stare at them, you can see them, but no, nobody knows.
12:49SPEAKERThey, they, especially in the darker frames, they look like a, like a piece of flare on
12:53SPEAKERthe glasses design almost like I lived this life back in the day when Google glass was
12:58SPEAKERa thing in like 2012, 2013, I think.
13:02SPEAKERAnd it was a wildly different experience.
13:05SPEAKERWell, okay.
13:06SPEAKERThere's two things.
13:07SPEAKEROne is that I see photos come up in my, in my archive of like the angle of me holding
13:13SPEAKERmy infant child from here, right?
13:16SPEAKERLike looking down from my head and seeing her in my arm in a way that you're never going
13:20SPEAKERto actually take a photo.
13:21SPEAKERCause you have to get your phone right up against your face and it's really weird.
13:24SPEAKERAnd like, that's pretty magic.
13:26SPEAKERIt's pretty amazing.
13:27SPEAKERBut at the same time, you know, the thing that you described is really awesome Jeremy
13:31SPEAKERabout the way you're taking pictures.
13:32SPEAKERBut I, I also really don't want to be walking around and having pictures taken of me.
13:37SPEAKERAnd I like, I realized public space has no right to privacy, yada, yada, yada.
13:40SPEAKERBut there's an expectation that a camera is a bulky device that people see.
13:44SPEAKERAnd as a result, as a result of that and, and glass holes back in the day, people were
13:51SPEAKERgetting kicked out of public spaces wearing the Google glass.
13:54SPEAKERI also feel awkward walking around with AirPods in, right?
13:57SPEAKERLike I, like if I'm, if I'm, if I'm in a social circle, like if I'm walking just by myself
14:01SPEAKERand go to the grocery store, I'm walking around the dog doing whatever AirPods in, no problem.
14:05SPEAKERIf I'm going to a bar or a party and even though the AirPods make, let me filter out
14:10SPEAKERthe background noise and hear the conversation better, I'm always going to feel like a jackass
14:14SPEAKERwearing the AirPods in that circumstance because it, it, I, I'm, my perception is that other
14:19SPEAKERpeople are going to think I'm not paying attention to them and I'm listening to like a podcast
14:23SPEAKERor something instead of like being present in the moment.
14:26SPEAKERAnd of course, you're not using the glasses audio in that case either.
14:29SPEAKERNo, no, no.
14:30SPEAKERI know.
14:31SPEAKERI know.
14:32SPEAKERRight.
14:33SPEAKERYeah.
14:34SPEAKERSo anyway, sometimes I am, I've had the glasses on streaming like YouTube TV and listening
14:38SPEAKERto a football game while doing craft projects.
14:42SPEAKERAnd I can double task and like I'm present, but no AirPods in my ears and you know, cheering
14:49SPEAKERwhen, when the night air score.
14:52SPEAKERThis is straight up, this is straight up the, the, the guy, the, the bad husband with the,
14:57SPEAKERwith the one AirPod in his ear in church, listening to the baseball game on Sunday morning.
15:01SPEAKERRight.
15:02SPEAKERIt's the same thing.
15:03SPEAKERWell, I think the analogy here is it's apt for what we're going to talk about soon is
15:07SPEAKERthat the, this type of open ear audio that has these kind of smart capabilities, whether
15:15SPEAKERit's streaming or with through AI the difference between this and transparency mode on AirPods,
15:22SPEAKERwhich for people who don't know, it lets you listen to outside world and filters out noise
15:26SPEAKERis kind of the same analogous to the difference between pass through video VR glasses versus
15:32SPEAKERoptical AR glasses in the sense that one is a reconstructed digital feed and one for video,
15:40SPEAKERone for audio and you know, VR for AirPods and open ear glasses kind of is that analog
15:46SPEAKERfor optically clear AR where information is passed, but you're kind of discreetly using
15:51SPEAKERit.
15:52SPEAKERYeah.
15:53SPEAKERThat makes sense.
15:54SPEAKEROkay.
15:55SPEAKERSo we're going to talk about Orion just a sec, but I want to talk about this Harvard
15:56SPEAKERthing before we get to that, because it more pertains to the Ray-Ban glasses that are out
16:00SPEAKERnow.
16:01SPEAKERSo the Ray-Ban glasses, just to be clear, at least Zuckerberg in the keynote presented
16:05SPEAKERthese as the predecessor to the AR glasses and, and the, the, the look of things to come.
16:11SPEAKERSo this, this group at Harvard took, and I looked at, I read the whole article.
16:15SPEAKERI read that what they did.
16:17SPEAKERIt didn't look particularly difficult.
16:19SPEAKERIt looked like they took a couple of existing technologies, glued them together and jammed
16:23SPEAKERit into jammed it into software that'll run your phone connected to these Ray-Ban glasses.
16:28SPEAKERAnd they used a third party face scanning thing and a constant stream of video, I think
16:33SPEAKERvideo or photos, I can't remember which.
16:35SPEAKERIt's video.
16:36SPEAKERAnd same thing, like one of the debatable things are now in the VR space is how much
16:41SPEAKERaccess developers have to the live camera speed.
16:45SPEAKERWe're talking about pass through video, it was announced that connect like developers
16:47SPEAKERwant it because it allows for stronger mixed reality experiences.
16:51SPEAKERThere's obviously some type of privacy concern because users may not know that when they're
16:55SPEAKERgetting pass through video or the cameras on whatever they're wearing are going to the
17:00SPEAKERimagery isn't just for mapping your room, but also you're going to get the imagery set
17:04SPEAKERover developers.
17:05SPEAKERAnd so for these glasses, there is no API for any developer to actually capture the
17:10SPEAKERvideo feed or the photo feed.
17:11SPEAKERSo what these Harvard students did is they kind of hacked their way into it because you
17:15SPEAKERcan live stream from these glasses.
17:18SPEAKERThey can live stream only to an Instagram account because it's meta and they own those
17:22SPEAKERplatforms.
17:23SPEAKERSo their back door way of doing it was actually streaming to a private Instagram account and
17:30SPEAKERthen remotely on a PC using computer vision on that video feed and then passing it that
17:38SPEAKERway through their processing facial recognition tracking and then feeding that information
17:44SPEAKERvery quickly, which is like the near real time back to the user's phone.
17:48SPEAKERKind of like if you watch like Mission Impossible, you know, Ethan Hunt is doing the same thing,
17:52SPEAKERsending video feeds to Simon Pegg and he's running them through facial recognition.
17:55SPEAKERBut this is actually happening in the real world done by college kids.
17:58SPEAKERWell, yeah, and in this case, Simon Pegg was a commercially available face recognition
18:03SPEAKERprogram that you can match a photo to a face in like, what, four seconds or something.
18:09SPEAKERIt was it was kind of crazy.
18:11SPEAKERSo they didn't release the code for this, obviously.
18:16SPEAKERI think there was a lot of it's funny.
18:18SPEAKERThe conversation around this dredged up a lot of the old talk back when this first started
18:21SPEAKERbeing possible, where Eric Schmidt said, yeah, this was probably the only thing we ever did
18:26SPEAKERat Google that we just killed because we thought it was a terrible idea and maybe bad for humanity.
18:32SPEAKERBut I mean, as we see more the number of cameras in the world as like car cars that can drive
18:38SPEAKERthemselves come online like your car has your cars each have six or seven cameras, right?
18:44SPEAKERMine has seven cameras.
18:45SPEAKERThey're always on when the car is running like this technology is there and it's just
18:50SPEAKERa matter of time until somebody starts turning it on is the problem, I think.
18:55SPEAKERYou've always been a little more cautious about privacy than than I have.
18:59SPEAKERI mean, in most respects, like I to this day will share my find my location with anybody
19:05SPEAKERwho wants it.
19:07SPEAKERFor years, you and I, I knew exactly where you were all the time.
19:09SPEAKERI just had I like would open my phone with, why do I I don't know where Jeremy is.
19:13SPEAKERAnd he's like, ah, yeah.
19:14SPEAKERAnd so so I don't feel hypocritical in wearing the glasses and taking my spy photos.
19:19SPEAKERLike I'm perfectly comfortable with with that if it if the roles were reversed.
19:24SPEAKERAnd but you're absolutely right.
19:26SPEAKERYou know that that is a new reality.
19:29SPEAKERAnd I also I'm not terribly I'm not even a little bit worried about well, what the scenario
19:38SPEAKERthat you describe the Harvard students producing seems inevitable, right?
19:42SPEAKERLike that is not something that anybody is going to be able to put the brakes on because
19:49SPEAKERthere's always going to be a way to do the kind of workarounds that they did the the
19:52SPEAKERanalog hole and be able to analyze a video feed that is supposed to be only for one platform
19:58SPEAKERand feed that back in.
20:00SPEAKERAnd it seems like meta in their announcements, like they're leaning into that very thing.
20:05SPEAKERLike they're just going to leverage they're going to do that themselves with the live
20:10SPEAKERvideo analysis that they're going to offer that allows you to, you know, engage with
20:14SPEAKERan A.I. that sees what you see in real time live.
20:20SPEAKERAnd that, you know, I feel like if they're moving towards open source, it's just they
20:24SPEAKERare probably going to be the first ones to enable people to do that legitimately, eventually.
20:31SPEAKERAnd that's just I feel like that's the reality.
20:34SPEAKERAnd we're going to have to live with it to some degree.
20:37SPEAKERAnd it's scary and exciting.
20:39SPEAKERIs that possible?
20:42SPEAKERI think you're right.
20:43SPEAKERThe other half of that equation is how much people's public data that they put on online.
20:48SPEAKERRight.
20:49SPEAKERLike this wouldn't work unless people upload their own photos and put themselves in videos.
20:55SPEAKERAnd maybe the distinction is that there's a whole generation of people who live through
20:59SPEAKERthat will understand that better.
21:00SPEAKERRight.
21:01SPEAKERNo kind of understand the corpus of information they've been uploading.
21:04SPEAKERBut the thing they might not understand is how indexable that is.
21:06SPEAKERThere's a difference between how many photos that you put yourself up on and maybe your
21:10SPEAKERpublic profile page and what you think might be actually like indexable because you've
21:17SPEAKERbeen in videos, you know, in places that you don't remember.
21:21SPEAKERRight.
21:22SPEAKERSo that's the scary part.
21:24SPEAKERWell, and then the other thing is there's a lot of video and imagery that has to be
21:30SPEAKERposted.
21:31SPEAKERLike you have no way to avoid posting.
21:33SPEAKERIt's really hard to get a job right now if you don't have a LinkedIn profile with a photo
21:36SPEAKERon it.
21:37SPEAKERRight.
21:38SPEAKERSo you can't walk into a store, you can't walk into a retail location without 15 cameras
21:43SPEAKERtaking your picture.
21:45SPEAKERYou can't walk down the street without 50 cars taking your picture.
21:47SPEAKERRight.
21:48SPEAKERSo it's we've moved from a place where getting a photograph taken was a semi-consensual engagement
21:56SPEAKERbetween two people, the person with the camera, the person whose picture is being taken to
21:59SPEAKERan area of ubiquitous surveillance.
22:03SPEAKERAnd like, I think I think this is what it like, honestly, as we talk, we're going to
22:06SPEAKERjust like I think this leads into the Orion talk really well, because like Facebook has
22:12SPEAKERa creepy problem, but Meta has a creepy problem.
22:16SPEAKERLike they've been creepy with people's data for a really long time.
22:19SPEAKERAnd I think if they want to sell these kinds of headsets, these kinds of ubiquitous computing
22:23SPEAKERdevices, that's that's the one of the hills that they have to climb over.
22:28SPEAKERAnd the way they do that isn't by doubling down on being creepy.
22:31SPEAKERThe way they do that is by like actively being less, less creepy as like their businesses
22:37SPEAKERseem to be moving away from advertising.
22:39SPEAKERIt makes it easier, I guess.
22:40SPEAKERI don't know.
22:42SPEAKERSo so let's let's talk about Orion.
22:44SPEAKERNorm, you you got to go down to connect.
22:47SPEAKERYou got to see Orion.
22:48SPEAKERYou got hands on.
22:49SPEAKERYou did an hour long video on on the test YouTube channel, which people should watch.
22:53SPEAKERWe're not going to like recap the whole thing.
22:55SPEAKERWe had a good talk with Boz.
22:57SPEAKERHe's the CTO.
22:58SPEAKERI always think of him as the hardware guy and talk about what what their goals were,
23:02SPEAKERwhere they where they felt like they were.
23:03SPEAKERAnd it felt like a very kind of open conversation compared to the way they've handled this kind
23:09SPEAKERof stuff in the in the recent past.
23:11SPEAKERSo it was I was I was stoked to hear it.
23:12SPEAKERI really enjoyed the video.
23:13SPEAKERIt was really good.
23:14SPEAKERSo we just want to start.
23:15SPEAKERI can go with the broad what it is, you know, why they made it.
23:19SPEAKERWhat questions you guys have?
23:21SPEAKERI think start with where it is right.
23:22SPEAKERLike that's that's what it is.
23:24SPEAKERLike it's yeah, it was not something I expected to see this year or next year or even probably
23:30SPEAKERthe year after that.
23:31SPEAKERYeah.
23:32SPEAKERYou probably won't.
23:33SPEAKERYeah.
23:34SPEAKERYeah.
23:35SPEAKERSo, you know, they call us a prototype and they've been working on this for almost 10
23:40SPEAKERyears now, five years, really in earnest.
23:43SPEAKERThe idea is that Mark Zuckerberg really wants to own the hardware platform that will be
23:49SPEAKERpost mobile.
23:50SPEAKERSo you might have seen a lot of headlines or like, OK, Mark Zuckerberg thinks this will
23:53SPEAKERreplace your phone.
23:54SPEAKERAnd if you actually listen to the interviews, he did a great one with Cleo Abrams and Alex
23:59SPEAKERHeath of The Verge.
24:01SPEAKERIt's like he doesn't think that people are going to wear glasses and not have phones.
24:05SPEAKERBut it's kind of like the.
24:08SPEAKERThe second or third device, we think of all the screens that you have, there's TV, you
24:11SPEAKERhave laptop, computer phones, mobile is first right now, there was a whole transitionary
24:15SPEAKERperiod that we all remember going from a traditional computer to a phone, but it's not like laptops
24:21SPEAKERwent away.
24:22SPEAKERHe doesn't think phones will go away, but he definitely thinks that there are things
24:25SPEAKERthat might do that he believes that he's that he'll spend billions on developing on glasses
24:32SPEAKERthat you would rather do on your glasses than on your phone.
24:35SPEAKERThat's his big bet.
24:36SPEAKERAnd the reason he wants to do that is, of course, he lost in the mobile wars, right?
24:40SPEAKERThey tried working on a phone, Apple and Google.
24:43SPEAKERThey're second class citizens.
24:44SPEAKERThere was actually a biggest app developer in the world, right?
24:47SPEAKERThat's what Facebook app is and Instagram and WhatsApp.
24:53SPEAKERSo they want something that is going to be socially acceptable, that people would wear
24:58SPEAKERand can do things that's like fundamentally browse feeds, browse, some type of social
25:03SPEAKERmedia feed, and to do that on hardware they own rather than someone else's hardware.
25:09SPEAKERAnd to get that point, they wonder, can they do optical AR pass through something that
25:14SPEAKERlooks kind of like glasses that can give you world tracking abilities, holographic objects
25:21SPEAKERfixed in the real world that have a wide field of view and are light?
25:26SPEAKERAnd so after five years of developing something, they have this prototype now that's under
25:31SPEAKER100 grams and that gives you a 70 degree diagonal field of view, which is the widest we've seen
25:38SPEAKERin any type of optical AR system, but also importantly in the glasses form factor.
25:46SPEAKERThey considered making it a product, they considered selling it, but I think two years
25:49SPEAKERago, they decided it's one, too expensive, they would rather make a development kit.
25:56SPEAKERThey didn't want to fail.
25:57SPEAKERThey didn't want to release this this year and have it bombed because of pricing, because
26:01SPEAKERof performance.
26:02SPEAKERAnd so it's not going to be a real product.
26:04SPEAKERAnd the important thing is what eventually will be a product will probably not have the
26:08SPEAKERspecs either.
26:09SPEAKERIt won't have the silicon carbide, won't have the 70 degree field of view either.
26:13SPEAKERIt's the shape of things to come, not the thing to come, right?
26:18SPEAKERI thought the interesting thing was, so there's three components, right?
26:21SPEAKERThere's the glasses, there's the compute puck, is I think what they called it, a little,
26:26SPEAKERit looked like it was a little brick of battery and arm SOC probably and probably some memory.
26:33SPEAKERAnd then there are the wristbands or wrist wrist wristband or wristbands, did you try
26:37SPEAKERto or did you just do one?
26:39SPEAKEROkay.
26:40SPEAKERCan, can you talk about the, like, so you said the glasses are under, they're 98 grams.
26:44SPEAKERThat's about what a playing deck, a deck of playing cards weighs, how'd they feel?
26:51SPEAKERThe weight balance is really nice.
26:52SPEAKERSo they are bulkier.
26:53SPEAKERI mean, they talked about how, you know, thick glasses are in now and, but everything they
26:58SPEAKERsaid was caveat by the fact that they, they knew the glasses, the rims would be thinner
27:03SPEAKERin a production version, the profile was going to be thinner, but they're acceptable.
27:09SPEAKERPeople weren't freaked out wearing them.
27:10SPEAKERThey're clearly wearing these, like, you know, a piece of technology.
27:14SPEAKERYou look like a huge dweeb.
27:15SPEAKERThere's no getting around that.
27:17SPEAKERYeah, absolutely.
27:19SPEAKERBut it's not HoloLens.
27:21SPEAKERIt's not magically, they're not, it's not, you wouldn't call it a headset.
27:24SPEAKERYou would call it classes or you would call it a headset.
27:29SPEAKERAnd they put a bunch of local compute in there.
27:32SPEAKERSo you know, like the vision pro has what's essentially a version of like Apple's M2 processor,
27:38SPEAKERbut also the equivalent in terms of die size, this R1 processor that does all the world
27:43SPEAKERtracking and reprojection and all the stuff that you need for, for slam here, there is
27:49SPEAKERno compute for applications.
27:52SPEAKERThat's all done on the puck, but there is compute for the slam tracking and all that
27:57SPEAKERfits into this small 98 gram power envelope and volume where you can get like two to three
28:04SPEAKERhours of battery life, but get the cameras working for hand tracking and also all the
28:10SPEAKERstuff that you need on the glasses for low latency performance.
28:14SPEAKERIs this so slam tracking for people don't know is the, is the way you take input from
28:18SPEAKERcameras and you compare the different views the cameras have to figure out where the headset
28:22SPEAKERis or the glasses in this case are in 3d space.
28:25SPEAKERIs the math on that?
28:26SPEAKERI guess you have to, I guess it's still pretty gnarly if you have to have a fairly big chunk
28:31SPEAKERof a M3 processor dedicated on the AVPs, right?
28:36SPEAKERI think it's probably way more for VR headsets is they're also pushing the actual rendering
28:42SPEAKERof the high resolution, right?
28:44SPEAKERThe pixels here, it's world tracking, hand tracking, eye tracking, some reprojection
28:50SPEAKERbecause they have to do a dynamic calibration of the very low latency or even things like
28:55SPEAKERvertical disparity between left and right eye.
28:58SPEAKERNow they said that the heat on the frames, the expansion of the, on a hot day expansion
29:04SPEAKERof just electronics could create alignment disparity that they would have to correct
29:09SPEAKERin real time.
29:10SPEAKERAnd so that all happens and wifi signal, you know, Bluetooth and a wifi six connection.
29:15SPEAKERYou know, these things sound like they should be simple and they are simple, but they seem
29:19SPEAKERreally impressed that they could get that down to the hundred of milliwatts in aggregate
29:25SPEAKERversus typically it would be hundreds of milliwatts for each of those components.
29:29SPEAKERYeah.
29:30SPEAKERI, I am completely like dumbfounded that they, that this thing runs for two hours.
29:36SPEAKERLike I don't even know where the batteries are.
29:39SPEAKERAre they in the stems here on this going back on the stems?
29:42SPEAKERYeah.
29:43SPEAKERAnd they're going to, the next version, they talk about having more curved stems because
29:47SPEAKERthey could, you know, have new battery shapes and technologies, you know, it was one of
29:52SPEAKERthose things where every, every little bit of volume, if they drop this in a fish tank,
29:57SPEAKERno air bubbles would come out because every bit of space is valuable and they don't want,
30:02SPEAKERand maybe it's a lesson with their partner with the Ray ban folks, but they don't want
30:07SPEAKERto have like a neck strap with compute.
30:10SPEAKERThey don't want to have anything that, that doesn't look like it's the glasses form factor.
30:13SPEAKERI mean, that's the right call for sure.
30:15SPEAKERI thought it was interesting in your talk with boss.
30:18SPEAKERHe said like he went down a list of things that they were designing and building and
30:22SPEAKERmanufacturing themselves.
30:23SPEAKERThey've made, what did he say?
30:25SPEAKERA thousand or 10,000 of these so far, they're about a thousand of them and their bill materials
30:30SPEAKERis roughly $10,000 90% of which $9,000 out of $10,000 is just in the lensing, the manufacturer
30:38SPEAKERof the seven carbide.
30:39SPEAKERAnd that makes sense is like that kind of stuff is phenomenally expensive, but he said
30:43SPEAKERthey were making like the micro micro LEDs and stuff like they're, they're not just going
30:47SPEAKERto electronics markets in China and picking stuff that's off the shelf and talking to
30:51SPEAKERtheir suppliers and getting the stuff that's off, off the, off the shelf.
30:55SPEAKERThey're actually building components, not just like boards.
30:58SPEAKERAnd I was, I was really surprised by that.
31:00SPEAKERThat was, that's, that's unusual for a company like Facebook.
31:04SPEAKERI think they're probably building the controller boards, but they're still sourcing things
31:09SPEAKERlike displays because they don't have fabs for displays.
31:13SPEAKERCan we talk, can we talk about these lenses for a moment?
31:16SPEAKERBecause this is a thing that I don't, I don't understand why it's not the first thing everybody
31:20SPEAKERwants explained.
31:21SPEAKERBecause to me, this is like, what the heck is happening here?
31:25SPEAKERBecause normally in VR display, like the reason why it's so freaking bulky is because you
31:29SPEAKERhave these lenses that have to refract the light and it has to travel a certain amount
31:33SPEAKERof distance between the display and these lenses before it gets sort of redirected back
31:38SPEAKERinto your retinas.
31:39SPEAKERAnd that requires distance.
31:41SPEAKERHere we have these panes of silicon carbide that looks, you know, several millimeters
31:47SPEAKERthick and somehow my people's eyes are able to focus on this image.
31:55SPEAKERWhat is happening?
31:57SPEAKERSo focus is one thing, but actually, so we're talking about wave guides, right?
32:00SPEAKERAnd wave guides are what HoloLens and Magic Leap actually use as opposed to having a traditional
32:05SPEAKERoptical stack where you're shooting, cause you need a display somewhere, right?
32:09SPEAKERThere needs to be some type of way of routing.
32:11SPEAKERSo with the VR, the actual displays are right in front of the panels and it's an optical
32:15SPEAKERstack that pushes light, not bending it like multiple times, but focusing it to your retinas.
32:21SPEAKERHere the displays are off to the side and they're traveling along things like almost
32:26SPEAKERlike fiber optics.
32:27SPEAKERYou're traveling along these etchings on the lenses themselves that bounce, you know, multiple
32:33SPEAKERtimes and that go from the small little panel, you know, millimeters and millimeters in dimension
32:40SPEAKERto something that looks like an image to your eyes.
32:44SPEAKERAnd if you've used wave guides in HoloLens or Magic Leap before, they give you a fair
32:50SPEAKERamount of field of view.
32:51SPEAKERMagic Leap 2 especially has that 70 degree field of view, but you get a lot of like aberration.
32:58SPEAKERTheir visual artifacts are very clear when you see those wave guides.
33:01SPEAKEREverything looks shimmery.
33:03SPEAKERYeah.
33:04SPEAKERThere's definitely the chroma on the periphery.
33:07SPEAKERYou can tell that light is like traveling sideways along this panel before it gets to
33:13SPEAKERyour face.
33:14SPEAKERIt's not just glare from the outside world and glare actually doesn't look great from
33:17SPEAKERthe outside world either because of that.
33:19SPEAKERSo the benefits of silicon carbide, you get low visual distortion, you get the wide field
33:25SPEAKERof view because it has that high index of refraction, that 2.6 index of refraction.
33:31SPEAKERAnd then it's also super durable too.
33:33SPEAKERSo stronger than glass.
33:34SPEAKERAnd so all these things combined, lighter than glass, all those things are kind of necessary.
33:40SPEAKERThey're just so expensive to manufacture.
33:42SPEAKERThis might be a stupid question, Norm, but is this, are these glasses, like when you
33:48SPEAKERhold the glasses up in front of your face, they're not on your face.
33:50SPEAKERDo you see the display on the lens or is it like projecting back into your eyeball or
33:57SPEAKERis it something, is it just that you, because of the way the focus works on the up close
34:01SPEAKERand the refraction stuff that Jeremy was talking about, you can't focus on the image is on
34:05SPEAKERthe lens.
34:06SPEAKERYou just can't focus on it because it's other people can other people talking to you see
34:10SPEAKERany essence of that image?
34:11SPEAKERNo.
34:12SPEAKERAnd that's a good question.
34:13SPEAKERWhen someone's looking at you, they don't see any of what you see because the lights
34:18SPEAKERbouncing in one direction, but they do see kind of what looks like a, like a, um, uh,
34:24SPEAKERiridescent shimmer, um, and off access, you'll see a lot of the photos look like people's
34:30SPEAKEReyes are glazed over or the whole panel is glazed over with some type of iridescent light.
34:35SPEAKERThat's not, that's only like, I think a low angle looking up, I'm looking at someone's
34:40SPEAKERstraight on, you just see their eyes and the amount of transparency.
34:44SPEAKERSo this is why they're not using something like birdbath optics, which, you know, also
34:48SPEAKERa kind of a micro OLED display bounced off, like kind of a pepper ghost style into your
34:54SPEAKEReyeballs.
34:55SPEAKEREvery time you use that with something like, you know, I have, um, the, uh, the X real
34:59SPEAKERglasses, you lose a lot of light because you're basically using a one way mirror to block
35:04SPEAKERoff light to the outside world.
35:06SPEAKERAnd so those look like sunglasses.
35:08SPEAKERIf you're looking at someone wearing something with a, a combiner and birth bath optics here,
35:13SPEAKERyou had a lot of light pass through and a lot on both outside world, the outside world
35:17SPEAKERis just as bright.
35:19SPEAKERIt doesn't look like you're wearing sunglasses.
35:23SPEAKERAnd how opaque can the, uh, augmented images get, uh, they're glowy there, you know, it's,
35:30SPEAKERit's like they're opaque enough, but it's, I would say less important than the opacity
35:38SPEAKERis the quality of the image, the quality of the thing they do look like.
35:42SPEAKERThe reason you call them holograms is they look like these light forms.
35:45SPEAKERIt's not going to be flat shaded things.
35:47SPEAKERUh, you know, it looks like a kind of like a, you know, a floating LCD panel without
35:53SPEAKERthe backlight.
35:54SPEAKERWhen you're looking through the, through the lenses, like, do you see like on the hollow
35:59SPEAKERand the old hollow lenses, you could see the edges of the AR area, right?
36:02SPEAKERLike there was a rectangle in front of your field of view that was like, here's where
36:06SPEAKERthe magic happens.
36:07SPEAKERAnd then everything else looks like crap.
36:09SPEAKERIs that, is that present here or do you see more edge to edge with these?
36:13SPEAKERYou definitely don't get full edge to edge.
36:15SPEAKERIt's 60 degrees horizontal.
36:17SPEAKERAnd so I think it depends on the content because it's not VR.
36:20SPEAKERIt's not trying to fill up that full field of view.
36:22SPEAKERThese windows are floating in front of you, not off the side, but you, you, if you move
36:26SPEAKERyour head left and right, they will crop off.
36:29SPEAKERYou'll get that clipping off in your periphery, both inner and outer.
36:33SPEAKERBut if I'm okay, so I'm going to make a, I'm making a rectangle with my, with my index
36:36SPEAKERand thumbs and I'm putting them in front of me.
36:38SPEAKERHow far away should I put that rectangle until I see what, where the AR stuff lives further
36:43SPEAKERthan you think, what's, what's the thing with 70 degrees.
36:47SPEAKERAnd I know 70 sounds a lot less than 90 or 96 that we typically have in VR headsets.
36:53SPEAKERBut the fact is that you're also wearing glasses frames.
36:56SPEAKERAnd so your field of view is already being cropped by glasses frames.
37:00SPEAKERLike the three of us are all wearing glasses.
37:02SPEAKERAnd if I look at my, I can see the frames of my glasses as I'm wearing them.
37:06SPEAKERSo think about that's already less than 90 degrees.
37:09SPEAKERYou're not gonna get your full, you know, field of view anyway.
37:12SPEAKERAnd it's like reducing that just a little bit.
37:15SPEAKEROkay.
37:16SPEAKERSo now here's my next question.
37:17SPEAKERThere's logic in the glasses.
37:18SPEAKERThere's the compute puck to run the apps on.
37:21SPEAKERIs the rendering happening on the glasses or is it happening in the compute puck?
37:25SPEAKERI believe the compositing and all the rendering is happening on the glasses.
37:28SPEAKERCompute puck is anything, all app logic.
37:30SPEAKERSo anything that does not require low latency for comfort, but basically outside of your,
37:37SPEAKERyour basic OS, which they call AR OS and the surprising thing is AR OS has only been in
37:42SPEAKERdevelopment for the past three months.
37:44SPEAKERSo it's running Android, their version of Android apps kind of re-skinned.
37:50SPEAKERDidn't notice like latency, but also wasn't doing a ton of like, you know, going back
37:54SPEAKERand forth.
37:55SPEAKERIt was very kind of basic, basic MR slash MR UI, you know, a version of their quest
38:01SPEAKERUI with docs items on the bottom menu can pull down from the top, that kind of thing.
38:06SPEAKERThree panes, a maximum of three panes right now that are connected together.
38:10SPEAKERSo you're not separating the panes and windows and locking them into the wall or anything
38:14SPEAKERlike that.
38:15SPEAKERSo very early for their, for their OS.
38:17SPEAKERWhat is interesting for the compute puck though, is it's over a wifi six connection and the
38:22SPEAKERsoftware protocol that he built on top of that pulses the data from the glasses to the
38:30SPEAKERcompute puck and vice versa.
38:32SPEAKERSo it's not a persistent wifi six connection, which allows them greater distance that you
38:37SPEAKERcan have up to 12 feet.
38:38SPEAKERSo when I was doing demo, the puck wasn't actually in my pocket.
38:40SPEAKERIt was just in the middle of the room and also means a battery savings as well is if
38:47SPEAKERI'm wearing like Ray-Ban Meadows and I'm doing live streaming that persistent wifi connection
38:51SPEAKERkills battery on the buses so quickly.
38:53SPEAKERYeah.
38:54SPEAKERI mean, when, when you're talking about the thing that you use to see the world, like
38:57SPEAKERI think Jeremy, your vision is probably better than mine, Norm, you can go without your glasses.
39:02SPEAKERI can't see anything without my glasses on.
39:04SPEAKERSo if I have to take the glasses off the charge midway through the day, it's a fairly substantial
39:08SPEAKERhassle for me.
39:09SPEAKERThe other thing I want to say about the compute puck is there are a lot of people, you know,
39:12SPEAKERApple fans saying, well, Apple has this in the bag cause they got something you could
39:16SPEAKERput on your wrist.
39:17SPEAKERThey got something you could put in your pocket.
39:19SPEAKERAnd obviously the w we'll talk about the wristband and how that may be analogous to the Apple
39:25SPEAKERwatch.
39:26SPEAKERBut I do think that if Apple wanted to have a version of this, what you have in your pocket
39:32SPEAKERwith the iPhone would be more than sufficient as that compute puck, just to do app logic
39:39SPEAKERkind of the way it is the processor when you do a wired CarPlay connection.
39:43SPEAKERYeah.
39:44SPEAKERIt's, it's, it's, it's this or wireless CarPlay too.
39:46SPEAKERIt does the same thing.
39:47SPEAKERIt just blasts across wifi then.
39:49SPEAKERIt seems like Apple's made their bet though, on all in on, you know, single device though,
39:53SPEAKERoddly.
39:54SPEAKERI mean, given that a lot of people think one of the downsides of the Apple vision pro is
39:58SPEAKERthe weight.
39:59SPEAKERThey could have solved that by moving someone that compute to the phone, but they chose
40:03SPEAKERnot to.
40:04SPEAKERI wonder if they will reverse that decision.
40:06SPEAKERIt doesn't seem like it's been a resounding success for them.
40:09SPEAKERSo you know, who knows?
40:11SPEAKERI think the type of things that you do in the vision pro are so much more in order of
40:16SPEAKERmagnitude, more compute heavy than the type of compute you would do in these AR glasses,
40:21SPEAKERsingle feed.
40:22SPEAKERLike I think they made the right choice there.
40:24SPEAKERLike I would not like a vision pro, even if it was lighter to be limited to three windows
40:29SPEAKERor, you know, single, not real multitasking.
40:31SPEAKERI want that kind of laptop capability in the vision pro I have today.
40:35SPEAKERYeah.
40:36SPEAKERI was just saying, I've only spent a little bit of time with division pro, but the time
40:38SPEAKERI spent made me feel more like it would replace my Mac book than my phone.
40:42SPEAKEROh, iPad.
40:43SPEAKERRight.
40:44SPEAKERYeah.
40:45SPEAKEROur iPad.
40:46SPEAKERSame difference.
40:47SPEAKERRight.
40:48SPEAKERYeah.
40:49SPEAKERNo, I think that that's huge.
40:50SPEAKERThat's like, they, they realized that long after meta did and meta is kind of followed
40:52SPEAKERup with, you know, following their lead now with how much you should be able to do in
40:59SPEAKERVR.
41:00SPEAKERWhereas like they had originally sort of been more like the phone where you're launching
41:03SPEAKERan app and then you're in these amazing games, you go back to the OS momentarily, but it's
41:07SPEAKERa bad experience to be there.
41:09SPEAKERNow they've Apple sort of shown that the OS should actually be pretty great, sort of like
41:13SPEAKERa PC experience, maybe even further out because now you have this sort of infinite space,
41:18SPEAKERbut I'm glad to see Apple lead the charge on what an VR OS should be and see meta follow
41:27SPEAKERup.
41:28SPEAKERThe, the, the difference on the AR glasses and the VR glasses and the VR thing is that
41:34SPEAKERwhen you're putting on a VR headset, you're taking away access to your favorite thing
41:37SPEAKERin the world, which is, you know, I mean, arguably either your children or your phone,
41:41SPEAKERdepending on who you ask and, and like, it's a big ask, right?
41:45SPEAKERSo it means that you're not going to see text messages.
41:47SPEAKERIt means you're not going to be able to scroll Twitter while you're waiting for something
41:49SPEAKERto load.
41:50SPEAKERIt means you're not going to be able to do all the things that we'd normally do while
41:52SPEAKERwe're sitting idle now because we have hummingbird like attention spans.
41:56SPEAKERBut when you do the, when you're wearing the glasses, the normal glasses, and you're waiting
42:01SPEAKERfor something to load, it's not as big a lift because you just pick up, it still works.
42:04SPEAKERHow norm, I'm curious about the carbide lenses too.
42:08SPEAKERThe high refractivity means that I would think that the world would be distorted a little
42:11SPEAKERbit looking through them.
42:13SPEAKERWas that the case?
42:14SPEAKERNot at all.
42:15SPEAKERIt's still completely normal and there is an extra layer in the alto stack that combines
42:19SPEAKERwhat the carbide lens wave guides are doing.
42:22SPEAKERAnd also what you see of the world, it's not distorted at all.
42:25SPEAKEROkay.
42:26SPEAKEROkay.
42:27SPEAKERNorm, if you could imagine the experience using Orion with just a lot more compute and
42:34SPEAKERmore time spent on the user experience, is it compelling?
42:39SPEAKERYes.
42:40SPEAKERThat is one, if it, if they can achieve a smaller form factor, keep it the same way,
42:47SPEAKERbut that, you know, it's something that is half as thick in the profile, half as thick
42:51SPEAKERin the side, the thickness of the frames actually solve the UX experience.
42:56SPEAKERAnd I don't need to have a world tracking thing.
42:59SPEAKERI don't need to have name tags in front of everyone, but just give me the bare essentials
43:03SPEAKERof making a good communications device.
43:05SPEAKERYou know, phone calls and video calls or, and, and browsing the things I pick up my
43:11SPEAKERphone for most, what are the, if you look at your, your, look at your iPhone, look at
43:14SPEAKERyour app usage.
43:15SPEAKERWhat are the things that you're on your phone?
43:16SPEAKERMost it's emails and social media, right?
43:20SPEAKERIt's maybe watching some videos, listening to some music, browsing feeds of some kind.
43:24SPEAKERIf I can do that comfortably, ergonomically, and we'll talk about the ergonomics of the
43:30SPEAKERinput right with glasses, I would stop picking up my phone to browse a threads feed.
43:36SPEAKERWhat about the wristband?
43:37SPEAKERCan you talk about what's possible with that right now?
43:40SPEAKERAnd we're having used it, what you could imagine it being used for in the future.
43:45SPEAKERLet's start with what they are.
43:46SPEAKERCause we haven't really talked about them yet.
43:47SPEAKERAnd they're there.
43:48SPEAKERThis is, I think the most interesting, like the lenses to Jeremy's point, super interesting.
43:52SPEAKERThis is something I would use on a current device.
43:54SPEAKERAnd it's the most ready product because they purchased this company control labs back in
43:58SPEAKER2019, that was looking at EMG input, electromyography.
44:03SPEAKERSo it's basically the electrical signals that are, could sense your muscles as you move
44:09SPEAKERthe different, the many muscles that are on your wrist and hands.
44:12SPEAKERAnd so the wristband is kind of calibrated to you in terms of like, it's, it's tight.
44:16SPEAKERIt's like rubber band tight.
44:17SPEAKERIt's not, you know, some people like a loose watch band.
44:20SPEAKERIt's not a loose watch band.
44:21SPEAKERIt's tight enough that the eight sensors on there, it's slightly imprinted into your skin,
44:26SPEAKERright?
44:27SPEAKERAnd then off you would see the, the, the, it's slight indents.
44:29SPEAKERCause it like nubbin type electrodes, like little, little metal balls, pill shape, pill
44:34SPEAKERshapes.
44:35SPEAKERRight.
44:36SPEAKERBut the form factor is more like a watch strap than it is, you know, black widows, you know,
44:41SPEAKERdarts.
44:42SPEAKERYeah.
44:43SPEAKERThey're not, they're not super bulky.
44:44SPEAKERLow power.
44:45SPEAKERThere's a machine learning processor on there.
44:47SPEAKERI can also imagine that they would put some digital interface, like a watch, but it goes
44:51SPEAKERin a dominant hand.
44:52SPEAKERYou don't need it when you have, they are glasses because you just hold it up and it
44:54SPEAKERjust gets projected on the wrist.
44:56SPEAKERRight.
44:57SPEAKERIn a fashion, right?
44:58SPEAKERLike, you know, put some spikes on there, right?
44:59SPEAKERMake it and make it a, make it really palatable for the hot topic crowd.
45:02SPEAKERSo the amount, what you can do right now is the four basic gestures I was taught to do
45:09SPEAKERwas tap index finger to thumb and orientation of wrist also matters.
45:15SPEAKERSo index finger to thumb, turn my wrist up in a middle finger to thumb for a menu, scroll
45:21SPEAKERup and down, which is actually to rub my thumb against my index finger up and down, and then
45:26SPEAKERdouble tap my thumb against my index finger to activate meta AI.
45:30SPEAKERThose four things.
45:31SPEAKERSo home, left click, right click and scroll basically.
45:35SPEAKERSure.
45:36SPEAKERYeah.
45:37SPEAKERHold to tap, hold to tap also works.
45:39SPEAKEROkay.
45:40SPEAKERAnd, but these are uncomplicated.
45:41SPEAKERLike I'm not going through a whole regimen of like, you know, trying these different
45:46SPEAKERgestures, like flexing my hands.
45:48SPEAKERThe thing I really wanted to get a sense of was accuracy, like false positives versus
45:52SPEAKERmissing input, and also how much intent, like, do I need to be thinking about this?
45:58SPEAKERThese actions I need to be pressing real hard.
46:02SPEAKERAnd one accuracy rate was really impressive.
46:06SPEAKERAnd this is without training, right?
46:07SPEAKERYeah.
46:08SPEAKERWithout training, without calibration.
46:10SPEAKERThey walked me through it.
46:11SPEAKERI did the, did all the gestures once and the system was good to go.
46:16SPEAKERAnd I had the false positive of the only for other only false positive I had were occasionally
46:20SPEAKERI would be like, just fidgeting and tapping my thumb against my hand.
46:24SPEAKERAnd then the meta AI would activate.
46:27SPEAKERBut for anything like the tap, you know, this general tap gesture, I didn't need to push
46:32SPEAKERmy fingers hard against each other.
46:34SPEAKERIt was literally just a simple, simple tap.
46:37SPEAKERIt wasn't like an exaggerated movement, casual tap, got it.
46:41SPEAKERAnd if you think about that kind of micro gesture and how untaxing you want those gestures
46:47SPEAKERto be on your muscles and having that, just not needing the hand in front of you.
46:52SPEAKERSo it needs to be captured by some field of view, some computer vision lens, having your
46:58SPEAKERhands behind your back and doing this, having your hands in your pocket.
47:01SPEAKERThat was why they're doing this.
47:04SPEAKERYou did that?
47:05SPEAKERI did that.
47:06SPEAKERYep.
47:07SPEAKERHands like underneath the table and just resting on my lap, right?
47:09SPEAKERJust in a very casual, you know, arms folded right near and you could be tapping and it
47:13SPEAKERwould be working.
47:15SPEAKERThat's how I want like Apple vision pro tap gestures to work.
47:19SPEAKERYou know, just to be clear, they have to be in line of sight of one of the cameras in
47:23SPEAKERorder to work.
47:24SPEAKERRight.
47:25SPEAKERYeah.
47:26SPEAKERYeah.
47:27SPEAKERYeah.
47:28SPEAKERIt has special down facing cameras just for that.
47:29SPEAKERAnd facial tracking.
47:30SPEAKERYeah, they do.
47:31SPEAKERBut occlusion is a problem.
47:32SPEAKERI think for that.
47:34SPEAKERAnd they're not asking you like, I wasn't dragging anything.
47:37SPEAKERFor example, I wasn't using like that for dragging actions.
47:40SPEAKERI believe that would be using computer vision and you would be using your hand put out in
47:44SPEAKERfront of you.
47:45SPEAKERSo it's this kind of multimodal input for tap and just your basic mouse clicks and think
47:51SPEAKERof those.
47:52SPEAKERThey want to be with these wristbands or mouse drags.
47:55SPEAKERThey want to be with computer vision.
47:57SPEAKERJust to be clear, is the term neural interface over selling this a little bit like it?
48:03SPEAKERIt sounds to me like it's reading the movements in your tendons and your muscles.
48:08SPEAKERIt's reading the electrical signal to the muscles, right?
48:11SPEAKERIt's electrical signals, not movements.
48:13SPEAKERWhen I hear neural interface, I think central nervous system, I'm thinking like matrix style
48:16SPEAKERjacks in the back of the head or Elon Musk killing a whole bunch of sheep.
48:22SPEAKERThis feels like this is the peripheral is less sexy, but probably easier.
48:27SPEAKERI think it's an important distinction though, Jeremy, that you're making because it's not
48:31SPEAKERa physical movement that they're detecting, but it's also not a thought they're detecting.
48:36SPEAKERIt's they're detecting the signal that comes when you trigger the physical movement.
48:40SPEAKERTrue, true, right?
48:41SPEAKERIt's not like if I think about tapping my fingers, that's what's been captured.
48:45SPEAKERIt is a physiological thing that they are detecting, but the physiological element is
48:51SPEAKERan electrical signal, not a physical movement.
48:56SPEAKERSo one of the things, you had videos in your tested video, which again, everybody should
49:02SPEAKERgo watch, that were like a person typing on a keyboard that didn't exist using these wristbands.
49:11SPEAKERWas that, that was from the demo reel that the company that they bought showed, right?
49:14SPEAKERThat's not something they, cause it wasn't in the keynote.
49:16SPEAKERNo, but it's, it's, it's on like meta research, like reality labs research has published over
49:23SPEAKERthe past four years things on this neural wristband.
49:28SPEAKERSo they talk about, you know, be able to recognize poses, you know, like using a bow and arrow
49:32SPEAKERand the act of pulling back a string.
49:34SPEAKERThey can, they can do that there.
49:36SPEAKERThere is a little bit of haptic feedback.
49:37SPEAKERThey've done all the research.
49:38SPEAKERThat's why I'm saying like, this is the most consumer ready thing.
49:42SPEAKERAnd it's not me just progress skating.
49:43SPEAKERLike they've said as much, yeah, it looks like a consumer product.
49:46SPEAKERYeah.
49:47SPEAKERYeah.
49:48SPEAKERZuckerberg and Bosner almost without revealing, like announcing a product, they're basically
49:52SPEAKERconfirming that, but we'll be out.
49:54SPEAKERI'm willing to bet next year, one year from today is a more expensive version of the Ray-Ban
50:01SPEAKERmeadows with a display in them and using the EMG wristband as input and display being good.
50:08SPEAKERGoogle the last, like off the side, not world track, just a feed of information display.
50:13SPEAKERDo you think that they would do that with quest too?
50:15SPEAKERIt seems like, I mean, the quest, they do hand tracking to have done hand tracking for
50:18SPEAKERa long time, but it's basically like, I don't have what I don't have a quest three, so I
50:22SPEAKERdon't know, but it's, it's like, it's like fingers and basic gesture and placement, right?
50:27SPEAKERIt's basically would be like, they'd be able to do better than Apple can do with the, with
50:31SPEAKERvision, you know, which is exactly what you're saying.
50:33SPEAKERYou wish that Apple would be able to pull off a meta has the capability to do that with
50:37SPEAKERquest using this device.
50:39SPEAKERI think it's going to be very telling whether it's on the roadmap that this is incorporated
50:45SPEAKERas an input option in quest, obviously different OS stacks.
50:49SPEAKERSo they'd, and there's interoperability, like a lot of the slam learnings and a lot of the
50:52SPEAKERstuff that they hand tracking stuff they did, I did on horizon OS and quest is what makes
50:57SPEAKEROrion hand tracking possible, but they're going to be separate OS developments.
51:02SPEAKERIf we do not see EMG as a possible optional input or their VR division, it actually gets
51:11SPEAKERme really concerned about how much investment they want to put in VR versus AR going forward.
51:18SPEAKERCause that was one of the things that Boz called out.
51:20SPEAKERHe had, and this seemed like a shift from previous statements, which was that he, they
51:24SPEAKERview AR and VR as two, that's not two sides of the same coin anymore.
51:29SPEAKERThat when you're taking over the world, you have a different set of criteria and strengths
51:34SPEAKERand weaknesses than when you're just looking through glasses.
51:38SPEAKERDo you mean the same meta as taking over the world?
51:41SPEAKERI meant taking over your sensorium, sorry, not taking over the world.
51:46SPEAKERYeah.
51:48SPEAKERAnd I know they're, you know, connect is the long history of their relationship and the
51:51SPEAKERacquisition of Oculus and, and it has been VR first.
51:55SPEAKERAnd so there is an, you know, micro ABrash was conspicuously absent for all of connect.
52:00SPEAKERRight.
52:01SPEAKERHe didn't do his talk this year.
52:03SPEAKERNo, he wasn't there.
52:04SPEAKERI mean, not that I saw him at all.
52:05SPEAKERSo I just had him on his own podcast and they had a great show.
52:09SPEAKERYes.
52:10SPEAKERThat was a really good, a really good conversation as well.
52:13SPEAKERWith the heavy emphasis on AI and certainly the positive feedback they've had on metal
52:18SPEAKERRay-Ban Meadows and now Orion, I don't think they're giving up on VR and every indication
52:25SPEAKERthat they've given in the past is that they'll believe it's the two track world where they'll
52:30SPEAKERhave a VR system.
52:31SPEAKERThat's great for that's your laptop analog or your home theater analog.
52:35SPEAKERThat's your high end device with high compute and still mixed reality.
52:40SPEAKERBut then the optical system is the equivalent of what people would spend on a phone system.
52:44SPEAKERLike that's my dream and that's my hope for them.
52:47SPEAKERI could see the shininess of VR really losing its luster and the shininess of the idea of
52:53SPEAKERa connected digital metaverse losing its luster and Zuckerberg saying, let's just focus on
52:58SPEAKERthis AR world.
52:59SPEAKERCause that's the best chance of replacing the phone.
53:02SPEAKERGen Z aside, when people got phones, we didn't throw out our TVs, right?
53:07SPEAKERYeah.
53:08SPEAKERYeah.
53:09SPEAKERBut you know, everything they're saying of like the opening up of Verizon OS for third
53:11SPEAKERparty manufacturers, the now app lab going away and letting more developers like the
53:18SPEAKERless control they're exerting over the quest ecosystem and quest platform.
53:24SPEAKERSometimes you could read that one reading of it is them saying, okay, let's let this
53:28SPEAKERbe someone else's problem.
53:30SPEAKERA big part of the problem with the, with the meta platforms is that you can only stream
53:35SPEAKERthose Ray-Ban glasses to Instagram.
53:36SPEAKERYou can't stream them to Twitch.
53:37SPEAKERYou can't stream them to YouTube or wherever else, because you know, that's not the way
53:41SPEAKERmeta works.
53:42SPEAKERSo okay.
53:43SPEAKERI like, I'm, I'm really excited about the wristbands.
53:47SPEAKERThe ones that they showed, it's funny cause the, the video that you embedded with the
53:50SPEAKERtypewriter had much bigger chunkier pre looking more prototype looking things.
53:55SPEAKERThe ones that you were wearing in the video and the demos of the, of the, of the glasses,
54:00SPEAKERthey looked like almost like a, like a cloth fitbit, right?
54:03SPEAKERThey were like an inch and a half wide and tight to your wrist and, but it looked comfortable
54:08SPEAKERand looked like a thing you would wear without even thinking about it.
54:11SPEAKERYeah.
54:12SPEAKERWhat the magnetic class, like they've clearly put industrial design into it.
54:16SPEAKERThe, if they want to sell this, if the software is ready and put it on store shelves, they
54:20SPEAKERcould do it in the next six months.
54:22SPEAKERAnd then the compute puck is kind of the boring part of the whole thing, right?
54:25SPEAKERIt's a, it's got wifi and Bluetooth and presumably some RMSOC that runs Android with a battery.
54:30SPEAKERExactly.
54:31SPEAKERYeah, exactly.
54:32SPEAKERAnd I guess, I guess my, so, so you came away from this, like I remember doing old VR demos
54:38SPEAKERand some of them you'd walk away from you like, boy, I'm glad that was over.
54:41SPEAKERAnd some of you would walk away from it and you'd be thinking and talking and we'd think
54:44SPEAKERand talk about it for literal months.
54:46SPEAKERAnd I wonder where this fits on the scale of that.
54:49SPEAKERYeah.
54:50SPEAKERThis feels to me like it was too, this is totally, I obviously haven't seen the thing,
54:54SPEAKERbut I feel like I'm surprised they showed this and I want to know why they showed it.
54:59SPEAKERHonestly, like I'd love to be inside the room where they decided to make this call.
55:03SPEAKERCause it's clearly like a, some sort of stake in the ground.
55:06SPEAKERLike this is, look, we, we are unveiling this kind of product that we're working on.
55:11SPEAKEREven if Apple, Google, maybe even Microsoft might have similar things in the works, but
55:17SPEAKERthey would never show something off that's this early.
55:19SPEAKERThey would wait until it's ready to be a product and there's no way this is going to be a product.
55:23SPEAKERLike there's too many things that are too expensive about it and unfinished.
55:27SPEAKERSo whatever ends up coming to the market is going to be substantially different.
55:33SPEAKERSo yeah, I'll, I'll, I'll tell you what I think is why, you know, they made the decision
55:38SPEAKERtwo years ago to not make this a product they've put in the estimates of $5 billion.
55:43SPEAKERAnd this, I think it's still kind of like, and wavy confirm that we're chatting with
55:47SPEAKERAlex Heath and that's a lot of money to, to, and they, and now AI is a thing that if they're,
55:54SPEAKERthey don't feel like they're all in AI investors are going to start getting freaked out.
55:57SPEAKERSo they've looked at, you know, maybe the past year since Apple showed their cards,
56:03SPEAKERit's a cat and mouse game, right?
56:04SPEAKERAnd Apple showed their cards with Apple vision pro.
56:07SPEAKERAnd so they know, okay, if Apple is going in with a real consumer product with Apple
56:09SPEAKERvision pro, they're not going to show optical AR for three or four years, but they see a
56:15SPEAKERmoment with the commercial success of Ray-Ban Metas to then pair that with the thing that
56:23SPEAKERthey've spent a ton of money in RD with, and then say, we have now a leapfrog moment, at
56:29SPEAKERleast in the understand for the media's understanding and as well as investors understanding of
56:34SPEAKERwhat we're doing to say bank on us, not them because of how, what the, you know, relatively
56:40SPEAKERlackluster response has been to the Apple vision pro.
56:44SPEAKERAnd they can look at, you know, headset sales because they, we have that option as well.
56:48SPEAKERWe have a $300 quest that we think is great.
56:50SPEAKERYou play all these games.
56:51SPEAKERYeah, they're coming from this like real nice position of strength.
56:55SPEAKERAt the same time, they're also trying to do a little bit of magic in front of you and
56:59SPEAKERsay, look at these 70 degree, you know, field of view AR glasses that will make smaller,
57:05SPEAKERbut reading between the lines, it's not going to be 70 degrees when those become a product.
57:11SPEAKERWalk out of the demo.
57:12SPEAKERThe thing I was most excited by was what the intermediate product in that spectrum is going
57:20SPEAKERto look like between Ray-Ban Metas and Orion, right?
57:24SPEAKERUsing your imagination.
57:25SPEAKERYeah.
57:26SPEAKERSo the thing next year, you know, if Ray-Ban Metas are 350 bucks, I would can imagine a
57:31SPEAKER$800 pair of glasses next year, you know, that will have Ray-Ban Meta form factor.
57:38SPEAKERThey're not Orion bulkiness be tethered to an app on your phone that will have some type
57:44SPEAKERof display, no world tracking, no slam, no computer vision, no hand tracking, but with
57:49SPEAKERthe EMG wristband that will be discreet and, and will be relatively socially acceptable.
57:54SPEAKERI think the world is ready for, you know, techies to be wearing that in the world and
57:59SPEAKERnot be shunned.
58:01SPEAKERThe question is, is the world ready to make the leap past that into something that really
58:07SPEAKERlooks like, you know, a piece of technology.
58:10SPEAKERAnd I also think like there's also a element of corporate wars here too, right?
58:16SPEAKERLike there's Google and Apple and Microsoft are also spending, maybe not Microsoft anymore,
58:21SPEAKERbut the other three companies are spending an enormous amount of money developing their
58:25SPEAKERown products in this space.
58:28SPEAKERAnd if they know that Silicon carbide is a dead end at this point, this is a hypothetical,
58:32SPEAKERnot something I actually believe or would have any expertise in, but if they know that
58:36SPEAKERSilicon carbide is a dead end, then they can encourage a whole lot of spending on stuff
58:41SPEAKERthat's not going to matter.
58:43SPEAKERYou know, if they feel like they're behind in the AI stuff, then this gives them an another
58:48SPEAKERfront in the, in the Corpo wars that Google and Apple and, and, and open AI and all the
58:54SPEAKERother places are going to spend money on.
58:56SPEAKERAnd that's a, that's a benefit for them because they have a deep war chest so they can spend
58:59SPEAKERmoney with reckless abandon.
59:02SPEAKERIf you look at the other companies in the space, Microsoft is not making hardware.
59:06SPEAKERThey just announced that there are, you know, mixed reality, windows mixed reality is going
59:11SPEAKERto be deprecated and going away with next version of windows.
59:15SPEAKERWhat's his name?
59:16SPEAKERWho's leading Microsoft hardware is gone.
59:18SPEAKERHe's running Amazon's devices division now.
59:24SPEAKERMicrosoft's business is in Azure and they were happy to be the backend that runs all
59:30SPEAKERthese, you know, model training programs.
59:33SPEAKERAnd they don't even have their own model, right?
59:35SPEAKERSo I think they're not going to be player in this Google.
59:37SPEAKERYes.
59:38SPEAKERCause they have their own AI Gemini and they have partners.
59:40SPEAKERAnd so the rumor is Samsung will have their meta Ray ban slash Orion competitor that will
59:46SPEAKERrun Gemini and maybe be tight, more tightly integrated or a Android devices.
59:52SPEAKERSo it's not like a second class citizen experience.
59:55SPEAKERThat's the one to look out for.
59:56SPEAKERThey've also quietly been spending billions of dollars building AR stuff since they shut
1:00:00SPEAKERdown their, their major VR initiative a few years ago.
1:00:03SPEAKERRight.
1:00:04SPEAKERAnd then Apple, like we still don't know what Apple's like is AVP.
1:00:09SPEAKERThe first, the first, the, the beta test product, is it the DK one of this world or is it the
1:00:14SPEAKERNewton?
1:00:15SPEAKERRight?
1:00:16SPEAKERWe don't know in their eyes.
1:00:18SPEAKERI think that Apple, I don't believe that AVP is the dev kit or optical glasses.
1:00:27SPEAKERAVP is a dev kit or pass through VR glasses with a more acceptable everyday wear form
1:00:34SPEAKERfactor for replacing the laptop or the Mac book and the iPad.
1:00:41SPEAKERI have no idea how far they might be along for optical classes, but just knowing how
1:00:46SPEAKERmuch processing you would need to get, to have a compelling, you know, actual world
1:00:53SPEAKERtrack persistent AR experience.
1:00:55SPEAKERLike that's still in the VR space for now.
1:00:59SPEAKEROkay.
1:01:00SPEAKERNorm, can you, what were the actual demos?
1:01:04SPEAKERYou didn't really talk about it a whole lot in your video, but I'm curious, like, were
1:01:07SPEAKERthere games you did some, you did some call stuff that you said was pretty compelling
1:01:11SPEAKERwith like virtual avatars.
1:01:12SPEAKERWhat, what can you just give us like quick hits on each of the things you saw?
1:01:16SPEAKERYeah.
1:01:17SPEAKERSo basic compute stuff.
1:01:19SPEAKERSo web browsing, watching a YouTube video.
1:01:22SPEAKERHow was the text rendering and the web browsing?
1:01:23SPEAKERIt's it's low res.
1:01:24SPEAKERI mean, this is 13 pixels per degree on the prototype I use though.
1:01:29SPEAKERIt's like, can you give us a comparison?
1:01:31SPEAKERYeah.
1:01:32SPEAKERIt's a, it's a, it's like rift one, rift one between rift one and a valve index.
1:01:37SPEAKERSo valve index, you can kind of read text in rift one.
1:01:40SPEAKERYou probably couldn't read text unless it was like oversized and chunky closer to valve
1:01:44SPEAKERindex than riff one.
1:01:45SPEAKEROkay.
1:01:46SPEAKERSo fuzzy texts everything a little bit Chloe and fuzzy video calls were, were decent.
1:01:53SPEAKERYou know, it's a rectangular window.
1:01:56SPEAKERWhen you're in a video call, they'll be using your either stylized avatar or codec avatar.
1:02:01SPEAKERSo that's all machine learned based on your voice.
1:02:04SPEAKERIt's not going to be the same as like vision pro or even quest pro tracking of your face
1:02:08SPEAKERbecause there are no face tracking cameras looking down.
1:02:12SPEAKERRight.
1:02:13SPEAKERIf you had to guess at the frame rate, I'm, it was stereographic, right?
1:02:16SPEAKERBut what was the frame rate pro seems, it seems like it'd probably be 60 like 60.
1:02:21SPEAKERIt felt more like 60 to 90.
1:02:22SPEAKERYep.
1:02:23SPEAKERYeah.
1:02:24SPEAKERIt was more, not about like latency for compute because it wasn't like how fast I could load
1:02:32SPEAKERinformation or first, but just like moving a window around and how fast things would
1:02:36SPEAKERload.
1:02:37SPEAKERIt felt like 60.
1:02:38SPEAKERAnd then what else?
1:02:40SPEAKERThere was one world tracking demo and this is that like looking at a bunch of items in
1:02:46SPEAKERfront of you, you know, asking about an AI to make a smoothie out of those ingredients
1:02:50SPEAKERand you look at the, the, the video, the B roll, those items are like clearly labeled
1:02:57SPEAKERlike a box of oats that says oats.
1:02:59SPEAKERThere were, yeah, there was text on all of them.
1:03:01SPEAKERIt was texts on the, exactly.
1:03:03SPEAKERIt was like the easiest machine learning operation to, but having done it twice, like the second
1:03:09SPEAKERtime it didn't get all the same objects as the first time.
1:03:11SPEAKERSo it wasn't on rails like they weren't programming it.
1:03:14SPEAKERIt was actually asking them at AI and the recipe was different each time.
1:03:18SPEAKERBut it was more impressive that the labels were floating on top of the objects and track
1:03:23SPEAKERon top of the objects themselves.
1:03:25SPEAKERThat was the more impressive thing.
1:03:27SPEAKERFirst off, I think that asking any large language model to make a recipe for objects in front
1:03:32SPEAKERof you is probably unwise and going to have bad mixed results at best.
1:03:37SPEAKERBut like, if you really wanted to impress me with that demo, you would have opened up
1:03:39SPEAKERa college fridge and been like, make me a Scalzi burrito with like the leftovers that
1:03:43SPEAKERare in here and figure it out.
1:03:44SPEAKERThat that's, that's, that's the cool version of that one.
1:03:47SPEAKERNone of that is cool.
1:03:48SPEAKERNone of these examples for like image based AI things I find more than just a tech demo.
1:03:56SPEAKERYou know, when, when Zuck did that Instagram video where he's like, you know, maybe I met
1:04:00SPEAKERus, tell me what to wear today, looking at his closet.
1:04:03SPEAKERNo one's really going to do that.
1:04:05SPEAKERNone of these are real things you do.
1:04:06SPEAKERPeople do in the real world.
1:04:07SPEAKERLook, man, you saw that jacket he was wearing at the keynote.
1:04:09SPEAKERI guarantee you he did that that morning.
1:04:12SPEAKERLike that is a hundred percent.
1:04:13SPEAKERHe asked the machine to tell what he should, how he should dress himself.
1:04:15SPEAKERThe Latin, even the stuff that Apple showed at WWDC about like using your iPhone camera
1:04:22SPEAKERto like look up a restaurant and look at the menu.
1:04:25SPEAKERLike these aren't things I think people are going to find more convenient than just Googling
1:04:30SPEAKERor using the basic, you know, text-based search on their phone.
1:04:35SPEAKEROn restaurants that you can just walk into without a reservation, they're going to have
1:04:38SPEAKERthe menu posted up on the wall outside the restaurant.
1:04:42SPEAKERExactly.
1:04:43SPEAKERThe translations bit that you just mentioned that that was another announcement that they
1:04:47SPEAKERmade for the Ray-Ban Metas.
1:04:49SPEAKERAnd yeah, live translation.
1:04:50SPEAKERI see that being useful.
1:04:52SPEAKERI question that a little bit because you have the problem with that while a cool tech demo
1:04:58SPEAKERand potentially very useful.
1:05:00SPEAKERYou need to have buy-in from both subjects, from both people engaging in that.
1:05:05SPEAKEROtherwise you're talking to me and I'm wearing my glasses and you don't, you don't, I don't
1:05:09SPEAKERspeak your language.
1:05:10SPEAKERI can't even explain to you what's happening.
1:05:12SPEAKERBut you have to know that I'm using my glasses to translate for me.
1:05:16SPEAKERAnd so that I think is something that it works on stage, but it won't be as practical in
1:05:21SPEAKERreality.
1:05:22SPEAKERThe universal gesture of like holding one finger up in the air, wait, point your glasses,
1:05:27SPEAKERwait, point your glasses, nod in your head, understanding.
1:05:32SPEAKERI think a phone is just more, you know, I think you're right there too.
1:05:37SPEAKERYou guys are thinking about this the wrong way.
1:05:39SPEAKERThey're picturing a world just like, just like the Zune experts that were like, Hey,
1:05:42SPEAKERwe're going to have squirt our files from Zune to Zune when we're on the bus so we can
1:05:46SPEAKERshare music we're listening to.
1:05:48SPEAKERThey're imagining the world where everyone has Ray-Ban glasses.
1:05:51SPEAKERSo you speak English and they speak Thai and it just works out.
1:05:55SPEAKERThat's right.
1:05:56SPEAKERYou're so right.
1:05:57SPEAKERYou are so right.
1:05:58SPEAKERAnd Zune squirt jokes will never get old.
1:06:00SPEAKERThank you for that.
1:06:01SPEAKERDo they have any games?
1:06:02SPEAKERNothing.
1:06:03SPEAKERThey showed some games.
1:06:04SPEAKEROkay.
1:06:05SPEAKERYeah.
1:06:06SPEAKERIt's called star gazer, which is kind of a, you know, eight bit shooter using a gaze and
1:06:11SPEAKERtap.
1:06:12SPEAKERSo it's, you know, it's, it's a last star fighter, right?
1:06:15SPEAKERYou're like, you're just looking like darting your eyes across and tapping with your hand.
1:06:20SPEAKERHow do you feel the view?
1:06:21SPEAKERAnd so it's like the most casual game of, of, you know, whack-a-mole.
1:06:25SPEAKERDid that feel fun?
1:06:26SPEAKERWas it fun?
1:06:27SPEAKERIt didn't look fun.
1:06:29SPEAKERIt was not taxing.
1:06:30SPEAKERThat was a point.
1:06:31SPEAKERLike I wasn't like waving my hands up or using a game pad.
1:06:34SPEAKERIt was like, you know, dart your eyes and whack the spaceships with lasers.
1:06:39SPEAKERBut the Pong one, the Pong one, which was both co-location as well as a remote play
1:06:43SPEAKERexperience.
1:06:45SPEAKERThat was the best use of stereo graphics because there was some depth there.
1:06:49SPEAKERThese two, like there's volumetric as well as the other player being completely visible,
1:06:56SPEAKERnot clear.
1:06:57SPEAKERThey were in glasses.
1:06:58SPEAKERSo I'm looking at another player there.
1:07:00SPEAKERWe're both wearing orions.
1:07:01SPEAKERWe both see the same Pong volumetric play area and we're talking to each other while
1:07:08SPEAKERplaying at the same time.
1:07:09SPEAKERSo you see the real person and the, and the spatialized virtual area, and they're in the
1:07:13SPEAKERsame place.
1:07:15SPEAKERAnd they are perfectly world locked and we're looking at the same thing and our attention
1:07:18SPEAKERis looking at the same thing.
1:07:21SPEAKERAnd so while the game was simple, it was like the holograms were real for us.
1:07:27SPEAKERIt's like that old 3d flash Pong game from back in the day, that was a, that was a banger.
1:07:33SPEAKERWhat what and was the performance good enough for like where the ball's moving fast or was
1:07:37SPEAKERit Pong speed?
1:07:38SPEAKERPong speed, a little bit of spin, you know, it was hand tracked, you know, you're waving
1:07:44SPEAKERyour hands out to spin the ball as it hit a virtual paddle.
1:07:48SPEAKERWas there any indication how many people can currently watch you guys play that using the
1:07:54SPEAKERglasses themselves?
1:07:56SPEAKERNo, no indication.
1:07:57SPEAKERAnd a lot of it was, if you look at the demo room, there's like this one demo room that
1:08:02SPEAKERall the demos are being done in on the left side of the room, which we were asked not
1:08:05SPEAKERto film is a bank of computers and technicians like monitoring and rebooting as needed.
1:08:11SPEAKERIt's such a scripted demo, a 100% it's such a scripted demo that, you know, they walk
1:08:15SPEAKERyou through the exact same sequence of these, you know, four to six things.
1:08:19SPEAKERAnd when you get to the point where you're sitting across from the person giving you
1:08:22SPEAKERthe demo at this eyeball table, this like this cocktail table, they're looking at a
1:08:29SPEAKERsmall monitor, the bottom next to the couch where they can see the what you're seeing
1:08:35SPEAKERso they can guide you through the demo.
1:08:37SPEAKERSo it was entirely scripted in terms of the progression of events.
1:08:42SPEAKERDid they say so that Pong thing is the first thing I think you've talked about that your
1:08:46SPEAKERhand position mattered more than like your finger movements.
1:08:49SPEAKERAre they tracking your hand with the SLAM cameras on the, on the glasses?
1:08:53SPEAKERAre they okay?
1:08:54SPEAKERThat makes sense.
1:08:55SPEAKERYeah.
1:08:56SPEAKEREye tracking, hand tracking, voice tracking, and the gesture input.
1:09:00SPEAKERDid they say how it's like, how are they charging the, I assume the dev kits, they just have
1:09:03SPEAKERlike dedicated chargers for, but that is the idea that in the real world, you'll have like
1:09:07SPEAKERa glasses case.
1:09:08SPEAKERYou put them in and, and everything charges up or we don't know yet.
1:09:11SPEAKERI hope USB, but you know, I don't know what takes more volume contactless charging docs
1:09:18SPEAKERor a USB port.
1:09:19SPEAKERThe Ray ban that is required putting in the case to charge, which I don't like contactless
1:09:23SPEAKERinside the case.
1:09:24SPEAKERRight.
1:09:25SPEAKERIs it like a key or something?
1:09:26SPEAKERIt snaps in there.
1:09:27SPEAKEROh, okay.
1:09:28SPEAKERThere there's just two, two touch points, you know, for yep.
1:09:30SPEAKERFor power ground and power.
1:09:33SPEAKERAnd then there's a battery in the case that you can charge off of when you're not.
1:09:35SPEAKERYeah.
1:09:36SPEAKERThat makes sense.
1:09:37SPEAKERHow long do they take to charge out of curiosity?
1:09:39SPEAKERThe existing ones.
1:09:40SPEAKEROh, because it's like AirPods, we take them off for 10 minutes and they're back to, but
1:09:45SPEAKERyou're back.
1:09:46SPEAKERGood to go.
1:09:47SPEAKERIt's not that fast charging.
1:09:48SPEAKEROkay.
1:09:49SPEAKERYeah.
1:09:50SPEAKERI think that's it for me.
1:09:51SPEAKERJeremy, you got any more questions for, for our guy norm?
1:09:54SPEAKERWhat'd you think of the quest S oh, the three S yeah, three S this is the new cheap quest
1:10:00SPEAKER$300 basically the quest quest three internals quest to optics.
1:10:06SPEAKERI think that if you can afford to spend a hundred more dollars, you could probably get
1:10:11SPEAKERa quest three refurbish or use right 128 gig model.
1:10:17SPEAKERAnd I would go with that because they stopped selling the two.
1:10:21SPEAKERThey stopped selling it to you.
1:10:22SPEAKERThis replaces the two in the marketplace though, but it has really good mixed reality, really
1:10:27SPEAKERgood pass through the, the world pass through is the same cameras as the quest three pastor,
1:10:33SPEAKERbut it's just using for now lenses and it's not even the resolution of the display.
1:10:37SPEAKERI'm fine with the quest two resolution.
1:10:39SPEAKERIt's the thickness of the lenses and the whole thing is just not having that pancake.
1:10:42SPEAKERYeah, we all know that the higher, the higher costs or price skews are the higher you know,
1:10:50SPEAKERthey make more profit on those.
1:10:52SPEAKERSo they, they brought the $650 skew down to 500 bucks for the quest three.
1:10:57SPEAKERYou would have thought, hopefully they could have brought that $500 skew back down to maybe
1:11:02SPEAKER400 and eliminated the lower tier line, but now they have this whole new thing.
1:11:07SPEAKERNow I it's just oddly fragmented and I'm so tired of people being sold a worse VR experience
1:11:14SPEAKERwhen even the top premium is not enough to hook people as much as it needs to.
1:11:20SPEAKERSo I, that, that's my takeaway is I just, I wish they'd stop selling cheap experiences
1:11:26SPEAKERjust to get people in because they are not sticky enough.
1:11:30SPEAKERAnd if this is going to ever take off, they need to make great experiences.
1:11:33SPEAKERAt the same time, the fact that they made a low end headset implies to me that they're
1:11:36SPEAKERnot going to give up on VR.
1:11:37SPEAKERTrue.
1:11:38SPEAKERNo, a hundred percent.
1:11:39SPEAKERYeah.
1:11:40SPEAKERZuck said he was extremely excited for the three S so, and he said it in a way that seemed
1:11:44SPEAKERauthentic.
1:11:45SPEAKERSo I think you're right.
1:11:46SPEAKERAnd even at 300 bucks, you're talking about retailers.
1:11:48SPEAKERWhat's the wholesale price, right?
1:11:49SPEAKERTargets can be buying these and Best Buy is going to be buying these from meta or like
1:11:54SPEAKER20% less than that, right?
1:11:56SPEAKERTwo to 20 or 250 or whatever it is.
1:11:59SPEAKERAnd so at some point they're going to be sales for, you know, retailer sales where people
1:12:03SPEAKERget 10% off, 15% off here.
1:12:06SPEAKERAnd if people are building up credit and buying sales and be able to buy not MSRP pricing,
1:12:11SPEAKERbut pay 250 bucks for a three S that's an incredible value.
1:12:15SPEAKERThat's how you kind of have to think about it.
1:12:16SPEAKERLike not just the MSRP value of these things.
1:12:19SPEAKERI just hope that that, that they fall in love with it, you know, as someone who's owned
1:12:23SPEAKERalmost every VR headset, I keep waiting for one that I think is going to break through
1:12:28SPEAKERand we're still not there.
1:12:30SPEAKER- I mean, I think, I think we've reached the thing that's interesting to me about the VR
1:12:35SPEAKERmarket now is that there's like four or five different silos of people, right?
1:12:39SPEAKERAnd it doesn't include core gamers really, right?
1:12:42SPEAKERIt's like the Beat Saber and fitness people.
1:12:45SPEAKERThere's people who do conferencing.
1:12:47SPEAKERThere's there's, there's people who just love VR stuff.
1:12:50SPEAKERI think you're in that category, probably Jeremy.
1:12:53SPEAKERAnd the rest of it is like it, the, the, the people playing games on their PS5 just kind
1:12:58SPEAKERof like we'd got pushed off to the side and, and aren't engaging in these, in these games.
1:13:04SPEAKERIt makes me really wish that instead of spending hundreds of millions of dollars making console
1:13:08SPEAKERgames for the first VR headsets they sold, they had spent that hundred, those hundreds
1:13:13SPEAKERof millions of dollars making VR games that are, and VR experiences that couldn't happen
1:13:18SPEAKERin other platforms.
1:13:19SPEAKER- Well, I think media too, right, there are going to be some people who buy these headsets
1:13:23SPEAKERthat don't want the cool presence of, social presence of going, playing Walkabout mini
1:13:29SPEAKERgolf or going in a rec room and just want to watch a big screen version of Lord of the
1:13:35SPEAKERRings and Amazon prime or VR 180 videos.
1:13:39SPEAKER- And I think the resolution bump, I think the resolution bump from quest two to quest
1:13:42SPEAKERthree, while on paper doesn't look like that much, it does cross a threshold where text
1:13:47SPEAKERbecomes legible and things, things are sharp.
1:13:49SPEAKER- And I think it's the lenses make a big difference too.
1:13:52SPEAKERIt was a noticeable, noticeable improvement to me when I've used them.
1:13:55SPEAKERNorm, how do, how do prescription lenses work on the Orion glasses?
1:13:59SPEAKERDid they say?
1:14:00SPEAKER- Yeah, so it won't be a snap on thing.
1:14:03SPEAKERIt'll be something that when you order them eventually, whatever product they'll be, you'll
1:14:08SPEAKERsend them your prescription and in the assembly process they'll have in the last layer of
1:14:11SPEAKERthe optical stack RX built in.
1:14:14SPEAKER- Okay.
1:14:15SPEAKERThat's nice.
1:14:16SPEAKERAnd, and like all ranges of prescription or is this like, okay, of course that makes sense.
1:14:22SPEAKERThis is fantastic.
1:14:23SPEAKERThank you guys so much for coming by.
1:14:24SPEAKERI really appreciate it and it was lovely to chat about this.
1:14:27SPEAKERI'm stoked to get to talk to somebody who spent hands on time with this because it seems
1:14:30SPEAKERlike they didn't give very many people access judging by the videos that are out on YouTube.
1:14:34SPEAKERNorm, where can people find you?
1:14:40SPEAKERWhere can people, you know, what, what, this is the part of the show where we plug stuff.
1:14:43SPEAKERSo
1:14:44SPEAKER- You know, just on, on tested youtube.com/tested.
1:14:47SPEAKER- Jeremy?
1:14:49SPEAKER- I have disappeared from social media, but I am at Jerware.
1:14:54SPEAKERIf you ever want to ping me, I usually go back and check a few times a month and you
1:14:59SPEAKERcan basically follow my, my, my career at digital eclipse now, which we're making video
1:15:06SPEAKERgames.
1:15:07SPEAKERWe're coming out with power Rangers this fall at some point this year.
1:15:10SPEAKERAnd we're very excited about it.
1:15:11SPEAKERBeen working on it for a couple of years.
1:15:13SPEAKER- I was stoked that there was just a content drop on the Atari 50th thing a few weeks ago,
1:15:18SPEAKERright?
1:15:19SPEAKERLike it seems like you made one of my favorite things of last year in that, by the way, I
1:15:23SPEAKERthink I shouted out on the show before, but the, but the kind of retro battle zone slash
1:15:31SPEAKERvector game inspired a game that you made for that is dope.
1:15:35SPEAKERIt's really good.
1:15:36SPEAKER- Thanks.
1:15:37SPEAKERThanks brother.
1:15:38SPEAKERVector sector.
1:15:39SPEAKERYeah.
1:15:40SPEAKERThere's a free update to that in the patch.
1:15:41SPEAKERAnd then there's the DLC that you mentioned that just came out for dry 50.
1:15:44SPEAKER- Thanks again to Norman Chan and Jeremy Williams for coming by.
1:15:46SPEAKERMake sure you go check out norms video over at a tested as YouTube channel that's youtube.com/tested.
1:15:53SPEAKERAnd we've reached the part of the show where we thank everybody for listening, but especially
1:15:57SPEAKERour patrons as always, Brad and Will made a tech pod is a listener supported show.
1:16:02SPEAKERWe literally would not be here without you, the listener.
1:16:05SPEAKERSo if you would like to support the show, you can go to patrion.com/tech pod.
1:16:09SPEAKERAgain it's patrion.com/tech pod.
1:16:11SPEAKERWe put that we put this at the end so we don't bug people at the beginning, but maybe that's
1:16:14SPEAKERa mistake.
1:16:15SPEAKERSo we appreciate the support for the show.
1:16:18SPEAKERBrad is going to be out for the next week or probably two, depending on when he gets
1:16:21SPEAKERpower back down in North Carolina, but I'm going to hold it down.
1:16:24SPEAKERI got another exciting guests lined up for next week.
1:16:27SPEAKERIn the meantime if you want to join the, the, the patron, if you want to join the community,
1:16:32SPEAKERyou can hop in a patrion.com/tech pod and for five bucks a month to get access to the
1:16:36SPEAKERdiscord, you get the patron exclusive episode.
1:16:39SPEAKERYou get access to a wonderful community full of beautiful nerds, just like you.
1:16:44SPEAKERAnd uh, we want to thank everybody who supports the show, uh, but especially our executive
1:16:48SPEAKERproducer tier patrons, including Andrew Slosky, Jordan Lippett, Bunny Fiend, Twinkle Twinkie,
1:16:54SPEAKERDavid Allen, James Kamik, and Pantheon makers of the HS3 high speed 3d printer.
1:17:01SPEAKERThank you all so, so much.
1:17:02SPEAKERI hope you have a lovely, lovely, lovely week and we will see you next week with another
1:17:08SPEAKERedition of the tech pod.
1:17:10SPEAKERSee you then.