Office Hours 2024-09-09 - Lightcraft Pro

Transcript

<!DOCTYPE html>

<head>

<title>Office Hours 2024-09-09</title>

</head>

<body>

<h1>Office Hours 2024-09-09</h1>

[00:00:00] Eliot: Hello.

[00:00:06] Joe: Hey, good to see you. Hold on, let me just keep getting that going.

All right, much better.

[00:00:20] Eliot: How you doing today? Good, good. What

[00:00:24] Joe: do we

[00:00:24] Eliot: got?

[00:00:25] Joe: So, uh, I got my Ethernet, uh, adapter, so I was able to get everything hard lined in. So, And I think it's good. So I kind of just wanted to check that and make sure that was, that's all the problem. And then I kind of just wanted to run a test of framing up a shot, shooting a shot, and then getting it to sink because like the movement exactly has been a little funky.

So I just want to make sure I'm not. So I figured that would be like a controlled thing. Yeah, that sounds good. Let's do it. Um, Yeah, sweet. So, okay. Um, and you've got my, let me see, I got my monitor up here. I can share that, but

[00:01:05] Eliot: that did the network seem like it got most of the, the, the hard lining get most definitely

[00:01:11] Joe: looked much more, uh, like rock solid from like the bouncing, like, I, I definitely think it solved that.

And yeah, cause I've got it wired into the phone. And then for a second there, I got a little, I was worried. Cause the power, so on that dongle from the Axiom, it won't power it. At least for me, it doesn't. So it's like, I have that Belkin adapter and I got the a hundred watt charger one. So it's like when I.

I used to be able to plug the Axiom directly into the phone and then it would get power. But so with this one, what I had to do was just sort of like, you know, always, I just had to re I put an extra battery pack here and then. I have that going into the Belkin to charge it. So,

[00:02:04] Eliot: and then that charges the phone while we're running on the network.

[00:02:06] Joe: Correct. That's running on the network. So I was a little concerned there about that, but that was at least a work around, you know, for now, makes

[00:02:14] Eliot: sense.

[00:02:16] Joe: And, uh, but just so that anybody else, if anybody else is watching and they stumble into that, um, so let me do a little sink, sink all this here. Okay. Time code sinking.

All right. That's

[00:02:31] Eliot: matching. Actually. Can you, if you can send me the link to the, actually, let me just look for the Belkin and I'll put it at that on our recommended kind of pieces. So Belkin?

[00:02:41] Joe: Yeah, the Belkin Ethernet. Uh, hundred watt adapter.

[00:02:46] Eliot: All right. Belkin, USB to ethernet.

[00:02:53] Joe: All right. Yeah, mine's the USBC one. So

[00:03:04] Eliot: connect uss BC to ethernet and post adapter. All right, so's. Let's see. Let's see. Uh,

[00:03:14] Joe: alright. Okay, so now we come. So I'm opening this up. And I, like, re, I kind of, like, just redid it again from scratch just to, like, get it in my bones.

[00:03:31] Speaker: Sure,

[00:03:31] Joe: sure. So I did it here, so I think it should all still be working. And it was good, because it didn't take me very long.

Like, once you kind of go through, um, you know, everything that you were talking about, then it's like, you know, stuff's pretty, uh, stuff's pretty fast.

[00:03:47] Eliot: Yeah. Yeah. Once there's once you understand there's a logical set of things that need to get flipped, it took a few round trips to, Oh yeah.

[00:03:56] Joe: Um, okay. There we go.

Um, yeah, I've been trying to sneak. Okay. So, all right, let me get rid of this guy. Okay. So now. Let's re toggle that, re toggle timecode.

[00:04:15] Eliot: Now before you, before the timecode is going to work, you're going to need to get the, uh, the video, the live video input. Oh, right. Yeah. Up and running.

[00:04:23] Joe: Correct. So that should be pretty straightforward.

Let's see, usually just one of these will do it.

Oh, right. I have to have this open.

[00:04:48] Speaker: Nope.

[00:04:49] Joe: It

[00:04:51] Eliot: always does this. Okay, let's see it. Now which, which input do you have it on? I have

[00:04:58] Joe: it. Um, I have it on Declan four UHD, progressive 24. Got it. All right. Which is right, which it should mon usually, oh, here's four. I have to do this. I always forget on my camera. Sometimes the SDI thing, I have to,

and then hopefully that does it

on.

[00:05:45] Eliot: I also, I'll also say that trying to push the engine at, at 4K is usually you hit some limits, .

[00:05:53] Joe: Oh. Even with the video input.

[00:05:55] Eliot: Yeah, I mean, I would, um, I would err on the side of, even if you're going to be recording inside the Blackmagic, inside the, um, camera at 4k, can you set your, oh, there we go. Now we've got, oh,

[00:06:08] Joe: okay.

So I, so it doesn't matter. I can still record in 4K and then use it in here in 10 days. Oh,

[00:06:14] Eliot: amazing. Yeah. Yeah. Oh, that's so much

[00:06:16] Joe: better.

[00:06:16] Eliot: I, I think you can do that. I, I'm, I'm fairly certain you can, and I, that'll, that'll make your life a lot easier because UHC is a lot of pixels, . Yeah. A lot of that's.

[00:06:26] Joe: No, that was really good here and I'm going to put my output from the camera to 1080.

Hopefully that so

[00:06:32] Eliot: it's yeah, you're going to need to reset your configuration in there. Uh, because right now, okay, there we go. It's 1080, 1080p 24. Okay. Um,

[00:06:41] Joe: let's see one where I need to do you

[00:06:44] Eliot: again. 1080. There we go.

[00:06:46] Joe: Ah, why are you doing that? Okay. That's interesting.

Really? Are you going to just, just, ah, interesting. That's weird.

[00:07:01] Eliot: Me, me, me. That's the, that's the first thing we need. Everything else gets downhill from, uh, is kind of falls outta that, that live input from the media. Um,

[00:07:12] Joe: let me try output.

Okay. Is that cool? Okay. I guess for some reason when I output 10 80 from the camera, it doesn't like it, but I out, I'm, now, I'm outputting 4K from the camera, but it's in, and it's coming in at 10 80, so I guess it's working. Whatever, .

[00:07:33] Eliot: Well, actually, but we, we really do wanna understand that 'cause it's gonna bite us.

Because it, it's, oh, okay. It's really persnickety about having, 'cause this is where all the timing is gonna come in from. So we, so it's,

[00:07:44] Joe: it's 4K 24 or 10 80 24. That's what it can output. So the, so the time, so the frame rate's still the same, it's just the resolution that's different.

[00:07:52] Eliot: Yeah. Frame rate's is going to be the, the same.

Um, but in terms of it coming in from the SDI signal, let's get the, it's a very precise way. SDI will bring, bring in the frame buffers and then get the information from that. So,

[00:08:05] Joe: okay. So I just put it back to 10 80. Let's hope, are you going to work

[00:08:09] Speaker: now?

[00:08:10] Joe: Yeah. See, that's funny. It goes away like that. Oh, that's probably because I have it coming in at, uh, there we go.

[00:08:22] Eliot: Actually right now you've got it on video output. Um,

[00:08:27] Joe: Oh, right. Okay. Yeah. Let me change the connectors. Oh, okay. So I guess it just takes it. Video in. Yeah. Okay. That was what I just wanted to see. That was the right now

[00:08:40] Eliot: you've got an output format as to his 2160. Um, just let's take a look at that black magic thing again.

Oh no, no. Okay. That it changed the output format. Okay. Okay.

[00:08:51] Joe: So yeah, that looks like it should be. Okay. So now let's try switching you again and seeing if you stick. No, that's interesting.

[00:09:07] Eliot: Cause that's where every, every other, okay. Like all the custom time steps and the timecode and stuff are derived from that, that media, that media input. I, I really want to understand what, what it's doing

[00:09:21] Joe: before we

[00:09:22] Eliot: extend further out. Oh,

[00:09:24] Joe: okay. That, okay. So that's probably why, hold on. I need to go to my, all the other time step and those there to, um,

[00:09:38] Eliot: Yeah, we need to do the time

[00:09:39] Joe: steps.

Oh, no, that was in media. Minecraft. So now let's change these. Yeah, that's, there we go. So now we'll make that right now.

[00:09:49] Eliot: It's on deck link. Well, 4, 1080, progressive, 24, bit C, fly, K. Sounds good. Make sure you save, compile and save that guy. And save. Oh,

[00:10:02] Joe: yeah. 4. Okay. Save.

[00:10:05] Eliot: Okay. Time step. Time step. 4, 1080, K.

[00:10:17] Joe: Save that guy.

Okay. So now, let's see. Are you doing the same? Nope. Okay. Good. So that's sticking to 1080. Okay. So now, there we go. Let's see. There we go. Beautiful. Okay. Okay. Okay. So that's coming in at 10 80.

[00:10:37] Eliot: That's, that's interesting. So I didn't realize that we would have to actually have all those working before the media input's even gonna work.

Okay. That she's, this stuff is coupled out, right? ? Yeah. Okay. That's a, that's actually a fantastic thing to understand. Yeah, no, I'm

[00:10:54] Joe: glad I thought about it. I was like, wait a minute, what about the timecode? So that's why it's been good for me to like, do it a couple times, you know what I mean? Right, right.

That's so I understand the built, the, I'm trying to get it in my bones. Um, okay, so now we should be able to come to Alto Schultz. Let me turn Jet Set on, that might be good.

Okay, now. And I guess apparently I tried loading, um, 5. 4, but I guess like USD support isn't ready with 5. 4. So it's like, I wasn't really able to do anything.

[00:11:36] Eliot: I don't think NVIDIA has the omniverse connectors done that there is a USD. It's, it's called USD import. It's a plugin in, uh, in Unreal that that's, that's their default USD exporter, and you can try it.

I just have had choppy results with it. You know, sometimes it works simple. It won't, won't even gimme an

[00:11:51] Joe: option. It doesn't even have it as an export option in 5.4 I'll show you. And I think that's because I look back at Omniverse and Omniverse doesn't, they don't have a 5.4 kit yet for that. It only goes up to, I only saw they only have up to 5.3, so I guess they're just getting around to it, so,

[00:12:07] Speaker: okay.

Okay. But

[00:12:08] Joe: no big deal. Five three's. Totally, totally good. Um, okay, so now we've got, we're synced here. So now, I mean, I should, let's see. Okay. So I think everything is okay. Let's let's redo you and you. Okay. That. Okay, cool. Time code. Everything seems to be matching good. Okay. The camera jet set camera is not moving.

[00:12:37] Eliot: So let's see. Is jet set, uh, sending, sending data. Um, let's see. So we would want to go

[00:12:44] Joe: to the vive legs. No,

[00:12:48] Eliot: nope. There's your problem.

[00:12:49] Joe: Um, so let's, uh, see what's going on.

[00:13:02] Eliot: Actually, let's go back to the settings page for a second. Uh, or the, I'm sorry. The, uh, the, the base, um, the, uh, I'm sorry. The, the digital page. Oh, sure. Yeah, that was

[00:13:15] Joe: one thing that I noticed you guys caught was the time code doesn't match.

[00:13:20] Eliot: Yeah, that's where, yeah, that, that's, that's a bug. Um, I think we're, we're close to getting that one, fix it.

One of the upcoming ones, but okay. So at least the time code is moving. Um, okay. Is

[00:13:32] Joe: this supposed to be right? Or is this also part of the bug?

[00:13:37] Eliot: Actually? Oh, that's actually something we should, we should correct. So that's what, that's what JetSets currently set to is 2160 P and, uh, that pushes the phone crazy hard.

Okay. Um, cause right now you're, you, you don't, and so when you're shooting Jet Set Cine and the, you, uh, you actually don't cause the Cine camera is going to have a high resolution, right? And this is the Jet Set setting. So you actually don't need, yeah, it's for the phone. So, and we should make that a little more clear.

I got to come up with a way to. Do that. Um, no, this is, I'm

[00:14:09] Joe: loving that. I don't have to be stressing the machine. Like this is all just going to make it much more reliable. The fact that I don't have to push 4k through this. So I am very, very happy about it. So let's go,

[00:14:21] Eliot: go into the jet sets, the main settings, the recording settings in the main menu and jet set.

And let's drop it down to like 10, 10 ADP or seven 20 or something. Cause that's in the camera,

[00:14:31] Joe: right? Is that in the camera or the Cine calibration?

[00:14:33] Eliot: Uh, it's in, um, let's see. It's in the main menu and let me check on mine. Okay, I changed it. So now it says 30 1080p 30. There we go. Yeah, there we go. That'll buy us some, uh, some, uh, some bandwidth on the phone.

Yeah.

[00:14:48] Speaker: Comp

[00:14:49] Eliot: 540p. Sure. That's fine. Uh, cause again with the Cine, Cine version, all the, the, the big pixels are being recorded under Cine camera. Uh, and so we can preserve the phone's, uh, GPU cycles for other things. That'll, this'll help. Um, okay. And then let's click on settings and let's make sure, uh, Oh, you know what?

I know what, what happened when you plug into your ethernet, you change your DHCP address. So let's, uh,

[00:15:12] Joe: so I changed that. Cause it's true. It did do that. So I did already change it here. I updated that

[00:15:20] Eliot: and click set to this PC. Let's just make sure we're doing the same there again. Okay. And copy that copy of the external tracking destination as usual.

Let's just, like, go through the process.

[00:15:30] Joe: Sure. Oh, yeah, see, Jet Set's back. Okay,

[00:15:33] Eliot: hey, there we go. That's a good thing. Yeah, because

[00:15:34] Joe: I, like, I did, I, I had done that before, so I was like, it should have saved that. Um, but yeah, that's a good call. So, let's see. So, Jet Set's back, baby. Um.

[00:15:43] Eliot: All right.

[00:15:44] Joe: Although, is it not?

Oh, that's why. Um, oops. Don't want to do that. Uh, that's because I need to set my track part. Boom, there we go. Um, isn't that?

Okay. That's interesting. Yeah. That looks nice. Um, Oh, that's why let me reset the origin points

and now it should be higher. Yeah, there we go. Great. Cause I was like, that's not matching. Okay. Perfect. I love it, man. Your guy's stuff, it's making sense. You know what I mean? I like it. Oh, good. This is good. All right. Cool. So now that looks pretty good. Um, let's check our time data and see how we're doing.

Yeah.

[00:16:35] Eliot: Ah, looks stable. Okay, now we're talking now. Now we are talking now. Okay, now we can . That is a wild . Okay. So I think the

[00:16:46] Joe: 4K was a big, I think that was a big part.

[00:16:49] Eliot: Yeah, I'm really glad.

[00:16:50] Joe: Thank you for that. Um, that was, I'm gonna make a

[00:16:52] Eliot: note of, um, okay. So actually, so let's see. So jet set, uh, jet set cam, uh,

[00:17:02] Joe: because that's probably, that could have been a reason why it wasn't getting synced.

footage when I was trying to shoot the test. It was probably like a little late in your buddy. So this is really how, okay, that's, that's a big one.

[00:17:15] Eliot: And 21, 21, 16 feet. Okay. I'm going to, I'm going to put a couple, we need to put something in the UI to warrant because, because otherwise you would inherently think, Hey, I'm shooting forecast should do it.

I said, absolutely. The natural instinct is to

[00:17:30] Joe: match settings.

[00:17:31] Eliot: So,

[00:17:31] Joe: um, but yeah, I think just a note saying, you know, if you're using a cine camera. You know, don't, don't worry about that. You know, something like that. So you guys will,

[00:17:40] Eliot: um, because it, it literally smokes the GGPU by a factor of four. Yeah, exactly.

And it's a solution. Exactly. You know, we're, we're counting cycles. Um, okay, let's, uh, let's go ahead and click calibrate. And so let's line those, those, the, those pieces of data up. And, um, it's gonna ponder that for a little bit. All right. That looks, that looks good. And then you can pull a key and let's, let's see how we're, uh, uh, let's see how we're, how we're lining This looks.

Head and shoulders above where we were.

[00:18:08] Joe: Oh, yeah, absolutely. Um, okay, so for the key Oh, I didn't apply the key on this one. Um, I forgot how to, uh, what was that? No, just go ahead

[00:18:18] Eliot: and click the, uh, the gri yep, that's the one. And then go, let's go down to, uh, the transform compositing passes and details. There we go.

Transform passes and open up Chroma Keying. And let's see, we're going to want to view, um,

[00:18:35] Joe: do I need to duplicate this, this chroma material?

[00:18:39] Eliot: Oh, and actually hang on for just a second. So we're not getting our, we have the live data feed going into that guy. So what we want is new input. I just, let me look at this, uh, a couple of warnings there.

Time code. Okay. Time code. Okay. Okay. Let's, let's click clear. And, uh, let's, uh, let's, uh, calibrate again. Let's just see where we're at. There we go.

[00:19:11] Joe: Yeah. Right. Cause now that calibration, you made it worse, I guess.

[00:19:14] Eliot: Well, it was, uh, instead of evaluating in a time code, we had it evaluating in seconds.

[00:19:21] Joe: Ah, okay.

[00:19:22] Eliot: Okay, wait. No interval can be found. Uh, no. The buffer size required for input should be enough.

Ah, okay.

[00:19:37] Joe: Do we need to increase this?

[00:19:41] Eliot: Let's see. So they're both at the same time code. Um, Let us, okay, so let's, let's crack one thing at a time. Let's, let's get the, uh, the display going through the video display, going through the Jet Set Cine, um, that, that green, the composure element. So, uh, let's, let's click on the composure element, uh, up in the, uh, outliner panel,

[00:20:05] Joe: outliner panel.

[00:20:06] Eliot: Yeah. And I got to remember exactly how we, uh, how we set that. Um, Oh, I see under composure. Let's do the, go down to, in the details is open up the input. There we go. Oops, uh, input spinner. It's, uh, that's the guy. And open up the input. And media source. And, uh, for media source, let's, uh, click on that instead of, uh, LC media texture.

There should be, like, uh, yeah, there we go. That's it. Okay.

[00:20:35] Joe: Okay.

[00:20:36] Eliot: So now we've got a comp. So if you click on the under Jetset, under the, uh, Jetset Cine shot, uh, one above. Uh, And we should, there we go. So we should, that's kind of weird looking. Um,

uh, Oh, I know what's going on. You have your, um, uh, see that, uh, your media element is visible by your camera. So you don't want to grab your media element and like just move it somewhere out of, out of frame.

[00:21:04] Joe: Yeah. I just, I just, like, if I do that, then it just goes away. I just.

[00:21:08] Eliot: Okay. That works. So now what happens when we pan back and forth?

We should, it's probably gonna work mostly, but jitter. Okay. Hey, okay. Okay. So we're, we're coming in, coming in the home stretch. All right. So now we've got them, got them there. So let us add a global frame offset of let's try plus two. And I, I'm guessing here on a little bit, cause I, I, and let's click calibrate.

Uh, cause I, I want to get it to the point where it sees both of the signals and it can line them up. And I, I think we're close. Okay. But we're also in an area that I'm, I'm the most fuzzy about with how Unreal thinks on this stuff.

[00:21:46] Joe: What is

[00:21:47] Speaker: this?

[00:21:48] Eliot: I know interval could be found. Uh, buffer size is required for black magic should be enough.

Okay. Cause of course it's a black magic. And then what I, what I tested on everything was an AJA. Um, right. Okay.

Cause we're hitting a buffer underflow on the black magic. So how this is going to work. So, okay. What happens when we pan back and forth? So we have, oh they're close, um, we're going in and out of sync. So, let's see. Let's do a buffer on it. Let's do a time correction of plus two on the, um, the black magic media.

There we go. And let's hit calibrate

thinking it's

weird. So we had it working and then it's acting up now. Okay. Yeah. That was

[00:23:08] Joe: whenever I did the calibration, like the last time, anytime I done calibrate, it

[00:23:12] Eliot: seemed to affect

[00:23:15] Joe: it that way.

[00:23:17] Eliot: Okay. So let's see. So let's look at this timing diagram. Um, Oh, it's close. All right, so let's, let's go ahead and click on our, um,

in the upper left hand corner, there's what looks like a color chip chart, and that's gonna, that has the, uh, Are you talking about here? Uh, no, uh, in the, uh, time data monitor, uh, there's a, just about, there it is, an inch to the right, uh, one, one, one to the right. Yeah. Open the buffer visualizer. So this is where the data is coming in.

So, okay. So jet set is coming in ahead of the video. Um, okay. Okay. I see what's going on. So let's go back to time data monitor. All right. And then what we actually can do, you know, let's just up the buffer size. Uh, let's remove the time correction on, uh, on the, on the black magic. And we're going to up the buffer size to 20 because that's, that's easy.

Okay.

All right, and then let's click calibrate and let's see if that does it.

So we just doubled our buffer size. So that should buy us more time to have things line up.

No, that's kind of weird. Um, let's go ahead and click that, that same color chip chart thing. Again, the buffer visualizer is jet set and video.

What are you doing? Cause we just increased the buffer size by a lot. But it's still somehow coming in on the end of it. Okay, so let's look at the time data monitor again. Um, so I'm getting a buffer underrun. Okay, so

I was running out of time, then adding buffers should have added more time.

[00:25:31] Joe: Should we try adding more to see if it does it, or? I

[00:25:34] Eliot: don't think, that's a lot of buffers, so that's already a decent amount. So, let's um, okay. Let's for the, let's remove our 2 frame offset from our global frame offset. Um, just for kicks, let's go the other direction.

Let's go minus 2, let's calibrate. I'm trying to get and understand a correlation, and I am fuzzy in this part. This is the part that just kind of worked when I did it. And so now, you know, now we're having to debug and figure out what's, what's going on. Yeah. No,

[00:26:05] Joe: I mean, Hey, this is what it's all about.

[00:26:07] Eliot: Yeah.

[00:26:07] Joe: And I know black magic is a pretty common one too. So it's, you know, pretty pricey. So I'm sure there's going to be a lot of black magic users out there. Uh, we want to figure this out. Made it

[00:26:18] Eliot: worse. So that made it worse. Okay. Let's actually change the global offset. Okay. Let's go in the other direction.

Like There we go. And let's hit calibrate. Cause I think, okay, now things are starting to make more sense. Black magic. I mean, yeah, if

[00:26:33] Joe: it goes back the other direction, you know, if it counters what it just did, then hopefully that should get us where we need to be or closer, I guess.

[00:26:45] Eliot: The thing, the thing that I'm curious about is it's saying no buffer interval could be found.

Um, okay. Let me look at that error message. Hey, no interval found

[00:27:04] Joe: size required.

Hopefully you guys are going to have those Gaussian splats sorted soon. So seriously,

[00:27:27] Eliot: I mean, seriously, it's, I've been

[00:27:29] Joe: looking more into that. I've been diving more into that. It's crazy shit. I totally understand. Like it's wild. It's very wild.

[00:27:36] Eliot: I'm going to make a bet. I want to go through this. I want to dial this for you.

And I'm going to bet that it's going to be a lot easier when you're doing your shoots to punch out a thousand frames in Unreal, train it, and run a splat on the phone when you're doing your shoots than all the stuff that we're doing. Yeah.

[00:27:52] Joe: Oh, I have no doubt. I mean, like, I was seeing this guy making a splat.

He was just taking a live video camera and just like spinning around the character. I was like, oh, that. So I was just like, okay, I would do it more detailed than that. You know what I mean? But I was like, whoa, the fact that you can get something pretty cool by just spinning, moving your, your camera around and being like, okay, I thought it was like a way more involved process.

Yeah. Yeah.

[00:28:15] Eliot: It's it's okay. Time didn't monitor. Let me, let me look at this. Cause I'm, I'm going to be looking at,

[00:28:27] Joe: but this is good to know. Cause there's always going to be people who are going to want. Unreal, you know what I mean? Or just want the buzzy word on real, even, you know, God, I think the splats will take a minute to catch because everybody's so like hooked on Unreal, they don't even know the hassle. So, and plus I, you know, I, I mean, cause I actually want to, I have something that I wanted to run by you.

Um, cause we're actually doing an Unreal project here on the 23rd. I am. And like this artist that he's bringing into mentor, we're doing like a live performance music video in virtual like, well, I am like just bought a whole bunch of fucking screens from this studio that closed down. So one of the things, and I'm waiting to see who the Unreal people are.

Um, cause I wasn't sure if like, I imagine they probably are using the vibe Mars at the moment, but I don't know, can Matt send tracking data from the camera out like you guys can.

[00:29:23] Eliot: Yeah, they, they, they both, that's going to be the same sort of stuff in Unreal. No, but I mean, to

[00:29:27] Joe: like other programs, like, so it's like, if I wanted that tracking data from the vibe to like, so that I could spit it to after effects.

[00:29:35] Eliot: Oh, no, I, maybe I've, I've honestly, I've never used it. I mean, you guys can't write. So I didn't win

[00:29:44] Joe: that. That's what I, okay. So that was my, that, that was sort of where I was hoping that you were going to say that the vibe Mars can't, because I didn't think it could. So my thought, because I was like, I really want, I was like, that's huge for like posts, like having a camera that's tracked the whole time so that I can overlay elements over the virtual environment.

Huge. So now I was pitching that to them and they were like, Oh my God, that sounds amazing. I just don't feel confident enough. to like oversee that on such a big project. So I was just curious, you know, it's in Hollywood. It's like, you know, it's right here. We can talk about that after we get this sorted, but it's a pretty big opportunity.

And I think it could be an awesome way for us to like show Jet Set on like a big production. That's going to like go out. Cause they're like, Uproxx is going to push it. And like, you know, Will is like one of the main ones behind it. It's about his space. So we could talk about that after, but I just wanted to plant that bug in your ear a little bit that, um, I really, cause I want to flex hard, like that'll be a really, no, I'm serious.

Like I want to flex hard and I want to say, Hey, this is a main difference between a lot of these other programs. It's like, yes, I was able to track the camera. You know, you could use it to track it for Unreal, but like I needed stuff for foreground and to like composite over. And now I was able to just, I was also able to get that camera data out of there, which Like that's like a clear differentiator from like other offerings from other products.

So, right. Close

[00:31:12] Eliot: tracking. It's, it's important. No, it's huge.

[00:31:15] Joe: So yeah, we'll get back. Sorry. I didn't mean to distract you. Let's look at one problem

[00:31:18] Eliot: at a time. So let me look at, so what I'll tell you what I'm doing right now is I'm, I'm, uh, the, I'm kind of just going through, cause I want to, I want to see how somebody sets up the time data monitor in black magic.

So I'm just going to actually go through this and under, uh, looking at this thing by, by Aidan Wilson. So I'm going to look at this.

[00:32:50] Joe: Oh, sure.

[00:39:06] Eliot: Okay, so I have an idea. So let's, um, can you share your, uh, Unreal screen and what I'm sorry, go ahead and share your Unreal screen and, um,

[00:39:20] Joe: Oh, is it not sharing it?

[00:39:22] Eliot: Yeah, it's not showing it right now. Oh, okay. Let me,

[00:39:32] Joe: uh, Zoom just froze. Hold on, let me go back in.

Sorry, just going back in.

I think it froze for a sec there.

Alright, so now

Are you, do you see it now?

[00:40:25] Eliot: Yeah, yeah, I see it now. I'm just trying to, I'm making sure I understand. The thing that's interesting is that we're trying to do everything with timecode. That's interesting. Whoa. Now we're getting away. Why are we getting the big jumps in our network again? No, let me just

[00:40:44] Joe: Sorry, I was playing around with settings, so I was just like Oh, no

[00:40:47] Eliot: worries.

[00:40:47] Joe: But yeah, let me, I'll just start from scratch here and then hopefully we'll see what's going on.

So this is with no offset. I was just toggling around. So it's

[00:41:07] Eliot: okay. So let's, um,

I don't understand is I see it. I see it jumping around again, synchronized, non synchronized. And you still have the network connection on. Yep.

[00:41:27] Joe: Yeah, I've been changing it

[00:41:28] Eliot: on. On the Live Link, let's try, let's try doing something. Uh, let's click on the Live Link, uh, on your Live Link window. And let's click on the Low Net 2 Live Link.

Uh, the, the, the, yeah, that's it. Let's scroll down a little bit. Let's take a look, let's take a look at this. Detected frame rate, um, and then, let's undo, let's open up advanced. Let me just see what we're doing here. Keep it one, at least one frame. Alright, let's scroll up a little bit. Alright, let's try setting an offset.

And let's set our offset to, we'll give ourselves a two frame offset. 24 frames a second, that's, Right. Yeah, actually, but that's, that's seconds. So it's um, oh yeah. So, uh, milliseconds is 41.9, so it's, um, 0.0 8 3 0 8 3. Yes. That's, that's seconds. And then let's take a look at what our time data monitor. Okay.

We're still Okay. Let's add a bigger setting. Let's add a 0.5 offset to that. I wanna, what I wanna understand is. Where we are adjusting in time, okay, yeah, let's go to 0. 5, let's just give it a larger, yeah, and let's look at the time data monitor to see, is that, no, that's not affecting it. Okay. All right. So let's go, let's, let's go, go back to our live link and we'll change that back to zero.

[00:43:05] Joe: Oh, so it looks like that was just doing what this is here.

[00:43:08] Eliot: Okay. Time. Oh, because it's in that case, it's thinking in terms of frames, cause we have the evaluation set to frames. Okay. Well, in that case, let's, let's keep upping. Let's go back to our live link, uh, our time data monitor. So we see where we're, what we're doing.

I want to see, let's increase our time correction on the low net to live link. Okay. I want to see what it takes to get that thing to go over to the right. That, that white, that big fat white bar should, there we go. Okay, that's what's going on. Now, now the world makes sense. So we're adding a time correction and we just put in like 10.

0 or something like that. You know, so 10 frames. Okay, okay. Now we should be able to click calibrate.

[00:43:52] Joe: Okay.

Come on. Show me potato salad. Let's go. Make me rich. Make me very, very rich.

[00:44:06] Eliot: Okay. Why is it still going in and out of range? No interval could be found. The buffer.

This is baffling. Cause that is clearly inside the spot. I bet what it is, is it's having a hard time matching the time code. Okay. Cause we are on

[00:44:31] Joe: the tentacle,

[00:44:34] Eliot: you know, that's not a bad idea because we, let's take a look at our technical lens. Let's work our way, work our way down. You know,

[00:44:43] Joe: I'm sinking to this device again.

Sink all. Okay. So now it should be

[00:44:51] Eliot: on the tentacle. Can you show me your screen on the tentacle? Because the tentacle should be generating a time code. And if it's not, then, then there's that's probably, can you see that? It says generating time code. Okay, let me, uh, pin that, pin that close.

[00:45:07] Joe: Are you able to see this?

Sorry.

[00:45:08] Eliot: Hang on,

[00:45:09] Joe: yeah, let me

[00:45:09] Eliot: pin that, uh, close for it. Alright, so let's see. Timecode.

[00:45:15] Joe: Is it better if it's, if I hold it that way?

[00:45:17] Eliot: Yeah, it's a little bit better. Okay, so, and what's our timecode? 01

[00:45:24] Joe: 015417. Okay. So the low net is a little behind now. Oh, but that's because we minused the 10 frames, I guess.

[00:45:33] Eliot: Yeah, 01 54 27 18.

Okay, so let's see. Alright, and then inside, let's, inside Jet Set, can you just make sure that we're seeing the same timecode in the little Jet Set preview? Absolutely. Yeah, there we go. Let's pop open the little, uh, alright, so, Tentacle, 01 54, okay. Oh, wait!

[00:45:56] Joe: Okay.

[00:45:58] Eliot: So that's 24 frames a second. Okay. So that's good.

So then let's, let's look back at, at Unreal.

[00:46:03] John: Yeah. Whatever you guys just did changed Unreal and they finally linked up the time. Good. Hi, I'm John. I just joined in.

[00:46:12] Joe: Sorry. What were you saying?

[00:46:13] John: Uh, your timecodes in Unreal finally synced up. They were, they were off. Oh, yeah. Yes.

[00:46:26] Eliot: So we should share context. Okay. So the, what we're working through, okay. Hey, John. Yeah, we're, we're going through a, a little dusty corner of, of, uh, Unreal Syncron, of Unreal Synchronization. Because I did our tutorial on the Aja, and this is, uh, and Joe's got a Blackmagic, and of course, something's a little bit different.

And we're figuring out exactly what it is. I don't know, John, have you gone through much on this time data monitor stuff?

[00:46:55] John: I am a cinematographer, so I'm not going to be as much help on the Unreal side of things. All right, no

[00:46:59] Joe: worries. Well, just so you, just so you know, like this is, we've got this, uh, uh, uh, cine camera hooked up to this.

So the idea is that as I'm moving this camera, it's moving in virtual space. Just so, you know, just so you realize, like, we're not just doing like a virtual camera thing. So there is a cinematography element, I guess.

[00:47:23] John: Your issue now is the delay or what?

[00:47:25] Joe: Well, the delay, I think we know there's a way to offset that, but it's just not staying.

It should just be rock solid here and like staying in sync.

[00:47:34] John: It was when you turned in there. Like I saw the virtual camera move and pan exactly as you. Oh yeah,

[00:47:41] Joe: no, it does. It's a little off, which, which, so we've had this, like this we've had set or, or like this has been possible. It's just like, for some reason it keeps going in and out of range.

Like you can see here, it's like, you know, which I, which will affect, uh, the accuracy of our sync.

[00:47:58] John: Well, I was kind of curious because I was reading over your guys tutorials yesterday. Um, and the Blackmagic is for sure set to 24 fips, not 23. 98. Yes.

[00:48:10] Joe: Yeah, we did that. Um, we, we checked that as well, cause that is coming in.

That's all you have 24, yeah, 24, uh, 24.

[00:48:23] John: And then what are you using to bring the video signal in just like a AJA box?

[00:48:27] Joe: Uh, a black magic, uh, capture card. Yeah. Got it. So, yeah, it's, and it's the black magic. It's the new 8K G2, you know, it's, it's a, it's a nice one. Um, so it should be working and I just got SDI out of the camera going right into the capture card.

[00:48:50] John: And forgive me for my ignorance, but the Lonet Live Link, just your, uh, time code there, where you're generating that from a tentacle box as well? From a tentacle

[00:48:58] Joe: box, correct. Yeah. Is it a

[00:49:00] John: tentacle box for both or are you doing the tentacle box on each? Just one for both. Oh, two, with two different cables coming from the same box?

[00:49:09] Joe: Uh, no, because the, the, the tentacle syncs to the phone.

[00:49:13] John: Yeah.

[00:49:14] Joe: Which then, and syncs the camera. Because I have the, uh, oh, right. So it's, yes, I have it. You can kind of see here. I have it, a wire going into the camera for time code right here. Yeah. So the tentacle is giving time code sync to the camera. Yeah.

And then through Bluetooth, it sends it to the Jet Set app.

[00:49:34] John: I mean, that could be your delay right there. Bluetooth protocol is not going to be like true on that. Like, I mean, I know in production side, when we're doing multicam and stuff, it's like, you really want a tentacle on each device individually that have been synced from a slate.

[00:49:49] Eliot: This is a slightly different case. So we're, we're, we're driving and the Bluetooth, we have a directly synced of the, uh, uh, Over to Jetset so we can actually talk to it in software and understand it. So they're, they're both generating timecode and the part that is curious, and we can see the timecode coming in, um, you know, in there, in the, uh, you know, like the, the last sample time, you can see them.

And it's, you know, it's slightly offset, but, you know, that's, that's to be expected from a couple different machines. The part that's very curious, Is that, um, normally when you click the calibrate button in the Unreal time data monitor, it looks at the incoming frame cue for both of these and, uh, looks, looks at the time code and both of these and looks for a lineup, right?

So as you know, I've got like 20 frames of this and 10 frames of this, is there some combination of, of that where I get the matched, matched frame and it's having a hard time finding it.

[00:50:44] John: Yeah, because it's drifting. I can see a drifting over the course of this conversation. They've been, it's not, it's like a variable rate of change happening on there.

It's not a standard offset. Like they were synced when we started this, when you rejammed and now they've drifted, which is why I was asking about 2398 versus two 24 and making sure everything is like set up in that. Cause like something is off on your time. Co cause it's, you can see it's still changing.

It's not consistent.

[00:51:11] Joe: Yeah, no, you're right. We, we had that issue with, um, with that. And, and I was just like, I had to change it to 24 and like, redo it through, uh, through low net. So, cause we had that issue where it was coming in 23 and 24. And then I realized on the tentacle, it was a 23, nine, eight. And then once it was added to 24, then this was all going.

So everything is a 24 right now.

[00:51:35] John: Um, But look at that see how it's shifting back like there's some sort of like it's not a consistent offset like every fifth frame it's going back to normal and everything and it's drifting like two or three frames off.

[00:51:50] Eliot: It's interesting.

[00:51:51] John: I'm curious what's going on with that.

Um. Which is why I was asking the Bluetooth possibly. I don't know. Sorry. I don't know. I'm not a programmer. I'm just.

[00:52:03] Eliot: No, I gotcha on that one. Um, this is a little strange. I wonder if, how much buffer that Blackmagic has. So, let's um, okay. Let's uh, for the time correction, let's, let's bring our buffer size on the low net back down to 10.

Uh, sorry. Buffer size

[00:52:21] Joe: back to 10. Yeah. And

[00:52:26] Eliot: let's remove our time correction on it. Just go back to zero. So I think what it's looking for is some combination where they line up. So let's adjust our time correction on the black magic. Is there, I'm not sure if it's going to let us set buffer size in this, but let's, um, let's, let's, let's try increasing briefly.

And I, but I think it's going to take us farther to the right. Okay. Yeah. Let's, let's go minus let's go minus on our time correction

more. And let's, let's try, let's try to go both directions. What I, what I don't fully understand is how Unreal buffers things internally. Because that, that red line is the evaluation line. That's where it's actually triggering, triggering buffers. Um, oh, you know what? Okay. Okay. Let's set both of those back to zero.

Uh, time correction on both of those back to zero. And then what we're going to do is we're going to actually increase our global offset. There we go. Okay. Uh, let me, uh, yeah. There we go. And that should, let's try going in one direction. We'll go global offset. And what we're, there we go. So what we're seeing there.

Okay, so six frames global offset, that way they're actually both more in time. So now let's try clicking calibrate, because what we've done is we've offset the overall time frame over a little bit. Uh, and now we can tell it, see if we get enough overlapping buffers. Uh, now that we offset the overall time, time frame, um, there's, there's some questions in terms of how Unreal is buffering video frames because it's hard.

So now it

[00:54:03] Joe: says no buffer for Lonet and Blackmagic.

[00:54:05] Eliot: Blackmagic's

over there. Lonet's over there. Um, is

the global offset both the times at the same time? Now we can increase the buffer size Okay, hang on before we do this. I'm just trying to understand how its internal buffer timing system is thinking. Um, so we changed our overall time. Okay, the red line and the green evaluation time is, is where, is where things land.

So, okay, let's do two things. One, let's uh, let's uh, let's set our time correction to zero on those, on the both of those. There we go. And we're going to increase our buffer size and loan it, uh, from 10. We can go up to 20. Okay. So that gives that buys us a larger buffer. Okay. Then what we're going to do is we're going to, uh, shift, lower our global.

What bugs me is that if we're increasing our buffer, let's, let's briefly go to 30 on our, on our loan at buffer. Cause that's just tracking data. And that's a, you don't need to use it much memory. What bothers me with this is that if time goes to the right. Then our buffer and maybe, maybe, except maybe that's maybe time is time happens to the left, which is counterintuitive, but it could, it could be that way because we're adding buffer and then the, but the buffer is going farther to the left.

[00:55:42] Joe: The left side. Yeah.

[00:55:43] Eliot: Yeah. So, okay. In that case, let us, let's start changing our global frame offset. And what we're going to be looking at, let's go higher for a second. Okay. Nope. That's right. Let's go lower and we're going to go back until we can try to find some spot and let's keep going back to zero.

Yeah, there's, there's our problem is the problem is that we have a very narrow buffer. Oh wait, that's interesting. Okay.

Maybe that'll do it. Okay. Let's try to click and calibrate. Cause we can get an arbitrarily large buffer of timecode samples or of tracking data samples. Um, it can just store those all day. It's just like a couple of kilobytes per sample.

[00:56:37] Joe: Oh, oh, that's cool. It's

[00:56:41] Eliot: better,

[00:56:42] Joe: better. Yeah.

[00:56:43] Eliot: Okay. What happens when you pan back and forth?

So now we've got, we've got that synchronous. Okay. So that's okay. But now we can offset, right? Um, so, okay, so it's, it's many, let's not touch that one because that's the global one that's setting our overall timing. Now we're going to want to time correct the individual ones. So, right now, um, so our time correction.

What I worry is that the Blackmagic has a very small buffer.

[00:57:12] Joe: Yeah, I've noticed that.

[00:57:14] Eliot: Yeah, because it's buffering video, and video is huge, so let's see.

And when we shrunk our buffer down to match, then we get this huge delay. You know what? Okay. Let's try a different approach. Instead of fighting this timecode thing, we're gonna, we're gonna approach the delay with just seconds. So go ahead and drop the buffer size back down to 10. We're gonna, we're gonna, this is more of a manual method.

And the timecode is great when it just magically works, but it's, it's giving us, uh, so for our evaluation method, instead of timecode, um, let's, uh, click both of those and we're gonna go to, uh, platform, platform time. There we go, platform time. Okay. All right. So now

[00:58:00] Joe: see now that gave it a whole lot more buffer.

[00:58:03] Eliot: Yeah. Yeah. Cause I think we'll, we'll give it a second to kind of like resolve. Okay, so now what we can do is we're just going to change our time correction in the low net, low net, um, and let's, let's, let's pan back and forth a little bit just to, just to begin. Uh, yeah, just to see how far we off. So, okay, so our, our low net is lagging.

So in fact, our time correction, we should have the negative. So let's try minus 0. 5 or minus one. That's fine.

[00:58:36] Joe: Okay. And calibrate?

[00:58:39] Eliot: Uh, no, just let's pan back and forth. I want to understand where this thing is behaving. Okay, so we're outside range. Okay, so let's go ahead and click calibrate. Then it should snap our green line back.

Okay.

[00:58:56] Joe: Not evaluated in time mode.

[00:59:00] Eliot: So, does that mean that our There's a possibility that our video is coming in very quickly. We have a very short video buffer and the video comes in quickly. We run out of buffer before we get tracking data in that I think is what's going on. Okay. So let, in that case, let me look at, uh, increasing buffer and, uh, black

[00:59:30] Joe: real.

This,

this night's going to feel good to crack.

[00:59:39] Eliot: Yeah, like I'm, I'm starting to understand how their time data monitor behaves. Um, And it was, I mean, I, sometimes you get lucky the first time and it just, everything just worked on, on the first time. So, but of course, then you go back through and, and, uh, understand how it works better.

So let's see. So, uh, I'm checking on this.

Yeah, I think we need more buffer. Um, let's. Let's see. Okay. Let me check on this, this guy.

[01:00:28] Joe: And I had the Blackmagic 8K G2, which is, that's the model of the capture card. I mean, it's, it's pretty new. It's a brand new card.

[01:00:37] Eliot: Yeah. Okay. Let's actually go. Um,

let's check something. Let's go into our, um, let's go to our, the core media, um, media input bundle. Uh, it's under the content browser. And yeah, there's Nidia, Lightcraft, and then there's the ZCAM live feed. Yeah, the ZCAM live feed. So let's see if we have an option in there of increasing the number, or buffer count.

Configuration, let's uh, okay, so bit C, that's fine. Um, let's scroll down a little bit and let's see if it, if it, cause it's loading stuff in software. So it should give us somewhere and maybe under a just in time rendering. Now actually Pat had, we had always had us check just in time rendering. Let's go ahead and check that for a second.

Copy. Okay.

[01:01:44] Joe: Okay.

[01:01:47] Eliot: And let's, uh, let's also look under advanced. Ideally, there'd be something which tells you how many buffers we get. And let's keep scrolling down.

[01:01:56] Joe: All

[01:01:58] Eliot: right. Platform time. Okay. Interesting. Sample evasion time. There we go. That's more or less what I want. That's what I want. Okay. So let's give us ourselves a time delay

and let's give ourselves a time delay of, Oh, I don't know. A quarter of a second. Yeah. 0. 25. Oh, that's interesting. So you can see it shifting over there. Okay. That's, that's actually, that's just, so let's go back to zero. I want to watch that, how it changes in the time data monitor. So that's, wait, what, what just happened there?

Uh, Oh,

[01:02:39] Joe: sorry. I just, I, the low net, I thought you said go back to zero. Oh yeah.

[01:02:43] Eliot: Let's let's, let's bring everything back to zero. Okay. Okay. So all that's doing is delaying. Oh, so

[01:02:50] Joe: that's just the time delay thing. Yeah. What

[01:02:52] Eliot: happens if we increase that delay a little bit? Well, all of a sudden, like what we're now we're synchronized.

Right. What just happened? Okay.

[01:02:59] Joe: So I just, let's increase it to 0. 5. Okay.

[01:03:03] Eliot: I'm I'm on 0. 25

[01:03:06] Joe: seems to be, I guess it seems to be holding.

[01:03:10] Eliot: Yeah. I'm starting to understand this. Okay. So we added 0. 25 to delay of that guy. Okay, so let's, let's, uh, let's, uh, save that. I'm curious

[01:03:23] Joe: if zero, just maybe because we also did click this just in time, and I guess that's Oh!

[01:03:30] Eliot: Okay, let's try that. Because I remember Pat had me click on that, and I, I gotta say, I don't fully understand what that is. So should

[01:03:36] Joe: I keep the 25 time delay, or

[01:03:37] Eliot: Let's keep it at zero, actually. And, uh, hold over the just in time rendering. Can you, uh, can you hold over that? I want to make sure I understand exactly.

Okay. Uh, and let's bring it over the late, the label, um, that's a eight sample picking friend, or I'm going to need to look up on that just to understand there's a lot of details of what they're doing in the engine. Um, okay. Frame lock helped at all. Uh, you know, I'm not going to worry about that quite yet.

Sample of it. And, um, I mean, this seems to be holding.

[01:04:07] Joe: Yeah. So

[01:04:08] Eliot: now, now let's see if we can get things to sync. Now. So the first thing we have now, we have some degree of. Of timing sync. Um,

[01:04:17] Joe: that's great.

[01:04:18] Eliot: Okay. What happens when we if we pan back and forth? I think we're gonna still have some delay. There we go.

Yeah, it's a little bit of delay. Okay, so And we're right on the edge of our available buffers. Cause the green line, that vertical green line is of our evaluation in the middle. Yeah. So let's, I mean, if

[01:04:36] Joe: we did like, Oh, should we adjust any of this stuff? Or, I mean, we didn't calibrate since we did that.

[01:04:43] Eliot: Yeah, let's go ahead and calibrate. Let's see, see where we're at.

Okay.

[01:04:50] Joe: Not evaluated in time. Should we switch it back to time code and see what happened?

[01:04:54] Eliot: Yeah, let's try that time code time code

[01:05:01] Joe: and now calibrate.

[01:05:03] Eliot: Yeah, let's go ahead and calibrate. And I bet we're gonna have to adjust our

Okay. Oh,

[01:05:14] Speaker: okay,

[01:05:15] Joe: that's better.

[01:05:17] Eliot: Yeah, we're synchronized. So now So we're seeing some buffers coming. So what happens when we pan back and forth now?

Oh, way later. Way later. But it's, uh, the live action's not moving. Why is the live action not moving? Wait, did it get frozen up?

[01:05:38] Joe: Oh, yeah. What is going on there? Um, let's see. Maybe, let me go. That's weird. Um.

Did that work now? There we go.

[01:06:03] Eliot: Okay, so at least they're smooth. Okay. So, one of them is, of course, delayed by a lot. So, let's, let's start. Just feels like I'm missing something. Okay. And a core of all this. Um, okay. Let's drop our, our time correction. Let's, uh, let's drop it a little bit and let's watch those.

We should be watching the buffers line up. Yeah. Okay. And then let's, let's, uh, pan back and forth a little bit. See where we're at. And we lost our,

[01:06:38] Joe: why is he

[01:06:40] Eliot: watching their live actions? At

a certain point, what I'm going to do is I'm going to go buy black magic card and put it in our shop and then, and then work through it. Yeah,

[01:06:56] Joe: that would be awesome. I

[01:06:58] Eliot: think probably what I'll do next. Because at a certain point it's just It's kind of, it's too many, too many round trips trying to debug, debug it on your, your machine.

Cause there's clearly something, something's different, you know, like the object, I hit it and it worked, but like something's different in how, how Blackmagic is doing it. And I want to understand it. Um, and I may just, it frustrates me. Oh yeah. You can say you're not both on timecode. There you go. Yeah.

Yeah. Cause I think what's going on, I think there's a core problem, which is that.

[01:07:31] Joe: Yeah. Frame drops too. That's a lot of frame drops.

[01:07:34] Eliot: I think the black magic probably has a very small amount of buffer and we're not getting our tracking data in from low net quick enough, let's try something. Let's, let's, let's drop the latency.

Okay. Let's go, let's go back to jet set and there's a, there's a setting in jet set that may, may be useful for this. Okay. And, and I'm okay.

[01:07:59] Joe: I'm, I'm, uh,

[01:07:59] Eliot: I'm right here. All right. So let's see. Let me check on mine where that's at. Settings, so let's get main menu settings, settings, uh, and then,

okay. And then in settings switch, there's a, uh, under advanced, there's a cine camera, 60fps, no comp. Let's turn that on briefly. Copy on. All right. And it may end up re telling you to re uh, may, it may force you to, uh, restart and set your, um, set your origin. No problem. I'm just gonna put

[01:08:46] Joe: this down for a second.

Okay. Origin is set. Okay.

[01:08:58] Eliot: Okay.

[01:08:59] Joe: Okay. Origin is set.

[01:09:01] Eliot: All right. So what happens when we pan? And now I'm noticing that we're losing the Blackmagic feed again.

[01:09:06] Joe: What? Why is that doing?

I don't know why it keeps making me update it like that. It's really lame.

[01:09:22] Eliot: Okay. So now let's, let's set our time correction, uh, on the low net two back down to zero. Now let's, uh, let's say, yeah, set them both to time code. Actually. Oh yeah. Sorry. And the Blackmagic one set that time code as well. And all right.

Yeah,

[01:09:45] Joe: see, every time I do that, it seems to be turning it off. Okay. So here, look, let's see what happens here. If I do this, I think that's a problem there. So here I do that and look, see, it switches it back.

[01:10:03] John: Okay. Hold on that black magic deck link screen for a second.

[01:10:09] Joe: Uh, sorry, which one? The one I was just,

[01:10:12] John: when you go to reset it.

[01:10:31] Eliot: Oh wait. Oh, oh, there it is. Under video, max number of video frame buffer. That's it. That's it. There it is. Up that, up that. Can I, how far can we up that? Okay. Maybe not 20, but let's, let's, let's, let's go up to like 16. That should do the trick. Yeah. 16. Okay. Okay. Yeah.

[01:10:50] Joe: Oh, nice.

[01:10:52] Eliot: Okay. Now the world starts to make sense.

That's it. So, okay, now let's, let's, uh, exit out of there. Switching both to time code correction. Um, cause then that should have just bought us enough extra frames that things are going to light up. So now we can hit, uh, calibrate and it should drop us both. That green line should drop in the middle of, okay, there we go.

Now we're seeing good. It

[01:11:14] Joe: didn't get the buffer error that time.

[01:11:16] Eliot: Yeah. Okay. This is, this is what's going on. So what's going on is this, this is what's happening in time. Is that the black magic frames are coming in and out, oh, let's, let's get it, let's get it active again before you declare victory. Yeah.

That guy. I'm wondering why it keeps shutting off. That's a little crazy. Well, it keeps switching it. I noticed they keep switching to here. Okay, let's scroll down a little bit. Why, why is it doing that? Let's scroll down in the settings panel, uh, a little bit. Uh,

[01:11:42] Joe: settings, uh, wait, where, what setting panel for where?

[01:11:45] Eliot: Uh, for the black magic, uh, media, media thing as you can, my feel. Let's scroll down. Why is it, let's scroll down a little bit. Maybe a setting that we Oh, okay. Sample evaluation type instead of platform time. Let's switch to time code. All right. Uh, uh, time. Yeah. Time code. There we go. Sample ation. Okay. Okay, let's save, and that should stay parked on timecode now.

[01:12:09] Joe: So now it got rid of it.

[01:12:17] Speaker: What?

[01:12:21] Eliot: Okay. Let's go back to that media panel.

Okay.

[01:12:30] Joe: Yep. And then it's back.

[01:12:34] Eliot: Time synchronization. Let's, let's turn off just in time rendering because that was one of the settings we changed.

[01:12:41] Joe: And then that narrowed the buffer a lot.

[01:12:45] Eliot: I thought we added six, we're still up to maximum 16. Let's go first.

[01:12:55] John: Take it. Oh, wait, under color

[01:12:56] Eliot: conversion settings. We have right now it says an invalid conversion.

Can you open up that, that, that, um. Where? There's a spin down that says color conversion settings.

[01:13:12] Speaker: They're doing here

[01:13:21] John: now that you've turned off just in time rendering. What if you go back to time code out of platform time? Let's take a look at that

[01:13:29] Joe: and it goes away.

[01:13:30] Eliot: It goes away. Oh, wait. Okay. Platform time. Okay. Okay. All right Nonetheless what we have Okay, I think that we can make this work. So let's go ahead and click save Um, and it wants to be on platform time.

It's baffling Alright, but we still bought ourselves a chunk of extra buffer. So now we're actually in the right, right place. So let's try offsetting this in time. Um,

[01:13:58] Joe: Oh, okay. So time code seems to be working now again,

[01:14:02] Eliot: for some reason. Yeah, now what bugs me is that, okay, give me a second. One of them's in, in, I think if they're both in time and we're going to be evaluating it, um, uh, let's okay.

Let's click calibrate once again, is that should align both of those things, at least in there in the middle of their kind of sweet spots of what we have available buffers for. What bothers me is that we had a big area of buffer. We added a bunch of buffers to the,

[01:14:34] Joe: yeah. And that's the buffer size again.

This

[01:14:41] Eliot: is the weirdest thing.

[01:14:43] Joe: And if you change it to platform time, then it has a much bigger buffer.

[01:14:47] Eliot: Yeah, give it a second because I think it's, it's, it's kind of thinking that timing diagram. Yeah, then it's going to come down

[01:14:54] Joe: to

[01:14:55] Eliot: where it's at, but at least that's now. And let's, let's change our evaluation.

Let's just change them both to, um, to platform time because then we're at least in the same, same world. So then let's, um, uh, what happens when we pan back and forth? Okay. Oh, much smoother. Yeah, we're getting closer. Okay, so let's do a slight time. Okay, let's uh, and it looks like the CG background is lagging.

Can you pan that back and forth again? Absolutely. Yeah. Background is lagging. So let's, um, add the tracking data. We can add in a floating point delay. So let's, uh, in our time correction, we can add in, um, let's try a minus, minus point up. We just blew out a range, uh, minus point 0. 015 or something like that because we're in, we're in seconds world.

Okay. So then how are we doing? Okay.

Okay. That might be about as close as we're going to get on this thing. Um, let's uh, let's play calibrate and see if those things line up in their, in their worlds. And I may, uh, if this kind of behaves Hi! He learned to live net is All right. This makes me a little crazy, but it's also It's synchronized and it's stable.

Okay. I may declare victory briefly in this, and I think I'm going to go order a black, a black magic card, because I There's too many things. Settings that I don't fully understand what it's doing. Like we, I think we increased the buffer that I think I understand the underlying core reason. The black magic card had this much samples of buffer and our tracking data was coming in here.

So we did not have a time overlap and the thing that would, it would make sense, make sense to work. So we increased our buffer sizes, the black magic. So now, now it's overlapping. Um, it still looks like a very tiny buffer, which I would expect the, their timing diagram to have a larger range. Um, if we increase that, yeah, I don't know what, what it's, what it's doing there.

Well, on the other hand, the good news is I have this recorded. Um, and I can talk to, I can talk to a couple of people about it to understand better what's going on. What's going on? Um,

[01:17:23] Joe: should I increase the buffer more for the video? Does that do anything?

[01:17:27] Eliot: You know, you can try that. Um, it's probably can't hurt.

I got, you know, I'm like 24 or something like that. 32. Um, it doesn't seem to be changing it that much in the,

[01:17:37] Joe: well, that's cause let's see if we did time code and maybe

it's

[01:17:47] Eliot: time. Cause it's making it act really wacky.

[01:17:50] Joe: Yeah. Now

the platform time seems to be. And

[01:18:04] John: then you go back to time code on both. And then on the black magic delay, 0. 0 and five.

[01:18:17] Eliot: Yeah. And time code, you're probably gonna have to go by multiple integer multiples.

[01:18:21] Joe: Oh yeah. Well, because

[01:18:22] Eliot: it'll be like, Oh, I'm sorry. Sorry. Your second, uh, on the, on the right, um, that little pull down menu with an S that evaluates whether you're in seconds or frames mode and that'll that's then, then you're okay.

If you're, if you're in frames, then you can change by integer multiples. Oh, yes. Then you're going to have to be like minus one or plus one or something like that. Yeah.

And then once you've moved it, you probably have to click the, uh, the calibrate again.

[01:18:54] Joe: I'm just seeing what it's doing, but I guess let's just try that and see what happens.

[01:19:11] Eliot: Well, at least our green evaluation line is, is now in the middle of, uh, Sort of. Sort of. It bugs me that it's popping in and out of range.

[01:19:24] Joe: Yeah. I don't,

[01:19:24] Eliot: I don't understand what's, what's going on there.

[01:19:27] Joe: And then if we do it in

[01:19:29] Eliot: this,

[01:19:30] Joe: then

[01:19:31] Eliot: Oh, I'm going to stop here. That's way more

[01:19:33] Joe: solid.

[01:19:35] Eliot: Yeah. I think, I think we may be, we may be in, in the, using

[01:19:40] Joe: End game of this for now?

[01:19:43] Eliot: Yeah, I need to understand better. Again and again, I just clicked timecode on the agile, hit a button and it worked. Um, and I didn't think about it. So, you know, there, there, there we go. So now I'm going to go figure out the Blackmagic side of things. And I'm going to be very, very grateful that Nelson Splats have come to exist.

[01:20:02] John: Because it was closer than 0. 02.

[01:20:06] Joe: What's that?

[01:20:06] John: You 0.03 on that one you're working on right now?

[01:20:10] Joe: Point oh three oh oh. So wait, maybe

[01:20:16] John: what? What you were just adjusting. Don't change. Oh, all right.

[01:20:18] Joe: Oh, wait, sorry, what were you saying?

[01:20:20] John: Your delay, you're adjusting, you're playing with that delay. Go to 0.03. It was getting closer when you went to 0.02.

[01:20:26] Joe: Oh, okay. Cool.

[01:20:30] John: So now it's too much.

[01:20:33] Joe: So maybe 0.025.

[01:20:42] Eliot: Yeah, we're in the time domain, so a little bit's gonna go a long way. Each frame at 24 is 41.7 milliseconds, so it's in just by small increments.

[01:20:51] Joe: Yeah, so let's do seven.

[01:20:58] John: Nah, it's gotta go the other way.

[01:21:00] Joe: Gotta go the other way, so let's

[01:21:08] Eliot: Alright. Yeah. Why don't you play with that a little bit? Yeah, yeah, yeah, exactly. And actually, let me double check on the supported equipment, just to make sure that we're actually working on real black magic. Supported. Let me see what your cards, cards, they, they support on this guy. Um,

okay. Let me check on something here.

This sounds vivid. Okay. Black magic. Let me look at this.

Yeah, I know exactly what goes. I'm going to ping on this. Uh, cause this is, this is a, this is an inner, there's an intersection aspect between black magic and Unreal engine, which is like, I don't program. I, you know, we're not programming either. I mean, we're, we're subject to how they, they do it. So I need to understand how they, how they do it.

Um, all right. That's that. All right. So Joe, let's, let's do that. Let me go and let me go look up on Unreal. There's a list of their supported cards and this one should work, right? It's a, what is it? What did you get? It's a, I got it. It's a black

[01:22:51] Joe: magic, a K G two.

[01:22:53] Eliot: A K G two. Okay. That should be fine. And it's modern card.

Um, and black magic. I'll quick start. Okay.

Yeah. There's the 5. 4 documentation. Okay. Here there's the, um, I'm going to put this on the chat because this is what I'll, I'll be going through as well. Is there, is there a reference point for the project? Um, and I'm sure it's something like, Oh, it's this three levels down and two over you click this checkbox and, and everything will make sense, right?

Um, it's just interesting to understand where it, where they have it. Where it's going to be hidden. Um, okay. All right, John.

All right. Mine

[01:23:42] John: might be easy. We'll see.

[01:23:43] Eliot: Yeah. What are you doing?

[01:23:45] John: Um, I have a question. So when I'm in the jet set, I'm, I'm, I'm trying to set up a thing. My brother runs an event company and I'm trying to develop a new concept using you guys tech for that.

[01:23:55] Eliot: Excellent.

[01:23:56] John: Um, when I go in and use the phone and I switch it from AI, cause the AI is okay, but I get a lot of artifacting around that, like on a natural comp and I have green screen set up and I switched to green screen, it gives it like a sepia tone, no matter what color the green screen is.

And I can't, it's not giving me a clean key. Just in the phone.

[01:24:21] Eliot: Okay. Gimme a second to understand. Did you set

[01:24:23] John: your green screen, Joe? And maybe you, because you have like all this set up, maybe if you just pull I can play

[01:24:29] Joe: with it.

[01:24:29] Eliot: Yeah, yeah. Actually, tell you what, so what we'll do is, is I'll pop open, actually, let me uh, open up a QR code so you guys kinda, uh, so we can actually look at this directly on, on his screen because this is Oh, you

[01:24:41] Joe: can do that.

[01:24:42] Eliot: Oh yeah. So lemme uh, we did this before, right? The remote, remote thing. Okay. Okay. So, uh. Joe, you can go ahead and exit, exit Jet Set. And, um, and then I'm going to pop up this QR code and then point your normal iOS camera at it. And it'll pop up a little, uh, yellow buttons, like open Jet Set. It'll ask you if it's okay to screen record.

It's not actually screen recording, but it's just pulling the pixels over a server.

[01:25:09] Joe: Uh, so, uh, sorry. Did you want me to just look at it with my camera first? Like the regular one? Yeah, go

[01:25:13] Eliot: ahead and exit Jet Set and just point to your normal iOS camera at this. Oh,

[01:25:17] Joe: yep. Okay, cool. Got it.

[01:25:21] Eliot: It'll pop up a yellow button and I'll show you on a screen share.

Okay, great. So let me screen share and show you what we're seeing. Uh, stuff's share that, uh, where did it go? All right. One, two, three, four. Great. Okay. So let me share my screen. Cause this is, oh, this is. Cause I think

[01:25:42] Joe: I know what you're talking about, John. Um, and cause I think I've, uh, stumbled into that myself.

[01:25:49] Eliot: There we go. Okay. So Sharon, here's the screen. So that, so here we

[01:25:54] Joe: go. Right. I just got my, let me, so I know one of the first things. Yeah, because I remember messing up on this too was I don't know if you did this but in the so here I'll just set an origin point there and just come to my set. Okay. Right.

So, um, I know one of the things was, I don't know if you did this here. But with setting, oh, hold up, let me, I think I've got to go composite. What is it, mat? Okay, yeah, so the mat's on green right now, so let's change it to

[01:26:35] John: But why are you seeing off the green, because you're not filling the green with your image?

No,

[01:26:39] Joe: I'm filming the green, that's why I'm like, why am I not getting a Right now

[01:26:42] Eliot: your mat's set to CG, so let's go to green. There we go. Pick that. was

[01:26:47] Joe: still doing that too. And then let's go

[01:26:49] Eliot: to the set button. Let's click on that and refresh my screen.

[01:26:59] Joe: Yeah. For some reason, normally I see a green screen here, but it's not showing up for some

[01:27:03] Eliot: reason.

Go ahead and click, go ahead and click that green swatch. That's kind of weird. Um, okay. There we go. There we go. Click green swatch and then click and drag on the green. There we go. All right.

[01:27:17] Joe: That's weird. Uh, I mean, one of the things I remember would, Oh, see, yeah, that's weird that it's just going away like that.

That's strange. That is, it's never done that to me before.

[01:27:27] Eliot: Let's go click set and over on the, uh, click the set button again, and let's go over the infinite GS click clear. Let's just make sure there's no garbage mats around anywhere. Okay, okay, that's fine. And, um,

that's a new one. All right, so let's, uh, let's click okay for a second. I want to understand why we're getting that weird, getting there under the mat. Let's go look at, um, let's look at, and, you know, we can look at that. Okay, oh, oh, that's what's going on. Um, okay. It's because we switched to the record when we're, we tried the high speed recording stuff.

It's still set at 60 cam 70p. Okay. Yeah. Okay. So go main menu, recording, uh, settings. There we go. We were doing something experimental and let's scroll down to advanced and, uh, uncheck cine camera. There we go. And then while we're at it, let's, uh, let's, uh, also do, there we go. There we go.

[01:28:29] John: Now, now, okay. So yeah, yours is super clean.

[01:28:32] Joe: So when I do, so this was. So, you know, yeah, so I did that and then you can do this and then it adjusts where your screen, where, what, what it's registering is your green screen. So it's like, I go and I scan here and I add, so let's just say that's what I want. So now.

[01:28:51] John: Oh, you hit the green screen and you hit start.

Sorry. I missed what you actually hit.

[01:28:56] Joe: Oh yeah, sure. So I went to set right now and I'll just clear that. So it's seeing the green, right. You know, it's noticing the green screen. And I hit, you see infinite GS, I hit start, and then it's like, Oh, is that your green screen? So it's like, so I'm going to add that, right?

So I added that section and now I scroll down to add more of the green, right? Whatever I want to be registered, I add that. And then I just hit stop. And now it registers and then okay. It register. I didn't, I'm

[01:29:25] John: gonna sound like a dollar, but I didn't even realize that. Nope. No, this is, was a set, there was a set function on there for, I was like even just struggling to find that.

So yeah,

[01:29:36] Joe: I had a feeling that's what it was. I had a feel and I was like, I, I had that same issue.

[01:29:41] Eliot: No, no, this is, this is because this is, this is our fault because the, we, when we were trying to build a 3d garbage matting system, and this was our first go at it, which is, we thought we were going to be really super clever and we're at school, we can detect the vertical walls, the vertical planes.

And then we said, Oh wait, we can be clever and we can detect the interior parameter, the perimeter of that green screen. And we can generate this nice, you know, nice polygon that fits it and that kind of stuff and make these like clever garbage screen, garbage mats. And that's all great. And it works great for a certain category of things.

And it completely blows up when you start hitting like a, a larger setup with the stage where, especially where there's different variations that green screen lighting and it detects the variations in green screen lighting as the edge. I'm like, whoops, okay, we, we messed up. So what we're doing is we're actually rewriting this part.

So it still detects the, the green screen, you know, wall, where it knows it, where it is in 3D. And then it's just going to pop up a, a four, you know, corner polygon or something on it, where you just drag it to the four corners that you want it. Um, and then there's just nothing to go wrong.

[01:30:41] Joe: Oh, that's great.

And that's, that's

[01:30:43] Eliot: why there aren't extensive tutorials on this, because we built it and we're using it. I'm like, nope, this, this, this isn't going to work. Um, so this is, the reason you haven't seen it is because this part of the UI is, is not great and we're redoing it. To have a, we're gonna have a, a separate, you know, scan button and then there's gonna be a separate key button.

And if you go, you click scan, you're gonna go scan, and if you click key, you're gonna go to the key . Amazing. I know Crazy sounds, sounds,

[01:31:09] Speaker: sounds about right to me

[01:31:11] John: of a, a question too. I'm running mine on an i a 13 Pro. Is the performance difference, like if I switched to a 15 pro going to be that much, it is.

Okay. Just before I went out and dropped a grand on a new phone,

[01:31:23] Eliot: wait two weeks. And, and cause they're going to drop because the, the thing is the big jump was the 15 pro max. I mean, that was the big job, right? The, the GPU and the cooling and the, um, the USB C connector, like the throughput you can get on that is like, uh, like it's, it's, it's.

You know, the 12, the 12, 14 is better than 13 and the 15 is way better than the 14 on everything. And so I'm looking forward to seeing the 16. So, you know, if you're going to go out and buy it, buy a phone specifically for it, you know, the 15 is awesome. And it'll, it'll, the 15 pro max,

[01:31:55] Joe: the 15 pro max, the

[01:31:58] Eliot: cooling is so much better.

It's already like a win. And, but if they're going to drop a new phone in another two weeks, I'm like, Yeah. 'cause I'm just group

[01:32:05] John: of concepting right now. I'm selling, like I, I'm like, we're listing the concept out there to see if there's a buyer effectively. But if, like, I'm not, I'll wait until the 16 drops at that point because

[01:32:15] Eliot: Yeah, the, the 13 will be great for your testing.

Yeah, it is a 13 pro,

[01:32:20] John: I have a 13 pro, yeah. Okay. Yeah.

[01:32:21] Eliot: So it's got lighter on it, et cetera. Yeah, the latter. The letters is one of the other huge things that makes. It's, it's such a big deal in tracking. Um, so let's, let's take a look. I mean, do you, do you have your phone? We can turn it on right now and walk you through it.

Do you have a green screen?

[01:32:35] John: I don't, I tell my brothers because the green screen built up over there.

[01:32:40] Eliot: Well, I'll tell you what, what do you, so you just saw how we did it. Like we, I popped up a QR code, he linked to it and that, that QR code, uh, turns on a new switch inside jet set, which is just a screen share button.

Looks exactly like the screen share button you've always seen. There's a little TV with like waves going through it. So you hit the button and that bounces the, the, uh, the UI over our servers. So then I can see it. And then you, you saw what I'm seeing, right? That's our, that's how we can kind of remote in and look at what you're doing.

When you hit the screen share button. Uh, and then we can fix stuff. Then it's super easy to fix stuff. Is this real?

[01:33:11] John: I can see it. Yeah. Yeah. Do I, so I do I need to hit that, that image then just to have that in jet set or is that one off you do that every time you need to read like remote end?

[01:33:20] Eliot: Yeah. We'll just kind of do it every time we need to.

It's, it's a, it's kind of one of these internal internal things that we have for do exactly this kind of stuff. And at some point we'll. Probably productizes people could link to each other because it's it's a godsend of solving this stuff. So yeah, if you jump in on office hours and you're at your brother's with a green screen or even if you get a, you know, green screen out of your house or whatever, um, uh, because that's This, this is always my vote is, is just get a green screen and drop and park it behind you because we can literally test everything you're going to test on the stage, you know, by you turning 90 degrees to the left with a green screen in front of your computer and us working through it on office hours, you know, all this stuff we're doing,

[01:34:03] John: I'm going to do is I'm going to have three in event spaces, premium, right?

Like they have like a very 10 by 10 square of what they're allowed to use. Um, and so I'm going to build like an eight by eight back wall, then two sidewalls. So I effectively have like a, like a half of a stop sign, like a hexagon. And that way I can do imagery, like camera movements with it. And like real time with, you know, guests coming through and say like, Oh, you're at Wembley stadium or something, you know, and like, Well, just to give you

[01:34:33] Eliot: a sense of what, what is looming, um, and what's going to be, going to be possible with all this stuff, um, let's see if I can load this open, open up a, cause you, what were, what you see seeing us go through with Joe is, is thrashing with Unreal.

And, and Unreal is amazing, but there are 10 million switches on every single one of those switches have to be right. Exactly right for it to work. You just, you know, I mean, I have a brand

[01:35:02] John: new M three and I downloaded it and I was like, this is struggling so hard. I get a PC and everything. Yeah.

[01:35:09] Eliot: In Unreal.

Uh, yeah, yeah, yeah. So what we're, what we're introducing some is, and this is, it's not even shipping yet, but it's just so you see what's, what's coming up. Um, like there's something called Gaussian splats. And these things are for a lot of things. They're going to be a game changer, um, because they let you do stuff.

Um, see, this is going to set my origin set. There we go. Cause it lets you get a pretty high level of photorealism while in the phone. Um, so I'm going to share this. Let me share this screen here. What did you do? Where am I at?

[01:35:47] John: Yeah. Cause the other thing I'd be curious about is like, could you send like, wait, every time I record a video, could I have a protocol that would send that to A separate computer, right?

Because like what I will, I'll have to back program is something that like. Takes the video I take throws like a corporate logo over it. Effectively effectively. And then it ship sends to like either SMS or email for a user.

[01:36:10] Eliot: You know, that's, we, we already have something where you can, um, uh, you can airdrop right off of, right off of jet set.

If you're in the review tab, you hit the share button and it'll airdrop and all that kind of stuff. And I think

[01:36:21] John: that's the, that's the simple version for now, but it like automating it all would be epic as well.

[01:36:26] Eliot: Yeah. Yeah. That's, that's coming up in terms of. I think this is going to be a, it's come up a few times to do for corporate events, because this is, this is what you're looking for, right?

It's the, you know, it's something really cool people can look at. All right, so let me share the screen so you guys see this. The splats are.

[01:36:45] John: Yeah, because I had been doing research on doing like setting up a rear projector, kind of like a poor man's LED wall. 'cause I, I've used those down in volumetrics and I was like, it, it's just too expensive.

[01:36:56] Eliot: Yeah. Yeah. And, and it's hard to get to be robust in lighting. So this is, this is a Gaussian splat. So this is, uh. This is, uh, captured from, uh, it was a Chinese, uh, coffee shop. Uh, one of the Josh Cha over in Beijing, uh, took the photos, but this is, it's, it's fully 3d, right? Things work, the reflections work, and this is all rendering in my phone, right?

That that's the thing that makes this stuff crazy is now you can make it, make it look right. And you're not dealing with, you know, cause again, the traditional virtual production thing, it's, it's a little bit of a circus. Uh, cause you got so much equipment that all has to be linked together and working.

You know, just as exactly as we're seeing.

[01:37:35] John: Yeah, exactly. You turn it on. And that's kind of like, you know, my idea for him is like, set it up in a way that any user, you could ship it in a box, ship it in a pelican, it's got the batteries that you charge, that it runs the fan off of.

It runs the. Everything have it look like a bit of a show because as you know, half of cinematography or any of this stuff is the show you put on is beyond, you know, a guy showing up with an iPhone. So, but like, it's been interesting leading, doing this research for him, just leading me into just the virtual production space, you know, Joe, where you're at is like next level there and you're in your living room.

I don't think my wife would support. I've already got too much gear in the closet behind me.

[01:38:14] Joe: Yeah, no, I'm definitely I'm pushing on the meeting a studio space here again here pretty soon. Um, yeah, I mean, one of the things that I don't know, Elliot, maybe if we have time that we could kind of test this out real quick, um, which is a different functionality from what we've been, which is.

If I shot a quick just shot in front of the green screen here, can we, can, can you help like kind of run me through getting that into After Effects? Yeah,

[01:38:40] Eliot: yeah. I said the, because

[01:38:41] Joe: we don't need to deal with Unreal. Like, let's just say, I just want to, I shot a plate of anything, right? Let's say we're not any, no virtual production.

I just want, I just need the camera tracking data. So if I just shot something here quick, I just want to be able to then throw it in after effects and just have all that camera data. So I can start overlaying effects and creating nulls and just start attaching shit. Actually. Yeah, let's,

[01:39:02] Eliot: let's do this. I got to get off.

Cause I, I've got, I've got another call coming in pretty fast, but the, um, I've been wanting to actually do a full, after effects, the exporter works, right? You know, so the, the basic techniques that I already, already did on the, the, um, it was a BRAW. You know, there's a track you with Jetson and B Raw and getting into, into Blender.

That stuff will all work. Uh, and this is the perfect time to actually really dial in, uh, an After Effects project and an After Effects tutorial, because we've got all the other pieces are behaving pretty well. So if you have, um, let's, let's do it. Cause that, especially if you have something that I could use.

Uh, for to draft a tutorial around what we can just do is go through it with you. And I'll, you know, I'll walk you through it and then use whatever clip you're using as, as the tutorial master. And then that, that'll work great. Cause

[01:39:49] Speaker: it's

[01:39:52] Eliot: good. It's good timing because after effects now now supports ACEs and can read the ACEs CG files that we're generating and they, they just, and it does linear compositing and they fixed like.

So many things that didn't work for a long time that now they work. I'm like, great. Now the, the only thing they don't let us do is, and I hit my head against this. I hope maybe, you know, somebody I cannot script after effect with every other program, we tell it to write where we're going to take our inputs and where we're going to write our output.

So it's all standardized, you know, right. Our outputs to like comp EXR, render EXR, et cetera. And after effects, you cannot script the output. Can't do it. You can do these weird side load workarounds where you side load a previously generated comp file. And then with that, have the, has the, the, you know, the output settings already in it.

But, yeah.

[01:40:37] Joe: Well that's because I was curious 'cause I saw the way that you were getting it into was like, it, it's a copy from OUTTA Auto shot and pasted into the new program. And I'm like, wait. But I was like, can you do that in After Effects? So that's why. I was curious, like, okay, what's the different workflow with that?

[01:40:51] Eliot: Yeah, and this is, it's great timing because it's, it's built, you know, the exporter is built, but we should, we should do a few run throughs of it. So you can kind of see. Yeah, I mean, we

[01:41:00] Joe: can do something quick now. Like I'll just, I can just shoot a five second shot here real quick, just run through it. And then once I just kind of do that, if you want like a better shot to test it on, then like, I don't mind doing that again.

You don't have to go for it. Okay. So, so, uh, in terms of with Jet Set now, right? Yeah. So I, I've got everything, I guess, pretty much, like, I don't even need to worry about my background or model or anything then, right? Yeah,

[01:41:25] Eliot: have you gone through and done a Jet Set Cine take? You put the, you know, the, the, um, the digital site in front of it, hit record, and, uh, sync QR codes and all that kind of stuff?

[01:41:35] Joe: Um, yeah, I've done that. So it's like, I, I can, uh, And that was like to match back to the city footage, right? Yeah. Um, yes. So I can do all that. So here, let me, I guess that's probably because if you already

[01:41:48] Eliot: did that, we can just use that take. If you already did the one with the matchback and everything.

[01:41:53] Joe: Oh, uh, yeah, maybe I do.

I think I do actually. Yeah.

[01:41:57] Eliot: Cause that'll save you. I mean, otherwise it's going to be like, we're going to go through 10 minutes of Yeah, that's a good call. Let me see. I should

[01:42:04] Joe: be able, I should have that.

[01:42:07] Eliot: Cause as soon as you can see it show up and say blender or something like that, or Unreal or whatever, they all work the same.

And then we can just pop the same, same data, the same frames into After Effects. You have good time because we just found a bug that has eluded us where we were clipping, clipping highlights in, um, when we did our conversion. So we're just fixing that now. That was a bear.

[01:42:27] Joe: Okay.

[01:42:28] Eliot: Yeah. Color gamut work is no joke.

[01:42:31] Joe: This was just a test clip that I was like trying to mess around with. Like this was where I was like, Oh my, the tracking is like a little off, like it's getting there. So you know what I mean? So that was like part of where

[01:42:44] Eliot: you want to share screen that we can, we can see it and we can, uh,

[01:42:48] Joe: Oh, sorry. Is that not shared?

Oh yeah. I

[01:42:50] Eliot: probably switched over when I was doing, doing my stuff.

[01:42:53] Joe: No problem. Let me where,

okay. Uh, you're able to see that now. Yeah, yeah. So this was just like a test, you know, you can kind of, it's like the tracking is like off, but I was just like playing around, just trying to do that. So like, I do have, so it's like this one I shot, right. You know, so it's like, I do have that tracking data. Um, so let me go for a

[01:43:25] Eliot: second.

So did you already run the, uh, the, uh, Unreal or sort of the after effects exporter and import the JSX file? Is that how you go? No,

[01:43:32] Joe: no. So, uh, so what I, I mean, what I did was I, I, I, I synced it, right. I did the same and then it matched it in, um, it was able to match it in auto shot, um, but then I just took the frame clips here, like I just, I matched it.

Like this was the render from Unreal. And then I took the clip and I just put the first frame, you know, I was just like, okay, 1300 out, you know, I recorded, I gotcha. I

[01:44:00] Eliot: gotcha. So I can do better than

[01:44:02] Joe: that.

[01:44:03] Eliot: Okay. We, we can, we can, I think we can do better than that. Um, let's see. Can you load that, open up that, that clip in auto shot just so I see all the different pieces that are.

takes. Let's go back to that.

[01:44:14] Joe: So that one would be that here. Um, let's see. Yeah. I mean, it should be, why isn't it? Where are my takes?

[01:44:26] Eliot: Now your project folder. Oh, there you go. Oh, that's synced to today.

[01:44:32] Joe: So where would I find my earlier takes?

[01:44:35] Eliot: So, on your project folder, did you, uh, Oh,

[01:44:38] Joe: right, okay, right, okay, because this was a different, I, I, I, I told you I had remade it, just for practice, so let me just go back to the one that I, to the first one.

That, that should have all of that, which is your, that should be that. And then now, so do I need to sync that to, Oh, I need to change get set.

[01:45:05] Eliot: I think you already, you already, um, pulled the takes over. So now you've, so now you pick the correct project folder under your take selection. You can just pick which day you are, you're working on.

So, right.

[01:45:14] Joe: Yeah. I just need to switch it on jet set city so that it reads the proper. File that so that it reads the proper

[01:45:21] Eliot: one. Actually, do you, yeah, you already pulled the takes over. So before we do anything else, if you have undertake selection, did, or did, did you already pull the takes over? Did you click scan?

And they, cause it looks like you got, okay, it looks like you got takes. Okay, there we go. So, and it looks like you've got a city offset. So before we do anything else, do you have, do you have blender on there? I do. Okay. So, let's, um, I always like running, running a, running something through Blender because that's, that's, that's like our, our lingua franca that we test.

I don't

[01:45:52] Joe: know anything about Blender. I just downloaded it. So just FYI. I'm like,

[01:45:56] Eliot: Oh, okay. Okay. I got you. I got you. Um, all right. I tell you what, let's see. Let me think through this for a second. Um, all right. In that case, let's not worry about Blender just yet. Cause it looks like it found the Cine offset.

So what we're, what we'll do is, so is that the, uh, the clip that you ran through Unreal, like clip in frame 2010 and out, out is a three, three, one 13. Okay. We'll just work with that. So, uh, under the run values. Uh, instead of Unreal, what we're just going to switch this to, uh, others. There you go. And then just click save and run.

And what that's going to do is it's a, all right. So it just generated that there's the after effects script. So go ahead and click the, uh, AE scripts exported to F colon Unreal. It's, it's about halfway up the page. Oh,

[01:46:41] Joe: Sigma 4D After Effects. Okay. There

[01:46:44] Eliot: we go. And go ahead and click the, uh, and it's going to be up one.

Uh, the After Effects script is exported to F Unreal Engines, uh, yeah, up about two centimeters. Oop. Because that one is synthesized. And we're going to go ahead and click the, each one of those, those, uh, directories is active, those purple links. So you go up about two centimeters. There you go. And you click on that one.

Okay. Now, now we're in the After Effects directory. So go ahead and then click in the path of the After Effects directory. And the whole thing should, uh, should highlight. Uh, it's up in the This here

[01:47:16] Joe: you're talking about?

[01:47:17] Eliot: Yeah, just to the left of that. Uh, yeah, copy address, copy address. That's fine. Okay.

Now we can go over to After Effects. Boom. And, uh, do you have like A 2024 or something like that.

[01:47:31] Joe: Yeah, I got, uh, this is 2024 here.

[01:47:34] Eliot: Cause it's the only, the recent versions of it fixed the ACEs color structure is space stuff. Oh, okay. That's that went into like March of 2023. I was like, thank you. Thank you.

Adobe. He just made my life better. All right. So we can just, um,

[01:47:50] Joe: Oh, I guess that clip isn't in here. Huh? That's don't

[01:47:53] Eliot: worry about that yet. Cause the script is going to load everything else. So you can actually just close the project and. And, uh, there we go. And we're gonna do file and we're going, going to do scripts, scripts and run a script file.

And, uh, go ahead and paste the, uh, paste the address that we, they had, uh, highlight. Uh, it's already like loaded in there. Yeah, there you go. Copy address. And then just go up there and that area you can hit shift day or whatever and, and just highlight all that. Oh, that's, there we go. And then shift a to highlight all that and then paste it.

There we go. And then hit enter. Oops. Okay. Uh, let's try that again. Uh, it's shift a highlight everything. Control V paste. There we go. Oh, it's got a little tilding. There we go. Hit enter.

[01:48:51] Joe: Oh, is it at the beginning? It might be at the beginning. It's going to

[01:48:54] Eliot: go, is that doing that? That's weird. All right. Like, uh, let's remove two, like the AE and then, um, let's see if it goes to that directory.

You hit enter.

[01:49:09] Joe: Windows can't find it.

[01:49:11] Eliot: Is it something that's weird? Um,

let's, uh, let's go back to that, that first one. Um, the, the, where, where it popped up with the after effects directory and, uh, Yeah, there you go. Just go ahead and click, click that guy again. I want to sequences Japan. Okay. So then just highlight in that top bar, uh, where it says the cash after I just highlight, highlight in there, um, uh, to the right of the AE, uh, there we go.

Copy that copy addresses text. There we go. Let's try that.

Let's see what, and it can fill face. All right. There we go. That's the plan. All right. And let's go ahead and you can highlight that and just click open and it's going to execute script and should load in some frames,

maybe load in a lot of frames.

[01:50:21] Joe: Yeah. It was probably a little bit of a link to your clip.

[01:50:32] Eliot: I guess it's 11. It's like a thousand frames.

[01:50:37] Joe: Yeah. Yeah. It is like, yeah. Okay, cool. Okay.

[01:50:40] Eliot: So then, uh, you, if you double click on the calm, uh, then it'll, it should show the different pieces of it. And let's see. So there's, you'll have the, the camera originals, um, and let's, let's go, let's set the color space correctly on the camera.

So go ahead and highlight. There's an image sequence there. If you double click that, you're going to see your image sequence and interpret footage main. Yep. And, uh, before we even do that, let's, um, because we need to set the color, the project space. So let's go ahead and cancel out of this for a second.

Okay. And let's go under file and it's either file under edit.

[01:51:17] Joe: Uh, oh

[01:51:18] Eliot: yeah, file project settings,

[01:51:20] Joe: file project settings.

[01:51:21] Eliot: Yep. And we're going to change this to, uh, under color. You're going to change this from color engine from Adobe color managed and to OCIO color managed. That's it. That's the one. And then under our working color space.

Okay. There it is. Compositing linear ACES CG. That's fine. Yep. And that's, uh, now you can click. Okay, cool. And then we can, now we can right click on that EXR file and we're going to interpret footage and domain and color. And then we're going to go to our, uh, Okay. Now we don't want ACES 2065 to one. I'm going to pull that down and we want actually.

Can you pull down a list of that? It should be, um I'm

[01:52:08] Joe: trying.

[01:52:09] Eliot: That's kind of weird. Oh, there we go. Yeah, so then we're going to scroll down up to, uh, your end utility. So we just want ACES. Uh, it should be at the near, near the very top should be like ACES, uh, ACES CG or something like that.

[01:52:27] Joe: Uh, oh wait, those were down.

It's not these ACES.

[01:52:31] Eliot: No, it's ACES CG. It should be a utility.

[01:52:38] Joe: Utility. Oh, so it's saying the utility. Okay. So here's the utilities.

[01:52:46] Eliot: What'd you guys hide it this time? Oh, here.

[01:52:49] Joe: Is this a AIS emulation or

[01:52:52] Eliot: no? Let's keep going down. It should just be like ACEs cg. It's like the most generic of, of ACEs color spaces that exists and it, it is, gotta be in there. And maybe,

okay, let's keep scrolling down. ACEs, GB. No, that's it. Okay. Something strange. Let's, let's go back out here under our, uh, in, under our interpreted color space. So what, uh, maybe a color space. Why is it nine? Good Lord. Okay. This is just, uh, this is interesting. All right. I'm going to have to go, um, I'm gonna have to go through this cause I, I do this and it was, it was, Oh, wait, is

[01:53:42] Joe: this it?

Aliases, ACES.

[01:53:48] Eliot: The one we want though, there it is. Utility aliases, ACES CG, you know, a little bit below that. All right. Okay. And it's probably gonna look a little weird. Cause this is, this is exactly, you picked a good time to be doing this because we just found a bug and we're, we almost have it fixed. But, um, do I see the camera original?

Okay. There it is, but it's going to look weird.

[01:54:11] Joe: Okay. So for some reason it's, it's in a 10 80 comp.

[01:54:16] Eliot: Yeah. I think that's another one of these weird limitations we ran into.

[01:54:23] Joe: Okay, so now that matches.

[01:54:26] Eliot: All righty. Okay, so then we can put a key light key on that and it should, um

[01:54:33] Joe: Even though it doesn't show any green here? Let's see. You see that? It's showing the transparency.

[01:54:44] Eliot: Oh, that's right. Cause it's defaults to have a, if you spin open that layer, I think we defaulted it to add a key light, uh, setting on it.

So if you open up the layer in the comp, uh, and you just turn around the spinner underneath, underneath it, it's already got a setting on it, right? So we turn it off and there's, there's the effects. So now you can drop in. And if you have rendered frames in the normal place, I thought that this would pull in the rendered background, so you would automate the comp construction.

And it's not doing that, so we're going to have to do a little bit of debugging on this to figure it out. But that, this is, this is the basics of how it works. You know, we just kind of went through it. So you can go ahead and bring in, you can right click and import your rendered frames. Uh, in your project, uh, I think you do it in the project folder, uh, this guy up here.

[01:55:29] Joe: Oh, you mean for the background?

[01:55:31] Eliot: Yeah. Let's just try, try popping in the frames and seeing it, seeing the thing and seeing if things line up correctly.

[01:55:36] Joe: Cool. Yeah. Let me, cause I believe that the rendered frames, let me just spit it out of Premiere as a clip versus the, uh, Well, it should actually be in the Unreal rendered frames, right?

Cause if you render them out of Unreal,

[01:55:48] Eliot: then it'll be an image sequence.

[01:55:49] Joe: Correct. Yeah. That's, I was going to render it out as footage, but do you want the image sequence instead?

[01:55:54] Eliot: And guys, I think I, I got to go because I think we have our 11

[01:55:57] Speaker: o'clock coming in. Yeah,

[01:55:59] Joe: got it. Okay. Um, is there a quick way to just, cause I don't, I'm not worried about the background.

I just want to get camera tracking data here. Just make sure that works. So like the

[01:56:08] Eliot: camera tracking data,

[01:56:10] Joe: uh, should be, if you

[01:56:11] Eliot: spin that down, that should be, uh, transform. Oh, wow. Okay. So yeah,

[01:56:19] Speaker: that should be in there. So

[01:56:21] Joe: I can just create a, no, just link that right to here. And in terms of for the camera,

[01:56:28] Eliot: and

[01:56:29] Joe: then

[01:56:31] Eliot: because my knowledge of after effects is low,

[01:56:35] Joe: right?

Yeah, well, that's why I was just like trying to see like, okay, how can I take this camera data and get it so that I can actually

[01:56:41] Eliot: like something and something onto it? Yeah.

[01:56:43] Joe: Yeah, which is usually like on a null object. So it's like, that's cool that it's got the camera here. But I was like, how do I paste that camera data onto this null object so that it sticks?

Thanks. When I move around.

[01:56:54] Eliot: Okay, I'm going to have to get back to you on that one. Um, because that, this is, but this is perfect because we, we have the basic pieces of this, but we've been needing to sort of bang through all the details on the A workflow. And it's perfect timing because you're doing this and Alden's doing this and it's time to sort of, you know, get it, get it dialed.

Yeah, because I have

[01:57:15] Joe: that project, and this is the function that I need with it. So it's like, if I got my Unreal stuff all shot in camera, Now I want to do stuff into the foreground. I'm like, right. I want it to, I want it to be rock solid on that track.

[01:57:29] Eliot: Right. So it matches up.

[01:57:31] Joe: So it matches up. And I can just, you know, especially for like text effects, like if I want like lyrics to like follow with them as they're, you know, and stick in the frame, things like that.

[01:57:39] Eliot: Right, right. Um, mutual stuff. Um, all right. That makes, all right. That makes sense. I gotta jump to Yeah, yeah. Later.

[01:57:46] Joe: I'll shoot you an email. We can offline on that later.

[01:57:48] Eliot: All right. Sounds like a plan. Good to meet you guys. Yeah, take care.

</body>

</html>

Transcript

EMAIL INFO@LIGHTCRAFT.PRO