Tutorials

# Synchronized Rendering in Unreal ​[00:00:00] We are going to go through the process of setting up synchronized compositing in Unreal using a Jetset Cine. We have already set up in Jetset Cine with a integrated timecode from a Tentacle Syncc device. And we have an AJA IO 4K which is an SDI input going into Unreal. So we'll just go through the process of running the existing Live Render Preview script, and then we'll go through the steps required additionally to configure the AJA media bundle, configure a custom Genlock and timecode blueprint, and connect those to the Timed Data Monitor and so we have a correctly genlocked input and output. Okay, let's get to it So we're in Unreal, we're just going to start off with a new project. We'll just do the film, video, and live events. I'll just click a blank we'll do test AJA. There we go. And do create, and it's going to create a [00:01:00] project. We're going to make sure we have the correct plugins. We're going to go to edit and plugins and make sure we have the AJA installed, which we do. We have LONET and we're going to make sure you install that. And we're going to Composure, make sure we have that installed. And we're going to type Python and make sure we have our Python editor script plugin and the Sequencer scripting and plugins. And last we're going to add the Timed Data Monitor. There we go. All right. And they're going to restart. Okay. So that's done. We're going to go to Autoshot. We're going to come up and go to file and export Unreal Live Preview script. And we can leave this as a default for now. We'll just click generate. We're going to come into Unreal and we'll make sure that we are at the command console, not the Python console. And we're going to hit ctrl v and enter our Live Preview script. And if we look [00:02:00] under our outliner, we can see that we have created four new components, a media plate, a background shot, an overall Composure composite and a Live Link camera. And for organization, I'm just going to right click and create a folder and we'll call that Jetset. And we're going to put all the Jetset components in our Jetset folder. Okay. We're going to come back over to Autoshot. We're going to click to open the web page. This is the web page of the current Jetset device that we have running. And we have Jetset Cine running on this device. So we can click Settings. And we can come down to our tracking protocol. For Unreal, we're using LONET 2, we're going to click set to this PC. If this line was blank, it will now be set to match this existing PC as our tracking destination, and that's where we want the tracking packets to go to. So that is this local device at port 40, 000. We're tracking at 24 frames a second. I'm going to highlight that, [00:03:00] copy this, I'm going to click save to make sure all that is sent to Jetset. And then we can come back to Unreal. Okay. And in Unreal, we can come down to our Live Link and we're going to click Source and we're going to pick a LONET 2 Live Link Control A and Control V to paste and click OK. All right. So there we have our real time tracking data coming in from Jetset. We can see a green light and we're going to make a preset. So let's go ahead to the presets and we're going to save as a preset and we're just going to name it Jetset Live Link Preset and we're going to actually tell Unreal to use that when it starts back up. So we're going to go to Project Settings and we're going to go to Live Link. There we go. And we're going to pick our Jetset Live Link preset as our default one, and that way if we exit Unreal and bring it back, it will automatically choose this LONET 2 preset to be active. Next, we need to create a [00:04:00] media bundle to make sure that we have the correct media coming in. So under our media panel there's a Lightcraft folder, which is our default Lightcraft media player and media texture were created. And we can leave those there. That's fine. We can just right click and create here a media and a media bundle. On this device, we already have the AJA desktop app running and that, that needs to be running for this to work to make sure that the live video is coming in correctly. So we're just gonna call this AJA Media bundle. All right. And I'm going to double click this, We're going to pick our source as our AJA media source, so now we're going to expand the media source. We're going to go click AJA. For the configuration, we need to disable the automatic configuration. So we're going to remove the automatic setup, and we're going to be running at a 1080 progressive, and we're at 24 frames per second. So then we can apply. Then we're going to set our timecode format to VITC. . [00:05:00] Now we're going to open up our 10, our video. We're going to change this from a 8 bit YUV to 10 bit YUV and we're going to change our override, our source encoding from linear to sRGB. We'll open up synchronization and we'll undo just in time rendering. Once we've recorded those we can click Save and exit our media bundle. Now we've configured that media bundle, we can drag and drop it into the scene. There we go. There is our media bundle. I'll hit F to frame on that. But that is our live video in our scene. As Pat's moving that, we can see it tracking back and forth. Great. So that's the first step is done. Once we have that, we're now going to set up our genlock. And, we're once again going to go to our folder. We can right click in our folder and we're going to create a blueprint. We're gonna open all classes and we're going to search for AJA and We want actually the time step. Custom time step. There we go. From an SDI input. And I'm going [00:06:00] to select that blueprint. And we're going to rename it. We're going to call that uh, AJA time step. And we're going to double click that. And once again, we are going to change our configuration to HD, progressive, and 24p. And we're going to apply that. There we go. And we're going to set our timecode format to VITC. Once we've configured that, we're going to click compile and save, and we can exit that blueprint. We can click our Genlock and we see, Hey, there's no Genlock. So let's go to our Timed Data Monitor and under our Genlock, we're just going to click the timestep provider settings. So this will open up our project settings and we're going to type in time step. And for our custom timestep, we're going to pick the AJA timestep. Now make sure we pick the one we just created and [00:07:00] named. Not the AJA custom time step. This is an easy mistake to make. This is the one we just created that is dedicated to our exact frame rate and SDI setup. Now we can see immediately that our Genlock is now working. So it is now running at 24 frames a second, using our AJA time step. Great. That's the first thing we needed And next, we're going to we have to configure timecode. For our timecode, we're going to go back to our content browser, and we need a custom timecode blueprint. We're going to right click again, and we're going to make a blueprint, and we're going to, this time, type in AJA once again, and we don't want the time step, we actually want the timecode. We want the timecode provider from the SDI input. I'm going to select that. And let's rename that. Timecode. Okay. Once again, we're going to double click that. And we're going to in [00:08:00] our details, we're going to set our timecode configuration instead of auto. There we go. We're going to uncheck that. Once again, we're going to set it at 1080, progressive, 24 frames a second, and VITC. And we're going to apply that. And I think that's it. All we need from there. Okay, so once again, we're going to compile and save that blueprint. And now we're going to go back to our Timed Data Monitor, and we're going to pick our timecode provider. We're going to click the settings. We're going to type in timecode. Alright, and so for our timecode provider, we're going to pick our AJA timecode, There we go, we're at 24 frames per second. And we can also at this point type in frame rate, just to make sure that everything is working at the same rate. We can tell it to go at 24 frames per second. And then I'm going to exit that. And we're going to refresh our timecode provider. So let's take a look at [00:09:00] our timecode. Did we get something wrong here? Okay, so this is something that's really easy to to run into. Because we, we did this, somehow Unreal has changed our timecode. Ha! So we're going to go back! To our timecode blueprint, and you'll see that it's changed our, behind our back, it changed our timecode configuration. Going to click this, once again, set it to 1080, progressive, 24fps, VITC, and we're going to apply that. Alright, now we're going to compile it, and save it. Gotta watch out for this, because sometimes it'll do that. Come back here and go to our timecode provider going to reinitialize. All right, so now we have our timecode and Pat, is that the timecode we have coming in on the camera? Okay, fantastic. So, since we've configured Jetset Cine as per the tutorial, now we have a Tentacle Sync that is both driving the timecode going into Jetset via Bluetooth and the timecode going into our Blackmagic camera [00:10:00] over its audio connector jack and is now being embedded into the HDMI output of the camera and converted into SDI via a HDMI to SDI converter by Blackmagic that preserves the timecode. So this means when we go to Timed Data Monitor, now we have two streams coming in, both the LONET2 Live Link and the AJA Media that both have embedded timecode. So now they can show up at the same point and this says synchronized, but right now our little green bar right here shows that we're on the edge of things. So let's do a couple of things and set this up more clearly. We're going to change from evaluation. We're going to go to timecode for both of these and we're going to change from seconds to frames and then we're going to click calibrate. And what calibrate does is it looks at the incoming signals and it offsets them in time. Okay. So that there is enough room for them to match. Okay, now we have synchronization. Let's go take a look at our composite. And as before, we'll click here. And this was our media [00:11:00] plate. And this was set by default, if you remember from the Live Render Preview script, but we've now added an AJA media bundle. So we're actually going to come down here and we're going to go to, into And go into Composure. And we're going to look at the input. And we're going to set the input media source. And we're going to change that to the AJA Media Bundle. There we go. So now we have a live input. Let's fix our keyer a little bit. We're going to go to our Transform and Compositing passes. And go to our Chroma Keying. And we can go to our key colors. And we're going to open our index. And we can actually just sample off in the more, in the corner. It's probably a better way to do that. I'm going to click accept. Then we can adjust a couple of the parameters. And we'll come back and adjust this a little bit more later. But this is, will give us a basic key. Next we want to take a look at our CG background to make sure this is going to match, We're going to first take note [00:12:00] that our tracking data is aligned with the floor. By default, this object is set to, the floor object, is set to translation is 20 centimeters above the ground. So let's change that to zero. So that way our digital floor will actually line up with our live action floor. And next let us go back to our CineCG shot. If you move the camera back and forth. Camera is moving. Now, if it happens to not be moving then you'll need to connect the Jetset Live Link camera to the incoming LONET feed. We do that by clicking on the Jetset Live Link camera and coming down here to the Live Link component controller, and under our subject representation, we can just pick Jetset. There we go. And if it wasn't moving before, it will be moving. So now we have our background and our foreground. Let's look at, we're going to click at our cine shot. That is our composite. Okay. So now we have a composite, but clearly things are not synchronized properly. So let's go back to our Timed Data Monitor, and [00:13:00] we're going to remove our global offset for the time being. We're going to click calibrate. Okay, it has looked at the incoming timecode signals and it's given some initial offsets, but we need to change those. So as Pat's moving back and forth, we can see that our video is coming in ahead of our tracking data. So we're going to actually going to want to correct our timecode a little bit. So we can actually increase the time correction by one, two, three, three frames. There we go. Let's see how we're doing. Maybe one more. Four frames? Nope, that was a bit too much. Okay. And so as we did that, it looks like it changed the global frame offset to enable that to work. And that is synchronized. Let's go ahead and click calibrate again, just to make, put our sampling points in the middle of where we want them to be. in the middle of the sweet spot for the incoming data signals. [00:14:00] Okay, here we have our tracking data is now reasonably aligned. We have our genlock coming in at 24 frames per second, coming in from the AJA time step. We have our timecode coming in also at 24 frames per second. We have a time correction on the AJA of three frames, to move it ahead. And over here we can see that our data sources windows are, aligned so that we have enough room on the video to line up with the tracking data. Okay, so let's try to get a slightly better key and we're going to call that good. So we're going to go back to our Jetset Cine Camera. And we're going to come down to Composure. And we're going to come down to our Despill Pass. And we're going to open up our Material Parameters. Under our material we're going to pick and type in AVG. That is a despilled green by average. And then for our key color, we're going to click, hit a plus button. So just by selecting our key color, it is now despilled much [00:15:00] more nicely. And now we can come back and I'm going to highlight our Jetset Cine shot. There we go. I'm going to adjust the the matte a little bit to uh, I'm getting it to come up here and bring this up and pick the alpha channel. And let's try to solidify that just a little bit. And we're going to close up our despill. And come back to our, come back to our chroma keying and there we've got the material parameters and I'm going to up my alpha bias just a little bit. All right. And then let's take a look at how that worked. Red, green, and blue. Okay. So that seems to have improved matters a bit. All right. Okay, so we're going to exit that, click on the CineShot. Pat's going to set up the camera to move. And while he's doing that, I'm going to adjust the borders and remove [00:16:00] the vignetting. I'm going to go to our keyer and let's de vignette our outer. There we go. That removes the borders going back to our cine shot. Yeah, let's just go slowly by the ground. There we go. So we have a reasonably synchronized ground contact going on here. Alright, that can work with that. So that's a good summary of the setup of going from the Live Render Preview all the way to synchronized rendering in Unreal.