Copyright © 2026 Eugene Hsuan

Powered by Hexo

Modified based on Magnetic theme

Mainly maintained using AI

Notes on Aximmetry and Low-Cost XR Setup

Notes on Aximmetry and Low-Cost XR Setup

Project Background

I needed to build a very low-cost XR solution. During my research I looked at several domestic options and mainstream platforms like Hecoos, and decided to use Aximmetry as the demo solution during the evaluation period. What follows is everything I actually used from Aximmetry during this test.

Differences Between Editions

  1. Studio Edition: Suited for very small or home studios — think YouTube creators and the like. It lets you use multiple cameras on a single PC through HDMI or USB capture devices, produce one or more video outputs via the GPU’s HDMI/DP/DVI ports, and stream output directly to YouTube, Facebook, Twitch, or any RTMP-based service. The watermarked Studio version can be used for non-commercial purposes indefinitely; a subscription plan removes the watermark.
  2. Professional Edition: Designed for small professional studios. Supports up to 4 SDI/NDI input/output ports on a single PC (e.g., 3 SDI camera inputs and 1 SDI composite output) without any camera tracking solution — it relies purely on fixed cameras and virtual camera movement.
  3. Broadcast Edition: For professional studios that need camera tracking. Supports an unlimited number of SDI/NDI input/output ports on a single PC (limited only by hardware), and also works in multi-PC configurations. It includes advanced mixed reality features: multi-camera tracking, industry-standard chroma keyers, high-quality real-time graphics, and automatic calibration of camera offset and lens distortion.

Installation and Setup

Environment Overview

Software Version Notes
Aximmetry Composer image-20251009155943445 The main application used for compositing and receiving tracking data.
Aximmetry UE image-20251009160407488 Aximmetry’s own fork of UE. They’ve modified the rendering pipeline (presumably to enforce the watermark). This version is based on UE 5.5. You can see two custom plugins in the engine’s plugin directory. The source is included and no external libraries are pulled in, but Aximmetry has altered the rendering pipeline — my guess is anti-piracy.
UE Project 5.5 A scene built with Aximmetry UE. I recommend also creating a vanilla UE 5.5 project alongside it, so you can recompile plugins and test them in a clean environment.

Creating a Basic Project

Project Settings

image-20251009155003718

In Composer’s startup settings, the main things to configure are: the input section on the left (your live-action camera input), the output size on the right (final composited output resolution), and the UE render settings at the bottom.

Node Setup

image-20251009161750537
  1. VirtualCam node: Add a camera to the project and wire it as shown.
  2. Main node: Add the camera and project here, and switch it to Live mode.
  3. Green: Add a test video clip here — a green screen clip works fine. Aximmetry will key it automatically.
  4. Camera Tracking: The camera tracking node. You’ll need to enter AximmetryEye in its options shortly.

Once set up, click the Play button in Composer and let the project run in editor mode. You should see a composited image immediately.

UE Scene Setup

image-20251011142232445

Add a camera to the scene.

Setting Up Camera Movement

Manual Control

image-20250930144118468

Here you can configure the camera’s motion properties. A and B represent the start and end points. Click the leftmost button to start the camera moving.

Clicking the edit button in “Camera/Render Settings” lets you drag the camera position directly in Aximmetry’s rendered output window. Use the ↑↓←→ buttons to fine-tune the camera coordinates.

iPhone Camera Tracking

image-20251013114229527 image-20251013114601929

Download the Aximmetry Eye app from the iOS App Store. Then connect via a wired ethernet cable (critical — wireless gave me issues). I’d also recommend attaching a heat sink to the phone; otherwise it gets too hot and disconnects, and tracking data goes haywire. When connecting, select Send Tracking Data (as shown) since we only need the camera pose data. Once connected, you’ll see the phone appear as an option in the Camera Tracking node’s detail panel on the Flow page.

image-20251013203732954

The position and orientation of the iPhone at initialization time becomes the scene’s origin point.

  • Refer to the reference links at the bottom — some include video demos.

Camera FOV Calibration

image-20251013202543178

FOV can also be calibrated manually, since the lens markings on most cameras aren’t particularly accurate. You need precise calibration.
Pan the camera left/right or up/down in the composite view: if the virtual image rotates faster than the real world, reduce Zoom; if it rotates slower, increase Zoom.

Manual Extrinsic Calibration for iPhone

image-20251027113433462

First, add an Actor above the camera and name it CameraLocation.

Aximmetry’s logic is: the camera spawns at the origin, then offsets by the delta from Aximmetry Eye on the phone to determine the camera’s runtime position.

So you need to update CameraLocation’s position to match the physical distance between the real camera and the origin.

Camera Calibration

Basic Concepts

Camera calibration covers two areas: lens calibration and tracking calibration.

Lens Calibration

The idea behind lens calibration is that real studio camera lenses always introduce distortions inherent to the lens — things like barrel distortion, center shift, etc. — while the virtual camera (the digital counterpart to the studio camera) renders a geometrically perfect image by default. Lens calibration aligns the two so they match. (Distortion calibration)

Tracking Calibration

The idea behind tracking calibration is that a tracking system can only measure the position and rotation of its tracking device, while virtual production requires the position and rotation of the studio camera’s no-parallax point (near the sensor). Tracking calibration computes the offset between these two transforms.

Calibration Tools

The official documentation on calibration mentions two tools: Basic Calibrator and Camera Calibrator. Both are found in the software installation directory.

image-20251009165746851 image-20251009165932446
Basic Calibrator

Basic Calibrator is Aximmetry’s manual camera calibration tool. It’s used to adjust intrinsic and extrinsic camera parameters so that the virtual scene and the real camera share consistent perspective and motion.

Official tutorial: Aximmetry Basic Calibrator Tutorial

Key Concepts
  • Z (Zoom): Represents the minimum magnification value — the lens zoom parameter.
  • F (Focus): Represents the lens focus distance.
Calibration Steps

1. Configure Camera Properties

Start by setting the camera’s basic parameters in Basic Calibrator:

  • Sensor Size: Enter the physical sensor dimensions.
    • Example: Sony FX3 sensor is 35.6 × 23.8 mm.
  • Camera Height: Measure and enter the actual height of the camera above the floor. This is critical for tracking calibration.

2. Set the Virtual Ground Plane

  • Pan Virtual: Sets the height of the ground plane in the virtual scene.
  • Make sure the virtual ground plane aligns with the real floor — this is the reference baseline for all subsequent calibration.

3. Add Calibration Points

  • Click Add to add calibration points.
  • Use at least 5 calibration points for better accuracy.
  • Important notes:
    • Avoid placing calibration points near the edges of the frame, where lens distortion is greatest.
    • Distribute points evenly across the central area of the image.
    • Choose clearly identifiable reference points in the scene (floor marks, wall corners, etc.).

4. Adjust Focal Length

This is the core step. You adjust based on how well the virtual grid tracks the real scene:

  • Criterion: Pan the camera left/right or up/down and watch how the virtual grid moves relative to real-world markers.
    • If the virtual grid rotates faster than the real markers: increase Focal Length.
    • If the virtual grid rotates slower than the real markers: decrease Focal Length.
  • Method: Incrementally tweak Focal Length and repeat the pan test until the virtual grid perfectly follows the real scene.

5. Verify the Calibration

  • Test camera movement from different positions and angles.
  • Confirm that the virtual and real scenes stay in sync in all directions.
  • If you see drift, add more calibration points or re-tune the parameters.
Tips
  • Use a stable tripod or fluid head — camera shake degrades calibration accuracy.
  • Calibrate in good lighting so reference points are clearly visible.
  • Re-calibrate periodically, especially after swapping lenses or changing focal length.
  • Save the calibration profile so you can reload it later.
Supported Tracking System Types

Basic Calibrator currently supports calibration for two types of tracking systems:

image-20251009170158715

  • 6DOF (6 Degrees of Freedom)

    • Definition: Captures the camera’s full spatial position (X, Y, Z) and rotation (Pitch, Yaw, Roll) simultaneously.
    • Characteristics: Full spatial motion tracking for the camera.
    • Use cases: High-precision virtual production and mixed reality.
    • Typical hardware: External professional tracking systems such as OptiTrack, Vicon, etc.

  • PTZ (Pan-Tilt-Zoom)

    • Definition: Cameras with pan (horizontal rotation), tilt (vertical rotation), and zoom capabilities.
    • Characteristics: No full spatial position tracking — primarily tracks rotation and zoom.
    • Use cases: Studio monitoring or simple virtual production setups.
    • Typical hardware: PTZ camera systems.
Device Configuration Example

Adding Aximmetry Eye in the Device Mapper:

image-20250930155931126
image-20250930164348165

Important: When using Aximmetry Eye, always connect via a wired ethernet cable to get stable tracking data. Wireless connections can introduce latency and connection drops.

Other Thoughts

The iPhone Sync Problem

Earlier versions of Aximmetry Eye didn’t support timecode or Genlock on iPhone, which creates a bunch of sync headaches.

This forum thread discusses the issue. An interesting question: now that iPhone 17 reportedly supports Genlock, can it be used directly as an external tracker?

This YouTube video mentions an external sync device for iPhone made by Blackmagic.

Problems I Ran Into

Engine Version Incompatibility

The official docs acknowledge this issue too. It affects recompiling third-party plugins and general compatibility.

Aximmetry’s UE build is not the full official source distribution, so recompiling complex C++ plugins — especially those that depend on private modules or rendering pipeline extensions — often fails.

Options:

  1. Use Blueprint-only or pure-asset plugins.
  2. Look for pre-compiled binaries built for this specific engine version.
  3. If you must use a C++ plugin, first verify it works in a vanilla UE 5.5 project, then try running it in Aximmetry’s engine.

Capture Card Format Incompatibility

When setting up the project you may run into issues where the capture card’s model or resolution isn’t detected, or the image goes black after the card disconnects.
In that case, unplug and replug the capture card, then restart the software.

Building Our Own XR Tool

We have in-house development needs. Given that Aximmetry’s approach — placing a capture card feed as a flat plane inside the scene — is actually pretty simple logic, I decided to hand-roll a small XR tool. The goal is to use it as a prototype for our larger application: an editor-side tool that handles the entire compositing and rendering pipeline inside UE, without Aximmetry.

The feature set is intentionally minimal:

  1. Ingest a green screen image
  2. Keep the green screen plane always facing the virtual camera (billboard)
  3. Multi-camera switching and multi-camera motion
  4. Full-screen output on a chosen display

Ingesting the Green Screen Image

image-20251104155838931

Create a new level, then add a MediaPlate to the scene.

Import the green screen video (you can drag any UE-supported format directly into the editor panel) and connect it to the MediaPlate.

image-20251104160039501

Duplicate the MediaPlate’s material into a local copy, add an MF_ChromaKeyer node, wire the material graph as shown, and assign the copied material back to the MediaPlate.

image-20251104160012519

Edit the sub-material to select the green color range for keying. Save, and the chroma key is done. I’m using a temporary video clip for now — in production this will be replaced by a live feed from a capture card.

image-20251104160147691

At this point we can display a keyed plate in the scene.

However, the level can’t be saved in this state — the MediaPlate loses its source material when you close and reopen the scene.

So let’s make the material setup persistent:

image-20251104165145468

Create a MediaPlayer asset and set its source to the green screen video file you imported.

image-20251104165246306

Create a MediaTexture asset and point its media source to the MediaPlayer you just created.

image-20251104165327820

Drag the MediaTexture into the MediaPlate’s material and update the input source. After saving, the MediaPlate works correctly on reload.
At this point, green screen ingestion is complete.

Keeping the Green Screen Facing the Camera (Billboard)

There’s a problem: when we switch camera angles, the flat plate doesn’t follow. We need to configure the plate so it always faces the active camera.

image-20251104171602887
image-20251104172150884

Without it, you get the obvious flat-plane look shown above.

Multi-Camera Switching and Movement

Since we’re controlling UE cameras directly, camera movement works in both Play mode and Editor mode via UE’s Sequencer. However, cut switching between cameras doesn’t work as smoothly out of the box, so I built a small tool to handle fast camera cuts.

image-20251104174529386

At runtime, click the control panel to switch between cameras.

Full-Screen Output on a Chosen Display

image-20251104174853388

Select the target display and output resolution, then push the render to full-screen. During a live broadcast you can have OBS capture that window.

With this, we’ve bypassed Aximmetry and its custom engine entirely — compositing and rendering happen inside standard UE, and we’re free to use stock UE and its open-source plugin ecosystem instead of being locked to Aximmetry’s special engine build.

Of course, this plugin still has a lot of rough edges. Just for fun. There are plenty of issues to solve before it’s production-ready, calibration being the most obvious one.

Full Source Code

Plugin Structure Overview

image-20251104180744538

Source

STcgXRViewport.cpp

File path: Private\STcgXRViewport.cpp

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
#include "STcgXRViewport.h"
#include "TcgXRViewportClient.h"
#include "PreviewScene.h"
#include "Editor.h"
#include "LevelEditor.h"
#include "Engine/World.h"
#include "Engine/Level.h"
#include "GameFramework/Actor.h"
#include "CineCameraActor.h"
#include "CineCameraComponent.h"
#include "LevelEditorViewport.h" // FLevelEditorViewportClient
#include "TcgXR.h"

// Constructor — initializes the custom Viewport
void STcgXRViewport::Construct(const FArguments& InArgs)
{
    SEditorViewport::Construct(SEditorViewport::FArguments());
}

// Creates the custom ViewportClient
TSharedRef<FEditorViewportClient> STcgXRViewport::MakeEditorViewportClient()
{
    if (!PreviewScene.IsValid())
    {
        PreviewScene = MakeUnique<FPreviewScene>(FPreviewScene::ConstructionValues());
    }

    ViewportClient = MakeShared<FTcgXRViewportClient>(PreviewScene.Get(), StaticCastSharedRef<SEditorViewport>(SharedThis(this)));
    ViewportClient->SetViewMode(VMI_Lit);

    // Copy camera parameters from the first perspective editor viewport
    if (GEditor)
    {
        for (FEditorViewportClient* VC : GEditor->GetAllViewportClients())
        {
            if (VC && VC->IsPerspective())
            {
                const FVector Loc = VC->GetViewLocation();
                const FRotator Rot = VC->GetViewRotation();
                const bool bIsOrtho = VC->IsOrtho();
                const float FOV = VC->ViewFOV; // use source viewport FOV
                const float OrthoWidth = 2048.f; // default ortho width
                ViewportClient->SetCameraFrom(Loc, Rot, FOV, OrthoWidth, bIsOrtho);
                break;
            }
        }
    }

    // Apply any cached parameters (won't crash if UI triggered setters before Client was created)
    ApplyCachedParams();

    return ViewportClient.ToSharedRef();
}

void STcgXRViewport::ApplyCachedParams()
{
    if (!ViewportClient.IsValid()) return;
    if (CachedFOV.IsSet()) { ViewportClient->SetFOV(CachedFOV.GetValue()); CachedFOV.Reset(); }
    if (CachedExposure.IsSet()) { ViewportClient->SetExposure(CachedExposure.GetValue()); CachedExposure.Reset(); }
    if (CachedFocalLength.IsSet()) { ViewportClient->SetFocalLength(CachedFocalLength.GetValue()); CachedFocalLength.Reset(); }
    if (CachedAperture.IsSet()) { ViewportClient->SetAperture(CachedAperture.GetValue()); CachedAperture.Reset(); }
    if (CachedFocusDistance.IsSet()) { ViewportClient->SetFocusDistance(CachedFocusDistance.GetValue()); CachedFocusDistance.Reset(); }
}

// XR state control interface
void STcgXRViewport::SetShowKeyed(bool b) { if (ViewportClient) ViewportClient->SetShowKeyed(b); }
void STcgXRViewport::SetVirtualCamMove(bool b) { if (ViewportClient) ViewportClient->SetVirtualCamMove(b); }
void STcgXRViewport::SetMixedRealityBlend(bool b) { if (ViewportClient) ViewportClient->SetMixedRealityBlend(b); }

bool STcgXRViewport::IsShowKeyed() const { return ViewportClient ? ViewportClient->IsShowKeyed() : false; }
bool STcgXRViewport::IsVirtualCamMove() const { return ViewportClient ? ViewportClient->IsVirtualCamMove() : false; }
bool STcgXRViewport::IsMixedRealityBlend() const { return ViewportClient ? ViewportClient->IsMixedRealityBlend() : true; }

// Utility: get the CineCameraComponent currently locked in any level viewport (Pilot/locked state)
static UCineCameraComponent* GetLockedCineCamera()
{
    if (!GEditor) return nullptr;
    const TArray<FLevelEditorViewportClient*>& Clients = GEditor->GetLevelViewportClients();
    for (FLevelEditorViewportClient* LVC : Clients)
    {
        if (!LVC || !LVC->IsPerspective()) continue;
        AActor* LockedActor = nullptr;
        if (AActor* CineLock = LVC->GetCinematicActorLock().GetLockedActor())
        {
            LockedActor = CineLock;
        }
        else if (LVC->GetActiveActorLock().IsValid())
        {
            LockedActor = LVC->GetActiveActorLock().Get();
        }
        if (LockedActor)
        {
            if (ACineCameraActor* Cine = Cast<ACineCameraActor>(LockedActor))
            {
                return Cine->GetCineCameraComponent();
            }
        }
    }
    return nullptr;
}

// Capture keyboard input and dispatch to the module for camera switching
FReply STcgXRViewport::OnKeyDown(const FGeometry& MyGeometry, const FKeyEvent& InKeyEvent)
{
    if (FTcgXRModule* Mod = FModuleManager::GetModulePtr<FTcgXRModule>("TcgXR"))
    {
        if (Mod->HandleKeyDown(InKeyEvent.GetKey()))
        {
            return FReply::Handled();
        }
    }
    return SEditorViewport::OnKeyDown(MyGeometry, InKeyEvent);
}

// Safe parameter setters: prefer modifying the camera locked to the level viewport;
// fall back to local cache / ViewportClient otherwise
void STcgXRViewport::SetFOVParam(float InFOV)
{
    if (UCineCameraComponent* CineComp = GetLockedCineCamera())
    {
        CineComp->SetFieldOfView(InFOV);
        return;
    }
    if (ViewportClient.IsValid()) { ViewportClient->SetFOV(InFOV); }
    else { CachedFOV = InFOV; }
}

void STcgXRViewport::SetExposureParam(float InExposure)
{
    if (UCineCameraComponent* CineComp = GetLockedCineCamera())
    {
        FPostProcessSettings& PPS = CineComp->PostProcessSettings;
        PPS.bOverride_AutoExposureBias = true;
        PPS.AutoExposureBias = InExposure;
        CineComp->PostProcessBlendWeight = 1.0f;
        CineComp->MarkRenderStateDirty();
        return;
    }
    if (ViewportClient.IsValid()) { ViewportClient->SetExposure(InExposure); }
    else { CachedExposure = InExposure; }
}

void STcgXRViewport::SetFocalLengthParam(float InFocal)
{
    if (UCineCameraComponent* CineComp = GetLockedCineCamera())
    {
        CineComp->SetCurrentFocalLength(InFocal);
        return;
    }
    if (ViewportClient.IsValid()) { ViewportClient->SetFocalLength(InFocal); }
    else { CachedFocalLength = InFocal; }
}

void STcgXRViewport::SetApertureParam(float InAperture)
{
    if (UCineCameraComponent* CineComp = GetLockedCineCamera())
    {
        CineComp->SetCurrentAperture(InAperture);
        return;
    }
    if (ViewportClient.IsValid()) { ViewportClient->SetAperture(InAperture); }
    else { CachedAperture = InAperture; }
}

void STcgXRViewport::SetFocusDistanceParam(float InFocus)
{
    if (UCineCameraComponent* CineComp = GetLockedCineCamera())
    {
        CineComp->FocusSettings.ManualFocusDistance = InFocus;
        CineComp->SetFocusSettings(CineComp->FocusSettings);
        return;
    }
    if (ViewportClient.IsValid()) { ViewportClient->SetFocusDistance(InFocus); }
    else { CachedFocusDistance = InFocus; }
}

TcgXR.cpp

File path: Private\TcgXR.cpp

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
// Copyright Epic Games, Inc. All Rights Reserved.
// This file is the main module for the TcgXR plugin, handling initialization, UI layout, and command registration.

#include "TcgXR.h"
#include "TcgXRStyle.h"
#include "TcgXRCommands.h"
#include "ToolMenus.h"
#include "Framework/Docking/TabManager.h"
#include "Widgets/Docking/SDockTab.h"
#include "Widgets/Layout/SBorder.h"
#include "Widgets/SBoxPanel.h"
#include "Widgets/Input/SCheckBox.h"
#include "Widgets/Input/SButton.h"
#include "Widgets/Input/SComboBox.h"
#include "Widgets/Input/SSlider.h"
#include "Widgets/Input/SSpinBox.h"
#include "Widgets/Input/SComboButton.h"
#include "Widgets/Text/STextBlock.h"
#include "Widgets/SWindow.h"
#include "Framework/Application/SlateApplication.h"
#include "STcgXRViewport.h"
#include "TcgXRViewportClient.h"
#include "LevelEditor.h"
#include "LevelEditorViewport.h"
#include "SLevelViewport.h"
#include "EngineUtils.h"
#include "Camera/CameraActor.h"
#include "Editor.h" // FEditorDelegates

static const FName TcgXRTabName("TcgXRWindow");

#define LOCTEXT_NAMESPACE "FTcgXRModule"

void FTcgXRModule::StartupModule()
{
	FTcgXRStyle::Initialize();
	FTcgXRStyle::ReloadTextures();

	FTcgXRCommands::Register();
	
	PluginCommands = MakeShareable(new FUICommandList);
	PluginCommands->MapAction(
		FTcgXRCommands::Get().PluginAction,
		FExecuteAction::CreateRaw(this, &FTcgXRModule::PluginButtonClicked),
		FCanExecuteAction());

	FGlobalTabmanager::Get()->RegisterNomadTabSpawner(TcgXRTabName,
		FOnSpawnTab::CreateRaw(this, &FTcgXRModule::OnSpawnPluginTab))
		.SetDisplayName(LOCTEXT("TcgXRTabTitle", "TcgXR"))
		.SetMenuType(ETabSpawnerMenuType::Hidden);

	UToolMenus::RegisterStartupCallback(FSimpleMulticastDelegate::FDelegate::CreateRaw(this, &FTcgXRModule::RegisterMenus));

	if (GEngine)
	{
		ActorAddedHandle = GEngine->OnLevelActorAdded().AddLambda([this](AActor* /*Actor*/){ RefreshCameraList(); });
		ActorDeletedHandle = GEngine->OnLevelActorDeleted().AddLambda([this](AActor* /*Actor*/){ RefreshCameraList(); });
	}
	// Refresh "currently controlled" text when editor camera moves or lock changes
	FEditorDelegates::OnEditorCameraMoved.AddLambda([this](const FVector&, const FRotator&, ELevelViewportType, int32){ RebuildCameraHotkeyUI(); });
}

void FTcgXRModule::ShutdownModule()
{
	UToolMenus::UnRegisterStartupCallback(this);
	UToolMenus::UnregisterOwner(this);

	FGlobalTabmanager::Get()->UnregisterNomadTabSpawner(TcgXRTabName);

	if (GEngine)
	{
		GEngine->OnLevelActorAdded().Remove(ActorAddedHandle);
		GEngine->OnLevelActorDeleted().Remove(ActorDeletedHandle);
	}

	FTcgXRStyle::Shutdown();
	FTcgXRCommands::Unregister();
}

void FTcgXRModule::PluginButtonClicked()
{
	FGlobalTabmanager::Get()->TryInvokeTab(TcgXRTabName);
}

void FTcgXRModule::RegisterMenus()
{
	FToolMenuOwnerScoped OwnerScoped(this);

	if (UToolMenu* Menu = UToolMenus::Get()->ExtendMenu("LevelEditor.MainMenu.Window"))
	{
		FToolMenuSection& Section = Menu->FindOrAddSection("WindowLayout");
		Section.AddMenuEntryWithCommandList(FTcgXRCommands::Get().PluginAction, PluginCommands);
	}

	if (UToolMenu* ToolbarMenu = UToolMenus::Get()->ExtendMenu("LevelEditor.LevelEditorToolBar.PlayToolBar"))
	{
		FToolMenuSection& Section = ToolbarMenu->FindOrAddSection("PluginTools");
		FToolMenuEntry& Entry = Section.AddEntry(FToolMenuEntry::InitToolBarButton(FTcgXRCommands::Get().PluginAction));
		Entry.SetCommandList(PluginCommands);
		Entry.Icon = FSlateIcon(FTcgXRStyle::GetStyleSetName(), TEXT("TcgXR.PluginAction.Small"), TEXT("TcgXR.PluginAction"));
	}
}

bool FTcgXRModule::HandleKeyDown(const FKey& Key)
{
	if (TWeakObjectPtr<ACameraActor>* Found = CameraHotkeyMap.Find(Key))
	{
		SwitchToCamera(*Found);
		RebuildCameraHotkeyUI();
		return true;
	}
	return false;
}

void FTcgXRModule::BindCameraHotkey(TWeakObjectPtr<ACameraActor> Camera, const FKey& Key)
{
	// Ensure a camera only has one binding: remove the old one first
	TArray<FKey> KeysToRemove;
	for (const auto& Pair : CameraHotkeyMap)
	{
		if (Pair.Value == Camera)
		{
			KeysToRemove.Add(Pair.Key);
		}
	}
	for (const FKey& OldKey : KeysToRemove)
	{
		CameraHotkeyMap.Remove(OldKey);
	}
	// Rebind the key to the new camera
	CameraHotkeyMap.Add(Key, Camera);
	RebuildCameraHotkeyUI();
}

void FTcgXRModule::SwitchToCamera(TWeakObjectPtr<ACameraActor> Camera)
{
	if (!Camera.IsValid()) return;
	FLevelEditorModule& LevelEditorModule = FModuleManager::LoadModuleChecked<FLevelEditorModule>("LevelEditor");
	TSharedPtr<SLevelViewport> ActiveViewport = LevelEditorModule.GetFirstActiveLevelViewport();
	if (!ActiveViewport.IsValid()) return;

	FLevelEditorViewportClient& LVC = ActiveViewport->GetLevelViewportClient();
	LVC.SetActorLock(Camera.Get());
	LVC.UpdateViewForLockedActor();
	if (!ActiveViewport->IsLockedCameraViewEnabled())
	{
		ActiveViewport->ToggleActorPilotCameraView();
	}
	LVC.Invalidate();
}

FText FTcgXRModule::GetCurrentControlledCameraText() const
{
	if (!GEditor) return LOCTEXT("NoEditor", "Currently controlling: none (editor not found)");
	const TArray<FLevelEditorViewportClient*>& Clients = GEditor->GetLevelViewportClients();
	for (FLevelEditorViewportClient* LVC : Clients)
	{
		if (!LVC || !LVC->IsPerspective()) continue;
		if (AActor* CineLock = LVC->GetCinematicActorLock().GetLockedActor())
		{
			return FText::Format(LOCTEXT("ControllingCam","Currently controlling: {0}"), FText::FromString(CineLock->GetActorLabel()));
		}
		if (LVC->GetActiveActorLock().IsValid())
		{
			AActor* Locked = LVC->GetActiveActorLock().Get();
			return FText::Format(LOCTEXT("ControllingActor","Currently controlling: {0}"), FText::FromString(Locked->GetActorLabel()));
		}
	}
	return LOCTEXT("NoCam","Currently controlling: none (no camera locked)");
}

FText FTcgXRModule::GetBoundKeyTextForCamera(TWeakObjectPtr<ACameraActor> Camera) const
{
	for (const auto& Pair : CameraHotkeyMap)
	{
		if (Pair.Value == Camera)
		{
			return FText::Format(LOCTEXT("BoundKeyFmt","Bound: {0}"), Pair.Key.GetDisplayName());
		}
	}
	return LOCTEXT("NoKeyBound","Not bound");
}

void FTcgXRModule::RefreshCameraList()
{
	CachedCameras.Reset();
	if (GEditor)
	{
		UWorld* World = GEditor->GetEditorWorldContext().World();
		if (World)
		{
			for (TActorIterator<ACameraActor> It(World); It; ++It)
			{
				CachedCameras.Add(*It);
			}
		}
	}
	RebuildCameraHotkeyUI();
}

void FTcgXRModule::RebuildCameraHotkeyUI()
{
	TSharedPtr<SVerticalBox> Panel = CameraListPanel.Pin();
	if (!Panel.IsValid()) return;
	Panel->ClearChildren();

	// Show currently controlled camera at the top
	Panel->AddSlot().AutoHeight().Padding(0,2)
	[
		SNew(STextBlock)
		.Text_Lambda([this](){ return GetCurrentControlledCameraText(); })
	];

	int32 Index = 1;
	for (TWeakObjectPtr<ACameraActor> Cam : CachedCameras)
	{
		TWeakObjectPtr<ACameraActor> LocalCam = Cam;
		FText Label = FText::FromString(FString::Printf(TEXT("Camera %d: %s"), Index, LocalCam.IsValid() ? *LocalCam->GetActorLabel() : TEXT("(none)")));
		Panel->AddSlot().AutoHeight().Padding(0,2)
		[
			SNew(SHorizontalBox)
			+ SHorizontalBox::Slot().AutoWidth().Padding(2)
			[
				SNew(STextBlock).Text(Label)
			]
			+ SHorizontalBox::Slot().AutoWidth().Padding(8,2)
			[
				SNew(STextBlock)
				.Text_Lambda([this, LocalCam](){ return GetBoundKeyTextForCamera(LocalCam); })
			]
			+ SHorizontalBox::Slot().AutoWidth().Padding(2)
			[
				SNew(SComboButton)
				.ButtonContent()[ SNew(STextBlock).Text(LOCTEXT("BindKey", "Bind Key")) ]
				.OnGetMenuContent_Lambda([this, LocalCam]()
				{
					FMenuBuilder MenuBuilder(true, nullptr);
					TArray<FKey> Keys; Keys.Reserve(9);
					Keys.Add(EKeys::One); Keys.Add(EKeys::Two); Keys.Add(EKeys::Three); Keys.Add(EKeys::Four);
					Keys.Add(EKeys::Five); Keys.Add(EKeys::Six); Keys.Add(EKeys::Seven); Keys.Add(EKeys::Eight); Keys.Add(EKeys::Nine);
					for (const FKey& K : Keys)
					{
						FText KeyName = K.GetDisplayName();
						MenuBuilder.AddMenuEntry(KeyName, LOCTEXT("BindKeyTip","Bind to this key"), FSlateIcon(), FUIAction(FExecuteAction::CreateLambda([this, LocalCam, K](){ BindCameraHotkey(LocalCam, K); })));
					}
					return MenuBuilder.MakeWidget();
				})
			]
			+ SHorizontalBox::Slot().AutoWidth().Padding(2)
			[
				SNew(SButton)
				.Text(LOCTEXT("SwitchNow","Switch to Camera"))
				.OnClicked_Lambda([this, LocalCam]()
				{
					SwitchToCamera(LocalCam);
					RebuildCameraHotkeyUI();
					return FReply::Handled();
				})
			]
		];
		++Index;
	}
}

void FTcgXRModule::CloseAllFullscreenWindows()
{
	for (TWeakPtr<SWindow>& W : FullscreenWindows)
	{
		if (TSharedPtr<SWindow> SW = W.Pin())
		{
			SW->RequestDestroyWindow();
		}
	}
	FullscreenWindows.Reset();
}

// Spawns the main window Tab: Viewport + right-side camera control panel + bottom fullscreen buttons
TSharedRef<SDockTab> FTcgXRModule::OnSpawnPluginTab(const FSpawnTabArgs& Args)
{
	TSharedPtr<STcgXRViewport> ViewportWidget;

	static float ManualExposure = 0.0f;
	static float ManualFOV = 90.0f;
	static float ManualFocal = 35.0f;
	static float ManualAperture = 2.8f;
	static float ManualFocus = 1000.0f;
	static bool bUseLiveLink = false;
	static TArray<TSharedPtr<FString>> ResolutionOptions;
	ResolutionOptions.Reset();
	ResolutionOptions.Add(MakeShared<FString>(TEXT("1920x1080")));
	ResolutionOptions.Add(MakeShared<FString>(TEXT("2560x1440")));
	ResolutionOptions.Add(MakeShared<FString>(TEXT("3840x2160")));
	static TArray<TSharedPtr<FString>> MonitorOptions;
	static int32 SelectedMonitorIndex = 0;
	static TSharedPtr<FString> SelectedResolution = ResolutionOptions.Num() > 0 ? ResolutionOptions[0] : nullptr;

	MonitorOptions.Reset();
	{
		FDisplayMetrics Metrics;
		FSlateApplication::Get().GetDisplayMetrics(Metrics);
		MonitorOptions.Add(MakeShared<FString>(FString::Printf(TEXT("Primary Display %dx%d"), Metrics.PrimaryDisplayWidth, Metrics.PrimaryDisplayHeight)));
		for (int32 i = 0; i < Metrics.MonitorInfo.Num(); ++i)
		{
			const auto& M = Metrics.MonitorInfo[i];
			FPlatformRect Rect = M.WorkArea;
			MonitorOptions.Add(MakeShared<FString>(FString::Printf(TEXT("Monitor %d %dx%d @(%d,%d)"), i, Rect.Right-Rect.Left, Rect.Bottom-Rect.Top, Rect.Left, Rect.Top)));
		}
		if (MonitorOptions.Num() == 0)
		{
			MonitorOptions.Add(MakeShared<FString>(TEXT("Monitor 0")));
		}
	}

	TSharedRef<SDockTab> Tab = SNew(SDockTab)
		.TabRole(ETabRole::NomadTab)
		[
			SNew(SHorizontalBox)
			+ SHorizontalBox::Slot().FillWidth(1.f)
			[
				SAssignNew(ViewportWidget, STcgXRViewport)
			]
			+ SHorizontalBox::Slot().AutoWidth().Padding(8.f,0.f)
			[
				SNew(SVerticalBox)
				+ SVerticalBox::Slot().AutoHeight().Padding(0,2)
				[
					SAssignNew(CameraListPanel, SVerticalBox)
				]
				// Top parameter row: FOV and exposure removed
				+ SVerticalBox::Slot().AutoHeight().Padding(0, 2)
				[
					SNew(SHorizontalBox)
					// Focal Length
					+ SHorizontalBox::Slot().AutoWidth().Padding(2)
					[
						SNew(STextBlock).Text(LOCTEXT("Focal", "Focal Length"))
					]
					+ SHorizontalBox::Slot().AutoWidth().Padding(2)
					[
						SNew(SSpinBox<float>).MinValue(1.f).MaxValue(300.f)
						.Value_Lambda([&]{ return ManualFocal; })
						.OnValueChanged_Lambda([&](float V){ ManualFocal = V; if(ViewportWidget.IsValid()) ViewportWidget->SetFocalLengthParam(V); })
					]
					// Aperture
					+ SHorizontalBox::Slot().AutoWidth().Padding(8,2)
					[
						SNew(STextBlock).Text(LOCTEXT("Aperture", "Aperture"))
					]
					+ SHorizontalBox::Slot().AutoWidth().Padding(2)
					[
						SNew(SSpinBox<float>).MinValue(0.7f).MaxValue(32.f)
						.Value_Lambda([&]{ return ManualAperture; })
						.OnValueChanged_Lambda([&](float V){ ManualAperture = V; if(ViewportWidget.IsValid()) ViewportWidget->SetApertureParam(V); })
					]
					// Focus Distance
					+ SHorizontalBox::Slot().AutoWidth().Padding(8,2)
					[
						SNew(STextBlock).Text(LOCTEXT("Focus", "Focus"))
					]
					+ SHorizontalBox::Slot().AutoWidth().Padding(2)
					[
						SNew(SSpinBox<float>).MinValue(1.f).MaxValue(100000.f)
						.Value_Lambda([&]{ return ManualFocus; })
						.OnValueChanged_Lambda([&](float V){ ManualFocus = V; if(ViewportWidget.IsValid()) ViewportWidget->SetFocusDistanceParam(V); })
					]
				]

				+ SVerticalBox::Slot().AutoHeight().Padding(0,6,0,0)
				[
					SNew(SHorizontalBox)
					+ SHorizontalBox::Slot().AutoWidth().Padding(2.f)
					[
						SNew(SCheckBox)
						.IsChecked_Lambda([] { return bUseLiveLink ? ECheckBoxState::Checked : ECheckBoxState::Unchecked; })
						.OnCheckStateChanged_Lambda([](ECheckBoxState State) { bUseLiveLink = (State == ECheckBoxState::Checked); })
					]
					+ SHorizontalBox::Slot().AutoWidth().Padding(2.f)
					[
						SNew(STextBlock).Text(LOCTEXT("LiveLinkSwitch", "Use LiveLink External Camera"))
					]
				]

				+ SVerticalBox::Slot().AutoHeight().Padding(0, 8, 0, 0)
				[
					SNew(SVerticalBox)
					.Visibility_Lambda([] { return bUseLiveLink ? EVisibility::Visible : EVisibility::Collapsed; })
					+ SVerticalBox::Slot().AutoHeight()
					[
						SNew(STextBlock).Text(LOCTEXT("LiveLinkTitle", "LiveLink External Camera Parameters"))
					]
					+ SVerticalBox::Slot().AutoHeight().Padding(0, 2)
					[
						SNew(STextBlock).Text(LOCTEXT("LiveLinkStatus", "Status: Disconnected / Connected"))
					]
				]

				+ SVerticalBox::Slot().AutoHeight().VAlign(VAlign_Bottom).Padding(0, 16, 0, 0)
				[
					SNew(SVerticalBox)
					+ SVerticalBox::Slot().AutoHeight().Padding(0, 8, 0, 0)
					[
						SNew(SHorizontalBox)
						+ SHorizontalBox::Slot().AutoWidth().Padding(2.f)
						[
							SNew(STextBlock).Text(LOCTEXT("MonitorLabel", "Display"))
						]
						+ SHorizontalBox::Slot().AutoWidth().Padding(2.f)
						[
							SNew(SComboBox<TSharedPtr<FString>>)
							.OptionsSource(&MonitorOptions)
							.OnSelectionChanged_Lambda([&](TSharedPtr<FString> NewItem, ESelectInfo::Type){ SelectedMonitorIndex = FMath::Clamp(MonitorOptions.IndexOfByKey(NewItem), 0, MonitorOptions.Num()-1); })
							.OnGenerateWidget_Lambda([](TSharedPtr<FString> InItem){ return SNew(STextBlock).Text(FText::FromString(*InItem)); })
							[
								SNew(STextBlock).Text_Lambda([&]{ return MonitorOptions.IsValidIndex(SelectedMonitorIndex) ? FText::FromString(*MonitorOptions[SelectedMonitorIndex]) : LOCTEXT("MonitorUnknown","Unknown Display"); })
							]
						]
					]
					+ SVerticalBox::Slot().AutoHeight().Padding(0, 4, 0, 0)
					[
						SNew(SHorizontalBox)
						+ SHorizontalBox::Slot().AutoWidth().Padding(2.f)
						[
							SNew(STextBlock).Text(LOCTEXT("ResolutionLabel", "Output Resolution"))
						]
						+ SHorizontalBox::Slot().AutoWidth().Padding(2.f)
						[
							SNew(SComboBox<TSharedPtr<FString>>)
							.OptionsSource(&ResolutionOptions)
							.OnSelectionChanged_Lambda([&](TSharedPtr<FString> NewItem, ESelectInfo::Type) { SelectedResolution = NewItem; })
							.OnGenerateWidget_Lambda([](TSharedPtr<FString> InItem) { return SNew(STextBlock).Text(FText::FromString(*InItem)); })
							[
								SNew(STextBlock).Text_Lambda([&] { return SelectedResolution.IsValid() ? FText::FromString(*SelectedResolution) : LOCTEXT("ResUnknown","Unknown"); })
							]
						]
					]
					+ SVerticalBox::Slot().AutoHeight().Padding(0, 6, 0, 0)
					[
						SNew(SHorizontalBox)
						+ SHorizontalBox::Slot().AutoWidth().Padding(2)
						[
							SNew(SButton)
							.Text(LOCTEXT("FullscreenBtn", "Fullscreen Output"))
							.OnClicked_Lambda([this, ViewportWidget]() {
								if(!ViewportWidget.IsValid()) return FReply::Handled();
								TSharedPtr<FString> LocalSelectedResolution = SelectedResolution;
								int32 LocalSelectedMonitorIndex = SelectedMonitorIndex;
								int32 W=1920, H=1080;
								{
									FString Res= LocalSelectedResolution.IsValid() ? *LocalSelectedResolution : FString(TEXT("1920x1080"));
									FString LW, LH;
									if(Res.Split(TEXT("x"), &LW, &LH)) { W = FCString::Atoi(*LW); H = FCString::Atoi(*LH); }
								}
								TSharedPtr<SWindow> FullscreenWindow = SNew(SWindow)
								.Title(LOCTEXT("XRFullscreen", "XR Fullscreen Output"))
								.ClientSize(FVector2D(W, H))
								.SizingRule(ESizingRule::FixedSize)
								.SupportsMaximize(false)
								.SupportsMinimize(false)
								.HasCloseButton(true)
								.UseOSWindowBorder(false)
								.CreateTitleBar(false);

								TSharedPtr<STcgXRViewport> FSViewport;
								FullscreenWindow->SetContent(SAssignNew(FSViewport, STcgXRViewport));

								if (FSlateApplication::IsInitialized())
								{
									FSlateApplication::Get().AddWindow(FullscreenWindow.ToSharedRef());
									FDisplayMetrics Metrics; FSlateApplication::Get().GetDisplayMetrics(Metrics);
									FVector2D TargetPos(0,0);
									if (Metrics.MonitorInfo.IsValidIndex(LocalSelectedMonitorIndex))
									{
										const auto& M = Metrics.MonitorInfo[LocalSelectedMonitorIndex];
										TargetPos = FVector2D(M.WorkArea.Left, M.WorkArea.Top);
									}
									FullscreenWindow->MoveWindowTo(TargetPos);
								}

								FullscreenWindows.Add(FullscreenWindow);

								if (ViewportWidget->GetClient().IsValid() && FSViewport.IsValid())
								{
									auto Src = ViewportWidget->GetClient();
									auto Dst = FSViewport->GetClient();
									Dst->SetCameraFrom(Src->GetViewLocation(), Src->GetViewRotation(), Src->GetFOV(), 2048.f, false);
									Dst->SetFOV(Src->GetFOV());
									Dst->SetExposure(Src->GetExposure());
									Dst->SetFocalLength(Src->GetFocalLength());
									Dst->SetAperture(Src->GetAperture());
									Dst->SetFocusDistance(Src->GetFocusDistance());
								}

								return FReply::Handled();
							})
						]
						+ SHorizontalBox::Slot().AutoWidth().Padding(2)
						[
							SNew(SButton)
							.Text(LOCTEXT("CloseFullscreenBtn", "Close All Fullscreen"))
							.OnClicked_Lambda([this]()
							{
								CloseAllFullscreenWindows();
								return FReply::Handled();
							})
						]
					]
				]
			]
		];

	RefreshCameraList();
	return Tab;
}

#undef LOCTEXT_NAMESPACE

IMPLEMENT_MODULE(FTcgXRModule, TcgXR)

TcgXRCommands.cpp

File path: Private\TcgXRCommands.cpp

1
2
3
4
5
6
7
8
9
10
11
12
// Copyright Epic Games, Inc. All Rights Reserved.

#include "TcgXRCommands.h"

#define LOCTEXT_NAMESPACE "FTcgXRModule"

void FTcgXRCommands::RegisterCommands()
{
	UI_COMMAND(PluginAction, "TcgXR", "Execute TcgXR action", EUserInterfaceActionType::Button, FInputChord());
}

#undef LOCTEXT_NAMESPACE

TcgXRStyle.cpp

File path: Private\TcgXRStyle.cpp

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
// Copyright Epic Games, Inc. All Rights Reserved.

#include "TcgXRStyle.h"
#include "TcgXR.h"
#include "Framework/Application/SlateApplication.h"
#include "Styling/SlateStyleRegistry.h"
#include "Slate/SlateGameResources.h"
#include "Interfaces/IPluginManager.h"
#include "Styling/SlateStyleMacros.h"

#define RootToContentDir Style->RootToContentDir

TSharedPtr<FSlateStyleSet> FTcgXRStyle::StyleInstance = nullptr;

void FTcgXRStyle::Initialize()
{
	if (!StyleInstance.IsValid())
	{
		StyleInstance = Create();
		FSlateStyleRegistry::RegisterSlateStyle(*StyleInstance);
	}
}

void FTcgXRStyle::Shutdown()
{
	FSlateStyleRegistry::UnRegisterSlateStyle(*StyleInstance);
	ensure(StyleInstance.IsUnique());
	StyleInstance.Reset();
}

FName FTcgXRStyle::GetStyleSetName()
{
	static FName StyleSetName(TEXT("TcgXRStyle"));
	return StyleSetName;
}


const FVector2D Icon16x16(16.0f, 16.0f);
const FVector2D Icon20x20(20.0f, 20.0f);
const FVector2D Icon40x40(40.0f, 40.0f);

TSharedRef< FSlateStyleSet > FTcgXRStyle::Create()
{
	TSharedRef< FSlateStyleSet > Style = MakeShareable(new FSlateStyleSet("TcgXRStyle"));
	Style->SetContentRoot(IPluginManager::Get().FindPlugin("TcgXR")->GetBaseDir() / TEXT("Resources"));

	// Use vector image brushes from Resources/TcgXR.svg for toolbar/menu icons
	Style->Set("TcgXR.PluginAction", new IMAGE_BRUSH_SVG(TEXT("TcgXR"), Icon40x40));
	Style->Set("TcgXR.PluginAction.Small", new IMAGE_BRUSH_SVG(TEXT("TcgXR"), Icon20x20));
	return Style;
}

void FTcgXRStyle::ReloadTextures()
{
	if (FSlateApplication::IsInitialized())
	{
		FSlateApplication::Get().GetRenderer()->ReloadTextureResources();
	}
}

const ISlateStyle& FTcgXRStyle::Get()
{
	return *StyleInstance;
}

TcgXRViewportClient.cpp

File path: Private\TcgXRViewportClient.cpp

1
2
3
// FTcgXRViewportClient implementation has been moved inline to TcgXRViewportClient.h
// This translation unit is intentionally left empty to avoid duplicate definitions.

STcgXRViewport.h

File path: Public\STcgXRViewport.h

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
#pragma once

#include "CoreMinimal.h"
#include "SEditorViewport.h"

class FTcgXRViewportClient;
class FPreviewScene;

// TcgXR custom Viewport widget — supports XR scene display and camera control
class STcgXRViewport : public SEditorViewport
{
public:
    SLATE_BEGIN_ARGS(STcgXRViewport) {}
    SLATE_END_ARGS()

    // Constructor
    void Construct(const FArguments& InArgs);

    // XR state control interface
    void SetShowKeyed(bool b); // toggle chroma key display
    void SetVirtualCamMove(bool b); // toggle virtual camera movement
    void SetMixedRealityBlend(bool b); // toggle mixed reality blend

    bool IsShowKeyed() const; // is chroma key display active?
    bool IsVirtualCamMove() const; // is virtual camera movement active?
    bool IsMixedRealityBlend() const; // is mixed reality blend active?

    // Access the underlying ViewportClient (used to copy camera/params to other viewports)
    TSharedPtr<FTcgXRViewportClient> GetClient() const { return ViewportClient; }

    // Safe camera parameter setters: if the Client isn't ready yet, cache values for later application
    void SetFOVParam(float InFOV);
    void SetExposureParam(float InExposure);
    void SetFocalLengthParam(float InFocal);
    void SetApertureParam(float InAperture);
    void SetFocusDistanceParam(float InFocus);

protected:
    // Creates the custom ViewportClient
    virtual TSharedRef<FEditorViewportClient> MakeEditorViewportClient() override;
    // Hide the default toolbar
    virtual TSharedPtr<SWidget> MakeViewportToolbar() override { return SNullWidget::NullWidget; }

    // Capture keyboard input and dispatch to module (for hotkey camera switching)
    virtual FReply OnKeyDown(const FGeometry& MyGeometry, const FKeyEvent& InKeyEvent) override;

private:
    // Apply all cached camera parameters
    void ApplyCachedParams();

private:
    TSharedPtr<FTcgXRViewportClient> ViewportClient; // viewport client
    TUniquePtr<FPreviewScene> PreviewScene; // preview scene

    // Cached camera parameters (temporary storage when Client isn't ready yet)
    TOptional<float> CachedFOV;
    TOptional<float> CachedExposure;
    TOptional<float> CachedFocalLength;
    TOptional<float> CachedAperture;
    TOptional<float> CachedFocusDistance;
};

TcgXR.h

File path: Public\TcgXR.h

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
// Copyright Epic Games, Inc. All Rights Reserved.

#pragma once

#include "CoreMinimal.h"
#include "Modules/ModuleManager.h"

class FToolBarBuilder;
class FMenuBuilder;
class ACinemaCameraActor;
class ACameraActor;
class SWindow;
class SVerticalBox;

class FTcgXRModule : public IModuleInterface
{
public:
	/** IModuleInterface implementation */
	virtual void StartupModule() override;
	virtual void ShutdownModule() override;
	
	/** This function will be bound to Command. */
	void PluginButtonClicked();

	// Hotkeys: handle a key press (called by STcgXRViewport), returns true if handled
	bool HandleKeyDown(const FKey& Key);
	// Hotkeys: bind a camera to a specific key
	void BindCameraHotkey(TWeakObjectPtr<ACameraActor> Camera, const FKey& Key);
	// Switch the level viewport to a specific camera
	void SwitchToCamera(TWeakObjectPtr<ACameraActor> Camera);
	// Refresh camera list and rebuild UI
	void RefreshCameraList();

	// Close all fullscreen windows
	void CloseAllFullscreenWindows();

	// Display the currently controlled camera name
	FText GetCurrentControlledCameraText() const;
	// Get the display text for a camera's bound hotkey
	FText GetBoundKeyTextForCamera(TWeakObjectPtr<ACameraActor> Camera) const;
	
private:
	void RegisterMenus();

	// Spawn the XR viewport tab
	TSharedRef<class SDockTab> OnSpawnPluginTab(const class FSpawnTabArgs& Args);

	// Rebuild the camera hotkey binding UI
	void RebuildCameraHotkeyUI();

private:
	TSharedPtr<class FUICommandList> PluginCommands;

	// Camera-to-hotkey mapping
	TMap<FKey, TWeakObjectPtr<ACameraActor>> CameraHotkeyMap;
	TArray<TWeakObjectPtr<ACameraActor>> CachedCameras;
	TWeakPtr<SVerticalBox> CameraListPanel;

	// Delegate handles
	FDelegateHandle ActorAddedHandle;
	FDelegateHandle ActorDeletedHandle;

	// Fullscreen window list
	TArray<TWeakPtr<SWindow>> FullscreenWindows;
};

TcgXRCommands.h

File path: Public\TcgXRCommands.h

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
// Copyright Epic Games, Inc. All Rights Reserved.

#pragma once

#include "CoreMinimal.h"
#include "Framework/Commands/Commands.h"
#include "TcgXRStyle.h"

class FTcgXRCommands : public TCommands<FTcgXRCommands>
{
public:

	FTcgXRCommands()
		: TCommands<FTcgXRCommands>(TEXT("TcgXR"), NSLOCTEXT("Contexts", "TcgXR", "TcgXR Plugin"), NAME_None, FTcgXRStyle::GetStyleSetName())
	{
	}

	// TCommands<> interface
	virtual void RegisterCommands() override;

public:
	TSharedPtr< FUICommandInfo > PluginAction;
};

TcgXRStyle.h

File path: Public\TcgXRStyle.h

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
// Copyright Epic Games, Inc. All Rights Reserved.

#pragma once

#include "CoreMinimal.h"
#include "Styling/SlateStyle.h"

class FTcgXRStyle
{
public:

	static void Initialize();

	static void Shutdown();

	/** reloads textures used by slate renderer */
	static void ReloadTextures();

	/** @return The Slate style set for the Shooter game */
	static const ISlateStyle& Get();

	static FName GetStyleSetName();

private:

	static TSharedRef< class FSlateStyleSet > Create();

private:

	static TSharedPtr< class FSlateStyleSet > StyleInstance;
};

TcgXRViewportClient.h

File path: Public\TcgXRViewportClient.h

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
#pragma once

#include "CoreMinimal.h"
#include "EditorViewportClient.h"
#include "PreviewScene.h"
#include "SEditorViewport.h"
#include "Editor.h"
#include "SceneView.h"

// TcgXR custom ViewportClient — handles camera control, XR display, etc.
class FTcgXRViewportClient : public FEditorViewportClient
{
public:
    // Constructor — initializes viewport parameters
    FTcgXRViewportClient(FPreviewScene* InPreviewScene, const TSharedRef<SEditorViewport>& InEditorViewport)
        : FEditorViewportClient(nullptr, InPreviewScene, InEditorViewport)
    {
        SetViewLocation(FVector(0, -300, 200));
        SetViewRotation(FRotator(-10, 0, 0));
        SetRealtime(true);
        EngineShowFlags.SetGrid(true);
        EngineShowFlags.EnableAdvancedFeatures();
    }

    // Render the main editor world so this viewport stays in sync with the main scene
    virtual UWorld* GetWorld() const override
    {
        return (GEditor) ? GEditor->GetEditorWorldContext().World() : nullptr;
    }

    // XR toggles
    void SetShowKeyed(bool bValue)
    {
        bShowKeyed = bValue;
        EngineShowFlags.PostProcessing = bShowKeyed; // example: toggle post-processing
        Invalidate();
    }

    void SetVirtualCamMove(bool bValue)
    {
        bVirtualCamMove = bValue;
    }

    void SetMixedRealityBlend(bool bValue)
    {
        bMixedRealityBlend = bValue;
        Invalidate();
    }

    // Camera parameter control
    void SetFOV(float InFOV)
    {
        ViewFOV = FMath::Clamp(InFOV, 15.f, 170.f);
        Invalidate();
    }

    float GetFOV() const { return ViewFOV; }

    void SetExposure(float InExposure)
    {
        ManualExposureBias = FMath::Clamp(InExposure, -10.f, 10.f);
        bApplyExposureBias = true; // enable exposure compensation
        Invalidate();
    }

    float GetExposure() const { return ManualExposureBias; }

    void SetFocalLength(float InFocalLength)
    {
        FocalLength = FMath::Max(1.f, InFocalLength);
        // Map focal length to FOV using the standard 36mm film width
        const float SensorWidth = 36.0f; // mm
        const float HFOVRadians = 2.f * FMath::Atan((SensorWidth * 0.5f) / FocalLength);
        SetFOV(FMath::RadiansToDegrees(HFOVRadians));
    }

    float GetFocalLength() const { return FocalLength; }

    void SetAperture(float InAperture)
    {
        Aperture = FMath::Clamp(InAperture, 0.7f, 32.f);
        bApplyDOF = true; // enable DOF override
        Invalidate();
    }

    float GetAperture() const { return Aperture; }

    void SetFocusDistance(float InDistance)
    {
        FocusDistance = FMath::Max(1.f, InDistance);
        bApplyDOF = true;
        Invalidate();
    }

    float GetFocusDistance() const { return FocusDistance; }

    // Copy camera parameters from an external viewport
    void SetCameraFrom(const FVector& InLoc, const FRotator& InRot, float InFOV, float InOrthoWidth, bool bInIsOrtho)
    {
        SetViewLocation(InLoc);
        SetViewRotation(InRot);
        if (bInIsOrtho)
        {
            SetViewportType(LVT_OrthoFreelook);
            SetOrthoZoom(InOrthoWidth);
        }
        else
        {
            SetViewportType(LVT_Perspective);
            ViewFOV = InFOV;
        }
        Invalidate();
    }

    virtual void Tick(float DeltaSeconds) override
    {
        FEditorViewportClient::Tick(DeltaSeconds);

        if (!bVirtualCamMove && GEditor)
        {
            for (FEditorViewportClient* VC : GEditor->GetAllViewportClients())
            {
                if (VC && VC->IsPerspective())
                {
                    const FVector Loc = VC->GetViewLocation();
                    const FRotator Rot = VC->GetViewRotation();
                    const bool bIsOrtho = VC->IsOrtho();
                    const float FOV = ViewFOV; // use current FOV
                    const float OrthoWidth = 2048.f;
                    SetCameraFrom(Loc, Rot, FOV, OrthoWidth, bIsOrtho);
                    break;
                }
            }
        }

        if (bVirtualCamMove)
        {
            const float Speed = 15.f;
            FRotator R = GetViewRotation();
            R.Yaw += Speed * DeltaSeconds;
            SetViewRotation(R);
        }
    }

    // Override post-process settings: only apply exposure bias / DOF when needed,
    // otherwise stay consistent with the editor viewport
    virtual void OverridePostProcessSettings(FSceneView& View) override
    {
        FEditorViewportClient::OverridePostProcessSettings(View);

        if (bApplyExposureBias)
        {
            View.FinalPostProcessSettings.bOverride_AutoExposureBias = true;
            View.FinalPostProcessSettings.AutoExposureBias = ManualExposureBias;
        }

        if (bApplyDOF)
        {
            View.FinalPostProcessSettings.bOverride_DepthOfFieldFstop = true;
            View.FinalPostProcessSettings.DepthOfFieldFstop = Aperture;
            View.FinalPostProcessSettings.bOverride_DepthOfFieldFocalDistance = true;
            View.FinalPostProcessSettings.DepthOfFieldFocalDistance = FocusDistance;
        }
    }

    // XR state queries
    bool IsShowKeyed() const { return bShowKeyed; }
    bool IsVirtualCamMove() const { return bVirtualCamMove; }
    bool IsMixedRealityBlend() const { return bMixedRealityBlend; }

private:
    bool bShowKeyed = false; // chroma key display active
    bool bVirtualCamMove = false; // virtual camera movement active
    bool bMixedRealityBlend = true; // mixed reality blend active

    // Adjustable camera parameters
    float ManualExposureBias = 0.f; // exposure compensation (log2 EV)
    bool  bApplyExposureBias = false;
    float FocalLength = 35.f;   // focal length (mm)
    float Aperture = 2.8f;      // aperture (f-stop)
    float FocusDistance = 1000.f; // focus distance
    bool  bApplyDOF = false;
};

TcgXR.Build.cs

File path: TcgXR.Build.cs

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
// Copyright Epic Games, Inc. All Rights Reserved.

using UnrealBuildTool;

public class TcgXR : ModuleRules
{
	public TcgXR(ReadOnlyTargetRules Target) : base(Target)
	{
		PCHUsage = ModuleRules.PCHUsageMode.UseExplicitOrSharedPCHs;
		
		PublicIncludePaths.AddRange(
			new string[] {
				// ... add public include paths required here ...
			}
			);
				
		
		PrivateIncludePaths.AddRange(
			new string[] {
				// ... add other private include paths required here ...
			}
			);
			
		
		PublicDependencyModuleNames.AddRange(
			new string[]
			{
				"Core",
				// ... add other public dependencies that you statically link with here ...
			}
			);
			
		
		PrivateDependencyModuleNames.AddRange(
			new string[]
			{
				"Projects",
				"InputCore",
				"EditorFramework",
				"UnrealEd",
				"ToolMenus",
				"CoreUObject",
				"Engine",
				"Slate",
				"SlateCore",
				"AppFramework",
				"LevelEditor",
				"CinematicCamera", // cinematic camera support
				// ... add private dependencies that you statically link with here ...	
			}
			);
		
		
		DynamicallyLoadedModuleNames.AddRange(
			new string[]
			{
				// ... add any modules that your module loads dynamically here ...
			}
			);
	}
}

References

  1. Basic Overview Video: The video I used to get started with Aximmetry.
  2. Aximmetry Eye Camera Tracking Device: Also covers how to deal with timecode sync.
  3. Virtual Production demo with Aximmetry Eye mobile app: Discusses how to stream iPhone data back into the pipeline.
  4. Aximmetry Eye Virtual Camera Tracker: Covers calibration, and mentions that WiFi 6 might be fast enough for wireless tracking.
XR Aximmetry