Merge pull request 'unreal_int' (#5) from unreal_int into main

Reviewed-on: #5
This commit is contained in:
Niklas Aumueller 2026-03-27 07:04:13 +00:00
commit dbf05ba92c
6 changed files with 188 additions and 4 deletions

View File

@ -155,6 +155,7 @@ src/main/resources
- [Configuration](docs/CONFIGURATION.md) - [Configuration](docs/CONFIGURATION.md)
- [Testing](docs/TESTING.md) - [Testing](docs/TESTING.md)
- [Source Map](docs/SOURCE_MAP.md) - [Source Map](docs/SOURCE_MAP.md)
- [Unreal Avatar](docs/Prototyp1_README_EN.md)
- [Known Issues](docs/KNOWN_ISSUES.md) - [Known Issues](docs/KNOWN_ISSUES.md)
## Known Limitations ## Known Limitations

View File

@ -74,6 +74,8 @@ Persistence schema (`binary_event`):
- **External processes**: optional Python simulator and Unreal launcher script. - **External processes**: optional Python simulator and Unreal launcher script.
- **Animation output**: JSON file written to `animation.output.path`. - **Animation output**: JSON file written to `animation.output.path`.
For Unreal-specific implementation details, see [Unreal Avatar Prototype](Prototyp1_README_EN.md).
## Shutdown Behavior ## Shutdown Behavior
`ApplicationShutdownManager` performs: `ApplicationShutdownManager` performs:

View File

@ -7,10 +7,11 @@ This folder contains detailed technical documentation for the project.
1. [Project Review](PROJECT_REVIEW.md) 1. [Project Review](PROJECT_REVIEW.md)
2. [Architecture](ARCHITECTURE.md) 2. [Architecture](ARCHITECTURE.md)
3. [Setup and Run](SETUP_AND_RUN.md) 3. [Setup and Run](SETUP_AND_RUN.md)
4. [Configuration](CONFIGURATION.md) 4. [Unreal Avatar Prototype](Prototyp1_README_EN.md)
5. [Testing](TESTING.md) 5. [Configuration](CONFIGURATION.md)
6. [Known Issues](KNOWN_ISSUES.md) 6. [Testing](TESTING.md)
7. [Source Map](SOURCE_MAP.md) 7. [Known Issues](KNOWN_ISSUES.md)
8. [Source Map](SOURCE_MAP.md)
## Document Guide ## Document Guide
@ -20,6 +21,8 @@ This folder contains detailed technical documentation for the project.
Component responsibilities, runtime data flow, and integration boundaries. Component responsibilities, runtime data flow, and integration boundaries.
- [Setup and Run](SETUP_AND_RUN.md) - [Setup and Run](SETUP_AND_RUN.md)
Reproducible setup for minimal and full integration modes. Reproducible setup for minimal and full integration modes.
- [Unreal Avatar Prototype](Prototyp1_README_EN.md)
Unreal Engine prototype details (MetaHuman setup, assets, animation pipeline, and plugin requirements).
- [Configuration](CONFIGURATION.md) - [Configuration](CONFIGURATION.md)
Complete key reference, validation behavior, and script environment mapping. Complete key reference, validation behavior, and script environment mapping.
- [Testing](TESTING.md) - [Testing](TESTING.md)

176
docs/Prototyp1_README_EN.md Normal file
View File

@ -0,0 +1,176 @@
# Prototyp1 Unreal Engine Project
## Overview
**Prototyp1** is a Blueprint-based project built with **Unreal Engine 5.6.1**.
It utilizes MetaHuman technology and external Fab assets to create a medical environment featuring an interactive character.
The MetaHuman character serves as avatar for the application. His job is to perform as interface between the User and the AI that is analyzing the cognitive load of the user. The MetaHuman explains the status of the cognitive Load from the driver. The main reason he exists is to improve the acceptance for advises that are given from a machine to a driver in a real car.
ToDos that can be done for the future:
- Implementation of an integrated TTS-Program.
- Tests of various styles of avatars.
---
## Project Structure
```
All/
├── Content/
│ ├── Dr_Doofy/
│ ├── Fab/
│ │ ├── Hospital_Wall
│ │ ├── Lowpoly_Medical_Room/
│ │ ├── Medical_X-ray_Image_Viewer_-_200_Uploads_/
│ │ ├── Modular3D_Hospital_Environment/
│ ├── MetaHuman/
│ │ ├── Common/
│ │ ├── Dr_Doofy/
│ │ ├── Body/
│ │ ├── Clothes/
│ │ ├── Face/
│ │ ├── Grooms/
```
---
## Key Components
### MetaHuman: Dr_Doofy
The central character of the project.
The character Blueprint is the main logic blueprint.
**Contains:**
- Full skeletal rig
- Blueprint class (core logic)
- Clothing assets
- Facial animations
- Groom assets (hair, etc.)
- Voice samples & performances
- Materials (color of clothes etc.)
All character logic is handled inside the Blueprint class.
Current blueprint image: [docs/blueprint.png](docs/blueprint.png)
---
### External Assets (Fab)
The following asset packs were integrated via Fab:
- Hospital_Wall_LowPoly
- Medical_Room
- Medical_X-ray_Image_Viewer_-_200_Uploads_
- Modular3D_Hospital_Environment
These assets provide the visual foundation of the environment.
---
### Camera System
A **static camera** is implemented in the project.
An external explanatory video serves as documentation for its functionality.
---
## Voice and Animation System
### Overview
The project uses externally generated voice samples combined with automated animations.
### Workflow
1. **Voice Generation**
- Voice samples are created using an external TTS program
- Corresponding animation sequences are generated based on the audio
in Unreal Engines
2. **AI Data Pipeline**
- An external AI determines the current hazard status
- The status is sent to an interface
- The interface writes the result into a file:
`Animation.json`
3. **Processing in Unreal Engine**
- After game start, the file is read every **60 seconds**
- The system distinguishes between **four predefined states**
- Based on the state, one of four prepared animations is played
4. **Animation Triggering**
- Animations are directly linked to generated voice samples
- Selection and playback are handled entirely via Blueprint logic
---
## Project Customization
All relevant content is located in the **Content folder**.
### Typical adjustments:
- Modify Blueprint logic in the MetaHuman Blueprint
- Replace or extend voice samples
- Adjust animations
- Replace or expand environment assets
---
## External Resources
### Project Download
🔗 [Download Project](https://gigamove.rwth-aachen.de/de/download/36f0d4f5c6bfd4eb91072b14f0be593c)
The link expires on 7th April 2026.
Password: 6gn56CZ6JyZzG4G
### Camera Explanation Video
🔗 [Camera Explanation](https://www.youtube.com/watch?v=UVEgpwYlQeA)
### TTS Tool (Voice Generation)
🔗 [External TTS Tool](https://ttsmaker.com/deE)
### Creating Facial Animation from voice samples
🔗 [Facial Animation](https://www.youtube.com/watch?v=XIrekfg_fmw)
---
## Requirements
- Unreal Engine **5.6.1**
- MetaHuman Plugin installed
- Access to Fab assets
- Main Level: Main-Level-J
---
## Notes
- Blueprint-only project (no C++)
- All logic implemented via Blueprints
- JSON-based external control pipeline
- Performance depends on MetaHuman complexity
---
## Required Plugins
- Blueprint File Utilities V 1.0
- Json Blueprint Utilities V 1.0
- Camera Shake Priviewer V 0.1
- Engine Camereas V 0.1
- Gameplay Cameras V 0.1
- MetaHuman Animator V 5.0
- MetaHuman Creator V 1.0
- MetaHuman SDK V 1.0
- MetaHuman Live Link V 1.0
- MetaHuman Core Tech V 1.0
- MetaHuman Animatore Calbration Processing V 1.0
---
## Status
Created on: 2026-03-24

View File

@ -28,6 +28,8 @@ unreal.enabled=false
### Mode B: Full integration run ### Mode B: Full integration run
Enable simulator and/or Unreal startup only after all referenced paths are valid on your machine. Enable simulator and/or Unreal startup only after all referenced paths are valid on your machine.
For Unreal Engine prototype setup details (MetaHuman, assets, plugins, and animation flow), see
[Unreal Avatar Prototype](Prototyp1_README_EN.md).
## Build ## Build

BIN
docs/docs/blueprint.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 485 KiB