diff --git a/README.md b/README.md index 5035cb3..3c2d3a0 100644 --- a/README.md +++ b/README.md @@ -155,6 +155,7 @@ src/main/resources - [Configuration](docs/CONFIGURATION.md) - [Testing](docs/TESTING.md) - [Source Map](docs/SOURCE_MAP.md) +- [Unreal Avatar](docs/Prototyp1_README_EN.md) - [Known Issues](docs/KNOWN_ISSUES.md) ## Known Limitations diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 6dc75c7..18b1a7e 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -74,6 +74,8 @@ Persistence schema (`binary_event`): - **External processes**: optional Python simulator and Unreal launcher script. - **Animation output**: JSON file written to `animation.output.path`. +For Unreal-specific implementation details, see [Unreal Avatar Prototype](Prototyp1_README_EN.md). + ## Shutdown Behavior `ApplicationShutdownManager` performs: diff --git a/docs/INDEX.md b/docs/INDEX.md index b7effeb..2f5f155 100644 --- a/docs/INDEX.md +++ b/docs/INDEX.md @@ -7,10 +7,11 @@ This folder contains detailed technical documentation for the project. 1. [Project Review](PROJECT_REVIEW.md) 2. [Architecture](ARCHITECTURE.md) 3. [Setup and Run](SETUP_AND_RUN.md) -4. [Configuration](CONFIGURATION.md) -5. [Testing](TESTING.md) -6. [Known Issues](KNOWN_ISSUES.md) -7. [Source Map](SOURCE_MAP.md) +4. [Unreal Avatar Prototype](Prototyp1_README_EN.md) +5. [Configuration](CONFIGURATION.md) +6. [Testing](TESTING.md) +7. [Known Issues](KNOWN_ISSUES.md) +8. [Source Map](SOURCE_MAP.md) ## Document Guide @@ -20,6 +21,8 @@ This folder contains detailed technical documentation for the project. Component responsibilities, runtime data flow, and integration boundaries. - [Setup and Run](SETUP_AND_RUN.md) Reproducible setup for minimal and full integration modes. +- [Unreal Avatar Prototype](Prototyp1_README_EN.md) + Unreal Engine prototype details (MetaHuman setup, assets, animation pipeline, and plugin requirements). - [Configuration](CONFIGURATION.md) Complete key reference, validation behavior, and script environment mapping. - [Testing](TESTING.md) diff --git a/docs/Prototyp1_README_EN.md b/docs/Prototyp1_README_EN.md new file mode 100644 index 0000000..619007b --- /dev/null +++ b/docs/Prototyp1_README_EN.md @@ -0,0 +1,176 @@ +# Prototyp1 – Unreal Engine Project + +## Overview +**Prototyp1** is a Blueprint-based project built with **Unreal Engine 5.6.1**. +It utilizes MetaHuman technology and external Fab assets to create a medical environment featuring an interactive character. +The MetaHuman character serves as avatar for the application. His job is to perform as interface between the User and the AI that is analyzing the cognitive load of the user. The MetaHuman explains the status of the cognitive Load from the driver. The main reason he exists is to improve the acceptance for advises that are given from a machine to a driver in a real car. + +ToDos that can be done for the future: + +- Implementation of an integrated TTS-Program. +- Tests of various styles of avatars. + +--- + +## Project Structure + +``` +All/ +├── Content/ +│ ├── Dr_Doofy/ +│ ├── Fab/ +│ │ ├── Hospital_Wall +│ │ ├── Lowpoly_Medical_Room/ +│ │ ├── Medical_X-ray_Image_Viewer_-_200_Uploads_/ +│ │ ├── Modular3D_Hospital_Environment/ +│ ├── MetaHuman/ +│ │ ├── Common/ +│ │ ├── Dr_Doofy/ +│ │ ├── Body/ +│ │ ├── Clothes/ +│ │ ├── Face/ +│ │ ├── Grooms/ +``` + +--- + +## Key Components + +### MetaHuman: Dr_Doofy +The central character of the project. +The character Blueprint is the main logic blueprint. + +**Contains:** +- Full skeletal rig +- Blueprint class (core logic) +- Clothing assets +- Facial animations +- Groom assets (hair, etc.) +- Voice samples & performances +- Materials (color of clothes etc.) + +All character logic is handled inside the Blueprint class. + +Current blueprint image: [docs/blueprint.png](docs/blueprint.png) + +--- + +### External Assets (Fab) +The following asset packs were integrated via Fab: + +- Hospital_Wall_LowPoly +- Medical_Room +- Medical_X-ray_Image_Viewer_-_200_Uploads_ +- Modular3D_Hospital_Environment + +These assets provide the visual foundation of the environment. + +--- + +### Camera System +A **static camera** is implemented in the project. + +An external explanatory video serves as documentation for its functionality. + +--- + +## Voice and Animation System + +### Overview +The project uses externally generated voice samples combined with automated animations. + +### Workflow + +1. **Voice Generation** + - Voice samples are created using an external TTS program + - Corresponding animation sequences are generated based on the audio + in Unreal Engines + +2. **AI Data Pipeline** + - An external AI determines the current hazard status + - The status is sent to an interface + - The interface writes the result into a file: + `Animation.json` + +3. **Processing in Unreal Engine** + - After game start, the file is read every **60 seconds** + - The system distinguishes between **four predefined states** + - Based on the state, one of four prepared animations is played + +4. **Animation Triggering** + - Animations are directly linked to generated voice samples + - Selection and playback are handled entirely via Blueprint logic + +--- + +## Project Customization + +All relevant content is located in the **Content folder**. + +### Typical adjustments: +- Modify Blueprint logic in the MetaHuman Blueprint +- Replace or extend voice samples +- Adjust animations +- Replace or expand environment assets + +--- + +## External Resources + +### Project Download +🔗 [Download Project](https://gigamove.rwth-aachen.de/de/download/36f0d4f5c6bfd4eb91072b14f0be593c) + +The link expires on 7th April 2026. + +Password: 6gn56CZ6JyZzG4G + +### Camera Explanation Video +🔗 [Camera Explanation](https://www.youtube.com/watch?v=UVEgpwYlQeA) + +### TTS Tool (Voice Generation) +🔗 [External TTS Tool](https://ttsmaker.com/deE) + +### Creating Facial Animation from voice samples + +🔗 [Facial Animation](https://www.youtube.com/watch?v=XIrekfg_fmw) + +--- + +## Requirements + +- Unreal Engine **5.6.1** +- MetaHuman Plugin installed +- Access to Fab assets +- Main Level: Main-Level-J + +--- + +## Notes + +- Blueprint-only project (no C++) +- All logic implemented via Blueprints +- JSON-based external control pipeline +- Performance depends on MetaHuman complexity + +--- + +## Required Plugins + +- Blueprint File Utilities V 1.0 +- Json Blueprint Utilities V 1.0 +- Camera Shake Priviewer V 0.1 +- Engine Camereas V 0.1 +- Gameplay Cameras V 0.1 +- MetaHuman Animator V 5.0 +- MetaHuman Creator V 1.0 +- MetaHuman SDK V 1.0 +- MetaHuman Live Link V 1.0 +- MetaHuman Core Tech V 1.0 +- MetaHuman Animatore Calbration Processing V 1.0 + +--- + +## Status + +Created on: 2026-03-24 + diff --git a/docs/SETUP_AND_RUN.md b/docs/SETUP_AND_RUN.md index b774f7a..0535892 100644 --- a/docs/SETUP_AND_RUN.md +++ b/docs/SETUP_AND_RUN.md @@ -28,6 +28,8 @@ unreal.enabled=false ### Mode B: Full integration run Enable simulator and/or Unreal startup only after all referenced paths are valid on your machine. +For Unreal Engine prototype setup details (MetaHuman, assets, plugins, and animation flow), see +[Unreal Avatar Prototype](Prototyp1_README_EN.md). ## Build diff --git a/docs/docs/blueprint.png b/docs/docs/blueprint.png new file mode 100644 index 0000000..e62e2f2 Binary files /dev/null and b/docs/docs/blueprint.png differ