Bridging AI and Game Engines Toward an Immersive Digital Workflow for Interior Design Visualization

The growth of Artificial Intelligence (AI) in the creative industries has significantly transformed how designers conceptualize, develop, and communicate their ideas. AI technologies have evolved beyond simple automation to become integral to the creative process, assisting designers with environmental, ergonomic, and functional data analysis. Through intelligent algorithms and data-driven insights, AI systems enable designers to experiment with a broader range of design alternatives, facilitating automated ideation, image synthesis, material simulation, and adaptive visualization. AI has thus emerged not merely as a supportive tool but as a catalyst for creativity that augments human intuition and enhances the aesthetic and experiential quality of interior environments. Despite this growth, a gap remains in understanding AI's actual usability, efficiency, and impact on real-world designs. Its integration into comprehensive, interactive digital environments remains underdeveloped, and designers often struggle to translate AI-generated concepts into spatially coherent visualizations that can be explored collaboratively with clients. To address this, the present study proposes a workflow integrating AI-driven generation using ComfyUI with Unreal Engine's real-time visualization, bridging creative ideation and spatial realization so designers can generate, manipulate, and experience design concepts interactively. While similar integrations have been informally explored, this concept has not yet received dedicated academic investigation. This research therefore demonstrates how the convergence of AI and game engine technologies can reshape digital design workflows, positioning AI as both a creative partner and a practical facilitator that redefines how interior spaces are imagined, developed, and communicated.

Contemporary interior design practice faces three interconnected challenges that justify this study. First, designers rely on multiple disconnected software platforms across project stages, producing fragmented, repetitive workflows; prior research has examined AI [2]–[4] and game engines for real-time visualization separately, but none treat the two as a single connected pipeline, leaving designers to either generate ideas or test them in real time, never both. Second, this fragmentation has real-world consequences for client communication: a 2022 Korea Consumer Agency report recorded 1,752 interior design complaint cases between 2018–2021, rising from 412 (2020) to 568 (2021), largely tied to dissatisfaction with materials and finishing, suggesting that static renders are insufficient for shared designer-client understanding before a space is built. Third, as AI-generated visuals and standardized pipelines become central to design production, regional materials, craftsmanship traditions, and culturally specific spatial customs risk being replaced by generic, algorithmically convenient outputs; a workflow that keeps the designer actively directing and refining AI outputs, rather than allowing full automation, helps preserve cultural and aesthetic specificity. Together, these gaps in workflow integration, client communication, and cultural preservation form the basis for the workflow proposed here.

This study aims to develop an integrated workflow combining AI and game engine technologies to improve work efficiency, support clearer communication of design intent, and enhance the quality of interior design presentation. Specifically, the objective is a connected process in which AI-generated textures, materials, and design variations can be produced and applied directly within a real-time, interactive visualization environment, allowing interior spaces and products to be showcased at a professional level.

Several studies have examined how AI can benefit interior design in isolation. Chen and Wang proposed AI-driven storage optimization for residential spaces [2], while Almaz et al. explored AI integration in design education to foster creativity and sustainability [3] — though both lack empirical testing with practicing designers. AlShkipi and Zahran's systematic review emphasized AI's role in personalization, sustainability, and efficiency, but relied heavily on English-language secondary literature, limiting generalizability [4]. Kahraman et al.'s experimental study found a nonsignificant change in outcomes, suggesting AI functions as an aid rather than a creative replacement [5], and Chen et al. introduced an AI system for generating aesthetically pleasing designs without user evaluation [6]. Collectively, these studies open possibilities for AI-design collaboration but focus on isolated technological features rather than integrated systems. This study departs from that approach by treating AI and game engines as one integrated system, proposing a grounded qualitative framework analyzing the affordances of combining ComfyUI and Unreal Engine into a unified interior design pipeline.

AI's growing role across industries [7] extends to the creative sector, reflected in initiatives such as South Korea's "Entertech, Seoul 2025" event promoting AI, XR, and metaverse convergence [8]. As these technologies advance, research points to the need for improved cognitive and meta-cognitive skills to adapt to AI-assisted design [9]. AI has been shown to generate design ideas rapidly, including layouts, materials, and moodboards, and to support predictive modeling — Çelik, for example, evaluated daylight performance of AI-generated housing plans across five climate zones [10]. However, such studies treat AI as a standalone tool rather than part of a connected system, limiting designers to either generating or testing ideas, rarely both. ComfyUI was selected as this study's generative foundation for its node-based visual system, which exposes and organizes each stage of the generative process — offering integration with custom models and ControlNet modules, iterative feedback loops with Unreal Engine, fine-tuned control over resolution and geometry, and clear workflow documentation. Game engines, meanwhile, provide the technical foundation for rendering graphics and simulating physics in interactive environments, and are increasingly used in architecture and interior design for real-time visualization and virtual walkthroughs. They support first-person and aerial viewing perspectives [11] as well as VR/AR walkthroughs, letting designers evaluate materials, lighting, and circulation interactively with clients. Kim and Hyun found VR enhances spatial perception and supports design activities [12], while related work highlights VR's broader value in urban and residential planning [13].

Interior design has shifted from manual drafting and static mood boards toward CAD, BIM, and 3D visualization tools that improve efficiency and accuracy, yet these technologies remain insufficiently integrated into everyday practice [14]. Designers still rely on multiple disconnected platforms, producing fragmented workflows, and the resulting communication gaps contribute to client dissatisfaction [15] — reflected in the complaint data discussed in the Rationale [16]. High-quality visual output also demands technical expertise and computational resources that may limit accessibility for less experienced designers, while the iterative nature of design projects can create inconsistency between initial concepts and final outputs. Preservation of design intent is a related concern: while globalization pressures design heritage toward standardization, technology also enables the preservation and dissemination of diverse cultural expressions [17]. As Fig. 1 illustrates, effective digital workflows must not only improve efficiency but ensure that generated visualizations remain faithful to the designer's intended language, spatial atmosphere, and cultural specificity throughout development and presentation.

This study employs a Research through Design (RtD) methodology, structured around four iterative steps: creating prototypes, testing them for usability and workflow efficiency, gathering feedback, and redesigning based on insights. The current stage remains conceptual — the mockup and initial AI-to-engine workflow have been developed, while full usability testing and iterative redesign are planned for future stages. Three qualitative methods carried out the Exploration and early Development phases: (1) Literature-Based Analysis, reviewing existing research to identify capabilities, limitations, and gaps; (2) Affordance Analysis of Tools, examining ComfyUI's and Unreal Engine's documented features and design affordances; and (3) Scenario-Based Conceptualization, using literature and tool insights to construct scenarios illustrating how the integrated workflow could function in everyday practice.

This research demonstrates how AI and game engine technologies can enhance the interior design workflow by generating design alternatives — optimized layouts, materials, furniture arrangements, and color schemes — while the game engine provides real-time rendering, lighting, and spatial simulation for multi-perspective exploration. ComfyUI was used to generate images, and a workflow was built using a GitHub plugin within Unreal Engine to generate images from prompts and instantly adjust interior material textures and colors. The workflow began by producing the interior product to be tested, followed by mockup creation, migration into Unreal Engine, ComfyUI workflow development, and final plugin migration for testing within the virtual environment.

The design process began with a mockup built in SketchUp, a design software widely used by architects and interior designers to visualize and modify three-dimensional forms. The design chosen was a chair, explored through various forms, structures, and material possibilities (Fig. 2), allowing each variation to be tested for comfort, stability, and proportion without commitment to a fixed outcome. Models were intentionally left untextured to maintain neutrality until materiality could be more accurately defined within Unreal Engine's real-time rendering environment. The completed mockup (Fig. 3a) was migrated into Unreal Engine using the Datasmith plugin, enabling seamless transfer of geometry and materials while preserving design integrity. Within Unreal Engine, the model was dismantled and reassembled (Fig. 3b) to allow texture and material adjustment in the final workflow, without requiring manual rebuilding — and the SketchUp connection remains available if further design changes are needed.

ComfyUI was integrated as a separate generative system supporting design development through AI-assisted outputs (Fig. 4), later incorporated into Unreal Engine as a plugin-based extension. This allowed generated visual assets, material concepts, and design variations to be transferred and implemented directly within Unreal Engine, enabling a flexible, experimental approach to visualization. By connecting both systems in a unified pipeline, the workflow bridges AI-generated design exploration with real-time rendering, enhancing both creativity and technical execution. After the workflow was established, 3D models and AI-generated outputs were organized and optimized for compatibility before being migrated into Unreal Engine via Datasmith, transferring geometry, scene hierarchy, and design elements into an interactive visualization space. There, the design was further enhanced through lighting, rendering, and environmental setup. Within Unreal Engine, prompt-based and input-driven systems allowed materials, styles, and visual outcomes to be modified without rebuilding the model (Fig. 5), creating a more efficient feedback loop between design intention and visual output.

This workflow is not limited to a single toolset. While SketchUp (2022 version) was used here for quick mockups, alternative tools such as 3ds Max or Blender could serve the same purpose depending on designer preference. Migration into Unreal Engine does not strictly require Datasmith — OBJ and FBX formats offer straightforward alternatives that preserve structural integrity while allowing further in-engine refinement. Established procedural pipelines, drawn from existing tutorials and communities, can be consolidated into a more automated system once installed and configured within ComfyUI and connected to Unreal Engine through a plugin-based setup. Embedding material definitions early makes the workflow more efficient once connected, allowing properties to be adjusted dynamically through prompts for faster iteration. Overall, this mechanism combines traditional 3D modeling tools, standardized file formats, procedural workflows, and AI-driven prompt systems into a streamlined, adaptive, and responsive design pipeline.

This study has several limitations. Integration between ComfyUI and Unreal Engine depends on external resources such as GitHub repositories and plugin installations, and version mismatches between the two systems can cause installation failures or reduced functionality. The workflow remains experimental and not yet fully optimized: only a single perspective of the model was successfully displayed, restricting multi-angle evaluation, and file migration errors — improper exporting, incorrect formats, or misconfigured settings — can produce missing assets or unstable performance, requiring careful file management. As the workflow has not yet reached a fully stable system, these challenges are treated as opportunities for continued refinement rather than fundamental flaws. Future development is expected to improve plugin stability and version compatibility, reducing reliance on trial-and-error setup and making the workflow more accessible to a wider range of users and easier to document for client collaboration. The system is also expected to support more advanced multi-perspective visualization, enabling evaluation of form, material, and lighting from multiple angles and distances. These improvements would enhance both the technical efficiency of the workflow and the quality of design analysis and presentation, while future iterations continue to explore alternative tools and approaches within the pipeline.

This project explored the development of an integrated design workflow combining 3D modeling tools, ComfyUI, and Unreal Engine to create a more dynamic and flexible visualization process. Traditional modeling methods remain valid, but emerging technologies extend their capabilities — approaches once dependent on lengthy rendering times can now be visualized in real time, with AI positioned not as a replacement for design but as a means of improving efficiency and creative freedom. The workflow's adaptability is central to its value: tools such as SketchUp, 3ds Max, or Blender can be used interchangeably for mockup creation, file formats like FBX or OBJ enable efficient migration into Unreal Engine, and Datasmith offers a direct alternative connection between the two. The integration of ComfyUI adds a further layer of generative control, allowing prompts to adjust materials, forms, and visual outcomes directly within Unreal Engine, enhancing the ability to test and refine design variations in real time. While the system still faces limitations — including version compatibility, incomplete workflow stability, and restricted multi-perspective visualization — it provides a strong foundation for future development, with continued iterations expected to achieve smoother plugin integration, more stable performance, and a more immersive, multi-angle design experience.

This study was made possible with support from Dongseo University, Busan, South Korea. The authors thank the professors, faculty, and all those involved in making this research possible despite its limitations.