New classes of imitation-based learningand self-learning paradigms zooming in on the design-development processes instead of the results (to be) achieved must be welcome to „customize“ the most suitable pattern to the requested final proposal. Two kinds of patterns are to be considered. A pattern of an existing architectural solution – an existing building, and a pattern of the process of how the solution came to existence. Fitting well with the so-far ruling supervised-learning image processing paradigms, the first strategy suffers two bottlenecks: computing power needed, further elaborated in the previous subsection, and training database availability as annotated in section (4). As the links put it, any attempt to develop with this strategy an architectural design of a building from scratch to a 3D model, even to layout drafts only, is doomed to fail. However, the task does not need to be so comprehensive. In the production architecture realm, most often a similar solution already exists and can be accessed. Launching the design process, this is a step undertaken by an experienced professional as a rule. After, the comprehensiveness of the task is reduced to bridging the gaps between the pattern and the current brief. Even a simple choice of a convenient sample from a limited database provides a significant reduction in the comprehensiveness of the task to be fulfilled. And it is easy to imagine that a machine can be fully competent to make the choice. Then, to bridge the remaining deviation from the current brief is easier, indeed.
The sequence of steps to develop the design up to the requested solution – a 3D model of the building or its layouts – is the second pattern. Here, the experience of AI-driven robots will apply. Not the way Figure-1 has trained the robot to make coffee – taking videos watching humans at the aktivity; unlike an autonomous vehicle, a „design robot“ would not learn from real-world scenes but „watching“ the design-development steps in a CAD software o ran into VR/XR extended environment (in the parametric stages of architectural design or even before), their sequence, and sequels. This way, either a general algorithm to develop designs from a given situation according to the brief or a „private“ agent to do this can be designed and trained. A general algorithm would learn how diverse designs have been developed, a „private“ agent would be an apprentice of a particular (human) architect or a studio; the nomenclature of reinforcement learning becomes true to the real-life – hence world imitation learning concept. Reinforcement learning, self-learning, meta-learning, and other new approaches supporting imitation-based- and transfer learning, tree of thoughts, auto-associative, and other revolutionary algorithms that active and proactive agents perform – welcome to the sphere of architecture and the built environment development. The possibility of adopting this class of applications has so far escaped the attention of the branch; nonetheless, these new strategies appear to be very adherent to architectural design processes.
World imitation learning turns out to be a promising strategy with CAD BIM environment as the world. Uniquely parameterized both in terms of geometry and physical properties, i.e. attributes of entities of this world, and in terms of their making, i.e. commands and steps of software work, the world of BIM software environment is an ideal living or working environment for reinforcement-learning (RL) agents. In implementing the strategy, the motivation of the RL agents is crucial. Examples of state/action-related intrinsic motivation techniques such as curiosity-driven exploration technique encouraging an RL agent to explore uncharted areas of a game world out of curiosity, even if there’s no immediate external reward associated with it, or novelty detection technique rewarding intrinsically the agent for encountering novel states or actions that it has not encountered before, resemble quite intimately the thoughts of an architect making architecture. Intrinsical rewarding of the deep-learning agents emerges as a specific issue at the heart of the models; nonetheless, policy-related motivation that modifies the agent’s policy to provide goals due to (extrinsic) rewards for learning specific behaviors shall not be ovelooked. The nature of the rewards fundamentally sets the performance of the models: the performance not only in quantitative terms but – more importantly – in terms of quality that interconnects the abstract world of AI model with the real world. This renders an essential quality for AI models to succeed in architecture, which most probably rests behind the horizon of our understanding so far. To launch R&D in this direction, lessons from Claude models fine-tuning using constitutional AI and reinforcement learning from human feedback (annotated in section (2) of the paper) seem promising, „from human feedback“ being the keyword. Also, a full and both-sides understanding of this problematics renders a precondition for bridging the gap between architectural and computer-science and -technic thinking that has been almost a rule in the realm of AI in architecture so far.
A strategy proposed already in 2016 by Nono Martinez (see section (2) of this paper), human-in-the-loop renders essential. Starting from following the human-in-the-loop in the phase of „customizing“, the algorithm shall learn by self-training to master the design process better than the man in the end – hopefully; the apprentice surpasses the master and, also uniquely, eventually becomes a personal or home apprentice, but his craft can draw on a global background at the same time.
The basis of the algorithm structure could be a pair of mutually interfering loops: a generative loop and an advice-whispering loop, or there can be more advice-whispering loops particularized according to the diverse natures of the parameters, which will be „switched-on“ only in a cascade. In the beginning, a suitable pattern will be selected from the database, which will be tested and optimized due to the specified outlines and with respect to a benchmark of independent parameters. The parameters of the spatial structure of the proposed building, the parameters of the physical properties of its constructions, and finally the parameters of the internal environment in the object will certainly come into consideration. A pragmatic optimization of the involved parameters structure appears a key task.
To take up such a challenge in the architectural design and building planning realm, lessons from the in section (2) reminded trailblazers such as AlexNet, Deep Blue, and especially AlphaGo Zero, Figure-1 and GR00T have to be learned. The evolutionary algorithms approaches and genetic programming have to undergo a deep survey to be subsequently considered as an option. Such a concept ought not to be refused or underestimated pointing out the gap between designing production (residential) buildings and playing chess or Go on the masters level: deployment of analogical algorithms deserves to be studied thoroughly first. After all, given the data format with which the AI algorithms can efficiently work in architecture and built environment development, the strategy can be quite straightforward: let BIM CAD software (or an into VR/XR extended platform, such as Wearrecho) be a robot! Concurrently, to facilitate the communication between the master and the apprentice, SMLs introduced in section (2) of this paper appear to have entered the scene just in time.
Let BIM CAD software be a robot!
Current achievements in robotics offer great and straightforwardly transferable inspiration for architectural designing. Towards the end of April 2024, headquartered in Vancouver, British Columbia, Sanctuary AI revealed a 7th-generation humanoid robot Phoenix that, after a week-long pilot test in a retail store, completed over 110 retail tasks, confirming Sanctuary AI´s mission to create the world’s first human-like intelligence in general-purpose robots [340,341]. Its pioneering AI control system Carbon allows the robot to learn and automate tasks in less than 24 hours, closely mimicking subsystems found in the human brain, including memory, sight, sound, and touch, and incorporating sophisticated human behavioral data-capture technologies; subsequently, this high quality, high-fidelity data accelerates the development of foundational AI models and customer deployments. Integrating state-of-the-art AI technologies for reasoning, task execution, and motion planning, Carbon translates natural language into real-world actions executing explainable and auditable reasoning, which ensures transparency and accountability, symbolic and logical reasoning coupled with deep learning and reinforcement learning, agency and goal-seeking behaviors, which enables autonomous decision-making, and human-in-the-loop supervision, which allows for fine-tuning and safety. Sanctuary AI envisions Phoenix as an augmentation to the human workforce; as such, the robot can be both directly piloted by humans for specific tasks, operated with a pilot-assistant combining human guidance and autonomy, or performing tasks autonomously when supervised by Carbon [342]. A cognitive architecture and software platform specifically designed for humanoid general-purpose robots, Carbon leverages symbolic reasoning, deep learning, and reinforcement learning to learn from data and improve over time adapting to different scenarios and making informed decisions; deep learning recognizes patterns, while reinforcement learning guides the robot´s decision-making [343]. Carbon exhibits adaptability and robustness when encountering uncertainty or novel situations during learning. It harnesses uncertainty; perhaps surprisingly, instruction with inherent uncertainty can promote learning and transfer more effectively than certain instruction. Thus, uncertainty triggers curiosity, which, in turn, facilitates learning [344]. To adapt to uncertain scenarios, Carbon employs adaptive neuro-fuzzy inference system (ANFIS), a hybrid model combining fuzzy logic and neural networks, and extreme learning machine (ELM), a single-layer feedforward neural network. These models allow Phoenix to learn from data and adjust its behavior based on uncertainty. Also, providing generalization ability, Carbon stands out in responses to novel situations performing goal-oriented behaviors, seeking to achieve objectives efficiently [345].
The experience of architectural practice combined with the ability to perceive and evaluate the processes of making an architectural design, and supported by knowledge of the fundamentals of machine and deep learning reads the previous lines as applicable to the work of CAD parametric, BIM software systems (or an into VR/XR extended platform, such as in section (2) said Wearrecho) in most, if not all categories and aspects. Moreover, a CAD software tool is well parameterized and digitized and, unlike a physical, humanoid robot, does not address gravity issues.
If understanding the nature of designing architecture and planning the built environment development, and comprehending the design- and planning workflow, an architect would appreciate an AI apprentice or assistant that would learn and automate tasks in less than 24 hours, closely mimicking the master´s attitudes and techniques of making, able to grasp natural language commands and transfer them into real-world-design actions, executing transparency and accountability, and, in addition, autonomous decision-making. An augmentation to the human workforce, such a design robot could be both directly human-piloted for specific tasks, operated with a pilot combining human guidance and autonomy, or would perform tasks autonomously. Leveraged its symbolic reasoning, deep learning, and reinforcement learning, the apprentice or assistant would learn from data and improve over time adapting to different scenarios, recognizing patterns, and making informed decisions, exhibiting adaptability and robustness when encountering uncertainty or novel situations during learning. On top of it, it would harness uncertainty when instruction with inherent uncertainty could promote learning and transfer more effectively than certain instruction; uncertainty would trigger curiosity, which, in turn, would facilitate learning. In general, by providing generalization ability, such an apprentice or assistant would stand out in responses to novel situations performing goal-oriented behaviors and seeking to achieve objectives efficiently. Where is the problem then?
Also, the new (in 2023 introduced) machine learning algorithms for 3D modeling and rendering – both new diffusion models and NeRFs deserve investigation in this relation, leaving for the moment aside the capability of these deep neural networks of generating high-quality, photorealistic images of complex scenes from multiple viewpoints that, as an unprecedented AI and VR/XR fusion, would mark out the next level of exploiting the immersive VR-environment for (among others) instant designing and communicating architecture as it deserves – in space and motion, diachronically, from spaces, and in „life-size“.
Introduced in section (4) of the paper, spatial computing shall also be surveyed and developed as another marriage between AI and VR/XR, that can push forward a state-of-the-art deployment of AI in architectural designing. Promising in regard to AI-aided designing architecture and planning of the development of the built environment may also show MeshDiffusion [38], appreciated for direct generating 3D meshes without any post-processing, and also LERF, the new marriage of NeRF with CLIP (contrastive language-image pre-training); with it, natural language queries in a 3D fashion can apply within NeRF, targeting different objects in the scene. And many other 3D objects considering outputs of AI development, though originally not focusing on architecture, need to be reviewed within the new R&D paradigm of AI´s deployment in architecture and the built environment.
Motivated not by architecture but by (the example of) machine learning industrial robots working in the isolation of individual production plants, the ERC Advanced Granted FRONTIER project led by Josef Sivic from the Czech Institute of Informatics, Robotics and Cybernetics of the Czech Technical University in Prague can also help to show the way to approach architectural design effectively eventually. Issues of computer vision and perception of the „environment“ can, if a suitable approach would be found, also benefit the field of architectural design and planning of the development of the built environment. Applications for this field could include new neural architectures that credibly represent physical and geometric structure as well as new algorithms that enable learning of complex multi-step tasks from just a few examples … like how humans can learn [346]. Algorithmic sharing of experiences between projects could address the problem of training database size and building, which, most probably, will be difficult and only slow given the already entrenched conservatism and autarky of the field. Similarly, the work of Josef Urban´s team (of the same institution) in the field of large-scale computer-assisted reasoning can eventually contribute to effective deployment of AI in architecture [347].
