Introduction
A range of technologies associated with the so-called Fourth Industrial Revolution (4ID) are projected to transform nations’ economies and revolutionize warfare (Schwab, 2017). Whilst rapid advances in artificial intelligence (AI) and robotic systems often capture much of the headlines when it comes to the changing character of future war, there are other technological developments that are set to have a major impact on military affairs (Rossiter and Layton, 2024). Immersive technologies like Extended Reality (XR) are increasingly becoming an essential part of the emerging defense technological landscape and promise to enhance military effectiveness across a range of performance dimensions.
XR is an umbrella term that includes Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). It encompasses, therefore, a spectrum of experiences, from fully virtual environments to overlays of digital content onto the real world. These technologies manipulate and extend our perception of reality, hence the term ‘extended’ (see Fig. 4.1).
Emerging trends in XR design focus on enhancing realism and user immersion. It is unsurprising, then, that the manipulation of reality is already proving useful for military training purposes (Boyce et al., 2022). Creating training scenarios that are difficult to simulate physically—outside of active combat—is inherently useful (Lele, 2013). Consider, for example, pilots’ visors displaying realistic real-world terrain from distant locations whilst training in the relative safety of home ranges. Any battlespace could potentially be replicated via XR technology back at the home station.
Advances in XR technology—alongside fast-paced developments in computing, data science, and analytics—are creating new opportunities for delivering realistic virtual training in a cost-effective way. While XR technology is carving out a strong use case in training delivery, it is also set to make a significant impact on other areas of military affairs. Indeed, several defense establishments and private firms are making considerable investments in XR to support and enhance command and control (C2) and battle management (BM).
Parsing C2 and BM
As a concept combining two interacting parts, C2 is tricky to define. In his 1838 book, Art of War, the nineteenth-century Swiss military thinker Antoine-Henri Jomini used the terms ‘Command’ and ‘Control of Operations’ (Jomini, 1838). Yet it was not until the 1980s that term became a mainstay term in NATO military doctrine. Although the term ‘command and control’ is widely used, there is no authoritative formal definition. Separated out, command can be thought of as an authority given to an individual of the armed forces for the direction, coordination, and control of military forces (UK MOD, 2017). Whereas control is this authority exercised.
Taken together, C2 can be thought of as a ‘dynamic and adaptive socio-technical system configured to design and execute joint action’ (UK MOD, 2017). The purpose of C2 is to focus military resources and activities so that they are integrated and maximized to deliver desired outcomes. It is understood that to deal with the growing complexity of operating environments, future C2 systems must be flexible and adaptable. One of the biggest challenges—as well as an opportunity—is the sheer volume of data that commanders must contend with. Advances in sensor and communication technologies provide opportunities to capture increased depth and breadth of data, including on complex problems. These changing dynamics are also apparent at the tactical level for commanders.
As with C2, BM requires a system that enhances situational awareness. A capable system will increase the speed of decision action cycles. It also allows for information to be exchanged between tactical commanders in a joint or coalition environment. Not only is this crucial for defeating an adversary, but an effective BM system is also necessary for reducing the chances of fratricide in a frenetic battlespace.
It is clear from these brief summaries of C2 and BM that an enhanced ability of individuals and organizations to digest and exploit data inputs could very well be a battle-deciding capability. The challenge for the future of C2 and BM is not a lack of data; data are plentiful in the future battlespace. But accessing the relevant data within a relevant timeframe will be one of the major factors for military success. The remainder of this paper seeks to anticipate some of the ways in which XR technology could revolutionize approaches to C2 and BM in this regard. Gauging the future impact of any maturing technology is fraught with difficulties, not least because of the uncertainty surrounding technological trajectories (Rossiter, 2023). The article, therefore, considers below broad ways in which XR—both the hardware and associated software—can support and enhance C2 and BM.
The immediate conceptual benefit of XR for commanders and staff is enhancing their ability to visualize the battlespace and optimize information flows for their decision-making.
Situational Awareness
Extended Reality provides a ‘tactical augmented reality’ to warfighters that organizes, filters, and displays incoming information (Liaropoulos, 2023). For warfighters, XR headsets could improve situational awareness on foot or when operating vehicles (Congressional Research Service, 2022). Papathanasiou and Karadimas (2023) note that warfighters ‘can receive specific and accurate information needed at the exact moment, which is either embedded in the augmented reality (AR) application itself or obtained from a remote operations center’ while in combat.
The above scenario applies to aircraft also, where AR can relay additional information to pilots that would not otherwise be displayed in the cockpit. The U.S. military’s Integrated Visual Augmentation System (IVAS), which modifies a Microsoft AR headset, has been the centerpiece of most military XR testing. IVAS can display several different types of information at once, like holographic maps, while taking on traditional headset roles like night vision (Kallberg et al., 2022). The platform also has the potential to leverage AI and other tools, though integrating more advanced features may tax the hardware and increase costs (Saballa, 2024).
XR has the potential to improve situation awareness for higher level commanders to aid C2. The immediate conceptual benefit of XR for commanders and staff is enhancing their ability to visualize the battlespace and optimize information flows for their decision-making. For instance, XR headsets allow users to visualize the battlespace in a 3D environment and display information in real-time. Visualization can also improve situational awareness beyond the battlespace. Researchers in 2018 used Microsoft HoloLens to display satellite coverage of the Earth, which allows the technology to provide a form of ‘space situational awareness’(Jenkins et al., 2018).
Commanders could also see in real-time where air assets are as well as information such as their loadout, remaining operation range, and proximity to ground forces. The customizability of their prototype meant that the HoloLens users could see a Common Operating Picture but also manipulate information streams to produce a User Defined Operating Picture that displays the most pertinent information for each viewer (Jenkins et al., 2018).
Situational awareness does not need to be linked solely to military sensors and communications channels for effective BM. In urban warfare, special operations, or other contexts where there is a large civilian presence, an AR display could also take in feeds from local social media or news sources to gauge civilian movement and reactions in real-time. Social media integration that is safe and legible to XR users would require significant processing power, but AI tools are already being trained to detect and analyze patterns and sentiment analysis.
However, XR platforms are not without their limitations and tradeoffs for Battle Management. Like many other technologies, XR will largely be dependent on the available bandwidth. NATO Allied Command Transformation tested using 5G to improve XR in October 2023 as part of a broader effort to improve communication among forces (NATO ACT, 2023). XR devices will place an ever-increasing demand on the bandwidth available to militaries. Also, XR platforms need to justify using their technology to display information in a 3D environment instead of a 2D screen, where information can also be rendered in 3D and updated in real-time at a lower cost and with less retraining. Researchers reported that test users tended to use XR prototypes to replicate what they already did at a normal desk: that is, operate multiple monitors simultaneously (Jenkins et al., 2018).
Adoption issues aside, 3D rendering and visualization require substantial processing power as the fidelity increases, meaning that XR users could have limited functionality compared to a traditional computer or end up tethered to a PC or laptop through a cable or wireless connection. Since some kinds of information, like text, also need to be optimized for XR headsets to be as easily readable as a 2D monitor, the drawbacks mean that militaries benefit more from adopting XR on a wide scale. Limited use of XR means that incoming information needs to be compatible with every potential kind of display, which creates more friction and points of failure.
Improving Decision Cycles
Modern militaries prioritize speeding up decision cycles to gain the initiative against adversaries. The enhanced situational awareness and integrated tools of XR facilitate faster decision cycles. In theory, relatively compact XR hardware can display the same information as several bulkier 2D monitors or help make that information more intuitive for users to understand. The primary benefit for commanders is that different data streams and methods of communication are integrated into one display or device for users to manipulate. ‘Applying an advanced visual perception of the environment, combined with digital communication for remote data transmission’, Papathanasiou and Karadimas note, ‘can…offer a shared experience to reduce the response time in critical situation’ ( 2023). Figure 2 illustrates a wide range of functions that could be created in an augmented environment.
An XR device on the wearer would potentially facilitate faster action by the user. In the future, live analysis of data feeds from battlespace could also be used in conjunction with XR devices to explore and visualize possible outcomes from possible Courses of Action (COA). In this way, XR could enable better real-time modeling and simulation to aid decision support to headquarters. The prospect that XR could make a complicated suite of tools portable has led firms like BAE Systems to assert that AR platforms will allow command centers to fit into the size of a briefcase, which reduces the time and space needed for commanders to set up and access information about the battlespace (BAE Systems, 2015). If so, this would clearly be a boon for C2 effectiveness and resilience. Indeed, with sufficient connection to wider networks, commanders could relocate quickly in fluid situations.
However, the concentration of information in a single device or set of devices also presents an opportunity for adversaries. Kallberg et al. (2022) argue that not only do the headsets have a potential to emit signals that adversaries can detect, but there is also a risk that they could manipulate and impede incoming information and reduce the effectiveness of commanders reliant on the headset. Therefore, militaries seeking to gain the benefits of centralizing information through XR will need to balance it against cybersecurity and data validity to ensure the system is helping decision-makers cut through the fog of war instead of adding to it (Watling, 2023).
Additional Benefits of Integrating Immersive Technologies
While the primary benefits of XR are improvements to situational awareness and decision-making speed, the technology also offers second-order benefits to BM. Firstly, improvements in situational awareness reduce the likelihood of fratricide and civilian casualties. A well-implemented XR system means commanders and squad leaders could be aware of the disposition of friendly units in the area as well as pertinent information such as civilian shelters or the activities of NGOs. In chaotic battle conditions, this could reduce the risk of a blue-on-blue engagement. When coordinating fires, XR-augmented pilots can help ensure they do not drop munitions into areas where they could harm friendly forces or civilians. Such a system could come in especially handy in densely populated areas or peacekeeping contexts.
However, the potential to reduce unwanted harm comes with several caveats. XR would not remove the responsibility for commanders to verify their data. Future users of XR would need to ensure that all relevant information is being fed to them, which might be difficult in areas where civilians are moving or sheltering in unpredictable ways, or when the adversary is aware of how the systems deconflict civilians and friendly forces from adversaries. In combined missions, forces from a partner nation may not have a compatible XR platform.
Lastly, extended reality systems can help personnel learn from previous missions and exercises. Just as XR can display a common or user-defined operating picture during missions, they can do so for after-action reports to help personnel visualize and interact with the sequence of events. For instance, teams from the U.S. Navy ‘created a virtual sand table that allows for easier post-mission analysis’ (NSWCDD Corporate Communications, 2022). Researchers for the U.S. military further tested AR in training by using flight data to display a rendered recording of an entity’s movements. (NSWCDD Corporate Communications, 2022). Because the data is timestamped, it can be fused with the corresponding time, which allows for flexibility and insight after a mission.
XR will enhance commanders’ situational awareness, enable speedier and perhaps improved (through visualized real-time modeling and simulation) decision-making, and provide additional advantages, such as the avoidance of fratricide or civilian casualties.
Conclusion
While some applications of XR are gaining in maturity— particularly those incorporating standalone AR—others are at a more nascent stage of development. We are some distance away from providing a user of XR, at the VR end of the XR spectrum, a perceived environment that is indistinguishable from a real-world experience. It is important to remember that much of the technological progress in XR will largely take place in the commercial sphere. Consumer gaming may develop superior VR technologies in the short term—at least until defense enterprises consider such technology to be essential to military effectiveness and begin funding VR programs to the same level.
Even those AR devices that are already at a relatively high technology readiness level (TRL) will still encounter persistent challenges in integration with other technologies and systems, as well as delays in fielding and testing. AR devices are already showing considerable promise, but we should be careful that our expectations about the near-term impact of XR technology do not exceed reality. Moreover, XR applications may have other adoption challenges for armed forces, such as producing greater demand for maintainers or IT and cybersecurity personnel. In addition, some analysts have raised concerns about the potential cybersecurity vulnerabilities of XR systems.
The article has attempted to show that XR technology has great potential to assist commanders with exploiting increasingly data-rich operating environments. XR will enhance commanders’ situational awareness, enable speedier and perhaps improved (through visualized real-time modeling and simulation) decision-making, and provide additional advantages, such as the avoidance of fratricide or civilian casualties. But it would be an error to believe that the prospects of exploiting data inputs in this way will revolutionize warfare per se. Military historian and theorist Martin Van Creveld (1985) argued some decades ago that military commanders were on a ‘quest for certainty’ that would reduce the risk of their decisions. There is unlikely to be anything approaching certainty on the battlefield. As Peter Roberts, professor of military sciences, contends, ‘There is nothing wrong with using more data to fix some of the imponderables of decisions in war. Still, there must also be pushback against any belief in perfection, eliminating risk, and deferring decisions to wait for the final piece of the puzzle’ (Roberts, 2023). XR may help commanders to visualize fragments of the battlespace, but it will not allow them to cut through the fog of war.
References
BAE Systems. (2015), Augmented reality systems set to revolutionise battlefield operations [Online] BAE Systems. Available from: https://www.baesystems.com/en-uk/article/augmented-reality-systems-set-to-revolutionise-battlefield-operations
BAE Systems. (2023), The future of military training: Synthetic environments and the military metaverse [Online] BAE Systems. Available from: https://www.baesystems.com/en-us/feature/the-future-of-military-training-synthetic-environments-and-the-military-metaverse
Boyce, M. W., Thomson, R. H., Cartwright, J. K., Feltner, D. T., Stainrod, C. R., Flynn, J., Ackermann, C., Emezie, J., Amburn, C. and Rovira, E. (2022) Enhancing military training using extended reality: A study of military tactics comprehension. Frontiers in Virtual Reality, 3, 754627
Congressional Research Service. (2022), Military Applications of Extended Reality [Online]. Available from: https://crsreports.congress.gov/product/pdf/IF/IF12010
Crevald, M. (1985). Command in War. Cambridge, Massachusetts : Harvard University Press.
Hugues, Olivier, Philippe Fuchs, and Olivier Nannipieri (2011). “New Augmented Reality Taxonomy: Technologies and Features of Augmented Environment”. In: Handbook of Augmented Reality. doi: 10.1007/978-1-4614- 0064-6_2. B. Furht, Ed. New York, NY, USA: Springer, pp. 47–63. isbn: 978-1-4614-0063-9.
Jomini, A. (1838). The Art of War New York. New York: Greenhill Press.
Jenkins, M., Wollocko, A., Negri, A. and Ficthl, T. (2018). Augmented Reality and Mixed Reality Prototypes for Enhanced Mission Command/Battle Management Command and Control (BMC2) Execution in J.Y.C. Chen and G. Fragomeni (eds) Virtual, Augmented and Mixed Reality: Applications in Health, Cultural Heritage, and Industry. Cham: Springer International Publishing, pp. 272–288. Available from: https://doi.org/10.1007/978-3-319-91584-5_22
Kallberg, D.J., Beitelman, V., Mitsuoka, V. and Pittman, J. (2022) The Tactical Considerations of Augmented and Mixed Reality Implementation, MILITARY REVIEW [Preprint].
Lele, A. (2013) Virtual reality and its military utility. Journal of Ambient Intelligence and Humanized Computing, 4, pp17-26.
Liaropoulos, A.N. (2023). Digitizing the Battlefield: Augmented and Virtual Reality Applications in Warfare in Routledge Handbook of the Future of Warfare. London: Routledge.
NATO ACT. (2023), NATO Allied Command Transformation and the Latvian Ministry of Defence Host Next Generation Communication Networks Technology Event [Online]. Available from: https://www.act.nato.int/article/act-latvian-mod-host-next-gen-technology-event/ [11 October 2024].
NSWCDD Corporate Communications. (2022), Virtual Reality Takes Foothold in Battle Management System [Online] Naval Sea Systems Command. Available from: https://www.navsea.navy.mil/Media/News/Article-View/Article/3226969/https%3A%2F%2Fwww.navsea.navy.mil%2FMedia%2FNews%2FArticle-View%2FArticle%2F3226969%2Fvirtual-reality-takes-foothold-in-battle-management-system%2F [08 October 2024].
Papathanasiou, C. and Karadimas, N. (2023). Augmented Reality and its Contribution to Enhance the Operational Capabilities of the Armed Forces. Earth sciences and human constructions, 3, pp. 49–55. Available from: https://doi.org/10.37394/232024.2023.3.5
Roberts, P. (2023), Command and Control: The Quest for Certainty [Online] Wavell Room Blog. Available from: https://wavellroom.com/2023/06/15/command-and-control-the-quest-for-certainty/
Robertson, A. (2023), The Meta Quest 3 is sharper, more powerful, and still trying to make mixed reality happen [Online] The Verge. Available from: https://www.theverge.com/2023/9/27/23890731/meta-quest-3-headset-hands-on-mixedreality-connect
Rossiter, A. (2023) Hyping emerging military technology: probing the causes and consequences of excessive expectations. International Relations, 00471178231186256.
Rossiter, A., and Layton, P. (2024). Warfare in the Robotics Age. Boulder, Colorado: Lynne Rienner Publishers.
Saballa, J. (2024), Anduril to Revive U.S. Army’s Troubled IVAS Program, Partners With Microsoft [Online] The Defense Post. Availablefrom: https://thedefensepost.com/2024/09/20/anduril-ivas-program-microsoft/
Schwab, K. (2017). The fourth industrial revolution. New York City: Crown Currency.
UK Ministry of Defence. (2017) Joint Concept Note 2/17: Future of Command and Control. Available from: https://assets.publishing.service.gov.uk/media/5a81c7a040f0b62305b90c42/concepts_uk_future_c2_jcn_2_17.pdf
Watling, J. (2023). The Arms of the Future: Technology and Close Combat in the Twenty-first Century. London: Bloomsbury.
