Our Blog

Enthusiasts have introduced a remarkable feature that combines Sora’s video-generating capabilities with ElevenLabs’ neural network for sound generation. The result? A mesmerizing fusion of professional 3D locations and lifelike sounds that promises to usher in an era of unparalleled creativity for game developers. How It Works In the context of game development, it should have looked like this: Capture Video with Sora: People start by capturing video content using Sora, a platform known for its advanced video generation capabilities. Luma Neuron Transformation: The captured video is then passed through the Luma neuron. This neural network works its magic, transforming the ordinary footage into a spectacular 3D location with professional finesse. Unity Integration: The transformed video is seamlessly imported into Unity, a widely-used game development engine. Unity’s versatility allows for the integration of the 3D video locations, creating an immersive visual experience that goes beyond the boundaries of traditional content creation. Voilà! The result is nothing short of extraordinary – a unique 3D location ready to captivate audiences and elevate the standards of digital content. A Harmonious Blend of Sights and Sounds But the innovation doesn’t stop there. Thanks to ElevenLabs and its state-of-the-art neural network for sound generation, users can now pair the visually stunning 3D locations with sounds that are virtually indistinguishable from reality. By simply describing the desired sound, the neural network works its magic to create a bespoke audio experience. This perfect synergy between Sora’s visual prowess and ElevenLabs’ sonic wizardry opens up a realm of possibilities for creators, allowing them to craft content that not only looks stunning but sounds authentic and immersive. OpenAI’s Sora & ElevenLabs: How Will They Impact Game Development? The emergence of tools like OpenAI’s Sora and ElevenLabs sparks discussions about their potential impact on the industry. Amidst the ongoing buzz about AI revolutionizing various fields, game developers find themselves at the forefront of this technological wave. However, the reality may not be as revolutionary as some might suggest. Concerns Amidst Excitement: Unraveling the Real Impact of AI Tools in Game Development Today’s AI discussions often echo the same sentiments: fears of job displacement and the idea that traditional roles within game development might become obsolete. Yet, for those entrenched in the day-to-day grind of creating games, the introduction of new tools is seen through a more pragmatic lens. For game developers, the process is straightforward – a new tool is introduced, tested, evaluated, and eventually integrated into the standard development pipeline. AI, including platforms like Sora and ElevenLabs, is perceived as just another tool in the toolkit, akin to game engines, version control systems, or video editing software. Navigating the Practical Integration of AI in Game Development The impact on game development, in practical terms, seems to be more about efficiency and expanded possibilities than a complete overhaul of the industry. Developers anticipate that AI will become part of the routine, allowing for more ambitious and intricate game designs. This shift could potentially lead to larger and more complex game projects, offering creators the time and resources to delve into more intricate aspects of game development. However, there’s a sense of weariness among developers regarding the constant discussion and hype surrounding AI. The sentiment is clear – rather than endlessly discussing the potential far-reaching impacts of AI, developers prefer practical engagement: testing, learning, integrating, and sharing insights on how these tools can be effectively utilized in the real world. OpenAI — for all its superlatives — acknowledges the model isn’t perfect. It writes: “[Sora] may struggle with accurately simulating the physics of a complex scene, and may not understand specific instances of cause and effect. For example, a person might take a bite out of a cookie, but afterward, the cookie may not have a bite mark. The model may also confuse spatial details of a prompt, for example, mixing up left and right, and may struggle with precise descriptions of events that take place over time, like following a specific camera trajectory.” So, AI can’t fully create games and its impact might be limited. While it could serve as a useful tool for quickly visualizing ideas and conveying them to a team, the core aspects of game development still require human ingenuity and creativity. In essence, the introduction of AI tools like Sora and ElevenLabs is seen as a natural progression – a means to enhance efficiency and open doors to new creative possibilities. Rather than a radical transformation, game developers anticipate incorporating AI seamlessly into their workflow, ultimately leading to more expansive and captivating gaming experiences.

This article was written by our CEO Olga Kryvchenko and originally published on Linkedin. To get more biweekly updates about extended reality, subscribe to Olga’s XR Frontiers LinkedIn newsletter. The power of touch has long been recognized as a potent sensory modality. With the rise of Extended Reality (XR) technologies, touch, or more precisely, haptic feedback, has found its profound significance. Especially in hazardous industries, where direct training poses risks, XR combined with haptic feedback offers a revolutionary approach. Understanding Haptic Feedback At its core, haptic feedback simulates the sense of touch and movement. This tactile feedback, when incorporated in digital interfaces or XR environments, provides users with realistic sensations ranging from a gentle breeze to a jolt from a virtual electric shock. The Evolution of Haptic Devices While the rudimentary concept of haptic technology revolves around vibrations, contemporary haptic devices offer far more intricate feedback. – Glove-based Systems: Pioneering the future of touch in XR, these gloves simulate intricate textures, temperatures, and resistances. Users can virtually touch a hot surface or feel the graininess of sand. – Vest Systems: These aren’t your average vests. They’re equipped to simulate everything from the impact of a bullet to the gentle tap on the shoulder. – Treadmills and Platforms: Beyond letting users walk or run in a virtual space, these devices offer gradients, and resistances, and even simulate different terrains. – Haptic Controllers: These handheld devices can simulate weight, resistance, and more. They’re often used in XR setups where precision is required, like machinery operation or surgical training. Spotlight: Some XR Haptic Innovations 1. Manus Meta Gloves: Advanced haptic gloves that simulate intricate tactile experiences within virtual spaces, ideal for hazardous industry simulations. 2. SenseGlove: Offering unparalleled haptic feedback, this device simulates realistic touch sensations, ensuring authentic training experiences in hazardous industries. 3. Electric Haptic Vest: A revolutionary vest that provides electric haptic feedback, enabling users to feel virtual impacts, touches, and temperature changes, enhancing training realism. These innovations are setting the benchmark for immersive XR training in high-risk sectors, ensuring both safety and efficacy. Incorporating Haptics in XR Training for Dangerous Industries Haptic feedback is not just a fancy addition; it’s a necessity in industries where mistakes can be catastrophic. 1. Realistic Hazard Simulation: XR environments equipped with haptic feedback allow trainees to understand and feel dangerous situations, like the rumble of an impending mine collapse or the heat from an electrical malfunction. 2. Skill Refinement: Fine motor skills can be honed in a virtual space. Imagine a trainee learning to operate a chainsaw; with haptic feedback, they can feel the tug, the resistance, and even the vibration, all without the real danger. 3. Emergency Protocols: Virtual emergencies can prepare employees for real-world crises. These drills aren’t just visual or auditory; haptic feedback ensures they are physically intuitive. Looking Ahead: The Future of Haptic XR Training The fusion of haptic devices and XR promises a future where training for even the most hazardous jobs is thorough, intuitive, and above all, safe. As the fidelity of haptic feedback continues to improve, the line between virtual training and real-world operation will further blur, leading to a workforce that’s adept and prepared for any challenges. In an era dominated by technological advancements, haptic feedback stands out as a game-changer for XR training in dangerous industries. It’s not just about seeing or hearing; it’s about feeling, and that tactile element ensures that trainees are not only knowledgeable but are also instinctively prepared for real-world scenarios. Image: Freepik

In this article, we will explore the impact of augmented and virtual reality on national parks, specifically how they are changing our experience of visiting natural landscapes. We focus on free virtual tours that allow us to see the beauty of national parks without leaving home. We will also reveal the main advantages of augmented reality, which enrich the visiting of real parks with the help of digital content with interesting information. In particular, we will share our experience of creating tourist VR excursions using the example of the Carpathian Mountains Virtual Tour and Medieval Fortress Virtual Journey. We will also explore other famous AR/VR use cases, e.g., the State Parks of California, giving us a unique perspective on our national parks’ historical and ecological significance. Discovering Benefits of AR and VR in National Parks Virtual and augmented reality offer such benefits to national parks as Getting informative content on the spot. When visiting parks and other historical places, visitors usually don’t know about this or that type of plant or even in which part of the park they are. With the help of an AR application, a tourist can simply point the screen at a particular plant and get additional information about it. Read also: The Ways AR Redefines Modern Tourism  Speaking about orientation in place, AR navigation will come in handy here, which guides the tourist in the right direction with the help of digital pointers. Read also: From Outdoors to Indoors: AR Navigation as Game-Changer Visiting places that aren’t accessible for you. During the COVID-19 pandemic and home isolation, virtual tours of museums or national parks were one of the popular alternatives to visiting real tourist locations. Even though the pandemic began to gradually fade away, virtual tours continue to be relevant. By visiting free tours on museums or park websites, an Internet user saves money on the ticket, car fuel, and transportation. Especially when a person does not always have an opportunity to get to the place where, perhaps, he would like to go. “The advantage is that you can go somewhere and experience the same feelings without leaving your home. Virtual guides have musical accompaniment, the effect of presence thus allows you to immerse yourself in the environment. This is all about VR. Plus, not all of us can go to Papua New Guinea or somewhere else to look at bamboo, baobabs, etc. Or go on safari. Therefore, these are opportunities to visit somewhere without spending additional funds,” said Oleksandr Ivanov, project manager of Qualium Systems. Drawing attention to ecology, rare plants and animals. As you know, there are more than 35,000 species of endangered plants and animals in the world, 99% of which are endangered due to human activities. With virtual and augmented reality for parks that are the home for these species of animals, it becomes possible to tell about these endangered species to a wider audience. As of July 2023, there are almost 7 billion people in the world who have smartphones, which is more than 86% of the entire population of the planet.  Qualium System’s Virtual Tours of National Parks Qualium Systems has great experience in developing VR excursions. Carpathian Mountains Virtual Tour is an application for Meta Quest, created as part of The World Of Carpathian Rosettes — Activities For Preserving The Cultural Uniqueness Of The Carpathians project by the Euroregion Carpathians — Ukraine association with the financial support of the European Union. The main goal of this program is to preserve the cultural uniqueness of the Ukrainian Carpathians. The main challenge in the development of this virtual tour was to introduce more than 260 photo and video materials into the program while providing the VR headset users with such an experience as if they were visiting a real museum. In this application, a tourist can explore virtual settlements in the Carpathians, both with the effect of presence in these settlements and from a bird’s eye view. There is also the option of a “bicycle trip” available. A tourist can “ride” along the Franko route, which is a path in the forest, famous Ukrainian poet Ivan Franko liked to travel. Also, our developers are working on another virtual tour of the medieval fortress in the Ukrainian town of Tustan. Unlike the previous tour, a VR headset user will not only be able to see the digital environment with exhibits but also interact with them. “With the help of VR, you can not only be there, in that museum, but also take the artifacts, examine, bring them up to your forehead. You could see everything down to every crack,” said Oleksandr Ivanov, project manager of Qualium Systems. Unveiling Diverse Use Cases of AR and VR in National Parks Our company pays attention to virtual tours and giving people the opportunity to see places that are inaccessible to them due to various circumstances. We constantly monitor the latest trends in the modern IT industry, and there are other famous examples that will be mentioned later. Virtual Reality Adventures in California State Parks The California Department of Parks and Recreation has introduced virtual and augmented reality tours for 9 parks located in the state of California. In the Virtual Adventurer mobile app, tourists get a guided tour that tells the story of each of the parks, including Colonel Allensworth State Historic Park, Montana de Oro State Park, Old Town San Diego Historic Park, and others. Moreover, futuristic holograms and 3D reconstructions are also involved. “The app also supports and enhances the department’s Reexamining Our Past Initiative by developing content for parks that tells a more complete, accurate, and inclusive history of people and places,” said California State Parks Director Armando Quintero.  Great Barrier Reef’s Virtual Realms As mentioned above, virtual excursions allow gadget users to get to places they would be unlikely to get to in real life. The same is true for the underwater attractions. Not every tourist wants to spend their time and money not only to reach, for example,…

In recent years, IT and immersive technologies are one of the most popular fields that people choose for their future careers. According to Statista, as of January 2023, the number of IT workers was 3.1 million, and by 2032 the number of workers will exceed 6 million. Among those starting to work in the field of IT and XR, there are both young professionals and those who have career experience in other fields and want to change their lives. And free online courses, which introduce a beginner to the essence of the matter in an interesting and accessible language, are useful for future developers. In this article, we offer an interesting selection of the best free XR training courses online. Whether you’re looking to develop XR experiences for Unity and Magic Leap 2, shoot a 360o film, or create your own metaverse, our selection of courses will provide you with the necessary skills and knowledge needed for those just starting their XR journey. Create with AR: Face Filters by Unity A training course for those new to AR application development, including creating face filters. This course is designed for those willing not only to learn the basics of programming applications on Unity but also to expand their portfolio with new cases, which will make it much easier to enter the field of XR. The course is divided into three parts: Get Started With AR. The future developer will learn the types of features in augmented reality, such as AR face filters, and AR markers that allow you to place digital content in real space. One will learn how to integrate a finished 3D model design into a game. Create a basic face filter. The main goal of this segment is to create your own customized AR face mask, including creating face trackers and using various textures, 3D models, and other components that can be used for an AR mask; Create an interactive face filter. In the final part of the course, the student has to make their mask interactive. In particular, developers also learn to add a user interface to the created mask and use visual scripts to program functions. A student is able to independently monitor their own progress and mark the completed lessons on the page. The entire course is available here: https://learn.unity.com/course/create-with-ar-face-filters?uv=2021.3  Virtual Reality Development Specialization by the University of London A virtual reality training program by the University of London is designed for beginners and consists of 5 courses. Its total duration is 6 months, 4 hours per week, with a flexible schedule. Each of the 5 training courses contains the following information: Introduction to virtual reality. Students get acquainted with the concept of virtual reality, its history, and basic principles; 3D Models for Virtual Reality. In this course, students will take the first steps in developing 3D models for VR, the basic principles of digital content development, and overlaying models in a digital environment. Students will also learn the basic principles of one of the most common VR engines, Unity3D; Interactive design with 3D models in VR. Students will learn to work with a key aspect of virtual reality, i.e., creating user interaction with the digital world. The course will cover the basic principles of creating interaction with virtual reality, taking into account the natural movements of the human body. In the end, students will receive useful advice from experts in the field of XR, and the main task to consolidate the result will be the creation of their own project where a person interacts with virtual reality; Creation of interactive 3D characters and dialogues. Students will learn to create communication between users of virtual reality, in particular the psychology of communication in VR, the basic principles of animation of 3D characters, in particular, game characters that will be used by the player and NPCs that will interact with a real player; Creating the First VR Game is the final course of the curriculum from the University of London, in which students have to create their first game in virtual reality and learn all the main stages of its development: idea, storyboarding, prototyping, testing, and implementation. The hosts of the Virtual Reality Development Specialization program are Professor Marco Gillies and Dr. Sylvia Xueni Pan. Gillies is an Academic Director of Distance Learning and Deputy Head of Computing at the University of London. Pan is a senior lecturer at the Virtual Reality department of the same university. You can sign up for the course here: https://www.coursera.org/specializations/virtual-reality  Extended Reality for Everybody by Michigan University  A learning program from the University of Michigan that introduces students to the general concept of extended reality. Its total duration is 3 months, 7 hours a week. The main goal of the program is to teach students the basic concepts of XR, the key problems of the development of augmented reality, and to correct them in further work with programs, as well as to learn how to create their own programs. The program consists of three main courses: Introduction to VR/AR/MR/XR, Technologies, Applications, and Key Issues. An introductory course to the concept of augmented reality where students will learn to understand the essential difference between virtual, augmented, mixed, and extended reality. Students will also learn more about the current state of XR, its main advantages, level of development, and problems; User experience and VR/AR/MR interaction design. In this course, students will learn the basic principles of creating XR experiences, including creating extended reality prototypes and putting them into operation. In addition, the issue of ethics in creating an interactive XR application for the user will be raised here; XR applications development with WebXR, Unity, and Unreal Engine. Students will gain practical skills in creating VR experiences in WebXR or Unity, AR applications with or without markers. The final course is where students will learn the basic concepts and techniques of creating complex applications in augmented reality, as well as learn to create their own apps taking into account the ethics, privacy,…

AI has already begun to be applied in areas where it is necessary to analyze and generate content. In particular, artificial intelligence is implemented in game and app development. AI is getting popular primarily due to its ability to significantly speed up and facilitate the content development process. Instead of spending several hours manually creating graphics for games and applications, you can use AI to generate the necessary components on its own simply. Then the developer will make their edits. Therefore, it is not surprising that the number of companies using AI is growing. According to  Forbes Advisor, by 2027 the artificial intelligence market will be estimated at 407 billion USD. And 64% of enterprises believe that artificial intelligence will increase their productivity. Unity, one of the most famous app manufacturers for portable gadgets, is among the companies that are already gradually introducing AI into their work.  In this article, we will take a detailed look and analyze the new artificial intelligence developments from Unity and what they mean for the development of XR applications. Leveraging Unity AI for App Development Unity has been using AI for a few years now, but, firstly, it was mostly for marketing and non-game content to a wider audience. Like, for example, Luna, a platform that aims to improve the advertising strategy of brands. Or Supersonic which is a stand-alone platform that automates and accelerates game advertising. Secondly, artificial intelligence and neural networks for gaming have previously been created only for developers who have special qualifications for working with applications. Like, for example, ML agents, which allow you not to code the behavior of digital elements, because these elements themselves (smart agents) are independently learning the algorithms of behavior in the environment. For example, in a game from Unity, you can hone the behavior of a digital corgi that runs after a stick. Or the behavior of racing cars, as in this video. Read also: Transforming Reality: How AI Revolutionizes XR for Next-Level Experiences “In the carting micro game, our ML agent scenes are typically set up with a few major components: the training environment, the level, that we want our agents to observe and familiarize with. Which, in our case, is our racetracks. The agents are the game objects we want to train. So, that they can accomplish a goal, such as driving without colliding with the track walls. And, finally — our academy, which collects all observations, done by our agents and trains them”, said Unity’s official video. In March 2023, Unity announced their own AI game and application development features. Then there was the first presentation of an ecosystem with artificial intelligence, which operates according to the principle of many AI generators, such as Midjourney and ChatGPT. A few days ago, new AI tools for developing games and applications were announced: Unity Muse and Unity Sentis, which we will discuss in more detail below.  Unity Muse: Empowering AI Creativity Unity Muse is the newest platform with artificial intelligence, the main purpose of which is to accelerate the creation of applications with 3D graphics. In particular, the option of this platform includes such basic functions as: automatic creation of animated characters and digital doubles; creation of the movements of the animated model — for example, jumps or backflips; creating a texture for the environment; addition to ready-made 3D models — for example, the video shows how you can highlight the roof area on the digital model of the house, enter a prompt and select several options for a red roof from the database. “The Muse platform provides quick access to detailed information from the documentation and can also generate 2D sprites and 3D animations. This allows developers to quickly get the resources they need to create prototypes or new experiences without waiting for final versions of models and sprites. With Muse chat, developers can effectively communicate with other team members and get the help they need during development”, said Qualium Systems Unity tech lead Arcueid D’athemon. The main difference between Unity Muse and ML agents is ease of use. The newest platform looks like a chat with a neural network and the principle of operation resembles the sensational ChatGPT. As mentioned above, the user enters the required prompt to generate the 3D content required for the application. Harness the Power of Unity Sentis AI Unity Sentis is a cross-platform tool that integrates AI into Unity applications. According to Unity’s official announcement, Sentis enables in-app digital models to work flawlessly on all operating systems, browsers, and devices that support Unity, from smartphones and tablets to game consoles and XR glasses. Thus, applications with built-in AI models work more efficiently due to the fact that data is processed in real-time. However, according to Arcueid D’athemon, Unity tech lead at Qualium Systems, the range of  Sentis options is wider, and the platform can also generate voice acting and animation for a character. “It opens up opportunities to quickly create dynamically generated actions in the application that will depend on the environment or interaction with the user. With Sentis, developers can effectively create realistic character movements and high-quality voice acting that enriches the user experience”, said Arcueid D’athemon. In the end, the combination of the input of the own neural network can change the process of the application use. For example, the user can directly interact with an NPC, while would respond to him with lines that are not pre-recorded. It enhances the immersive experience of gameplay, using the example of a virtual AI character named Orb. Unity’s interest in AI in app development is evident with recent innovations such as Unity Muse and Unity Sentis. The first tool allows developers to quickly and efficiently create 3D models of game and non-game characters, environments, and textures. As for Sentis, this program allows you to modify the gameplay by introducing NPCs with their own artificial intelligence. Unity’s AI ecosystem continues to evolve, inspiring game and app developers to explore new horizons and reimagine what’s possible in…

Today, artificial intelligence is a driving force that transforms and facilitates a whole range of fields: from creating texts and images to improving the performance of entire applications. Moreover, AI is becoming an essential element in developing virtual and augmented reality. Artificial intelligence has already become an assistant for developers of such companies as Magic Leap, Unity, and others. In this article, you will learn more about the solutions and benefits that AI brings to the development of extended reality. AI and XR: Perfect Fusion for Immersive Experiences Companies involved in the development of extended reality applications can use artificial intelligence for various purposes. Here are some ways in which AI can be applied. Personalized Realities: Power of AI in Customizing XR Experiences AI can analyze data on the behavior and preferences of VR users to offer more personalized and immersive content. For example, AI can customize the virtual environment to a user’s individual needs, as well as suggest content or interactive elements that may be of interest to a user. Enhancing Interactions: AI’s Role in Developing XR Assistants AI can be used to develop virtual reality agents with advanced intelligence. These agents can interact with users, answer questions, provide information, and create more realistic scenarios. The use of AI makes it possible to create agents that can adapt to the user’s actions and demonstrate intellectual behavior. Magic Leap has already successfully implemented artificial intelligence to provide better and more realistic XR experiences to improve human interaction. They are already using AI algorithms to develop digital models in AR, one of which is the MICA hyperrealistic human. Using these AI algorithms, the creators managed to give their digital model eye movements, facial expressions, and behavior that are as close as possible to a living person. “AI components were added to track a user and look them in the eye. Additional AI elements were added for body language and posture. Our team focused on natural facial expressions, we wanted MICA to emote in believable ways,” said John Monos, vice president of Human-Centered AI at Magic Leap. Power of AI in Analytics and Development AI can analyze the collected data while using virtual reality applications to identify patterns, trends, and problems. This information can be used to improve the development process and optimize the user experience. For example, AI can help identify the most popular features or discover bottlenecks in an application that need optimization. Alex Volkov, Qualium Systems head of XR development, wrote an article about AI implementation in game development and how it optimizes and speeds up the work of a game developer, using ChatGPT as an example. “In fact, it can save up to 60–70% of the time required for task implementation. Being a professional developer, I knew how to implement a task from scratch, but still needed to write each line of code, optimize it, and debug it. However, with ChatGPT, I managed to save a lot of time on the first point and focus on the last two points,” said Volkov.  You can read the full article about game development using AI here.   Rise of AI in XR: Enhancing Content Analytics and Recognition AI can be used to recognize and analyze content in virtual reality. For example, AI can automatically analyze video and audio materials, detect objects or events in VR scenes, and also provide speech recognition and natural language processing functions. AI-Enhanced XR: Revolutionizing Safety and Management Practices AI can help manage and secure the virtual space. For example, artificial intelligence can control and optimize resources, manage virtual objects, and keep users safe by preventing unwanted situations or interactions. For example, Unity is working on developing its own AI algorithm for honing the work of 3D models. “Unity enables the real-time training of models based on unique datasets produced in the creation and operation of RT3D experiences. Through this training, we can build ever-richer services on top of Unity and provide extraordinary capabilities for our partners to leverage Unity as a data creation, simulation, and training engine for their own needs. Natural-language AI models incorporated into the Unity Editor and runtime train on real code and images. That real-usage training data is abstracted from its initial use (it’s not captured or recorded as-is), however, this learning enables Unity’s customers to substantially increase their productivity,” said a blog post.   These are only a few examples of how XR companies can use artificial intelligence. The application of AI depends on the specific needs and goals of the company, as well as on available technologies and resources. Image: Unsplash.

Brief overview of ChatGPT and its potential uses in game development ChatGPT (Generative Pre-trained Transformer) is an artificial intelligence language model that has been pre-trained on a massive amount of text data. It is capable of generating human-like language and can be used for a variety of natural language processing tasks, including text completion, summarization, and translation. In game development, ChatGPT can be a valuable tool for generating code and providing suggestions or prompts for developers. By inputting a description of a desired game feature or behavior, ChatGPT can generate code that can help developers save time and improve the quality of their work. For example, ChatGPT could be used to generate code for complex AI behaviors, physics simulations, or other game mechanics. It can also be used to suggest improvements or optimizations to existing code. While ChatGPT is not a replacement for skilled game developers, it can be a valuable tool to help streamline the development process and allow developers to focus on the creative aspects of game development. As AI and machine learning continue to advance, it’s likely that ChatGPT and other similar tools will become increasingly important in game development and other fields. Introduction to the specific task of creating floating stones that change speed based on the player’s distance In an existing game, I was tasked with implementing a group of floating stones that would change behavior as the player moved closer to them. In their idle state, the stones should float smoothly and slowly, but as the player approaches, they should start to jitter more and more. This required creating a class, implementing dependencies, and other code that couldn’t be achieved through animator controllers or libraries. While this wasn’t a “nightmare” level task, it was still time-consuming. ChatGPT proved to be a useful tool for generating code snippets and saving time in the development process. Explanation of how ChatGPT can be used to generate code for game development tasks When working with ChatGPT, it’s important to start with the right context. In my case, I began with a promo message about the technology I planned to use. It’s important to keep in mind that ChatGPT is just a tool and won’t generate ideas or code on its own. You need to provide clear and detailed input for the best results. That said, ChatGPT can be a real-time-saver. When using the tool, it’s essential to provide a detailed description of what you’re trying to achieve. The more information you provide, the better the output will be. It’s important to note that using ChatGPT should take less time than achieving the same result without it. So, don’t be afraid to put in the effort to provide a detailed description. With ChatGPT, you can streamline your development process and focus on other aspects of your project. Example prompts and code snippets used to generate the necessary code for this task Let’s dive into the practical use of ChatGPT. As I mentioned earlier, I started by providing context about my game engine and coding language. I want to ask you to help us create some code for my game based on Unity SDK, C# code language. ChatGPT responded kindly, and I moved on to the next step — providing a detailed description of the task and its conditions. A few words about the context of the task. In the game, players can find floating stones. These stones have random directions of jittering, but this jittering is very smooth. If the player appears within around 10 meters of the stone, the jittering speed becomes faster and the speed depends on the player’s distance to the stone (more if closer). As a result, ChatGPT provided me with a basic realization of the class I needed to implement. While the code wasn’t perfect, it covered around 80% of what I needed to do. using UnityEngine; public class FloatingStone : MonoBehaviour { public float maxSpeed = 1f; public float minDistance = 10f; private Vector3 initialPosition; private float initialSpeed; private bool isPlayerNearby; private void Start() { initialPosition = transform.position; initialSpeed = Random.Range(0.2f, 0.5f); } private void Update() { float speed = isPlayerNearby ? CalculateJitteringSpeed() : initialSpeed; transform.position = initialPosition + new Vector3( Mathf.PerlinNoise(Time.time * speed, 0) – 0.5f, Mathf.PerlinNoise(0, Time.time * speed) – 0.5f, Mathf.PerlinNoise(Time.time * speed, Time.time * speed) – 0.5f ); } private float CalculateJitteringSpeed() { float distanceToPlayer = Vector3.Distance(transform.position, PlayerController.instance.transform.position); float speed = Mathf.Lerp(initialSpeed, maxSpeed, (minDistance – distanceToPlayer) / minDistance); return Mathf.Clamp(speed, initialSpeed, maxSpeed); } private void OnTriggerEnter(Collider other) { if (other.CompareTag("Player")) { isPlayerNearby = true; } } private void OnTriggerExit(Collider other) { if (other.CompareTag("Player")) { isPlayerNearby = false; } } } Next, I made some corrections — I added context about using Zenject for player injection and refactored the class to align with project conventions. I then asked ChatGPT to use my updated code version and add jittering rotation to the object based on the same conditions. After this step, I received the final version of the code that I could use in my project. I made some minor changes on my end, and in the last step, I asked ChatGPT to add XML comments to the final class and received the desired result. The final result of the code public class FloatingStone : MonoBehaviour { [SerializeField] private SpeedSettings speedSettings; [SerializeField] private float minDistance = 10f; [SerializeField] private float amplitude = 0.5f; [SerializeField] private float rotationAmplitude = 1f; [SerializeField] private new SphereCollider collider; private Vector3 initialPosition; private Quaternion initialRotation; private float initialSpeed; private bool isPlayerNearby; private Transform player; [Inject] private void Constructor(IPlayer player) { this.player = player.Transform; } private void Awake() { initialPosition = transform.position; initialRotation = transform.rotation; initialSpeed = speedSettings.GetRandomSpeed(); collider.radius = minDistance; } private void Update() { float speed = isPlayerNearby ? CalculateJitteringSpeed() : initialSpeed; Vector3 newPosition = initialPosition + new Vector3( Mathf.PerlinNoise(Time.time * speed, 0) – amplitude, Mathf.PerlinNoise(0, Time.time * speed) – amplitude, Mathf.PerlinNoise(Time.time * speed, Time.time * speed) – amplitude ); Quaternion newRotation = initialRotation * Quaternion.Euler( Mathf.PerlinNoise(Time.time * speed, 0) * rotationAmplitude – rotationAmplitude…

Virtual reality is gradually becoming more widespread in many fields, including music. With its help, newcomers learn how to play musical instruments in a more exciting and immersive environment, that facilitates the learning process. Students can acquire a certain musical instrument in virtual reality since VR is not just about fancy environments. Some apps allow a headset user to use real musical instruments, with digital clues overlaid on them. Seeing these clues, a student masters their skills much faster.  This article aims to showcase in detail various virtual solutions that facilitate music learning. Experience the Future of Music Education  In general, there are two types of VR music apps: games and learning apps. VR music games involve a headset user performing rhythmic tasks to the music. Beat Saber is a bright example of this game. While playing Beat Saber, a user hits digital cubes with lightsabers to the music. Meanwhile, virtual reality learning apps are developed for newcomers who plan to learn to play real musical instruments. The basic principle of VR music apps for learning is it’s interactive experience, where a student uses hand gestures and controllers to press virtual piano keys, pluck digital guitar strings, or beat virtual drums with digital sticks. Take Your Music to Next Level  Experience the Joy of Piano Playing  In a VR piano app, a VR headset user plays on a virtual or physical instrument, with digital clues are overlaid on it. These clues tell a student what piano key they should press on. VR-XR Piano App by ArtMaster allows a newcomer to learn how to play the piano in an unusual set. In the program, you can also choose a location that fits your mood: an opera, Moon surface, cliff, etc. There’s also a selection of songs and compositions to play and the option that allows you to adjust a composition’s speed and the length of notes. Moreover, this program also involves a physical piano, thus creating an additional immersion effect and facilitating the learning process. We at Qualium Systems are also developing a VR piano app. In this app, a VR headset user gets transferred into a virtual music hall with a digital piano. Similar to the previous use case, playing music on virtual piano keys requires the use of hands. The app also includes three levels of learning: easy, medium, and hard. The playlist of our future VR piano app includes both classic and modern songs. Enhance Your Drumming Skills  When dealing with VR drums, a headset user can apply controllers to use a digital stick to beat virtual drums and recreate movements while playing the real drums. Smash Drums is an example of a virtual reality app designed to teach how to play drums. It’s suitable for both newcomers and professional musicians. With a VR headset, users can try their hand at drumming on various difficulty levels, ranging from easy to challenging. A few months ago, the app got an update that includes an additional, highly challenging level that realistically recreates the movements of a drummer. Step Up Your Guitar Game  Virtual programs for learning how to play the guitar involve controller-free hands to pull digital guitar strings. Unplugged for Oculus Quest is an example of an app that uses similar principles. By the way, Tetiana, the Ukrainian host of Disco VR YouTube channel, attempted to play Unplugged, using a real electric guitar. Guitar Strummer is another use case, where a headset user can learn how to play the acoustic guitar. There are two ways to create, learn, and play new chords:  choosing notes that look like virtual balls and playing them with a drummer stick choosing a chord and playing with hands. In this app, you can create and save your own chords. After every play, a program estimates your playing chord performance.  Transforming Music Education: Advantages of VR Technology in Music Learning Virtual reality is becoming increasingly popular in music education due to its numerous advantages for learners, such as A personalized learning experience for musicians. Students can develop their skills in a digital environment that is suitable for them and isolates them from various distractions. Moreover, virtual clues are overlaid directly on a real instrument and allow a student to master a played composition. Virtual apps also provide a user with the possibility to choose the speed and note length of a composition. Compactness and constant access. Virtual instruments help you save some space in your home and play a musical instrument virtually at any suitable time. Only you can hear the music. Your neighbors won’t complain about music being too loud when you learn how to play piano or another musical instrument. Inspiring virtual environment. Sometimes, when a musician plays an instrument, they can find themselves in a not-so-supportive environment with a lot of distractions. Wearing a virtual reality headset, a musician can choose the most comfortable and inspiring place to learn music. Virtual reality has become a convenient tool for learning music by offering a convenient and personalized experience of mastering musical instruments, such as guitar, piano, drums, etc. These apps enhance users’ music skills and improve their performance in an exciting and inspiring digital environment. 

Our experience in XR project development shows that the applications our clients order often contain similar features, for example, it can be 2D drawing or interaction with objects. In each project, we developed features from scratch, even if we had already implemented them before. In order to optimize this process, we decided to develop a special library, the advantages of which we want to briefly tell you about in this post. This will be the first in a series in which we will focus on the benefits of this solution for our customers. In the future, we will talk in more detail about the structure of the library and its components. In short, the main idea of ​​our library is the development of typical features that we encounter from project to project. The library will contain the API with the core logic of the features and their implementation for specific Platforms, such as Oculus, Pico, iOS, Android, etc. There will be about 20 features in total that will be further supplemented. Why is it beneficial for our customers to have such a library? 1. The speed of project development increases, as a result, it becomes possible to bring the product to the market faster Due to the fact that we have already developed the main features for various platforms, there is no need to develop them from scratch for a new project. This saves time and therefore costs for our customers. 2. The reliability and quality of feature implementation is growing, while risks, code problems, and testing costs are being reduced All the features that we add to the library are constantly used in our projects. As a result, you get reliable functionality, perfected on various projects, which is constantly optimized. 3. The available functionality is expanding, as a result, it becomes possible to use all the properties of the features Clients need different functionality, for example, 2D drawing can be used in a VR smart office to draw on a flipchart, or it can be used in a 2D android application for signing documents. Thanks to our library, you will get all the available functionality of a feature, for example, the same 2D drawing, from which you can use the features you need. 4. Continuous improvement, resulting in the ability to receive feature updates even after the completion of the project After we have finished working on your project, the improvement of the library does not stop. You can also enjoy these benefits. We provide our customers with feature updates with new possibilities. To sum up, we want to once again dwell on the main advantages of the library we are developing: development speed, reliability and quality, accessible functionality, and continuous improvement. That’s all for today. Follow our future materials, they will contain more details on the internal filling of the library.