Flutter with Unity Tutorial: Can They Work Together? [Our Integration Experience]

Flutter and Unity Integration: Our Step-by-Step Experience

In this Flutter Unity tutorial, we’ll explore how these two tools can be effectively combined. Whether you need a real-world Flutter Unity example or insights on how Flutter and Unity integration works, we’ll guide you through our experience.

One of our previous posts discussed how these two technologies can work together and why they can be effective solutions from a business perspective. These included the possibilities of adding 2D/3D and AR/VR solutions to our existing application and reducing development costs by splitting work between a lightweight cross-platform framework like Flutter and the advanced Unity engine.

Let’s assume that we are already one step further before deciding on the final technology stack for the project. We have chosen Flutter for obvious reasons – it’s a strongly developing cross-platform technology that meets the performance expectations of the most demanding tasks. But how does it work in conjunction with Unity? What do we need to know, and how can we prepare for any potential challenges that we might face? 

After many months of work combining Flutter and Unity, we have made the mistakes and identified the essential issues for you to help answer all of these questions.

Flutter Unity Widget: How We Embedded Unity in Flutter

From all of the available solutions out there, we found the flutter_unity_widget library, built by juicycleff, to be the most impressive. Not only did we test its operation and scroll through any potential questions and problems that could arise, but our developers also actively contributed to the community by answering questions from other developers on GitHub and Discord.

How is the Unity project connected to the mobile app in Flutter?

While both Flutter and Unity projects can be developed separately, they have two aspects in common.

The first is communication, which takes place entirely in the background, and specifying what messages will be sent and processed by both environments in both directions is necessary. For example, commands to open or close a scene. 

The second is the building of the application. If we want to display scenes from Unity in our app, we first need to export the files required to build the project (Android and iOS) separately in Unity. When creating the app in Flutter, these files are then processed. For the app to run correctly, Android and iOS project configuration must then be carried out separately.

What does a Unity Developer need to know about working with Flutter?

Once the communication has been established and implemented in a Unity project, Flutter Developers can then go on to create a different environment where a Unity developer can build an application from within Flutter to test it.

This makes it possible for the Unity Developer to develop his part of the app separately and independently, and his knowledge of Flutter can be reduced to simply using ready-made commands to build the app.

What does a Flutter Developer need to know about working with Unity?

Setting up a Unity project and getting to the point of successfully building files for iOS and Android doesn’t have to be complicated and doesn’t even require as much additional work as you might think. Everything can be set up before making the application to limit potential errors, allowing you to go through everything step by step according to the established path you’ve already gone down many times before.

However, it’s important to note that it can be very time-consuming when a Flutter Developer is still learning to configure the project under Unity and is still discovering bugs. That’s why they will need to create an internal document to write down all the steps to deal with each case. 

Keep reading to learn more about essential aspects of specific issues.

How does it work moving between Unity and Flutter in an app? 

Once you launch Unity, it already exists in the background until you shut down the application. Moving between Flutter and Unity screens works by swapping the displayed screen, and we show the Flutter application or display the entire scene from Unity when needed. It’s also possible to contain a set from Unity on a portion of the screen in Flutter. Proper management of Unity in Flutter doesn’t cause any problems, ensuring the process looks and feels smooth. Here’s an example from our existing application to show you what it looks like when you move between the two:

Performance Insights for Flutter with Unity AR

Rendering Unity in Flutter is about as efficient as rendering a native view on Android or iOS. Projects are OK with performance issues as long as resources from Unity are correctly managed. 

How to manage the performance of Unity in Flutter?

If we already know that Unity runs in the application’s background when launched, then we want to reduce resources as much as possible when they aren’t needed. To do this, it’s best to prepare an empty scene in Unity that frees up the resources used when displaying other demanding scenes. Turn it on in Unity each time just before calling the command that displays the screen in Flutter and closes the Unity scene. The next time Unity calls the screen, the resources needed for the scene will reload.

But what if we want an immediate effect after displaying the scene from Unity? In that case, if we previously load the scene and leave it open, it will immediately show the next time we change between screens.

The second key issue is the first loading of the Unity project. Our team decided to initialize Unity right at the start of the application so that later when a particular game is first launched, only the scene content is loaded. 

What version control systems should you use for Flutter and Unity in one project?

Unity projects often consist of many assets of considerable size, so version control management should be divided so that these files skip this for the main project and avoid unnecessarily downloading a lot of data and creating large repositories. Secondly, these teams usually work independently, meaning that the work in Unity should be managed separately. For the mobile app, we use a distributed GIT system and Plastic SCM software for the Unity project. Only when we are ready on both sides do we update the files through Plastic SCM and build the project together with the latest changes across the project.

What are the limitations when combining Unity with Flutter? 

Only one of the two environments detects screen interaction. If we have interactive elements in Unity that we want to use, then all screen interaction is managed in Unity. But what if we want to use the interactive features on Flutter and have them simultaneously react on Unity? In that case, the information for this screen interaction will have to be sent through the background communication path of both environments. 

Our team has not encountered any more limitations, but we will provide an update if more appear. 

What advice would we give when combining Unity with Flutter in a commercial project? 

First

Properly separating work on the application and then combining it with Unity can save developers a lot of time and hassle. Dividing the project into a section without Unity and a section that connects the two will allow you to maintain a usual work pace. After creating new functionalities that are ready to go, throw them into the section that already has Unity plugged in to go further to your users. 

Second

Creating a CI/CD from the beginning can also save a lot of problems in the future. Using the Unity environment takes more additional resources and requires more powerful hardware. If possible, work on building the final build of the project, including Unity, sensibly shifting to a more powerful external machine. 

Summary

The use of both technologies in a single project proved to be an effective solution in making the most of the team’s development capabilities, even on a part-time or task basis. The biggest challenges proved to be time and work management in anticipation of avoiding problems when building the application. But ultimately, we can happily conclude that Futter and Unity interact very well with each other. 

Frequently Asked Questions (FAQs)

How do you integrate Unity with Flutter?

Unity can be integrated with Flutter using the Flutter Unity Widget. This widget allows developers to embed Unity scenes directly into a Flutter app, enabling seamless communication between the two environments. You can define commands that control the Unity scene from Flutter and vice versa, such as opening or closing a scene.

---

Can Flutter and Unity be used together for AR?

Yes, Flutter and Unity can be combined to create augmented reality (AR) applications. Unity’s robust AR capabilities can be embedded within a Flutter app, enabling you to utilize AR functionalities while maintaining the lightweight cross-platform benefits of Flutter.

---

What is the Flutter Unity Widget?

The Flutter Unity Widget is a library that facilitates the integration of Unity into Flutter applications. It allows developers to display Unity scenes within Flutter apps and control Unity from Flutter. The widget can be found on GitHub and offers documentation and example implementations, making it an essential tool for those looking to combine Flutter with Unity.

---

Is there a performance issue when embedding Unity in Flutter?

Performance can vary depending on how well resources are managed between Unity and Flutter. By ensuring that Unity scenes only load when necessary and freeing up resources when they aren’t needed, you can prevent most performance issues. Optimizing scene loading and resource management is key to maintaining smooth performance.

---

Can Unity WebGL be embedded in a Flutter app?

Yes, Unity WebGL can be embedded in Flutter applications, allowing Unity content to be rendered in a web environment within Flutter. This is particularly useful for developing web-based apps that require Unity’s advanced 3D rendering capabilities.

---

How can I get started with Flutter Unity integration?

To get started with Flutter Unity integration, you can begin by using the Flutter Unity Widget. Visit the GitHub repository for documentation, sample code, and step-by-step instructions on embedding Unity scenes in your Flutter app. Start small by embedding basic scenes and gradually expand to more complex integrations such as AR, WebGL, or 3D content.