Flutter under the hood

Today in this post I will explain why after many years developing with native technologies, Flutter for me is an unprecedented technological bet. But for that, we will have to go into the guts of this technology explaining why I consider that this technology can really compete with a native technology.

Dart

Many people mistakenly consider Flutter as a programming language but the programming language is called Dart. Flutter is the SDK for Dart to write applications that can be run on any screen.

To get this far, Dart has evolved a lot since its origins back in October 2011. The arrival of Dart 3 has been a journey of constant change and improvement.

Dart’s compiler technology lets you run code in different ways:

• Native platform: For apps targeting mobile and desktop devices, Dart includes both a Dart VM with just-in-time (JIT) compilation and an ahead-of-time (AOT) compiler for producing machine code.

• Web platform: For apps targeting the web, Dart can compile for development or production purposes. Its web compiler translates Dart into JavaScript or Web Assembly.

Two of the most interesting tools available in Flutter for development are known as Hot Reload and Hot Restart. These features are inherited from the Dart programming language for reasons that will be discussed later.

Hot Reload

Hot Reload will allow us to reload the hot view without having to recompile our project, reducing the time needed to see the result of a visual change.

Hot Restart

Hot restart will allow us to restore the state of the application to the initial state when it was first compiled without the need to recompile.

Dart Native (machine code JIT and AOT)

Dart offers two types of execution and compilation because the needs during development are not the same as in production. Hence JIT (Just-In-Time) and AOT (Ahead-of-Time) were born. 

During development, a fast developer cycle is critical for iteration. The Dart VM offers a just-in-time compiler (JIT) with incremental recompilation (enabling hot reload), live metrics collections (powering DevTools), and rich debugging support.

When apps are ready to be deployed to production—whether you’re publishing to an app store or deploying to a production backend—the Dart ahead-of-time (AOT) compiler can compile to native ARM or x64 machine code using dart compile tool. Your AOT-compiled app launches with consistent, short startup time.

JIT (Just-In-Time) and AOT (Ahead-of-Time) compilation are two different approaches to compiling code in Dart.

In JIT mode, the code is compiled at runtime, when the application is executed. The advantage of JIT mode is that it allows for faster development and testing, as you can make changes to the code and see the results immediately.

In AOT mode, the code is compiled ahead of time, before the application is executed. The advantage of AOT mode is that the compiled code runs faster than JIT-compiled code, as it does not require the additional overhead of compiling the code at runtime. This makes AOT mode ideal for deploying your application to production, as it provides a smooth, fast, and reliable user experience.

To perform the compilation Dart relies on the dart compile tool.

If you notice so far we haven't talked about how to graphically represent an element in the UI of an application in Dart. 

This is because we have so far dealt with all aspects at the language level. Different types of compilation, how to get our code efficiently to any architecture, debugging tools like hot reload / hot restart, but nothing related to the user interface. We left that to Flutter.

Flutter Architectural layer

Flutter is an open source framework for Dart developed by Google to create beautiful, cross-platform, natively compiled applications from a single code base. We have already seen how it manages to compile thanks to the Dart language. Now it's time to see how the SDK offers a set of tools to create the User Interface (UI).

Flutter is designed as an extensible, layered system. It exists as a series of independent libraries that each depend on the underlying layer. No layer has privileged access to the layer below, and every part of the framework level is designed to be optional and replaceable.

As we have seen, Flutter is composed of an independent set of 3 layered libraries. Let's take a closer look at them.

Framework Dart

The name of this library is given because the layer set of this part is written in Dart. Typically, developers interact with Flutter through the Flutter framework, which provides a modern, reactive framework written in the Dart language. It includes a rich set of platform, layout, and foundational libraries, composed of a series of layers. 

Material and Cupertino

The reason why in the top layer of the architecture we see two frameworks in the same layer is because Flutter proposes to use the same UI for all platforms. The Material and Cupertino libraries offer comprehensive sets of controls that use the widget layer’s composition primitives to implement the Material or iOS design languages.

• Material: With this package you can create visual, behavioral, and motion-rich widgets implementing the Material 3 design specification. 
• Cupertino: If for some reason we want to use native Apple (iOS-style) components, with this package we can create beautiful and high-fidelity widgets for current iOS design language.

Widgets Layer

The widgets layer is a composition abstraction. Each render object in the rendering layer has a corresponding class in the widgets layer. In addition, the widgets layer allows you to define combinations of classes that you can reuse. This is the layer at which the reactive programming model is introduced.

Flutter widgets are built using a modern framework that takes inspiration from React. The central idea is that you build your UI out of widgets. Widgets describe what their view should look like given their current configuration and state. When a widget’s state changes, the widget rebuilds its description, which the framework diffs against the previous description in order to determine the minimal changes needed in the underlying render tree to transition from one state to the next.

Rendering Layer

The rendering layer provides an abstraction for dealing with layout. With this layer, you can build a tree of renderable objects. You can manipulate these objects dynamically, with the tree automatically updating the layout to reflect your changes. 

That is, so as not to have to work directly with the rendering engine at this level. What it does is to compose our UI as an abstraction in the form of an optimised tree. So that when the state of the application changes our UI Flutter doesn't have to reload the whole view unnecessarily but only the part that it needs.

It should be noted that although its name may cause confusion, this layer is not rendered as such. Rather, it is organised as an abstraction and optimisation of what will be rendered layers below.

Fundation Layer with Animation, Painting and Gestures

Basic foundation classes and building block services such as animation, painting and gestures offer commonly used abstractions on the underlying foundation. In other words, foundation serves as an output to the engine but instead of working directly with the foundation classes, it is abstracted into 3 smaller packages which are animation, painting and gestures.

Engine C/C++

At the core of Flutter is the Flutter engine, which is mostly written in C++ and supports the primitives necessary to support all Flutter applications. The engine is responsible for rasterizing composited scenes whenever a new frame needs to be painted. It provides the low-level implementation of Flutter’s core API, including graphics (through Impeller on iOS and coming to Android, and Skia on other platforms) text layout, file and network I/O, accessibility support, plugin architecture, and a Dart runtime and compile toolchain.

The engine is exposed to the Flutter framework through dart:ui, which wraps the underlying C++ code in Dart classes. This library exposes the lowest-level primitives, such as classes for driving input, graphics, and text rendering subsystems.

In other words. What the engine does is to create a blank canvas using Skia/Impeller and paint pixel by pixel the tree of widgets that the previous Dart framework abstraction sent it. 

We can summarise the way it works as follows. When a widget needs to be updated, what it does is to propagate a signal upwards (its parent), if this also needs to change it will do the same to its parent, so on and so forth until someone says.... I should not change. Then the rendering engine thanks to the abstraction and optimization that was created will repaint ONLY that part of the widget tree. The optimised widget abstraction itself will prevent those widgets that have no states (StatelessWidget) from being updated, in those cases the rubbish collector will delete them and they will be created from scratch, as this will be faster. However, stateful widgets (StatefulWidget) will be updated. This is why it is very important that both the engine and the widget tree are optimised if we want to compete with native runtimes.

One of the reasons why I say that Flutter is highly optimised, apart from the fact that it has done an excellent job with the state tree abstraction, is because the Skia/Impeller rendering engine uses for each platform the native paint package it is most interested in. For example in Apple it uses Metal, in Android OpenGL, in Windows DirectX, etc... 

Skia 

Skia is the default Flutter rendering engine for Android and was the default engine for iOS until recently, up until version 3.10. Skia is a 2D graphics library used by Flutter for rendering the UI. It supports shaders, which are applied to different objects, shapes, or text. These shaders are compiled and executed on the GPU at runtime. The cause of the aforementioned problem lies precisely in this last sentence. 

The generation and compilation of shaders happen sequentially based on demand, and this process can take several hundred milliseconds, which is much longer than the 16 milliseconds required to render a single frame at 60 fps. As a result, the compilation process can cause significant delays and a drop in frame rate from 60 fps to 6 fps, known as "compilation jank." 

This problem has been present within the Flutter framework for quite some time, despite a considerable number of related issues on the GitHub repository. However, somehow this problem has been swept under the rug by the decision-makers who determine the priority of issues regarding the Flutter framework. That was the case until recently, specifically at the beginning of May last year (2022), when it was mentioned that there exists an experimental branch dedicated to a rewrite of their graphics backend. It was stated that it would take many months before that project becomes stable enough for production use. Luckily for us developers and all others who use our applications, that ambitious but much-needed project has reached its conclusion and has taken shape. This project or new Flutter rendering engine is called Impeller.

Impeller 

Impeller is a new Flutter rendering engine that the Flutter team claims solves the early-onset jank problem. It is designed as a replacement for Skia, with the goal of enabling better animations and addressing the "jank" issue, while also potentially providing support for 3D, which was not previously possible with Skia, as it exclusively supports 2D. Furthermore, it's important to note that Impeller is designed to fully harness the advantages of modern accelerated graphics APIs like Metal and Vulkan, maximizing their capabilities. What sets it apart from Impeller and essentially solves the mentioned problem is that Impeller performs the majority of shader compilation ahead of time. 

Unlike Skia, Impeller compiles shaders during the build process instead of at runtime. During the build process, the Impeller shader compiler (impellerc) is used to compile GLSL (OpenGL Shading Language) code into SPIRV. GLSL is a high-level shading language utilized for programming shaders in OpenGL and OpenGL ES. On the other hand, SPIRV is an efficient representation for shader code that is not limited to OpenGL and OpenGL ES but can be used with other graphics technologies as well. impellerc performs the conversion of GLSL code into SPIRV to enable cross-platform compatibility and efficient execution of shaders. The behavior of shader compilation is controlled by passing flags to impellerc. The output of the compilation is a binary blob in the form of a SPIRV module, which can be loaded and executed by a graphics driver or runtime. As a result, these SPIRV modules (binary blobs) can be executed on various platforms.

This significantly reduces the initial app startup time and eliminates shader compilation jank. Additionally, it is important to note that the Flutter team states that this approach reduces the APK/IPA size, which will be another significant benefit of this engine. 

As far as it is known, Impeller will have support for iOS, Android, Desktop, and Embedder API users, which primarily indicates support for all platforms targeted by the C++ engine. This suggests that Impeller is not intended to support the web platform. At least for now.

Embedder Platform-specific

To the underlying operating system, Flutter applications are packaged in the same way as any other native application. A platform-specific embedder provides an entrypoint; coordinates with the underlying operating system for access to services like rendering surfaces, accessibility, and input; and manages the message event loop. The embedder is written in a language that is appropriate for the platform: currently Java and C++ for Android, Objective-C/Objective-C++ for iOS and macOS, and C++ for Windows and Linux. Using the embedder, Flutter code can be integrated into an existing application as a module, or the code may be the entire content of the application. Flutter includes a number of embedders for common target platforms, but other embedders also exist.

Anatomy of a Flutter app

The following diagram gives an overview of the pieces that make up a regular Flutter app generated by flutter create command. It shows where the Flutter Engine sits in this stack, highlights API boundaries, and identifies the repositories where the individual pieces live. 

Dart App

• Composes widgets into the desired UI.
• Implements business logic.
• Owned by app developer.

Framework (source code)

• Provides higher-level API to build high-quality apps (for example, widgets, hit-testing, gesture detection, accessibility, text input).
• Composites the app’s widget tree into a scene.

Engine (source code)

• Responsible for rasterizing composited scenes.
• Provides low-level implementation of Flutter’s core APIs (for example, graphics, text layout, Dart runtime).
• Exposes its functionality to the framework using the dart:ui API.
• Integrates with a specific platform using the Engine’s Embedder API.

Embedder (source code)

• Coordinates with the underlying operating system for access to services like rendering surfaces, accessibility, and input.
• Manages the event loop.
• Exposes platform-specific API to integrate the Embedder into apps.

Runner

• Composes the pieces exposed by the platform-specific API of the Embedder into an app package runnable on the target platform.
• Part of app template generated by flutter create command, owned by app developer.

Creating a Flutter Project

If we create a Flutter project we can see its internal file structure. For this we can either use the flutter create command or use Android Studio and it will create it for us. In my case, I'm going to show it to you with Android Studio.

If we look at the file explorer we can see that we have one folder per platform selected. This is because Flutter creates a native project per platform.

• analysis_options.yaml file is used to configure the Flutter linter. This is something we have talked about before.
• pubspec.yaml file is used to indicate the dependencies of the project. A large community is behind the dependency manager (Pub dev) that brings us amazing packages.
• android, ios, linux, macos, web, windows folders are created for each project selected when creating our project. In them they are linked to our native code.
• lib folder we have all the .dart files common to all the projects.
• test folder we have a place to have all the unit tests in Dart.

If I had to say something against Flutter it would be the strong dependency on Objective-C in iOS. Although we mark Swift as language this will not prevent that the GeneratedPluginRegistrant, that is to say, the plugin that connects our iOS project to Flutter is written in Objective-C and imported through the Bridging-Header. 

In addition, we have CocoaPods. A shortcoming of using CocoaPods as a dependency manager compared to Swift Package Manager (SPM) is that the compilation of dependencies is sequential, which implies that they will be resolved sequentially one by one taking more time than if they are resolved in parallel as SPM does. We can observe it in the compilation times.

Conclusions

As you can see, there are many differences with any other cross-platform technology. As a mobility expert I have been interested to know React Native, Xamarin, .Net Maui or Kotlin Multiplatform but none of them is as optimized and advanced as this one. About this last one you can see an extensive review in this blog. That's why after many years developing in native today I spread and work daily with this technology. Besides, I still think that my knowledge in native is a very positive skill to develop in Flutter because as we saw, internally it has one native project per platform.