Rspack is a great technical topic and the future guest of the podcast. Let's have a quick overview here.
This time, we will be discussing the principles and practices of Rspack by ByteDance, presented by Yang Jian from ByteDance.
Yang Jian - Principles and Practices of Rspack by ByteDance
Official website: rspack.dev
Some of the content shared here is the original text and my personal feelings. I will indicate the original text and focus on my feelings.
Outline of the original text:
- why Rspack
- Ecosystem compatibility
- Out-of-the-box capabilities
- Business performance benefits
- Architecture design
- Migration guide
webpack is not good enough
There are many large-scale applications within the company, and the project startup, cold startup, and build process are slow.
The optimization methods of traditional webpack are outdated, such as babel -> swc-loader, esbuild/vite, etc.
These solutions have their advantages, but they cannot solve the problem completely. For example, swc-loader can speed up the loader, but it is not comprehensive.
Vite is fast in development, but its rollup build is not faster than webpack.
Esbuild is fast in production optimization, but it does not support dev-hmr, and its plugin capabilities for packaging are weaker, unable to meet the requirements of code splitting.
This answers the question of which is better between vite and rspack, but there is no definitive answer. It depends on the scenario and extreme performance. For example, development experience and server-side rendering (SSR), but there are differences in the build process for heavy projects.
However, rspack itself is more low-level, so the migration from vue-cli may be more troublesome.
What should a good build tool be like
It should meet the developers' demands:
- Good performance, fast cold startup, fast hot module replacement (HRM), fast build process even after multiple builds.
- Flexibility, flexible configuration to adapt to different usage scenarios. ToC (To Customer) build product size and ToB (To Business) scalability for large-scale applications.
- Performance of production artifacts: code splitting, tree shaking, first screen rendering, etc.
- Application scenarios: web app, Node.js, cross-platform.
- Low migration cost, no resistance from the business side.
From this perspective, webpack is the best, and tools like swc/esbuild/vite still have a long way to go for traditional projects and cross-platform projects.
Here comes rspack!
Rspack is about to be released, and it has many advantages:
- Implemented in Rust, with strong native capabilities.
- Incremental compilation, fast HRM.
- Compatible with the webpack ecosystem.
- Out-of-the-box support for ts/tsx/css/css modules, no need to search for plugins.
- Default optimization strategies, such as tree shaking, code splitting, code compression, etc.
Good compatibility, some examples are mentioned. We can see familiar plugins.
Also, no need to tinker with ts anymore, the default is fast and good. No need for babel-loader/ts-loader, it used to be a nightmare.
Dealing with CSS used to be troublesome, such as extracting CSS, using css/style loaders, etc. Now it is also built-in, fast and good. However, configuration is still needed for less/sass. There is a comparison screenshot, rspack is simpler, mainly because the output of less-loader is directly set as CSS and CSS modules for built-in processing.
After discussing compatibility and performance benefits, three indicators are mentioned: cold startup in development, HRM, and production build.
A graph is provided, but I won't include it here. It shows that rspack performs very well, especially in production builds compared to rollup.
After discussing the benchmark scores, let's talk about the actual implementation. The internal projects have seen significant improvements, with a 10x improvement compared to webpack.
For example, one HRM in webpack takes 20 seconds, which is unimaginable... but after optimization, it only takes 1 second.
Now we can answer some common questions:
Why not use Node.js?
- Webpack has been optimizing its single-threaded performance for many years, but there are limitations.
- Node.js itself is single-threaded and cannot support multi-threading.
- Using worker-thread to support multi-threading involves creating new V8 instances to simulate single-threading. The process of communication requires serialization and deserialization, which is not efficient for data sharing.
- Lack of concurrency ecosystem. Due to the limitations of V8's multi-threading support and high performance consumption, there is a lack of concurrency programming ecosystem.
Why not use Golang - esbuild?
- Golang has good performance.
- It does not have good support for NAPI, so it cannot meet the requirements.
- esbuild does not provide APIs for manipulating AST (Abstract Syntax Tree) and does not support downgrading to ES5.
- Good performance.
- Good support for NAPI, Rust macros, less boilerplate code.
- Rust has good support for WebAssembly, and it even pushes for more aggressive and advanced features.
- Rust's swc has rich support for manipulating AST and also supports downgrading to ES5.
Is swc enough?
Large companies have showcased their strength.
- It was found that swc has poor concurrent parsing performance. Profiling in the development environment revealed that parsing was the bottleneck, and further investigation found bugs in the underlying libraries.
How does it achieve speed?
- Architecture based on Webpack 5.
- Embracing Rust.
- Replacing Babel with swc, as loaders are slow. Many people use swc/esbuild to speed up the build process.
An architecture diagram is provided, which I have not seen before, but it is said to be similar to the architecture diagram of webpack. (I have not experienced this myself, I have only heard it twice)
The build process involves:
- Dependency graph generation in the make parse phase.
- Chunk generation in the seal phase.
- Optimization analysis, analyzing module usage and side effects.
- Chunk graph creation, including module graph and splitting.
- ID generation for modules and chunks.
- Module code generation.
- Runtime module.
- Chunk resource generation.
- Resource generation.
Package caching strategy, parallel loading of split packages.
User configuration for polyfilling ES5, ES6, ESNext, etc.
This part is quite exciting, it's the first time I've heard about it.
The yellow parts can all be processed in parallel, and Rust is faster. Webpack cannot handle multi-threading well, and even with thread-loader, communication is difficult. However, rspack has addressed both of these shortcomings.
The core principle is to parallelize wherever possible. With multi-core processors, it can achieve a 1.7x improvement, although there is still some overhead. But compared to webpack, the improvement is limited, such as a 1.1x-1.2x improvement.
Migration of Legacy Projects
- Utilize built-in features, such as js/ts/css/file-loader/html-plugin, etc.
- Third-party plugins are not supported yet, but they can be followed up.
Roadmap for the future:
- Compatibility with the webpack ecosystem.
- Support for more frameworks.
- Support for persistent cache and lazy compilation.
- More production optimizations.
- Support for module federation.
After listening to this outlook, I have gained a deeper understanding of webpack's architecture and identified areas for optimization. This is what rspack aims to achieve.
I have great respect for the extensive experience shared, although I still don't understand some details and need to learn more.