Not just smaller, but faster to launch too, thanks @personalizedrefriger
This should hold up, in relative terms, for all the desktop platforms.
dev ← personalizedrefrigerator:pr/desktop/bundle-the-app
opened 08:57PM - 30 May 25 UTC
# Summary
This pull request bundles the desktop app with [esbuild](https://es… build.github.io/). On Linux, this:
- Reduces startup time by roughly 46% (see test steps below).
- Reduces the `.AppImage` size from 219 MiB to 162 MiB.
This change was also observed to improve performance on Windows 11:
- Reduces the portable app startup time by roughly 44%.
- Reduces the size of the portable application's .exe file from roughly 319 MiB to about 239 MiB.
> [!NOTE]
>
> This pull request selects ESBuild over Webpack for two reasons:
> - I'm more familiar with the APIs for customizing imports and postprocessing files — `js-draw` uses esbuild.
> - [esbuild should be much faster than Webpack](https://esbuild.github.io/).
# Notes
- Bundling the application means that most `require`s and imports are resolved at compile time. This allows moving many dependencies from `"dependencies"` to `"devDependencies"`, significantly reducing the application size. However, this means that `shim.requireDynamic` 1) usually cannot require relative paths and 2) can only require packages present in `"dependencies"`.
- Changes were made to remove several `shim.requireDynamic` calls.
# Testing
## Testing performance: Linux
I'm using the following script to compare the performance after this change with the performance of the 3.4.1 release:
<details><summary>Script</summary>
```bash
#!/bin/bash
# Commit with bundling
TO_COMMIT=61b6fe29aa23c3468cd1c931060ea90ecedc8281
# v3.4.1
FROM_COMMIT=24df67472623c4124a5262ee1631c4f205183683
# A diff that adds support for a --testing-start-perf flag.
# When the flag is provided, the app is closed just after the
# MainScreen component mounts (which happens after the main start
# logic runs).
START_PERF_DIFF=$(cat <<"EOF"
diff --git a/packages/app-desktop/gui/MainScreen.tsx b/packages/app-desktop/gui/MainScreen.tsx
index 2358def52..a0dbb4224 100644
--- a/packages/app-desktop/gui/MainScreen.tsx
+++ b/packages/app-desktop/gui/MainScreen.tsx
@@ -47,6 +47,17 @@ import PluginService from '@joplin/lib/services/plugins/PluginService';
const ipcRenderer = require('electron').ipcRenderer;
+let testingStartPerf: boolean|null = null;
+const closeIfTestingStartPerf = () => {
+ testingStartPerf ??= bridge().processArgv().includes('--testing-start-perf');
+
+ // When testing start performance, close the app after everything is finished loading.
+ if (testingStartPerf) {
+ // Wait until the next iteration of the event loop, then close
+ requestAnimationFrame(() => bridge().electronApp().quit());
+ }
+};
+
interface Props {
plugins: PluginStates;
pluginHtmlContents: PluginHtmlContents;
@@ -361,6 +372,7 @@ class MainScreenComponent extends React.Component<Props, State> {
public componentDidMount() {
window.addEventListener('keydown', this.layoutModeListenerKeyDown);
+ closeIfTestingStartPerf();
}
public componentWillUnmount() {
diff --git a/packages/lib/utils/processStartFlags.ts b/packages/lib/utils/processStartFlags.ts
index 3b6830c6f..3dd68bd6a 100644
--- a/packages/lib/utils/processStartFlags.ts
+++ b/packages/lib/utils/processStartFlags.ts
@@ -79,6 +79,12 @@ const processStartFlags = async (argv: string[], setDefaults = true) => {
continue;
}
+ if (arg === '--testing-start-perf') {
+ // Closes the app after startup completes
+ argv.splice(0, 1);
+ continue;
+ }
+
if (arg === '--update-geolocation-disabled') {
Note.updateGeolocationEnabled_ = false;
argv.splice(0, 1);
EOF
)
TEMP_DIR=/tmp/joplin-perf-test
mkdir "$TEMP_DIR"
cd ./joplin/
# Not actually CI, but this allows building the OneNote converter.
export IS_CONTINUOUS_INTEGRATION=1
function do_build() {
echo "Building..."
STATUS="$(git status --porcelain)"
if [ "$STATUS" != "" ] ; then
echo "ERROR: Found unstaged files $STATUS. Exiting."
exit 1
fi
# Do a full rebuild to help ensure consistency
git clean -dXf
git reset --hard HEAD
echo "Applying diff..."
echo "$START_PERF_DIFF" | git apply -
yarn
if [ "$?" != "0" ] ; then
echo "ERROR: yarn install failed"
exit 1
fi
cd packages/app-desktop/ && yarn dist
if [ "$?" != 0 ] ; then
echo "ERROR: yarn dist failed"
exit 1
fi
cd ../../
mv packages/app-desktop/dist/Joplin-*.AppImage "$TEMP_DIR/Joplin.AppImage"
# Clean any local changes
git reset --hard HEAD
}
# Building
if [ ! -f "$TEMP_DIR/Joplin.old.AppImage" ]; then
git checkout "$FROM_COMMIT" && do_build
mv "$TEMP_DIR/Joplin.AppImage" "$TEMP_DIR/Joplin.old.AppImage"
else
echo "Already built: Joplin.old.AppImage"
fi
if [ ! -f "$TEMP_DIR/Joplin.AppImage" ]; then
git checkout "$TO_COMMIT" && do_build
else
echo "Already built: Joplin.AppImage"
fi
# Performance testing
cd $TEMP_DIR
mkdir ./profile
PROFILE_DIR="$TEMP_DIR/profile"
echo "Setting up the profile..."
./Joplin.AppImage --testing-start-perf --profile $PROFILE_DIR
function timeStartup() {
command time -f "%e" \
--append "--output=./results$1" -- \
./Joplin$2.AppImage --testing-start-perf --profile $PROFILE_DIR
if [ "$?" != 0 ] ; then
echo "ERROR: Failed."
exit 1
fi
}
# Reset the result files
echo "" > ./results-old
echo "" > ./results-new
for i in $(seq 1 10) ; do
echo "Testing new..."
timeStartup -new
echo "Testing old..."
timeStartup -old .old
done
echo "OLD"
cat ./results-old
echo "NEW"
cat ./results-new
```
</details>
At a high level, the script:
1. Builds a copy of the app from 24df67472623c4124a5262ee1631c4f205183683.
2. Builds a copy of the app from 61b6fe29aa23c3468cd1c931060ea90ecedc8281.
3. Launches the app built in step 1 and times how long it takes for the main component to mount.
4. Launches the app built in step 2 and times how long it takes for the main component to mount.
5. Repeats steps 3-4 nine more times.
## Results
<details><summary>Results on Fedora 42 — OS "Power Mode" set to "Performance"</summary>
**Before**: Startup time in seconds for Joplin v3.4.1:
```
5.25
5.10
5.00
5.10
4.97
5.08
5.14
5.22
5.40
5.10
Average: 5.14
```
**After**: Startup time in seconds for Joplin built from this pull request:
```
2.90
2.76
2.67
2.62
2.72
2.64
2.71
2.84
2.99
2.96
Average: 2.78
```
**Percent change**: $$\frac{\text{final}-\text{initial}}{\text{initial}} \times 100% = \frac{2.78-5.14}{5.14} \times 100\%$$, which is a 45.9% decrease.
</details>
<details><summary>Results on Fedora 42 — OS "Power Mode" set to "Power Saver"</summary>
**Before**: Startup time in seconds for Joplin v3.4.1:
```
13.95
13.93
14.08
16.35
14.09
16.79
14.92
15.86
16.07
15.27
Average: 15.13
```
**After**: Startup time in seconds for Joplin built from this pull request:
```
8.03
7.85
7.55
7.89
7.95
7.26
7.24
10.02
7.75
7.83
Average: 7.94
```
**Percent change**: $$\frac{\text{final}-\text{initial}}{\text{initial}} \times 100% = \frac{7.94-15.13}{15.13} \times 100\%$$, which is a 47.5% decrease.
</details>
On average, the startup time decreased by `46.7%`.
**Application size change**: Comparing the before and after builds created by the above script,
- **Before (Joplin.old.AppImage)**: 219 MiB
- **After (Joplin.AppImage)**: 162 MiB
## Testing performance: Windows
Similar tests were done on Windows. Tests were done using [this script](https://github.com/user-attachments/files/20527523/windows-perf-test.sh.txt), run in `git-bash.exe`.
<details><summary>Results (Joplin portable on Windows 11): Roughly a 44% decrease in startup time</summary>
**Before**: Startup time in seconds for Joplin built from c9eb9af7417ea5b42c658693785070591592c3b9:
```
10.48
12.39
10.54
9.29
9.25
10.69
12.33
10.14
9.05
10.71
Average: 10.49
```
**After**: Startup time in seconds for Joplin built from this pull request:
```
5.64
6.74
6.42
5.54
4.73
5.89
6.89
6.26
5.26
5.58
Average: 5.90
```
**Percent change**: $$\frac{\text{final}-\text{initial}}{\text{initial}} \times 100% = \frac{5.90-10.49}{10.49} \times 100\%$$, which is roughly a 43.8% decrease.
</details>
**Size change**: Comparing versions of Joplin Portable built by the above script:
- **Before (`JoplinPortable.exe`)**: 319 MiB
- **After (`JoplinPortable.exe`)**: 239 MiB
## Regression testing
At present, this change mostly relies on automated tests. However, it has been manually verified that:
- On Fedora 42 (with an earlier commit) using the built .AppImage,
- It is still possible to import a OneNote notebook (using one of the test OneNote notebooks in the `tests/support` subdirectory of the CLI app).
- It is still possible to export a note to a single HTML file.
- On Windows 11 using the installed `.exe`:
- It's possible to import `app-cli/tests/enex_to_md/images_with_and_without_size.enex` using file > import > "ENEX - Evernote Export File (as Markdown)".
- It's possible to open the imported note in an external editor.
- Making a change from the external editor (VSCode) and saving updates the original note.
- Double-clicking the note opens it in a new window.
Planned manual testing to check for regressions in modified code:
- [x] Check exporting from the CLI app:
- [x] JEX
- ~~HTML~~ (not supported in the CLI app).
- [x] Check JEX export from the Android app.
This is not the case anymore. The size of upcoming v3.7 AppImage is nearly 500MB, meanwhile v3.6 was just 150MB.
Does anybody know why did it grow so much?
Worth raising it as an issue on GitHub, the Windows installer has increased in size too but the .zip files and Mac package haven't experienced the same, so it's likely just stuff being packed into the formats unexpectedly that isn't strictly needed rather than the app ballooning.
It seems to occur between 3.7.1 and 3.7.3
