Extracting appimage to improve io/cpu startup performance

Lounge
1

Since it's probably not good enough to share as a pull request, I thought to share my attempt at adding extract flag in Joplin update/installation file. I would like to gauge the interest to this feature from the community

Why

It reduces (by several times) cpu and io load during startup times

Usage

  1. Download the script
  2. run it from terminal ./Joplin_install_and_update.sh

Disclaimer

This script will update your current installation just like original install/update script. If it doesn't work and for some reason you're left with broken installation, run regular install/update file

3 Likes
2

If it improves the startup time, is there any reason not to make this flag the default?

1 Like
3

You're right, since there's no immediate disadvantage to extract appimage (at least I couldn't find one), it would be better as a default.

@james-carroll, maybe you have any opinion on disadvantages of extracted appimage relevant to Joplin?

My main concern is thorough testing since we no longer treat joplin.appimage as binary (binary is ~/.joplin/squashfs-root/@joplinapp-desktop). I, by myself, know too little what distros, DEs and Joplin features this change might break.

In any case, I'll rewrite extracted version to be default with optional --nonextract flag and carry on testing / addressing to-do's in the file.

4

Aside from the bit where 1 file is easier to manage than 1 folder (albeit not by much, assuming it truly is one folder); my main concern would be is there a difference in environment variables between running directly and running via the AppImage stub? Stuff like $LD_LIBRARY_PATH would be concerning.

Essentially the AppImage does the exact same thing the Snap does, decompress an archive and manipulate it (arguably I think AppImage might be a little worse, snap mounts in place, whereas AppImage effectively extracts and runs from /tmp, meaning the snap defers some of the runtime penalty but the AppImage has it all at once).

But as the "snaps being slow" thing goes, the main complaints at the time were from using XZ compression. (Good compression ratio, bad CPU). This got swapped to LZO for compatibility reasons, worse ratio but much better speed making people happy. The next phase would be to use ZSTD (now more kernels support it natively), for better ratio and speed (worse than XZ but better than LZO).

So no, I don't see a problem with the extraction that's immediately obvious, but I wonder if it could be worth an AppImage pre-release using ZSTD compression over LZO to gain some gains without extra complexity?

3 Likes
5

Yes, dynamic library resolution could be a problem, just like here: Outdated libnotify.so.4 in Appimage (2.14.17) masks the correct system library (although in this particular case it might have been a solution if the system library is resolved first :slight_smile:

1 Like
6

This is what is inside the AppRun shell script (of 2.14.20 AppImage ):

export PATH="${APPDIR}:${APPDIR}/usr/sbin:${PATH}"
export XDG_DATA_DIRS="./share/:/usr/share/gnome:/usr/local/share/:/usr/share/:${XDG_DATA_DIRS}"
export LD_LIBRARY_PATH="${APPDIR}/usr/lib:${LD_LIBRARY_PATH}"
export XDG_DATA_DIRS="${APPDIR}"/usr/share/:"${XDG_DATA_DIRS}":/usr/share/gnome/:/usr/local/share/:/usr/share/
export GSETTINGS_SCHEMA_DIR="${APPDIR}/usr/share/glib-2.0/schemas:${GSETTINGS_SCHEMA_DIR}"

BIN="$APPDIR/@joplinapp-desktop"

But since the install script bypasses it (why go directly to @joplinapp-desktop ?) the system LD_LIBRARY_PATH will come first.

The GNOME settings schema folder will also be bypassed (nothing in the in the fresh install folder actually, but if written to, it will go the the systemwide I guess )

1 Like
7

Thank you very much for responses, @james-carroll and @QuantiusBenignus. I can see that before settling on extraction path we need to try to move to zstd compression first.

At this time, I will drop my extraction approach and work on changing the compression.

I moved extraction gist to a git branch to ease the inspection of diff and updated it to make it a bit safer for those who might be interested to use it. First post with link is updated.

4 Likes