Sync is blocked while Joplin on Android slowly deletes remote items

mrjo118 · 26 May 2025 06:29

From my observations in the code, disabling note history means that client will neither create any new note history, nor will it clean up existing note history.

If you enable note history on other clients with a given amount of expiry days, that client will still create new note history and cleanup after the set amount of days, regardless of what other clients have set.

So if you combine 90 days expiry on desktop clients and disable note history on mobile clients, it will mean that the cleanup will not happen on mobile, however any cleanup that happened on desktop which has been synced will still need to be applied to your mobile client(s) whenever new cleanup has happened.

It's worth giving it a go right now to see if disabling note history on your mobile makes the deletions settle down, but if the deletions are actually due to your mass import and mass deletions of all your notes, then you would need to complete the synchronisation for that on your mobile regardless of the note history settings.

mrjo118 · 26 May 2025 06:32

In my case I have under 500 items and I have note history set to 7 days to keep it low

tomasz86 · 26 May 2025 07:38

Yeah, I've got too profiles - one has 1,500 and the other 3,000 files, and syncing them is slow but still manageable. In the past, I had 7,000 files in a single profile, and syncing that was unbearably slow . In fact, the slow sync was the reason why I split my notes into multiple profiles in the first place.

mrjo118 · 26 May 2025 22:40

I'm not denying the file access speed is the bottlenock with syncing on your device, but I can't quite understand from all the material I've read why in the case of Joplin, there should be such slowness with file access on newer versions of Android.

Yes there are speed limitations to access files using SAF / FUSE on Android 11 and above, but I have not found any indication that making use of the slow file access api's are necessary for reading / writing files in an app's own data directory, which is the case for Joplin (except if you are using file system sync).

This post on the syncthing-android github suggests that either accessing files in the app's own directory or making use of the MediaProvider api (and therefore making use of the MANAGE_EXTERNAL_STORAGE permission) will bypass any such speed limitations:

github.com/Catfriend1/syncthing-android

Explanation for slow sync on Android w/lots of files + request for fix

opened 01:52AM - 13 Feb 24 UTC

closed 10:42PM - 18 Feb 24 UTC

taoeffect

### Description of the issue Hi, SyncThing is perfect for syncing large amoun…ts of small files that don't change, e.g. photo albums and music libraries. However, on Android, it's a well known issue that the performance is almost prohibitively bad for this use-case. Example threads: - https://forum.syncthing.net/t/sync-becomes-very-slow-between-mac-and-android-device/21614 - https://forum.syncthing.net/t/syncing-to-android-is-incredibly-slow/20752 I reached out to the GrapheneOS developers to see if they knew why this might be happening. Here is what they had to say (reposted with their permission), I hope it is helpful to addressing this issue. Although their response might be taken as a bit abrasive, I ask that you please forgive them and look past it, in case what they have to say is useful for addressing this problem that users (like me and others) are facing: > It's almost certainly primarily caused by how the app is choosing to do it rather than an inherent issue with Android. Accessing files in shared storage is certainly slower than native filesystem access in the app's storage. That's why it would be best for an app like this to sync to their own storage and implement a content provider for accessing it instead of putting it directly into the user's shared home directory. It's likely the home directory performance will improve though. > > The app accessing their own files is native f2fs filesystem access which has the usual performance of f2fs with SELinux. SELinux adds a bit of overhead, but not an enormous amount. There are multiple ways to access shared storage, some performing better than other methods, but it's not direct filesystem access. An app using Storage Access Framework to access a huge number of files is certainly going to be a lot slower, while lower level access to shared storage isn't high overhead. > > They're blaming Android but it's how they designed their app. They chose to do it this way. They could have synced the data to their app storage by default and provided a storage provider for browsing it in the file manager. If they did that, it would appear as an entry in the file manager's list of storage providers where the top level home directory appears. > > Android focused on trying to get apps to stop misusing the home directory and demanding bulk storage accessed. > > The workarounds to provide backwards compatibility with apps depending on bulk storage access led to access to the home directory being slower than before. This app appears to have simply reworked their code to continue functioning with this instead of considering redesigning it to better fit into how Android intends for storage to be used. They could have native filesystem performance if they used the native filesystem access to their own app sandbox directory they have by default. > > They just need to start saving files to their own directory and expose their own directory as a storage provider so users can access it from the system file manager / system file picker. > > This app is primarily meant to sync a specific directory across computers, right? They should be using a directory in their app sandbox where they have native filesystem access and exposing that to users via a storage providers. > > They're just not doing this in a good way for Android. > > It is true that accessing the user's home folder is slower than native file access, but it shouldn't be as slow as their app, so something seems wrong with their implementation. It's also their choice to take that approach instead of the high performance approach. They ended up doing that since they were always doing it that way and the performance gap has always been there but used to be smaller. Security/privacy features slowed down home directory, but not their sandboxed directory. > > They're blaming Android but it's their choice to put files in a location that has always been slower to access and the gap in performance got much wider. Android wanted developers to stop using the home directory instead of using their sandboxed storage and exposing files to users and other apps via content/storage providers. The performance hit to home directory was a consequence of adding abstraction layers to provide fine grained access control with backwards compatibility. > > The reason we were able to implement our Storage Scopes feature in GrapheneOS is because Android abstracted home directory access in a way that enabled us to build that feature on top of what they provide. It was a good change. The problem is that app developers don't know the platform well and those that do don't want to rewrite significant parts of their app to work the way it always should have been done. They're paying for doing things in an invasive way to begin with.

The latter solution is unlikely to be an option for Joplin because it would require very strict criteria to be allowed to be published on the play store if it included that permission. However, I can use an alternative app such as ZArchiver (which uses the MANAGE_EXTERNAL_STORAGE permission, aka 'Access all files') on my Android 14 phone, and it will instantly list a directory of 3000+ photos in my camera directory. So that proves it is possible to list a directory with many files in via a third party app, without it being really slow.

So I suspect that as the GrapheneOS developer suggests in the linked post, the problem is not an inherent problem of the Android OS, but rather the file access is not implemented properly to be able to bypass SAF / FUSE within the Joplin app. As Joplin uses react native to manage access to the file system, I suspect the issue lies in a third party library being used. I'm guessing that would be the react-native-fs library.

mrjo118 · 28 May 2025 23:07

@tomasz86 I've had a look at the source for react-native-fs and it does look to be using 'app-specific files' type of file access on internal storage (using getFilesDir()) as defined in the Android developer documentation here:

In terms of how the Joplin code uses react-native-fs, it uses it in this file, using the rnfs module:

github.com/laurent22/joplin

packages/app-mobile/utils/fs-driver/fs-driver-rn.ts

a755f0903

import FsDriverBase, { ReadDirStatsOptions } from '@joplin/lib/fs-driver-base';
const RNFetchBlob = require('rn-fetch-blob').default;
import * as RNFS from 'react-native-fs';
import RNSAF, { DocumentFileDetail, openDocumentTree } from '@joplin/react-native-saf-x';
import { Platform } from 'react-native';
import tarCreate from './tarCreate';
import tarExtract from './tarExtract';
import JoplinError from '@joplin/lib/JoplinError';
const md5 = require('md5');
import { resolve } from 'path';


const ANDROID_URI_PREFIX = 'content://';

function isScopedUri(path: string) {
	return path.includes(ANDROID_URI_PREFIX);
}

// Encodings supported by rn-fetch-blob, RNSAF, and
// RNFS.

This file has been truncated. show original

This is used in combination with the react-native-saf-x library in that file also, using the rnsaf module. So actually the Joplin code clearly distinguishes whether or not to use the storage access framework, depending on the path that the file is located. From what I can tell, if you're not using file system sync, outside of importing / exporting data, only the rnfs module would be used by Joplin, and there should not be any poor performance implications with that on Android 11+. I wonder if the issue might be to do with how the file data is transferred between the javascript and java code through using react native, and if newer versions of Android might affect how that performs for some other reason. As I still can't find evidence to suggest the problem is caused by a limitation in Android. In the case of Syncthing, access to files in a variety of directories is required, so it's not surprising it could face issues due to needing to use SAF

EDIT: This bug report also suggests there should be no performance issues when accessing data within the private application folder:

mrjo118 · 2 June 2025 22:51

@jpservices I've reviewed the relevant code again today and actually I remembered incorrectly. Disabling note history will prevent new history being created on that client, but it wont stop the note history cleanup. I didn't realise when you disable note history that the note history expiry setting is still visible, and that still gets used for the the note history cleanup when history is disabled.

For now, if you are following my suggested workaround, you could set the expiry days to the maximum 730 days on your mobile clients. But I've created this PR Desktop, Mobile: Extend the maximum note history expiry days to 999999 by mrjo118 · Pull Request #12374 · laurent22/joplin · GitHub , which if accepted will allow setting the note history expiry days to 999999 days on mobile as well

system · 2 July 2025 22:51

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Adding android to linux/nextcloud set up, deletes notes Support	3	465	24 July 2021
Discussions about Sync Lounge	29	1693	25 June 2020
Unsyncable/Phantom Notebooks Between Linux and Android Devices Support	16	714	31 December 2023
Very disappointed on Joplin and its sync problems Lounge	20	589	5 May 2025
Days to sync on android (onedrive/joplin server test) Support	15	150	9 December 2024

Sync is blocked while Joplin on Android slowly deletes remote items

Related topics