Sync issues have been plaguing me for awhile and the last topic on this issue was closed only 8 hours after the last message, so no time to really do anything to update that topic even tho this post belongs there. (why such a short time).
The truth is filesystem sync was only meant to be a for testing and dev. I made it available to everybody, a bit reluctantly, at the demand of users but I've never liked it because I can't make it as reliable as I'd liked. Any reason you don't try an alternative sync method?
Prompted by this statement, I decided to give OneDrive a go, and encrypted all my notes to put them on there. However, my notes are STILL not synched, and I don't know how many times I've opened the app at this point, but let's just say it's been more than 5 and many times just to kickstart the sync again since it doesn't sync in the background.
Looking at sync status, I really don't have that many notes:
and I'm wondering what the problem is. really, it still seems unusable on android no matter which way I try to use it. I've asked before to support jex import on the android which would make the initial sync at least fast.
am I missing something still?
Could a potential fix that I export all my notes and reimport? I don't know if there's anything lingering that causes this slow import
in order to debug I'm experimenting after oneDrive doesn't seem to finish
steps to debug
perform a backup of my notes on my laptop
start up a local network joplin server instance
start up a dev instance on my laptop of joplin and load the backup of my notes
sync to the joplin server from my laptop (finished within 10 minutes)
wipe data on android and start afresh
sync from joplin server
observations so far...
it's been two hours and it's still not finished synching to android.
it looks busy on the android client, creating local items, fetching items, fetching resources, but it just doesn't finish
android app used to sync in a couple of minutes from the filesystem when that was working
I have less than 2,000 notes, and less than 2,000 resources, it shouldn't take that long given how fast it was to sync to the local server from the laptop
given that i don't see many people issues on android, I'm wondering if I'm the only one who has this problem, and why this is so
sync status
sync status on laptop:
Sync status (synced items / total items)
Note: 1896/1896
Folder: 93/93
Resource: 1580/1580
Tag: 99/99
NoteTag: 327/327
Revision: 0/5
Total: 3995/4000
Conflicted: 0
To delete: 0
sync status on android
Sync status (synced items / total items)
Note: 1901/1901
Folder: 94/94
Resource: 1583/1583
Tag: 99/99
NoteTag: 327/327
Revision: 0/0
Total: 4004/4004
Conflicted: 0
To delete: 0
The storage access on Android is slow (by design by Google for security reasons), so it's normal that synchronisation will take longer there in comparison to desktop operating systems. In addition, resources are fetched asynchronously, so you shouldn't be worried about the numbers moving even after the synchronisation itself has finished.
For kicks, I rolled back to Joplin Android version 2.4.3 the last known version for me that I knew worked quickly. Finished the complete sync in 724 seconds. I know this is before the forced updates by google, but I can't imagine google making a chance that would change sync to to take days to complete. (I put my screen timeout on 30 minutes and kept doing sync until I finally gave up)
It's been over a year now, and I sitll think something's off, but at least I know this is a workaround for now.
They actually did just that (see https://reddit.com/r/androiddev/comments/kpn68k/android_11_very_slow_file_access_performance). In the worst case scenario, file access can be ~200 times slower compared to the older versions. This applies to both the Android OS itself and apps which target older and newer versions of Android. However, Google forces apps to target only new versions of Android as a part of their Play Store policy.
This is about Android 11 and later. I'm sorry but I'm not going to spend more time on explaining this, but if you're interested, you can just look into FUSE filesystem on Android, SDCardFS deprecation, and their negative impact on storage I/O performance.
look, i'm not trying to argue with you here, but I'm trying to understand the issue.
I want to call out that 2.4.3 on my phone syncs the notes within 10 minutes. The current version of Joplin, I can't get it to sync because it takes so long I give up. Let's just say I've tried for weeks.
You mention FUSE, but that can't possibly be the issue, because then 2.4.3 wouldn't work fast either since FUSE operates at a lower level than SAF.
Regarding SDCardFS, I don't even have an SD card, so this is also not relevant. And same thing here, if SDCardFS was an issue, it would not explain how rolling back to 2.4.3 on the same phone would make sync work quickly.
you mention ~200 times but where do you get this number? from the reddit link? He mentions a 100 times increase, but again, this is ONE dude on reddit. This hardly explains the problem but merely provides 4 year old anecdotal evidence of an issue he had.
Yes, the older version of Joplin targets an older version of Android. Because of that, it's not limited by the current storage restrictions.
I'd guess you could try compiling the current version of Joplin with a lower Android OS target and still be able to achieve the same performance, however this isn't going to happen with the official app, because Google Play Store doesn't allow updating apps that target older Android versions.
That's why I suggested to look up SDCardFS. Despite the name, it has nothing to do with an SD card. It's the filesystem which was used on Android for the internal memory before Android 11. It allowed for direct filesystem access, hence it was much more performant than what we have today with FUSE.
I'm not sure what you mean by this. Targets an older version of Android? My phone is running Android 15, and only Android 15. The difference between the versions is API, not underlying filesystems.
I looked it up, and it said it was a replacement (at least originally) for FUSE to access SD cards, which suggests it is related to SD. What I got was that the original FUSE implementation in Android 7 introduced performance and deadlock issues. To address these, Android 8 introduced SDCardFS, which provided better performance. However, with the improved FUSE implementation in Android 11, the performance issues were resolved, making SDCardFS redundant.
if you were telling me that current API's are forced to use SAF i(or something like it) n order to be accepted into google play, and SAF makes it less performance, that would at least make sense. But FUSE is as I mentioned at a lower layer
It's not only about your version of Android, it's about which level of SDK the application targets (see https://developer.android.com/google/play/requirements/target-sdk). If the app targeted an older version of Android (e.g. before 11), then it would likely be able to access the storage with no performance penalty (like the old version of Joplin does).
Hobbyist.
I think this article explains the situation quite well:
Also the same 200x slower access as reported in the Reddit thread:
Just to avoid any confusion, the /sdcard partition on Android is the internal storage, not an SD card.
You understand that the OS has to use FUSE wether Joplin app is version 2.4.3 or 3.1.6, right? IOW, it still isn't FUSE. Also, given that the FS is ext4 and there's nothing special that needs to be mounted, it shouldn't be using FUSE at all.
I Found an older thread where Laurent says it's due to an update filesystem API so it could be the issue.
I'm wondering if this could be revisited at this point to figure out limitations. I know there are some limitations on FS where there are too many files in a folder, and I see some apps breaking this into subfolders to mitigate the issue (based on first few characters in the UUID string for instance).
