Sync process slower on macOS than on mobile

tessus · 2019-09-17T18:13:43.213Z

I’ve noticed that the sync process on macOS takes much longer than on mobile.

Please note that all clients are in the same network. In the first test nothing is actually to sync. Everything is up-to-date.
When I click on Synchronize on the desktop app (macOS) it takes about 4 seconds until I get the Completed message.
On mobile (Android and iOS) it takes one second.

For the 2nd test I don’t have any absolute numbers, but when syncing changes 5-10 notes, one can see the counter go up on macOS rather slowly. On mobile the counter and the sync operations is done in a fraction of the time.

iirc the perf on desktop was much better at one point.

I believe I found the reason: The desktop app uses HTTP/1.1, while the mobile apps use HTTP/2.0.

laurent · 2019-09-17T20:03:31.773Z

tessus:

I believe I found the reason: The desktop app uses HTTP/1.1, while the mobile apps use HTTP/2.0.

Weird it would have this impact. For testing can we try to make the desktop app use http/2.0? (not even sure if that can be setup somewhere)

tessus · 2019-09-17T21:04:33.072Z

laurent:

For testing can we try to make the desktop app use http/2.0?

I don't know. That was my point. I believe that either an electron update or some other change must have switched from h2 to http/1.1.

laurent:

Weird it would have this impact.

You implemented multiple connections for faster sync. This has a much higher impact on h2 than on http/1.1. I actually don't even know, if there is an impact on http/1.1.

tessus · 2019-11-05T03:25:40.613Z

@laurent according to the user agent strings okhttp/3.12.1 is used on mobile and node-fetch/1.0 on desktop.

Why are we using 2 different clients to connect and retrieve the data? In either case, the okhttp is using h2, which is a lot faster than http/1.1.

Is there any chance you could make the desktop to use h2 as well? Or where is the code that invokes these 2 different user agents?

laurent · 2019-11-05T07:11:10.700Z

okhttp is a native lib that RN use on mobile. On desktop we use the nodefetch package, which can be upgraded if needed.

tessus · 2019-11-05T07:23:57.138Z

Unfortunately node-fetch is only http/1.1, upgrading it won’t help. There’s no support for h2 in node-fetch.

The question is why are we using a package and not the native lib that Electron uses? Or did you run into any issues with it?

laurent · 2019-11-05T07:46:33.233Z

It’s just the same code between the cli app (which was done first) and the Electron one. I didn’t look if there was a better lib for Electron.

tessus · 2019-11-05T08:16:55.245Z

Ah, ok, I understand. However, this also means that CLI and desktop are using http/1.1 which is way slower in establishing connections. H2 can also help with the multiple connections Joplin can use for syncing. We should try to find an h2 option. I was looking into the following project, which looks promising:

GitHub

grantila/fetch-h2

HTTP/1+2 Fetch API client for Node.js. Contribute to grantila/fetch-h2 development by creating an account on GitHub.

laurent · 2019-11-05T08:48:39.611Z

Might be worth a try. But it’s a big change I think as we need to make sure all sync target types still work, and also need to check file upload which works a bit differently.

If there’s a benchmark that shows that this method is faster for Joplin (not in the general case), then maybe it’s worth investing time in this, but otherwise maybe not.

tessus · 2019-11-05T17:40:24.399Z

The only benchmark I can imagine is changing 50 items and check the log for the timing.

But the best benchmark would be to test this on Linux, since performance there is apparently really bad (according to several topics). Unfortunately I don’t have a Linux machine with a DE.

boring10 · 2019-11-08T01:31:37.283Z

@laurent @tessus If you would like I can help since I am running Linux. If you want to perform the changes to HTTP/2.0 then I can run some sync tests by syncing different numbers of items.

I can attest to the fact that Linux is excruciatingly slow. There are times when there is only 1 note change and it can take upwards of a minute to perform a sync. For reference, my sync target is Dropbox.

everett · 2022-03-10T02:18:08.836Z

I'm seeing the same difference between Android and desktop Linux.

With my small collection of 30-ish notes across maybe five notebooks, the desktop client takes 3-5x as long to "synchronize" when there's no changed content (basically check the server for changes). Doing the same syncs after adding a new note in either client doesn't noticeably change the time it takes.

Edit: I should specify that the server is Nextcloud; not sure if this makes a difference.

Daeraxa · 2022-03-10T02:23:48.780Z