Nextcloud sync fails on all but the default profile

roman_r_m · 2024-06-30T21:30:48.642Z

There's definitely something bizarre going on here. I was able to replicate a similar (although maybe not exactly the same issue).
On Android I can sync the default profile but when trying to sync a second profile with another NC account I get "WebDAV directory not found".
The desktop client has no problem syncing.

I am not sure what is going on, 2 requests from Android look exactly the same to me:

default profile - no issue

PROPFIND /remote.php/dav/files/admin/joplin/ HTTP/1.1
depth: 0
authorization: Basic <redacted>
cache-control: no-store
if-none-match: JoplinIgnore-45210
user-agent: Joplin/1.0
Content-Type: text/xml
Content-Length: 190
Host: 192.168.1.79:8087
Connection: Keep-Alive
Accept-Encoding: gzip
Cookie: cookie_test=test; oc_sessionPassphrase=<redacted>; nc_sameSiteCookielax=true; nc_sameSiteCookiestrict=true; oc2vt4y9tr8e=3ddcad8cd6ff9c7312b077958038496d

2nd profile on android - directory not found

PROPFIND /remote.php/dav/files/test/joplin/ HTTP/1.1
depth: 0
authorization: Basic <redacted>
cache-control: no-store
if-none-match: JoplinIgnore-6475
user-agent: Joplin/1.0
Content-Type: text/xml
Content-Length: 190
Host: 192.168.1.79:8087
Connection: Keep-Alive
Accept-Encoding: gzip
Cookie: oc_sessionPassphrase=<redacted>; nc_sameSiteCookielax=true; nc_sameSiteCookiestrict=true; oc2vt4y9tr8e=3ddcad8cd6ff9c7312b077958038496d

curl call from desktop - also succeeds

*   Trying 192.168.1.79:8087...
* Connected to 192.168.1.79 (192.168.1.79) port 8087
> PROPFIND /remote.php/dav/files/test/joplin/ HTTP/1.1
> Host: 192.168.1.79:8087
> Accept: */*
> Depth: 0
> Authorization: Basic <redacted>
> Content-Type: text/xml
> Cache-Control: no-store
> If-None-Match: JoplinIgnore-67263
> User-Agent: Joplin/1.0
> Content-Length: 190

roman_r_m · 2024-06-30T21:32:08.045Z

I guess one difference I can see is the successful requests all have this cookie set cookie_test=test. No idea where this is coming from, certainly not from Joplin.

UPD
actually, I'm pretty sure it's set by Nextcloud. I can see it mentioned in its source code a few times.

UPD2
not sure how this can be coming from NC given I capture network traffic on the wire, hmm

roman_r_m · 2024-06-30T21:57:54.744Z

I suspect there's some bug in how Joplin handles cookies on Android. I am seeing issues even without 2 profiles like this:

start with a clean state
enter nextcloud details and press "Check Synchronization Configuration"
now without leaving the screen change the details to another user on the same server
check configuration succeeds but when you try to actually sync NC returns "permission denied"

I think some cookie specific to a particular nextcloud account gets saved on step 2 and then incorrectly used for all subsequent steps for a different account.

laurent · 2024-07-01T09:04:03.343Z

That could be related to cookies but as I know we don't look at cookies or explicitly pass them around. Auth is just basic-auth, so it's email+password that's sent with each request (hopefully over https)

roman_r_m · 2024-07-01T09:15:25.079Z

laurent:

we don't look at cookies or explicitly pass them around

I suppose it's the underlying network library that attaches these cookies. RN fetch blob certainly does this automatically.

laurent · 2024-07-01T10:51:16.533Z

Yes that's probably it. I wonder if this test cookie is some kind of security measure - maybe it's testing that it's being sent back, but normally we shouldn't have to send anything back other than the auth credentials.

roman_r_m · 2024-07-01T12:24:28.490Z

I think it's not the test cookie that is the problem, it's another one, not sure which.
Joplin shouldn't send any cookies at all, not sure how easy it is to do though.

moontan · 2024-07-01T15:17:36.163Z

Thank you @Coffee – Very happy to see my enquiry generated such a vividly dedicated line of responses. I wish I lnew more about the inner workings of both Joplin and Nextcloud, so I could contribute some more valuable details.

One addendum I have is that apparently I jumped to conclusions about the failure pattern in regard to which one of the accounts syncs and which don't. So I think I need to summarise the situation once more:

I have these accounts:
Joplin user 0 (default profile) sync set up with Nextcloud user 0
Joplin user 1 sync set up with Nextcloud user 1
Joplin user 2 sync set up with Nextcloud user 2
Joplin user 3 sync set up with Nextcloud user 3

All accounts sync fine on the Joplin client for macOS. When setting up sync on the Android client, hitting the button for testing the configuration button produces a positive result, and during my first attempts the user 0 default profile synced alright, while user 1, user 2, and user 3 produced the errors from the screenshot of my original post. Then the profile that synced fine was all of a sudden user 1, while user 0, user 2, and user 3 began to generate the aforementioned errors. I thought the switch to a different profile working was caused by a reboot, but I know now that that can be ruled out. Currently the working sync is on the user 3 profile, but I have no idea what I did to effect this.

Thank you kindly for helping me get to the bottom of this issue!

roman_r_m · 2024-07-01T15:25:56.279Z

I think that's because those cookies have a timeout. So after a while they become invalid and from that point whichever account syncs first works, all the others do not.

laurent · 2024-07-01T15:40:25.286Z

By the way, how do you connect to your account @moontan @Coffee, is that with email/password or with an application password? Do you have MFA enabled in Nextcloud?

Coffee · 2024-07-01T16:11:50.860Z

laurent:

how do you connect to your account

email/password

MFA not enabled

roman_r_m · 2024-07-01T20:40:55.050Z

Clearing cookies before each request (using react-native-cookies · GitHub) seems to fix the issue. Although this alone may not be enough because some requests are done using rn-fetch-blob - this may need to be handled separately.

But at least now it seems pretty clear what the issue was.

moontan · 2024-07-01T23:29:25.131Z

laurent:

By the way, how do you connect to your account @moontan @Coffee, is that with email/password or with an application password? Do you have MFA enabled in Nextcloud?

Application password generated by NC.
MFA enabled.

Coffee · 2024-07-02T09:37:43.332Z

roman_r_m:

But at least now it seems pretty clear what the issue was.

I love good news!
Or am I being too optimistic again and getting my hopes up too soon?

roman_r_m · 2024-07-03T11:06:31.556Z

I only did a quick check but it seemed that deleting cookies before every request has solved the issue.

Someone still need to implement the fix properly, unfortunately I don't have the time to do it myself at this time.

Coffee · 2024-07-03T15:26:51.752Z

roman_r_m:

I suppose it's the underlying network library that attaches these cookies

I also assume that it is the behaviour of an underlying network library of Android.
My words as a layman for it were WebDAV Stack.

laurent · 2024-07-03T15:43:50.593Z

I wonder if the bug should be reported to Nextcloud. It looks like it's looking at the cookies (that we didn't set) and reject the request based on this, even though we provide valid Auth credentials with that same request.

Also cookies have nothing to do with WebDAV as far as I know, at least I can't think of any implementation that uses them.

roman_r_m · 2024-07-03T16:08:48.864Z

Coffee:

I also assume that it is the behaviour of an underlying network library of Android.

It's not OS-level, just a 3rd party lib used by React Native called okhttp

roman_r_m · 2024-07-03T16:13:02.317Z

laurent:

It looks like it's looking at the cookies (that we didn't set)

But from their point we do set them. Or rather okhttp does.

I guess whether it's a bug or not isn't entirely clear. I don't remember if WebDAV specification says anything about cookies at all.

Also, it depends on whether this cookies requirement is specified anywhere in their docs. If it is, then I'd argue it's a bug in Joplin, otherwise it's less clear.

Coffee · 2024-07-03T17:49:30.464Z

roman_r_m:

It's not OS-level, just a 3rd party lib used by React Native called okhttp

In my opinion, the important thing in this case is whose decision it was to install this library on our Android.
Was it Google's or Samsung's decision to deliver this component with the operating system, or was it Joplin's decision to deliver this library with Joplin?

roman_r_m:

okhttp

Is it a matter of https ://square.github.io/okhttp/ ?