Nextcloud sync fails on all but the default profile

moontan · 2024-07-01T15:17:36.163Z

Thank you @Coffee – Very happy to see my enquiry generated such a vividly dedicated line of responses. I wish I lnew more about the inner workings of both Joplin and Nextcloud, so I could contribute some more valuable details.

One addendum I have is that apparently I jumped to conclusions about the failure pattern in regard to which one of the accounts syncs and which don't. So I think I need to summarise the situation once more:

I have these accounts:
Joplin user 0 (default profile) sync set up with Nextcloud user 0
Joplin user 1 sync set up with Nextcloud user 1
Joplin user 2 sync set up with Nextcloud user 2
Joplin user 3 sync set up with Nextcloud user 3

All accounts sync fine on the Joplin client for macOS. When setting up sync on the Android client, hitting the button for testing the configuration button produces a positive result, and during my first attempts the user 0 default profile synced alright, while user 1, user 2, and user 3 produced the errors from the screenshot of my original post. Then the profile that synced fine was all of a sudden user 1, while user 0, user 2, and user 3 began to generate the aforementioned errors. I thought the switch to a different profile working was caused by a reboot, but I know now that that can be ruled out. Currently the working sync is on the user 3 profile, but I have no idea what I did to effect this.

Thank you kindly for helping me get to the bottom of this issue!

roman_r_m · 2024-07-01T15:25:56.279Z

I think that's because those cookies have a timeout. So after a while they become invalid and from that point whichever account syncs first works, all the others do not.

laurent · 2024-07-01T15:40:25.286Z

By the way, how do you connect to your account @moontan @Coffee, is that with email/password or with an application password? Do you have MFA enabled in Nextcloud?

Coffee · 2024-07-01T16:11:50.860Z

laurent:

how do you connect to your account

email/password

MFA not enabled

roman_r_m · 2024-07-01T20:40:55.050Z

Clearing cookies before each request (using react-native-cookies · GitHub) seems to fix the issue. Although this alone may not be enough because some requests are done using rn-fetch-blob - this may need to be handled separately.

But at least now it seems pretty clear what the issue was.

moontan · 2024-07-01T23:29:25.131Z

laurent:

By the way, how do you connect to your account @moontan @Coffee, is that with email/password or with an application password? Do you have MFA enabled in Nextcloud?

Application password generated by NC.
MFA enabled.

Coffee · 2024-07-02T09:37:43.332Z

roman_r_m:

But at least now it seems pretty clear what the issue was.

I love good news!
Or am I being too optimistic again and getting my hopes up too soon?

roman_r_m · 2024-07-03T11:06:31.556Z

I only did a quick check but it seemed that deleting cookies before every request has solved the issue.

Someone still need to implement the fix properly, unfortunately I don't have the time to do it myself at this time.

Coffee · 2024-07-03T15:26:51.752Z

roman_r_m:

I suppose it's the underlying network library that attaches these cookies

I also assume that it is the behaviour of an underlying network library of Android.
My words as a layman for it were WebDAV Stack.

laurent · 2024-07-03T15:43:50.593Z

I wonder if the bug should be reported to Nextcloud. It looks like it's looking at the cookies (that we didn't set) and reject the request based on this, even though we provide valid Auth credentials with that same request.

Also cookies have nothing to do with WebDAV as far as I know, at least I can't think of any implementation that uses them.

roman_r_m · 2024-07-03T16:08:48.864Z

Coffee:

I also assume that it is the behaviour of an underlying network library of Android.

It's not OS-level, just a 3rd party lib used by React Native called okhttp

roman_r_m · 2024-07-03T16:13:02.317Z

laurent:

It looks like it's looking at the cookies (that we didn't set)

But from their point we do set them. Or rather okhttp does.

I guess whether it's a bug or not isn't entirely clear. I don't remember if WebDAV specification says anything about cookies at all.

Also, it depends on whether this cookies requirement is specified anywhere in their docs. If it is, then I'd argue it's a bug in Joplin, otherwise it's less clear.

Coffee · 2024-07-03T17:49:30.464Z

roman_r_m:

It's not OS-level, just a 3rd party lib used by React Native called okhttp

In my opinion, the important thing in this case is whose decision it was to install this library on our Android.
Was it Google's or Samsung's decision to deliver this component with the operating system, or was it Joplin's decision to deliver this library with Joplin?

roman_r_m:

okhttp

Is it a matter of https ://square.github.io/okhttp/ ?

laurent · 2024-07-03T18:00:28.265Z

roman_r_m:

I guess whether it's a bug or not isn't entirely clear. I don't remember if WebDAV specification says anything about cookies at all.

More generally I can't think of any WebDAV API that uses cookies for authentication, that's normally something meant for browsers.

It seems like they have some code somewhere that says "if invalid cookie => error". While they should actually check the headers for authentication, especially since it's how the connection was initiated.

Coffee · 2024-07-03T18:07:13.239Z

laurent:

I wonder if the bug should be reported to Nextcloud.

In my opinion, it would be a good idea to report the issue to NextCloud. However, this would have to be done by one of the Joplin experts. I would be willing to take part in the dialogue. However, I do not see myself in a position to represent this issue at NextCloud myself.

However, two aspects must be taken into account:

First:
Should the workaround suggested by roman_r_m, which could probably be implemented relatively easily in Joplin, be implemented relatively quickly at the same time?

roman_r_m:

Clearing cookies before each request

Second:
Could it be that it is not the NextCloud people who have to solve the problem in their own code, because it could also be a 3rd-party library on the server side?

roman_r_m · 2024-07-03T20:17:43.979Z

laurent:

More generally I can't think of any WebDAV API that uses cookies for authentication, that's normally something meant for browsers.

A quick google search shows it's possible:

webdavsystem.com

Creating WebDAV Server with Cookies Authentication

Describes how to create a WebDAV Server with pure cookies authentication and configure WebDAV Ajax Library to open documents

unix.stackexchange.com

Mount webdav on Linux with cookie authentication

webdav, davfs2, konqueror

asked by Clint on 10:39AM - 14 Jan 17 UTC

roman_r_m · 2024-07-03T20:19:20.503Z

Coffee:

Was it Google's or Samsung's decision to deliver this component with the operating system, or was it Joplin's decision to deliver this library with Joplin?

The dependency chain goes like this: Joplin - React Native - okhttp

But what does it matter? It's not like this is a bug in okhttp.

Coffee · 2024-07-04T07:17:10.903Z

roman_r_m:

It's not like this is a bug in okhttp.

Then, in this case, it doesn't matter.

Coffee · 2024-07-04T07:23:43.307Z

roman_r_m:

WebDAV Ajax Library to open documents
Creating WebDAV Server with Cookies Authentication

This could probably play a role because of https ://nextcloud.com/office/, especially if several users are to be able to edit the same document at the same time.

moontan · 2024-07-15T19:43:51.256Z

I'm really grateful you're putting so much effort into this. After updating both the desktop client and the Android client, now the sync seems to run smoothly – with one exception: When I hit the “check sync settings” button in one of my four profiles on the Android client, it returns this message:

“Error: Please check that URL, username, password, etc. are correct and that the sync target is accessible. The reported error was: WebDAV directory not found”

Since the sync with the same Nextcloud server works for all the other profiles (and for all profiles on the macOS client), there's a small chance the problem lies with the Nextcloud user account, which I am about to investigate, but I presume the Joplin app on Android might be the cause of the issue as well. When I ignore the failed sync test and initiate the sync it spits out a “completed” message followed by “Last error: Error: MKCOL locks/: Parent node does not exist (Exception Sabre\DAV\Exception\Conflict) (409): <?xml version="1.0" encoding="utf-8"?> <d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns"> <s:exception>Sabre\DAV\Exception\Conflict</s:exception> <s:message>Parent node does not exist</s:message> </d:error>

When I do a search on SabreDAV, its website says that it comes with an experimental WebDAV-Client that's a wrapper around curl. Can we rule out that the Joplin Android app implements alpha/beta stage code from that WebDAV client?

(I triple checked my config input in the Joplin Android app, and even retyped the URL when I noticed a bug that moves the cursor to the left of a character that I insert anywhere in the string but the very end.)