Since OneDrive sync error this morning each sync fetches all items?

sashab · 5 January 2021 00:45

Since the OneDrive sync error this morning, on each sync it appears to be fetching all items regardless of whether they have been modified.

Is this because of something still being broken with OneDrive?

Regards

PC
APP: Joplin 1.6.2 (prod, win32)
OS: Windows 10 Home 64-Bit

ANDROID
APP: Joplin 1.5.0
DB: Database v34
OS: Android 10
HW: Samsung Galaxy S20 ULTRA 5G

Sync Log • Part 1:

2021-01-05 09:53:02: "RevisionService::maintenance: Starting..."
2021-01-05 09:53:02: "RevisionService::maintenance: Service is enabled"
2021-01-05 09:53:03: "RevisionService::collectRevisions: Created revisions for 0 notes"
2021-01-05 09:53:03: "RevisionService::maintenance: Done in 114ms"
2021-01-05 09:53:07: "Running background sync on timer..."
2021-01-05 09:53:07: "Preparing scheduled sync"
2021-01-05 09:53:07: "Starting scheduled sync"
2021-01-05 09:53:07: "Token expired: refreshing..."
2021-01-05 09:53:08: "Saving updated OneDrive auth."
2021-01-05 09:53:10: "Sync target info:", "{"version":2}"
2021-01-05 09:53:14: "TaskQueue.stop: syncDownload: waiting for tasks to complete: 0"
2021-01-05 09:53:14: "TaskQueue.stop: syncDownload: Done, waited for 0"
2021-01-05 09:53:54: "SyntaxError: On file c0816967a5ea4da49e5f2a1a0cdf0b6a.md: OneDriveApi::exec: Cannot parse JSON error: <!DOCTYPE html PUBLIC '-//W3C//DTD XHTML 1.0 Transitional//EN' 'http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd'><html xmlns='http://www.w3.org/1999/xhtml'><head><meta content='text/html; charset=utf-8' http-equiv='content-type'/><style type='text/css'>body {font-family:Arial; margin-left:40px; }img  { border:0 none; }#content { margin-left: auto; margin-right: auto }#message h2 { font-size: 20px; font-weight: normal; color: #000000; margin: 34px 0px 0px 0px }#message p  { font-size: 13px; color: #000000; margin: 7px 0px 0px0px}#errorref { font-size: 11px; color: #737373; margin-top: 41px }</style><title>OneDrive</title></head><body><div id='content'><div id='message'><h2>Our services aren't available right now</h2><p>We're working to restore all services as soon as possible. Please check back soon.</p></div><div id='errorref'><span>Ref A: 9DB8BAA007AE44A8AB69EC9F96C605EA Ref B: SYD03EDGE1009 Ref C: 2021-01-04T22:53:53Z</span></div></div></body></html> Unexpected token < in JSON at position 0
SyntaxError: On file c0816967a5ea4da49e5f2a1a0cdf0b6a.md: OneDriveApi::exec: Cannot parse JSON error: <!DOCTYPE html PUBLIC '-//W3C//DTD XHTML 1.0 Transitional//EN' 'http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd'><html xmlns='http://www.w3.org/1999/xhtml'><head><meta content='text/html; charset=utf-8' http-equiv='content-type'/><style type='text/css'>body {font-family:Arial; margin-left:40px; }img  { border:0 none; }#content { margin-left: auto; margin-right: auto }#message h2 { font-size: 20px; font-weight: normal; color: #000000; margin: 34px 0px 0px 0px }#message p  { font-size: 13px; color: #000000; margin: 7px 0px 0px0px}#errorref { font-size: 11px; color: #737373; margin-top: 41px }</style><title>OneDrive</title></head><body><div id='content'><div id='message'><h2>Our services aren't available right now</h2><p>We're working to restore all services as soon as possible. Please check back soon.</p></div><div id='errorref'><span>Ref A: 9DB8BAA007AE44A8AB69EC9F96C605EA Ref B: SYD03EDGE1009 Ref C: 2021-01-04T22:53:53Z</span></div></div></body></html> Unexpected token < in JSON at position 0
    at JSON.parse (<anonymous>)
    at OneDriveApi.exec (C:\Users\%USER_PROFILE%\AppData\Local\Temp\1jyGINt2P9qlJFviWRHWTq7OzbH\resources\app.asar\node_modules\@joplin\lib\onedrive-api.js:270:27)
    at async OneDriveApi.execText (C:\Users\%USER_PROFILE%\AppData\Local\Temp\1jyGINt2P9qlJFviWRHWTq7OzbH\resources\app.asar\node_modules\@joplin\lib\onedrive-api.js:359:20)
    at async FileApiDriverOneDrive.get (C:\Users\%USER_PROFILE%\AppData\Local\Temp\1jyGINt2P9qlJFviWRHWTq7OzbH\resources\app.asar\node_modules\@joplin\lib\file-api-driver-onedrive.js:110:21)
    at async tryAndRepeat (C:\Users\%USER_PROFILE%\AppData\Local\Temp\1jyGINt2P9qlJFviWRHWTq7OzbH\resources\app.asar\node_modules\@joplin\lib\file-api.js:38:19)"
2021-01-05 09:53:54: "Operations completed: "
2021-01-05 09:53:54: "fetchingTotal: 586"
2021-01-05 09:53:54: "fetchingProcessed: 559"
2021-01-05 09:53:54: "Total folders: 216"
2021-01-05 09:53:54: "Total notes: 663"
2021-01-05 09:53:54: "Total resources: 13"
2021-01-05 09:53:54: "There was some errors:"
2021-01-05 09:53:54: "SyntaxError: On file c0816967a5ea4da49e5f2a1a0cdf0b6a.md: OneDriveApi::exec: Cannot parse JSON error: <!DOCTYPE html PUBLIC '-//W3C//DTD XHTML 1.0 Transitional//EN' 'http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd'><html xmlns='http://www.w3.org/1999/xhtml'><head><meta content='text/html; charset=utf-8' http-equiv='content-type'/><style type='text/css'>body {font-family:Arial; margin-left:40px; }img  { border:0 none; }#content { margin-left: auto; margin-right: auto }#message h2 { font-size: 20px; font-weight: normal; color: #000000; margin: 34px 0px 0px 0px }#message p  { font-size: 13px; color: #000000; margin: 7px 0px 0px0px}#errorref { font-size: 11px; color: #737373; margin-top: 41px }</style><title>OneDrive</title></head><body><div id='content'><div id='message'><h2>Our services aren't available right now</h2><p>We're working to restore all services as soon as possible. Please check back soon.</p></div><div id='errorref'><span>Ref A: 9DB8BAA007AE44A8AB69EC9F96C605EA Ref B: SYD03EDGE1009 Ref C: 2021-01-04T22:53:53Z</span></div></div></body></html> Unexpected token < in JSON at position 0
SyntaxError: On file c0816967a5ea4da49e5f2a1a0cdf0b6a.md: OneDriveApi::exec: Cannot parse JSON error: <!DOCTYPE html PUBLIC '-//W3C//DTD XHTML 1.0 Transitional//EN' 'http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd'><html xmlns='http://www.w3.org/1999/xhtml'><head><meta content='text/html; charset=utf-8' http-equiv='content-type'/><style type='text/css'>body {font-family:Arial; margin-left:40px; }img  { border:0 none; }#content { margin-left: auto; margin-right: auto }#message h2 { font-size: 20px; font-weight: normal; color: #000000; margin: 34px 0px 0px 0px }#message p  { font-size: 13px; color: #000000; margin: 7px 0px 0px0px}#errorref { font-size: 11px; color: #737373; margin-top: 41px }</style><title>OneDrive</title></head><body><div id='content'><div id='message'><h2>Our services aren't available right now</h2><p>We're working to restore all services as soon as possible. Please check back soon.</p></div><div id='errorref'><span>Ref A: 9DB8BAA007AE44A8AB69EC9F96C605EA Ref B: SYD03EDGE1009 Ref C: 2021-01-04T22:53:53Z</span></div></div></body></html> Unexpected token < in JSON at position 0
    at JSON.parse (<anonymous>)
    at OneDriveApi.exec (C:\Users\%USER_PROFILE%\AppData\Local\Temp\1jyGINt2P9qlJFviWRHWTq7OzbH\resources\app.asar\node_modules\@joplin\lib\onedrive-api.js:270:27)
    at async OneDriveApi.execText (C:\Users\%USER_PROFILE%\AppData\Local\Temp\1jyGINt2P9qlJFviWRHWTq7OzbH\resources\app.asar\node_modules\@joplin\lib\onedrive-api.js:359:20)
    at async FileApiDriverOneDrive.get (C:\Users\%USER_PROFILE%\AppData\Local\Temp\1jyGINt2P9qlJFviWRHWTq7OzbH\resources\app.asar\node_modules\@joplin\lib\file-api-driver-onedrive.js:110:21)
    at async tryAndRepeat (C:\Users\%USER_PROFILE%\AppData\Local\Temp\1jyGINt2P9qlJFviWRHWTq7OzbH\resources\app.asar\node_modules\@joplin\lib\file-api.js:38:19)"
2021-01-05 09:53:56: "DecryptionWorker: cannot start because no master key is currently loaded."

Sync Log • Part 2:

2021-01-05 11:03:03: "RevisionService::maintenance: Starting..."
2021-01-05 11:03:03: "RevisionService::maintenance: Service is enabled"
2021-01-05 11:03:03: "RevisionService::collectRevisions: Created revisions for 0 notes"
2021-01-05 11:03:03: "RevisionService::maintenance: Done in 111ms"
2021-01-05 11:04:04: "Running background sync on timer..."
2021-01-05 11:04:04: "Preparing scheduled sync"
2021-01-05 11:04:04: "Starting scheduled sync"
2021-01-05 11:04:06: "Sync target info:", "{"version":2}"
2021-01-05 11:04:11: "TaskQueue.stop: syncDownload: waiting for tasks to complete: 0"
2021-01-05 11:04:11: "TaskQueue.stop: syncDownload: Done, waited for 0"
2021-01-05 11:04:26: "Saving settings..."
2021-01-05 11:04:26: "Settings have been saved."
2021-01-05 11:04:38: "Saving settings..."
2021-01-05 11:04:38: "Settings have been saved."
2021-01-05 11:04:50: "Saving settings..."
2021-01-05 11:04:50: "Settings have been saved."
2021-01-05 11:05:03: "Saving settings..."
2021-01-05 11:05:03: "Settings have been saved."
2021-01-05 11:05:16: "Saving settings..."
2021-01-05 11:05:16: "Settings have been saved."
2021-01-05 11:05:32: "Saving settings..."
2021-01-05 11:05:32: "Settings have been saved."
2021-01-05 11:05:44: "Saving settings..."
2021-01-05 11:05:44: "Settings have been saved."
2021-01-05 11:05:56: "Saving settings..."
2021-01-05 11:05:56: "Settings have been saved."
2021-01-05 11:06:09: "Saving settings..."
2021-01-05 11:06:09: "Settings have been saved."
2021-01-05 11:06:22: "Saving settings..."
2021-01-05 11:06:22: "Settings have been saved."
2021-01-05 11:06:38: "Saving settings..."
2021-01-05 11:06:38: "Settings have been saved."
2021-01-05 11:06:41: "Saving settings..."
2021-01-05 11:06:41: "Settings have been saved."
2021-01-05 11:06:41: "Operations completed: "
2021-01-05 11:06:41: "fetchingTotal: 2186"
2021-01-05 11:06:41: "fetchingProcessed: 2186"
2021-01-05 11:06:41: "Total folders: 216"
2021-01-05 11:06:41: "Total notes: 663"
2021-01-05 11:06:41: "Total resources: 13"
2021-01-05 11:06:41: "Saving settings..."
2021-01-05 11:06:41: "Settings have been saved."
2021-01-05 11:06:42: "DecryptionWorker: cannot start because no master key is currently loaded."
2021-01-05 11:09:45: "Saving settings..."
2021-01-05 11:09:45: "Settings have been saved."
2021-01-05 11:10:50: "Saving settings..."
2021-01-05 11:10:50: "Settings have been saved."
2021-01-05 11:11:17: "Saving settings..."
2021-01-05 11:11:17: "Settings have been saved."
2021-01-05 11:11:23: "Saving settings..."
2021-01-05 11:11:23: "Settings have been saved."
2021-01-05 11:11:41: "Running background sync on timer..."
2021-01-05 11:11:41: "Preparing scheduled sync"
2021-01-05 11:11:41: "Starting scheduled sync"
2021-01-05 11:11:43: "Sync target info:", "{"version":2}"
2021-01-05 11:11:49: "TaskQueue.stop: syncDownload: waiting for tasks to complete: 0"
2021-01-05 11:11:49: "TaskQueue.stop: syncDownload: Done, waited for 0"
2021-01-05 11:12:04: "Saving settings..."
2021-01-05 11:12:04: "Settings have been saved."
2021-01-05 11:12:19: "Saving settings..."
2021-01-05 11:12:19: "Settings have been saved."
2021-01-05 11:12:31: "Saving settings..."
2021-01-05 11:12:31: "Settings have been saved."
2021-01-05 11:12:45: "Saving settings..."
2021-01-05 11:12:46: "Settings have been saved."
2021-01-05 11:12:59: "Saving settings..."
2021-01-05 11:12:59: "Settings have been saved."
2021-01-05 11:13:03: "Garbage collecting alarms..."
2021-01-05 11:13:03: "RevisionService::maintenance: Starting..."
2021-01-05 11:13:03: "RevisionService::maintenance: Service is enabled"
2021-01-05 11:13:03: "RevisionService::collectRevisions: Created revisions for 0 notes"
2021-01-05 11:13:03: "RevisionService::maintenance: Done in 111ms"
2021-01-05 11:13:14: "Saving settings..."
2021-01-05 11:13:14: "Settings have been saved."
2021-01-05 11:13:27: "Saving settings..."
2021-01-05 11:13:27: "Settings have been saved."
2021-01-05 11:13:40: "Saving settings..."
2021-01-05 11:13:40: "Settings have been saved."
2021-01-05 11:13:53: "Saving settings..."
2021-01-05 11:13:54: "Settings have been saved."
2021-01-05 11:14:06: "Saving settings..."
2021-01-05 11:14:06: "Settings have been saved."
2021-01-05 11:14:20: "Saving settings..."
2021-01-05 11:14:20: "Settings have been saved."
2021-01-05 11:14:21: "Operations completed: "
2021-01-05 11:14:21: "fetchingTotal: 2186"
2021-01-05 11:14:21: "fetchingProcessed: 2186"
2021-01-05 11:14:21: "Saving settings..."
2021-01-05 11:14:21: "Total folders: 216"
2021-01-05 11:14:21: "Total notes: 663"
2021-01-05 11:14:21: "Total resources: 13"
2021-01-05 11:14:22: "Settings have been saved."
2021-01-05 11:14:22: "Saving settings..."
2021-01-05 11:14:22: "Settings have been saved."
2021-01-05 11:14:23: "DecryptionWorker: cannot start because no master key is currently loaded."

Screenshot from Android:

Gamegenorator · 5 January 2021 03:17

Exact same problem here, I can't find anything regarding it.

michaelmaass · 5 January 2021 06:14

Is this a nee behaviour?

Gamegenorator · 5 January 2021 06:15

@michaelmaass

Is this a nee behaviour?

Yes, it started this morning.

JackGruber · 5 January 2021 08:59

From the error merssage:

chris-zh-ch · 5 January 2021 09:28

Same here. Because of that also higher CPU-Usage. All Items get synced everytime regardless of changes or no changes.
No Error Message at the end.
Updated yesterday to version 1.5.14 (Desktop portable) and since then this problem. Probably coincidence.
So we will have to wait until Microsoft is doing something?

johnny · 5 January 2021 09:56

Does anybody have the same problem on Joplin < 1.6.2? I am asking this because in 1.6.2 is a minor change in Joplin regarding OneDrive. I am pretty sure that this has nothing to do with the Error but it would be good to have a confirmation anyway.

sashab · 5 January 2021 12:32

I started getting the error while running 1.4.7.

I then installed 1.6.2 in a hope that it may resolve the issue, it didn't

It was at this point I created the forum thread about it.

I have since however also tried 1.5.14 which I'm currently on but still the issue persists.

It is also happening on my phone which is running currently Joplin Android 1.5.1, it however had originally been running Joplin Android 1.5.0.

sashab · 5 January 2021 12:36

Some more logs from Joplin 1.5.14 in case it is useful to someone:

210105_2255.12-2302.25 • Joplin Desktop 1.5.14 • Sync Log (Debugging Enabled).txt (358.8 KB) 210105_2305.12-2310.15 • Joplin Desktop 1.5.14 • Sync Log (Debugging Enabled).txt (360.1 KB)

johnny · 5 January 2021 13:27

Thanks. It seems to be a Microsoft problem (again). I think it is not possible to fix this from the joplin side.

sashab · 5 January 2021 18:00

Darn, oh well thanks for letting me know.

Now for the indefinite wait to start... LOL

sashab · 5 January 2021 23:30

Over on reddit there is somebody having the same issue yet they are using WebDAV to sync with their NAS box, which suggests Microsoft isn't at fault then.

syae · 6 January 2021 00:45

link to the Reddit thread for reference: https://www.reddit.com/r/joplinapp/comments/kr15ix/joplin_synchronises_everything_each_time/

rxliuli · 6 January 2021 05:35

I encountered the same problem. . .

srm39 · 6 January 2021 08:36

Similar issue here. Joplin 1.5.14 seems to never stop syncing on Windows or iPhone. OneDrive is sync target. Problem started a couple of days ago, I think.

I've recently cancelled my Evernote Premium account and switched to Joplin as it's the best option for my use cases. There have been two significant sync issues within a short time (appreciate one was MS not Joplin - as this one may also be - but it still affected use). I hope this isn't an ongoing source of pain as I really want Joplin to be able to replace Evernote for me, but it has to be reliable.

foglcz · 6 January 2021 10:00

Confirming the issue as well, however:

Windows desktop v1.5.14 synchronizes fine (although it does synchronize all of the notes). Total items as per sync status is 572.
Android app v1.5.1 synchronizes forever with ever increasing number of items.

The difference is also in the behaviour: desktop app starts syncing, and then fetches items x/197 -> x/375 -> x/567 -> x/579 -> 579/579, and then it completes.

Android app synces 3 items with ever increasing # of 3 items:
Fetching items x/3 -> x/6 -> x/9 -> x/12 and so on, until cancelled.

Below is the log from desktop from most recent sync on desktop:

2021-01-06 10:42:23: "Preparing scheduled sync"
2021-01-06 10:42:23: "Starting scheduled sync"
2021-01-06 10:42:25: "Sync target info:", "{"version":2}"
2021-01-06 10:42:27: "TaskQueue.stop: syncDownload: waiting for tasks to complete: 0"
2021-01-06 10:42:27: "TaskQueue.stop: syncDownload: Done, waited for 0"
2021-01-06 10:42:57: "Saving settings..."
2021-01-06 10:42:57: "Settings have been saved."
2021-01-06 10:43:00: "RevisionService::maintenance: Starting..."
2021-01-06 10:43:00: "RevisionService::maintenance: Service is enabled"
2021-01-06 10:43:00: "RevisionService::collectRevisions: Created revisions for 0 notes"
2021-01-06 10:43:00: "RevisionService::maintenance: Done in 154ms"
2021-01-06 10:43:23: "Saving settings..."
2021-01-06 10:43:23: "Settings have been saved."
2021-01-06 10:44:00: "Saving settings..."
2021-01-06 10:44:00: "Settings have been saved."
2021-01-06 10:44:03: "Saving settings..."
2021-01-06 10:44:03: "Settings have been saved."
2021-01-06 10:44:03: "Operations completed: "
2021-01-06 10:44:03: "fetchingTotal: 579"
2021-01-06 10:44:03: "fetchingProcessed: 579"
2021-01-06 10:44:03: "updateLocal: 1"
2021-01-06 10:44:03: "Total folders: 24"
2021-01-06 10:44:03: "Total notes: 371"
2021-01-06 10:44:03: "Total resources: 27"
2021-01-06 10:44:03: "DecryptionWorker: starting decryption..."
2021-01-06 10:44:04: "Saving settings..."
2021-01-06 10:44:04: "DecryptionWorker: completed decryption."
2021-01-06 10:44:04: "ResourceFetcher: Auto-add resources: Mode: always"
2021-01-06 10:44:04: "ResourceFetcher: Auto-added resources: 0"
2021-01-06 10:44:04: "Settings have been saved."
2021-01-06 10:44:12: "SearchEngine: Updating FTS table..."
2021-01-06 10:44:12: "Saving settings..."
2021-01-06 10:44:12: "Settings have been saved."
2021-01-06 10:44:12: "SearchEngine: Updated FTS table in 120ms. Inserted: 1. Deleted: 0"

Whereas the log from android app contains two lines about savings settings infinitely, until sync is cancelled:

I was thinking about it this morning, and it feels to me that there might be a race condition, where Joplin fetches the delta link first, updates the settings file with next delta, and then starts syncing contents. But when requesting for the contents, it might see the settings file as modified (because it modified it underneath itself), saying "hang on, settings have changed, gotta restart the sync again" and that'd just repeat itself infinitely.

Then again:

Google Play shows Joplin app updated on December 27. Google play My Apps tab does not show Joplin updated yesterday, meaning the problem would have manifested sooner if it was a Joplin issue.
I have noticed the issue only yesterday on my android, when it was syncing infinitely
The fact that it works on desktop but does not work in android for me is odd. I presume both app would use different delta link, and as this was the source of last bug in OneDrive sync as well, I think it might be a good bet that it's on their end again.

Is there a way to force a full re-sync of the android app, discarding the deltalink?

foglcz · 6 January 2021 11:45

I've recently cancelled my Evernote Premium account and switched to Joplin as it's the best option for my use cases. There have been two significant sync issues within a short time (appreciate one was MS not Joplin - as this one may also be - but it still affected use). I hope this isn't an ongoing source of pain as I really want Joplin to be able to replace Evernote for me, but it has to be reliable.

BTW, I have migrated to Joplin more than a year ago after years of using Evernote, and in the year+ I had only two sync issues with joplin - this and the previous one. Since the previous one was caused by Microsoft themselves, I suspect this one will be too.

For me, using Joplin has been great, and I will stick with it despite the (microsoft's) sync issues. I suspect that Joplin Server will remove the issues eventually, and the end-to-end encryption is the big thing for me. While I would love to have OneDrive for business sync target, as I trust Microsoft datacenter security more than I trust security of my own house (and fully patched enthusiast router), alas, I'll likely just use the Joplin Server once I have time to set it up.

(I think that since I'm a common human, I'm not big or interesting enough to warrant anything more than automated hacks on my house. And I don't plan to open joplin server to the internet.)

If you do decide to switch in the end, and end up liking Joplin, please consider donating to Joplin development. Even 2 dollars help, as the software takes time and effort to write (and support).

srm39 · 6 January 2021 14:03

Once I've settled on Joplin, I would be happy to contribute.

Agree that the E2EE is essential. It's only when I started looking at alternatives to Evernote (due to the recent V10 issues), I've realised what's important to me and how few solutions offer everything I'm looking for.

Evernote, even if they made V10 amazing, isn't an option any more as they've removed local notebooks, which is a deal-breaker.

I hope the sync problem gets fixed soon as I have a paid MS account with 1TB included and it's not worth it (to me) to set up a self-hosted solution, particularly with the remote access I would need.

rxliuli · 6 January 2021 14:31

+1, I don’t know what the developers of onedrive are doing, there are too many problems with joplin+onedrive

srm39 · 6 January 2021 14:45

OK, now looking into non-OneDrive/Dropbox solutions. Can anyone recommend something hosted (not self-hosted) which would work and be relatively cheap? Have heard that Linode might be an option but although I am reasonably tech-able, I know nothing about Linux so it would need to be quite easy to set up.
Would prefer to avoid a dedicated device in the home which has to be on all the time for remote access (and open to the internet). If this info is available elsewhere, apologies for posting here.

Topic		Replies	Views
Mentally exploding, Joplin's sync is a shortcoming, and for me, it's fatal Support	5	1141	14 March 2021
Getting sync error with OneDrive Support	8	691	12 August 2022
Sync with onedrive error Support	4	470	30 April 2021
Joplin is trying to fetch too many items while syncing (sync seems broken) Support	6	1218	8 February 2021
Issues with onedrive Support	1	620	26 March 2020

Since OneDrive sync error this morning each sync fetches all items?

Related topics