Hi,
I use Dropbox to synchronize files between a few clients (e.g. Mac, Linux and Android). That worked fine, until on a Mac client I edited a file in the Joplin Dropbox directory. In the file I edited only the content, not metadata on the end of the file.
Since then, I see following error message in Mac clients:
Completed: 18/06/2020 22:59
Last error: Error: Missing required property: type_:
and in Linux one:
Fetched items: 5/9.
Completed: 19/06/2020 09:06
Last error: Error: On file 27805ddb4e5a4fde890cba9a3912d19b.md: Missing required property: type_:
The actual note file which content matches the content 27805ddb4e5a4fde890cba9a3912d19b.md cannot be synced. Actually, looks like nothing is synced after the break.
What should I do?
Version on Mac:
Copyright © 2016-2020 Laurent Cozic
Joplin 1.0.218 (prod, darwin)
Client ID: 353f5d2f6c1b4a6ca9ee73d4e2d8157c
Sync Version: 1
Profile Version: 30
Keychain Supported: Yes
Revision: 5be8c2c (master)
Version on Linux:
"Joplin 1.0.201 (prod, linux)
Client ID: b59333338c814e368d5c71c2fefe5320
Sync Version: 1
Profile Version: 28
Revision: e65af8c (master)"
There’s a plan to rename the .md files on the sync target to .bin, to show they shouldn’t be edited. In the meantime you can try to remove the last newline characters from your file.
Thanks for very prompt response. It helped!
Interesting, I didn’t see extra line in Vim, even if I looked for both CR and LF.
It is not an extra line, the last line may not be followed by a newline for reasons only the Joplin gods know.
When the file is edited with vi it should display “[noeol]”, e.g.
"127108ae05174b29a623f0226c689e76.md" [noeol] 13L, 431C
1 Like
Apologies for dual-posting this with Github, but I am not sure which is the correct issue forum to use.
=============
Hello! I am seeing this issue as well now, and it appears to be an issue with every file that I attempt to edit. I checked the .md file on NextCloud and all appeared to be in order. What I am noticing is adding new files works fine, but any subsequent edits generates a sync error. I have not attempted to edit anything outside of the UI.
Sync Log Error:
Error: On file e62af337869a4bcdade7cd58025368c3.md: Missing required property: type_: test
test
tttttt
id: e62af337869a4bcdade7cd58025368c3
parent_id: e231a3b071e1419abce813b44fb82715
created_time: 2020-06-24T20:56:20.508Z
updated_time: 2020-06-24T20:56:52.029Z
is_conflict: 0
latitude: 0.00000000
longitude: 0.00000000
altitude: 0.0000
author:
source_url:
is_todo: 0
todo_due: 0
todo_completed: 0
source: joplin-desktop
source_application: net.cozic.joplin-desktop
application_data:
order: 1593032180508
user_created_time: 2020-06-24T20:56:20.508Z
user_updated_time: 2020-06-24T20:56:52.029Z
encryption_cipher_text:
encryption_applied: 0
markup_language: 1
is_shared: 0
Here is the metadata block from that test note.
id: e62af337869a4bcdade7cd58025368c3 parent_id: e231a3b071e1419abce813b44fb82715 created_time: 2020-06-24T20:56:20.508Z updated_time: 2020-06-24T20:56:52.029Z is_conflict: 0 latitude: 0.00000000 longitude: 0.00000000 altitude: 0.0000 author: source_url: is_todo: 0 todo_due: 0 todo_completed: 0 source: joplin-desktop source_application: net.cozic.joplin-desktop application_data: order: 1593032180508 user_created_time: 2020-06-24T20:56:20.508Z user_updated_time: 2020-06-24T20:56:52.029Z encryption_cipher_text: encryption_applied: 0 markup_language: 1 is_shared: 0 type_: 1
Based on the error message above, it is almost like it is failing to load the type_ value from the metadata block.
One other note: I did just recently move to my own NextCloud instance hosted on a DigitalOcean Droplet, but never attempted to edit files via that mechsnism.
You are not supposed to edit the file on the sync target. (I’m not sure why people insist on screwing around with data on the sync target.) Laurent should change the format to binary. Are you going to hexedit then?
Also, did you miss the part where there MUST NOT BE an EOL (end of line = return / line feed) at EOF (end of file). e.g. When you do a cat file, the prompt will not be in the next line, but in the same as the last line of text. Or vi will tell you [noeol] as scurius has already mentioned.
I never edited any files. That is what I said in my last paragraph.
Apologies, I missed that part.
However, I bet if I checked one file on your sync target, it would have an EOL at EOF.
Maybe some other issue with your sync software. e.g. you nextcloud sync client. Did you by any chance sync that directory also to your local disk? Theoretically the SW should not touch the files… But you won’t believe what I’ve seen… Maybe a process changed the files and synced them back.
Some people had similar issues when using Syncthing on Android 10.
I have to buy groceries, but will be back in about 1h. Until then, you could try the following on one .md file that doesn’t sync (on the sync target):
truncate -s -1 filename.md
If it syncs, your files on the sync target have a problem. For what reason ever and we have to figure out why that is.
No worries! Thanks for the help. I did not even install the NextCloud sync on my windows machine because that NextCloud instance is solely for Joplin. What did happen is I was forced to move away from ServiceMetric as a NextCloud provider. I guess the $5/year was too good to be true and they shut down. This forced me to set up a small Ubuntu 20.04 droplet on Digital Ocean. I installed Snap version of NextCloud, configured it, and installed the Jopin Web App on it to support sync.
I then went to my Windows machine and did the following:
1.) Made a JEX backup
2.) Deleted everything in Joplin. NOTE: I did not delete my JoplinProfile folder, which I probably should have.
3.) Changed the Sync endpoint.
4.) Did a restore of the JEX file.
5.) Synchronized all files to NextCloud.
6.) On my phone, I deleted everything, udpated the endpoint, and resynced.
Everything seemed to work great. All content was pushed up to the server and came back down to the phone. No problem. That was Sunday.
What I noticed today was that any edits all stopped syncing with the errors I posted. New file creations sync fine. Any subsequent edits break. ALso, any attempt to edit an existing note also renders it broken.
I tried your suggesting of truncating one character from my test .md file. After examining it in vi, but not saving, the last character was a 1. type-:1
After truncating, the one was gone. I then attempted my local sync and I get the same error, but now the log no longer shows the 1. Seems like the EOL may not be the issue…
I also tried restoring the Joplin version from 1.0.220 as 1.0.224 dropped today. That displayed the same issue.
What I am going to do now is start from scratch.
1.) Delete my Joplin Profile
2.) Delete everything in NextCloud
3.) Restore my JEX backup on Windows
4.) Sync to NextCloud
Hope for the best!
This is really odd. Your steps to move the notes are fine and valid.
Since my truncate command removed the 1 your file had no EOL at EOF, so your test file should have synced normally.
Same exact problem. Here is the sequence that led to the error.
1.) Fresh import from JEX after deleting Joplin Profile and everything on the NextCloud server.
2.) Sync with NextCloud. All seemed to work normally.
3.) Edit a single note via the Windows Joplin UI (1.0.224)
4.) Sync to the server (appeared successful).
5.) Hit synchronize again and receive same type_: missing error.
Ugh. Not sure what my next steps should be. The only thing materially different from the past two years is I am hosting my own instance of NextCloud.
A few more bits of detail after some testing.
The following steps seem to lead up to the corruption:
1.) Take an existing note that has not been edited since I noticed this behavior.
2.) Edit it. This seems to sync with the server normally and I get a log message indicating one file synced.
3.) Attempt to Edit again. Synchronization produces the type missing error.
The other thing I noted is that I can take a corrupt file and delete. This seems to be synced to the server properly and i do not receive any more failures until I attempt to edit using the steps I mentioned above.
gamejunkie, at this step:
2.) Edit it. This seems to sync with the server normally and I get a log message indicating one file synced.
Could you get the file on the sync target and see if it is valid or not (you can upload it somewhere and we can check too).
That will help determine if the file is corrupted when it's uploaded or later, when it's downloaded.
Hi Laurent - I definitely looked at it and it seemed correct. There was a type_id there and it was of type “1”. @tessus asked that I run a truncate of one char on it from the command line, which I did, and what remained was type_: , which indicates the “1” was the last char and there was no extraneous EOL.
Here is some more info. I checked my test file on the server and here is what I could see in VI.
I then did a RAW EXPORT from my windows client and checked the same .md file. Here is what appears locally.
Thanks for checking. So it seems the file is indeed correct on server, but then it’s on download that it somehow gets truncated? That’s very strange. Could you post the log before and after this bug happens? https://joplinapp.org/debugging
Finally, here is what appears in the logs.
Here is a sanitized version of the log. I removed the actual note text and left the log messages.
log_error_example.txt (2.5 KB)
