Synch file format/EOL issues

Features
1

This concerns the implementation of the unserialize function of the BaseItem.

The issue:

If a saved/synchronized file contains newlines at the end of the file the file cannot be imported as a note.

The cause:

The unserialize method steps through the files split into lines in reverse and assumes it has reached the end of file metadata information once it reaches an empty line. If there is an empty line at the end of the file thus creating an empty object to be imported and rightly raising an error due to a missing type_ property.

This is not a problem as such, as it works fine as long as the user does not touch the saved/synchronized notes with an editor or other tools, which often times insert a newline at the end of text files. I only ran into this issue as I actively touched the synchronized files in an attempt to troubleshoot another issue (which was due to not having the latest version of the app).

I would be happy to improve the code so be more resilient to issues like this and maybe a bit simpler if you’d be interested. I didn’t really understand it the first time I had a look at it, due to a lot of string handling and conditions.

I could use some pointers:

  • is there any sort of test setup to make sure I’m not breaking anything, when changing the code (didn’t find anything on the fly)?
  • is there a particular reason, why the title and content of a note is at the top as cleartext only seperated by a newline and not just another property other than historical ones? I think the serialization and deserialization would be considerable simpler with a different format.
  • any ideas/plans on file format transformation or versioning to support multiple app versions in the future if there are new fields introduced/required? This would enable features, like showing the user a message saying they need to upgrade to a new app version, without them having to go to the forums to report a bug and/or allowing automatic transformations to a new file “schema” version.

What are your thoughts on this?

My post on the forum about the issue for reference

2

Yes, from /CliClient, you can launch ./run_test.sh to run the tests, or just /run_test.sh synchronizer for just the synchronizer tests.

is there a particular reason, why the title and content of a note is at the top as cleartext only seperated by a newline and not just another property other than historical ones? I think the serialization and deserialization would be considerable simpler with a different format.

It's for historical reasons, but also it makes it convenient to preview notes in thumbnails since it means you have the title first. You then also see the most relevant part of the note, the title and body, when you open it.

As editing notes directly in the sync folder is not a use-case that I wish to support (too many possibilities of errors), I'm also not planning to change the serialisation format.

any ideas/plans on file format transformation or versioning to support multiple app versions in the future if there are new fields introduced/required? This would enable features, like showing the user a message saying they need to upgrade to a new app version, without them having to go to the forums to report a bug and/or allowing automatic transformations to a new file “schema” version.

Yes that's indeed an issue. The database is protected from version issues (the app will exit if it finds a database version more recent than what it can handle), however the sync format indeed is not, which recently caused problems when a user would update one app but not the other.

I'm not sure how to best handle this, if you have any suggestion feel free to let me know. Maybe some version number in the sync file?