Joplin indirect reliance on Python problematic for Fedora 31

Development
1

I have been building Joplin on Fedora Linux for some time now – https://github.com/taw00/joplin-rpm – and it looks like Joplin is relying on Python 2.6.0 in some places. Fedora 31 is in beta (release in November I think? and will supply Python 3.8) and it will mark moving beyond mere deprecation to wholesale cleansing of Python 2.6.0. Or at least all the related packages: https://fedoraproject.org/wiki/Changes/F31_Mass_Python_2_Package_Removal

I have been experimenting with building Fedora 31 and it fails. If you want to see an example of what that looks like, go here (your browser should properly display this as text): https://copr-be.cloud.fedoraproject.org/results/taw/joplin/fedora-31-x86_64/01040718-joplin/builder-live.log.gz

Now, I don’t know if I grasp the reported errors at this point, but it looks like however sqllite3 is being compiled/installed seems to be a problem? Because it is calling for python 2.6.0 stuff?

I will keep investigating, but I wanted Laurent and the other folks working on Joplin to be aware that Linux distros will be moving away from this older variety of Python very soon now and there will likely be implications.

Thanks! -t

2

Joplin does not use python at all if I am not wrong.

Next year python 2 is dead and all distribution will need to support only python 3.

3

Joplin uses something that uses python. And python is used for Joplin’s build scripts. And then there is NPM which is a morass of almost unfathomable dependencies. I haven’t dug too deeply into it yet. Regardless, this is a word of warning about the impending complications that may be forthcoming for many many projects out there.

4

Joplin doesn’t directly use Python. Maybe some libs are built using Python but in that case they are the ones who will fix this kind of issue. On our side there’s no much we can do.

Based on the log it’s sqlite3 which uses Python but this is a widely used package, so for sure they’ll update it if Python 2 becomes discontinued.

5

Is there any news on this?

6

Yes. I finally got it to build for Fedora 31. Here were the issues:

  1. Python2 is still required by some of the cascading build scripts (not a Joplin issue, … well, until it is)
  2. sqlite3 is built and the OS’es deployment is ignored. I am not sure why it behaves this way, but it does.
  3. the sqlite3 version indicated in the two package.json files didn’t allow newer linuxes, like Fedora 31, to complete a compile.

What I did to get it to build on Fedora 31:

  1. Forced a python2 install and ensured it was available in the path
  2. Bumped the acceptable version of sqlite3 in the two package.json files (risky, but so far, so good) to match what the OS expects.

Note: I am not a javascript expert, trying to parse the tracebacks and find a root cause has been … challenging.

If you are a Fedora 31 user, please help me test the Joplin RPM builds. I am very confident they are fine, but I don’t currently have a desktop version of Fedora 31 available. Again, you can enable the Joplin repo via the instructions found here: https://github.com/taw00/joplin-rpm

1 Like
7

Hi Todd,

Thanks for sharing this. I will install and test today and see how we go!

1 Like
8

Hi again,

Seems to be working just fine right now. I haven’t found any issues on F31. Yay! Is it only the build that has the dependency on Python2, rather than the RPM?

If you are after any logs or other info, please let me know and I will do my best to assist. I am all in with Joplin now!

9

Hey, awesome, thanks!

Yeah, just the build requires it. No logs needed. Just let me know if you run into a problem that seems build related.

-t

10

This is what I get on Fedora 29:

Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.263N5O
+ umask 022
+ cd /builddir/build/BUILD
+ cd joplin-1.0
+ cd joplin-1.0.174
+ cd Tools
+ npm install
npm ERR! cb() never called!

npm ERR! This is an error with npm itself. Please report this error at:
npm ERR!     <https://npm.community>
11

The challenge with npm is that it spews so many errors as a matter of course I don't know what to take seriously. :wink:

The build completed though, correct? Because, I have built for RHEL 7, 8 and Fedora 29, 30, 31.

12

That was a known bug about a month ago that was fixed afterwards, so if you upgrade npm it should be fine. More info there: https://stackoverflow.com/questions/15393821/npm-err-cb-never-called

13

Yeah, I don’t see that error in my F29 builds.
Note, I use the mock build infrastructure. Every build is on a clean machine and always up-to-date.

I do see this in all the builds, but it has had no discernible impact thus far:

Compiling /builddir/build/BUILD/joplin-1.0/joplin-1.0.174/ElectronClient/app/plugins/GotoAnything.jsx...
fatal: not a git repository (or any of the parent directories): .git
Could not get git info { Error: Command failed: git rev-parse --abbrev-ref HEAD
fatal: not a git repository (or any of the parent directories): .git

    at checkExecSyncError (child_process.js:629:11)
    at execSync (child_process.js:666:13)
    at Object.<anonymous> (/builddir/build/BUILD/joplin-1.0/joplin-1.0.174/ElectronClient/app/compile-package-info.js:23:11)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:831:12)
    at startup (internal/bootstrap/node.js:283:19)
  status: 128,
  signal: null,
  output:
   [ null,
     <Buffer >,
     <Buffer 66 61 74 61 6c 3a 20 6e 6f 74 20 61 20 67 69 74 20 72 65 70 6f 73 69 74 6f 72 79 20 28 6f 72 20 61 6e 79 20 6f 66 20 74 68 65 20 70 61 72 65 6e 74 20 ... > ],
  pid: 337,
  stdout: <Buffer >,
  stderr:
   <Buffer 66 61 74 61 6c 3a 20 6e 6f 74 20 61 20 67 69 74 20 72 65 70 6f 73 69 74 6f 72 79 20 28 6f 72 20 61 6e 79 20 6f 66 20 74 68 65 20 70 61 72 65 6e 74 20 ... > }
14

I build rpms the official way using the mock building system. This implies that every build takes place in a freshly installed, clean environment, using only the packages specified in the spec file. For all packages the most recent officially released versions are used. That’s npm 6.9.0 and nodejs 10.16.3.

With these, it doesn’t build on F29.

15

Bizarre. Because I do the same. I’m unsure what we must be doing differently.

16

Can you pm me a zip with your log files (build.log, installed_pkgs.log, …)?

17

Just go to the COPR repo taw/Joplin as mentioned in the link in the beginning of the thread. All the build details are there.

Wait. Are you building using my source RPMs or using your own?

18

I use the tar.gz (joplin + contrib) and the spec file from https://github.com/taw00/joplin-rpm.

19

Hmm. Weird. I’ll look into it when I have time.
Oh. And mock has to have access to the internet. JavaScript magic Foo and all that.

20

I just duplicated your error.
You have to give build access to the internet. With these insane Javascript based builds there is no real way around it and hence, Red Hat for example, will probably never let these kinds of applications ship with the product (in particular electron based applications, but I digress). A truly proper build should never need access to the internet to complete. A source package should be self-contained. With NPM… this makes for super ugly source RPMs.

You have to edit your ~/.config/mock.cfg file and set this variable to True:

config_opts['rpmbuild_networking'] = True

Then it should build for you just fine.
-todd

Side note: I need to clean up that contrib tarball. The only thing it uses is the joplin.appdata.xml file. I really should contribute that to the joplin project directly, to be frank. Haven’t gotten around to it.