As you know, I have been building Joplin RPMs for Fedora, CentOS, RHEL, and OpenSUSE for some time. Years now. I generally have no great issues even though node can be super-challenging since there are so many moving parts. So, kudos for you guys in keeping it all together.
Anyway. That stopped with release 2.3.3 and now 2.3.5. All builds began failing for me.
The first change was I had to npm install npm on most versions of the various OSes because they mostly all default to v6. Alas, the filelockVersion in package-lock.json changed (because the format of the file changed with v7). OK. No big deal, but I have no idea if the follow-on issue(s) is(are) related.
I need help understanding the preferred nodejs versions you are using to build successfully. I get this error ... and I have no understanding why. Is that an npm version issue? A nodejs version issue? A node-gyp version issue? A python version issue? Or just a bug? Any thoughts?
lerna notice cli v3.22.1
lerna info versioning independent
lerna info Bootstrapping 15 packages
lerna info Installing external dependencies
lerna ERR! npm install exited 1 in '@joplin/tools'
lerna ERR! npm install stderr:
npm WARN deprecated urix@0.1.0: Please see https://github.com/lydell/urix#deprecated
npm WARN deprecated har-validator@5.1.3: this library is no longer supported
npm WARN deprecated resolve-url@0.2.1: https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated debug@3.2.6: Debug versions >=3.2.0 <3.2.7 || >=4 <4.3.1 have a low-severity ReDos regression when used in a Node.js environment. It is recommended you upgrade to 3.2.7 or 4.3.1. (https://github.com/visionmedia/debug/issues/797)
npm WARN deprecated chokidar@2.1.8: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details.
npm WARN deprecated request@2.88.0: request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated node-pre-gyp@0.11.0: Please upgrade to @mapbox/node-pre-gyp: the non-scoped node-pre-gyp package is deprecated and only the @mapbox scoped package will recieve updates in the future
npm WARN deprecated tar@2.2.2: This version of tar is no longer supported, and will not receive security updates. Please upgrade asap.
npm ERR! code 1
npm ERR! path /builddir/build/BUILD/joplin-2.3/joplin-2.3.5/packages/tools/node_modules/sharp
npm ERR! command failed
npm ERR! command sh -c (node install/libvips && node install/dll-copy && prebuild-install --runtime=napi) || (node-gyp rebuild && node install/dll-copy)
npm ERR! info sharp Downloading https://github.com/lovell/sharp-libvips/releases/download/v8.9.1/libvips-8.9.1-linux-x64.tar.gz
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@3.8.0
npm ERR! gyp info using node@14.17.0 | linux | x64
npm ERR! gyp ERR! configure error
npm ERR! gyp ERR! stack Error: Command failed: /usr/bin/python3 -c import sys; print "%s.%s.%s" % sys.version_info[:3];
npm ERR! gyp ERR! stack File "<string>", line 1
npm ERR! gyp ERR! stack import sys; print "%s.%s.%s" % sys.version_info[:3];
npm ERR! gyp ERR! stack ^
npm ERR! gyp ERR! stack SyntaxError: invalid syntax
npm ERR! gyp ERR! stack
npm ERR! gyp ERR! stack at ChildProcess.exithandler (child_process.js:319:12)
npm ERR! gyp ERR! stack at ChildProcess.emit (events.js:376:20)
npm ERR! gyp ERR! stack at maybeClose (internal/child_process.js:1055:16)
npm ERR! gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:288:5)
npm ERR! gyp ERR! System Linux 5.11.18-300.fc34.x86_64
npm ERR! gyp ERR! command "/usr/bin/node" "/builddir/build/BUILD/joplin-2.3/joplin-2.3.5/packages/tools/node_modules/.bin/node-gyp" "rebuild"
npm ERR! gyp ERR! cwd /builddir/build/BUILD/joplin-2.3/joplin-2.3.5/packages/tools/node_modules/sharp
npm ERR! gyp ERR! node -v v14.17.0
npm ERR! gyp ERR! node-gyp -v v3.8.0
npm ERR! gyp ERR! not ok
Any help is appreciated. A lot of people use these RPMs. They haven't noticed that I have not made them available yet.
As far as I'm aware, Joplin actually requires specifically Python 2 to build (or for me it has anyway).
For my builds, setting Python 3 modules via $PYTHONPATH broke it, and my build packages specifically mentions Python 2, so I assume past me had reasons for it.
I wonder if there's some confusion caused by having a python binary directly.
I think Debian/Ubuntu decided to ship without a python binary and use exclusively python3 in the default installation, leaving python implicitly Python 2. I'm unsure with FedorySuseCentEnterprise editions, but I could image if anything relied on #!/usr/bin/env python hashbangs or similar, it could cause differing results if python == python3
Indeed. Some scripts call, for example, python ... no version specified. On most modern OSes, that will be python 3. On RHEL 8 that was python 2. For the SQLite portion of the build, it called python, but expected python 3. So, for RHEL 8, I symlinked /usr/bin/python3 to $HOME/.local/bin/python. Kludgey and terrible, but worked. Now, there seems to be some other novelty going on within the builds that ... well ... ugh. Python 2 is dead and buried. It's been end-of-lifed for a good long while now. So, if this other script is requiring python 2 and then yet another wants python 3. I dunno. We'll see. I haven't been this stymied in a very long time.
So I've just experimented on my end, based on Ubuntu 20.04, without the Python 2 environment (and hence no python binary in $PATH), I got the following
npm ERR! gyp ERR! configure error
npm ERR! gyp ERR! stack Error: Can't find Python executable "python", you can set the PYTHON env variable.
npm ERR! gyp ERR! stack at PythonFinder.failNoPython (/root/parts/joplin/build/packages/tools/node_modules/node-gyp/lib/configure.js:484:19)
npm ERR! gyp ERR! stack at PythonFinder.<anonymous> (/root/parts/joplin/build/packages/tools/node_modules/node-gyp/lib/configure.js:406:16)
npm ERR! gyp ERR! stack at F (/root/parts/joplin/build/packages/tools/node_modules/node-gyp/node_modules/which/which.js:68:16)
npm ERR! gyp ERR! stack at E (/root/parts/joplin/build/packages/tools/node_modules/node-gyp/node_modules/which/which.js:80:29)
npm ERR! gyp ERR! stack at /root/parts/joplin/build/packages/tools/node_modules/node-gyp/node_modules/which/which.js:89:16
npm ERR! gyp ERR! stack at /root/parts/joplin/build/packages/tools/node_modules/isexe/index.js:42:5
npm ERR! gyp ERR! stack at /root/parts/joplin/build/packages/tools/node_modules/isexe/mode.js:8:5
npm ERR! gyp ERR! stack at FSReqCallback.oncomplete (node:fs:198:21)
npm ERR! gyp ERR! System Linux 5.8.0-63-generic
npm ERR! gyp ERR! command "/snap/node/5252/bin/node" "/root/parts/joplin/build/packages/tools/node_modules/.bin/node-gyp" "rebuild"
npm ERR! gyp ERR! cwd /root/parts/joplin/build/packages/tools/node_modules/sharp
npm ERR! gyp ERR! node -v v16.7.0
npm ERR! gyp ERR! node-gyp -v v3.8.0
npm ERR! gyp ERR! not ok
This was a pretty huge hint, so I set $PYTHON = python3, and then I got the same error as you did above. I tried again with both Python3 and Python2 in the environment, but `PYTHON=python", and in this situation the build worked properly.
So it seems to me, Joplin does have a strict build dependency on Python 2 which might be going un-noticed generally for the time being, but probably wants looking at given as you say, it's well past being EOL.
For the time being, it looks like you could set the PYTHON variable to point to a sideloaded Python2 installation and maybe it'll fix this for you, or potentially modify $PATH so that python == python2 for the Joplin build specifically. It's not the mix of Py2 and 3 generally that is breaking it, but probably just an odd instance of something having a specific dependency on Py2 and assuming it's simply python.
There is some deep magic going on the the RPM build environment. /usr/bin/python3 gets forced by "something" no matter what I do. Outside of RPM, sure ... no problem. But within, no. And Red Hat and Fedora spout in 1-zillion places: "Thou shalt not build with python2 anymore." Grr. Python 2 hasn't been used in any of the Joplin builds for a long time. Maybe a year or more. Ugh.
This may be the end of my maintenance of this RPM build set. A lot of folks use it, unfortunately. I'm tired of fighting nuances to language versions, and that one script called by a script within a script within some node tree of depends ... and twitchy build environments. Maybe for a short while I will just repackage the upstream binaries into RPMs. I hate to do that (not native built) but ... whatever.
I'm tired and grumpy. Maybe I will feel differently tomorrow. G'night.
More that, python 2 causes issues. Heck, OS venders don't even ship the full stack for that version. It's node module sharp (at least, I think that is the cuprit) that is triggering this. When I get back to a computer, I'm going to see if there is a newer version or some such that can be used.
UPDATE1: No ... sharp-libvips seems to not be where that python2 call is coming from.
The emails are starting to roll in noting an incompatibility between desktop and mobile. I will probably have to emergency package with the upstream binaries and then try to solve this another time.
This issue makes me realise that sync target upgrades will always be quite tricky. Even if I release all the apps at the same time, there are plenty of non-official channels that are updated with a delay.
I wonder if there would be a way to notify all package maintainers in advance, and then wait for them to be ready before doing the release. I guess just a thread on the forum wouldn't quite work since not everyone will read it. @t0dd, @james-carroll, do you have any suggestion on how I could let you know about these upgrades?
I use (at work) s/w versioning of major.minor.patch.githash (x.y.x.a) scheme. for me, the simplest approach would be to consider any sync target upgrades as a major change so the major version (x) would be incremented.
I don't think it's unreasonable to ask all the third party maintainers to make forum accounts, put them all in a group, and ping them once a year. If they're unwilling to put in the effort there, IMO they're treating the package manager responsibility poorly.
I know generally speaking the updates aren't usually incompatible in the way sync-target upgrades have been, but especially with Joplin Cloud (hopefully?) becoming one of the main ways of funding the project, it becomes more serious to consider that you're actually representing someone elses brand and work, and potentially, damaging someone elses brand and work.
I'm perfectly aware the majority of users of the snap for example can't usually tell it's a third party package, which is why I actually specifically come on and try to root out mistakes or problems it's creating uniquely, because it actually presents itself as Joplin Desktop. If they don't want to make themselves available like that, they should call themselves Joplisn't instead.
Personally however, I don't find it too hard to go through every project I'm partially responsible for and keep myself up to date with what's going on, so I don't think I've ever been more than a few hours out of sync except where I might be particularly busy on the day. It isn't that much to ask for, and CI systems exist for a reason anyway.
I'm pretty good about keeping up myself, but sometimes even I am AFK for a month occasionally (I once took 4 months off from computers, but that was years ago).
I package element (chat app) as well. They have a private chat room just for package maintainers. I suppose, like James said, you could identify those who do stuff like I do and have us all in one chat room, or private forum channel, or something, and just ping us when we need to know something particularly important. I have the github release page RSS feed tied to a chat channel and I watch for every release. But that is just me.
Maybe just gather our email addresses and send us an IMPORTANT!!! subject-lined email? Dunno.
UPDATE: I successfully built packages using the upstream binary. I can't seem to solve this particular issue, but at least my userbase is not dead in the water.
Thanks for the feedback. I think as a first step I'll make a list of third-party packages with point of contact so that if such situation happens it will be easier to reach out. I think the F-Droid and Chocolatey packages are fully automated but not sure about the Homebrew one in particular.
@t0dd, @james-carroll, how about your package? Is it published automatically when a new version is released or do you need to manually do it?
I could automate it the entire way but I like seeing the patch notes and doing some manual testing first, so it's been manual the past few months. It is probably overkill though, the builds have been reliable basically since the start aside from some odd changes to the build environment.
But I am considering arm64 builds and as long as GitHub doesn't have free runners for arm64, that adds some restrictions on my ability to automate the process.
