Help Packaging the Joplin for ArchLinux

laurent · 2022-01-08T18:01:22.679Z

Not really unfortunately because your script does a lot of things and patch stuff, so I don't really know where the problem can come from. It says that this particular fork-slug package is missing, so you'd need to check why it's not being included.

Actually you have a line that lists packages there and fork-uslug is not being included, so maybe that's where? arch-pkgbuilds/PKGBUILD at 60c7def2cb97251bd06a521a010c0aec8dc7fd3c · alfredopalhares/arch-pkgbuilds · GitHub

james-carroll · 2022-01-08T20:29:10.946Z

I'm adapting to the yarn changes too, (independent of the AUR) and uslug gives me some errors as well. For me, namely:

Error: uslug@https://github.com/laurent22/uslug.git#commit=8c12bc7678eaefa752e673ea9cfbc0b1a14d7237: EINVAL: invalid argument, rename '/tmp/xfs-5aa2219e' -> '/tmp/xfs-5aa2219e/package.tgz'

There's someone with the same issue on another package here; I'll try to find a resolution and maybe by lucky co-incidence it might help OP

laurent · 2022-01-08T21:40:19.136Z

This package used to be on a separate GitHub repo, but I've moved it to the monorepo now so it's strange you have this reference. Is it a matter of running npm run clean from the root?

james-carroll · 2022-01-08T21:55:36.282Z

It looks like my issue is probably unrelated, though it might cast some doubt on reliability of the node packages from various sources.

I'd been using Node installed as a Snap (which is actually an official release from the NodeJS foundation), and basically it just doesn't work in there. I can't figure it out yet, but frankly it's not the first time I've had extremely "minor" but crippling differences with the Snap package vs the other distributions of Node.

It worked absolutely fine for me when I'd installed Node from NodeSource · GitHub. I used to also just download the tarball release from the NodeJS site directly in the past.

It makes me wonder if there also be some subtle differences in the Arch Node package that just makes it different to other releases, for the purposes of the AUR build script, maybe downloading Node specifically from Download | Node.js - building with it, and discarding it after might produce different results.

(OFC I don't know AUR packaging guidelines so maybe they'd disapprove this)

leematos · 2022-01-09T23:58:11.871Z

I'm actually hitting a similar problem right now just trying to build dev branch on Mac for iOS debugging.

error: Error: Unable to resolve module @joplin/fork-uslug from /Users/l/joplin/packages/renderer/MdToHtml.js: @joplin/fork-uslug could not be found within the project or in these directories:
  ../renderer/node_modules
  ../../node_modules
  /Users/l/joplin/packages/app-mobile/node_modules/@joplin/fork-uslug

If you are sure the module exists, try these steps:
 1. Clear watchman watches: watchman watch-del-all
 2. Delete node_modules and run yarn install
 3. Reset Metro's cache: yarn start --reset-cache
 4. Remove the cache: rm -rf /tmp/metro-*
  34 |     source_map: require('./MdToHtml/rules/source_map').default,
  35 | };
> 36 | const uslug = require('@joplin/fork-uslug');
     |                        ^
  37 | const markdownItAnchor = require('markdown-it-anchor');
  38 | // The keys must match the corresponding entry in Setting.js
  39 | const plugins = {

For my app-mobile/node_modules/[at]joplin dir uslug isn't there:

node_modules % cd [at]joplin 
l@lees-mbp @joplin % ls -lha
total 0
drwxr-xr-x    11 l  staff   352B Jan  9 17:45 .
drwxr-xr-x  1033 l  staff    32K Jan  9 18:38 ..
lrwxr-xr-x     1 l  staff     5B Jan  9 17:45 app-mobile -> ../..
lrwxr-xr-x     1 l  staff    25B Jan  9 17:44 fork-htmlparser2 -> ../../../fork-htmlparser2
lrwxr-xr-x     1 l  staff    17B Jan  9 17:44 fork-sax -> ../../../fork-sax
lrwxr-xr-x     1 l  staff    17B Jan  9 17:44 htmlpack -> ../../../htmlpack
drwxr-xr-x   259 l  staff   8.1K Jan  9 17:45 lib
drwxr-xr-x    48 l  staff   1.5K Jan  9 17:45 renderer
drwxr-xr-x    83 l staff   2.6K Jan  9 17:45 tools
lrwxr-xr-x     1 l  staff    17B Jan  9 17:44 turndown -> ../../../turndown
lrwxr-xr-x     1 l  staff    28B Jan  9 17:44 turndown-plugin-gfm -> ../../../turndown-plugin-gfm

I looked at app-desktop/node_modules and I don't see it there either, but the desktop app builds for me now.

as a node/node packaging newb I assume there is magic here to build this type of package? (gonna go do some reading now)

laurent · 2022-01-10T01:04:08.075Z

npm run clean && yarn install from the root doesn't fix it?

leematos · 2022-01-10T02:48:13.806Z

Just tried upgrading node, making sure I'm on npm 7, and did:

npm clean
yarn install

still complained. Cleaned my iOS build dir and then

pod install

still erroring.

I'm gonna attempt a clean repo pull in a new directory tomorrow to see how that goes.

alfredopalhares · 2022-01-10T11:22:09.689Z

Cleaning the worksapce still produces the same error for me.

Is it possible that there is a reference somewhere in the code GitHub - laurent22/uslug: A permissive slug generator that works with unicode. Instead of the one in the monorepo?

Regards,
Alfredo Palhares

Daeraxa · 2022-01-11T12:12:53.992Z

Interestingly somebody raised an issue trying to build joplin-cli in a dockerfile which is also complaining about uslug using npm install-g joplin : npm install need a ssh private key · Issue #6003 · laurent22/joplin · GitHub

#6 45.39 npm ERR! code 128
#6 45.39 npm ERR! An unknown git error occurred
#6 45.39 npm ERR! command git --no-replace-objects ls-remote ssh://git@github.com/laurent22/uslug.git
#6 45.39 npm ERR! Warning: Permanently added 'github.com,20.205.243.166' (ECDSA) to the list of known hosts.
#6 45.39 npm ERR! git@github.com: Permission denied (publickey).
#6 45.39 npm ERR! fatal: Could not read from remote repository.
#6 45.39 npm ERR!
#6 45.39 npm ERR! Please make sure you have the correct access rights
#6 45.39 npm ERR! and the repository exists.

laurent · 2022-01-11T12:25:34.355Z

It's a big mystery this uslug package. How can a URL that's not referenced anywhere in the repository can still popup here and there.

roman_r_m · 2022-01-11T21:13:10.876Z

I am also seeing this trying to run Android app:

error: Error: Unable to resolve module @joplin/fork-uslug from /home/roman/dev/joplin/packages/renderer/MdToHtml.js: @joplin/fork-uslug could not be found within the project or in these directories:
  ../renderer/node_modules
  ../../node_modules
  /home/roman/dev/joplin/packages/app-mobile/node_modules/@joplin/fork-uslug

Daeraxa · 2022-01-11T21:39:56.935Z

laurent:

How can a URL that's not referenced anywhere in the repository can still popup here and there.

This might be just igorance and lack of knowledge about yarn or the project in general but there are a bunch of references to https://github.com/laurent22/uslug.git in the yarn.lock file.

jabraun · 2022-01-19T15:14:19.259Z

I am, unfortunately, not a developer, but I wanted to bump this thread in hopes that folks will continue looking for a solution. @alfredopalhares has done a lot of often-thankless work to package Joplin for the Arch ecosystem, and many of us on Arch/Manjaro depend on his efforts. Not to mention it sounds as though the issue is affecting packages for other systems, too.

laurent · 2022-01-19T15:20:06.895Z

Daeraxa:

there are a bunch of references to https://github.com/laurent22/uslug.git in the yarn.lock file.

I guess that's where it comes from then, although it's strange it goes get it from there in some cases and not others.

Daeraxa · 2022-02-04T10:39:05.747Z

Did you resolve it?
Got the same issue after I just tried building & running the android mobile app, fresh clone after an upstream fetch:

info Reloading app...
 BUNDLE  ./index.js 

error: Error: Unable to resolve module @joplin/fork-uslug from /home/dae/.local/share/Trash/files/joplin/packages/renderer/MdToHtml.js: @joplin/fork-uslug could not be found within the project or in these directories:
  ../renderer/node_modules
  ../../node_modules
  ../../../../../../../node_modules
  /home/dae/.local/share/Trash/files/joplin/packages/app-mobile/node_modules/@joplin/fork-uslug

If you are sure the module exists, try these steps:
 1. Clear watchman watches: watchman watch-del-all
 2. Delete node_modules and run yarn install
 3. Reset Metro's cache: yarn start --reset-cache
 4. Remove the cache: rm -rf /tmp/metro-*
  34 |     source_map: require('./MdToHtml/rules/source_map').default,
  35 | };
> 36 | const uslug = require('@joplin/fork-uslug');
     |                        ^
  37 | const markdownItAnchor = require('markdown-it-anchor');
  38 | // The keys must match the corresponding entry in Setting.js
  39 | const plugins = {
    at ModuleResolver.resolveDependency (/home/dae/.local/share/Trash/files/joplin/packages/app-mobile/node_modules/metro/src/node-haste/DependencyGraph/ModuleResolution.js:129:15)
    at DependencyGraph.resolveDependency (/home/dae/.local/share/Trash/files/joplin/packages/app-mobile/node_modules/metro/src/node-haste/DependencyGraph.js:288:43)
    at Object.resolve (/home/dae/.local/share/Trash/files/joplin/packages/app-mobile/node_modules/metro/src/lib/transformHelpers.js:129:24)
    at resolve (/home/dae/.local/share/Trash/files/joplin/packages/app-mobile/node_modules/metro/src/DeltaBundler/traverseDependencies.js:396:33)
    at /home/dae/.local/share/Trash/files/joplin/packages/app-mobile/node_modules/metro/src/DeltaBundler/traverseDependencies.js:412:26
    at Array.reduce (<anonymous>)
    at resolveDependencies (/home/dae/.local/share/Trash/files/joplin/packages/app-mobile/node_modules/metro/src/DeltaBundler/traverseDependencies.js:411:33)
    at processModule (/home/dae/.local/share/Trash/files/joplin/packages/app-mobile/node_modules/metro/src/DeltaBundler/traverseDependencies.js:140:31)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)

roman_r_m · 2022-02-04T10:46:47.797Z

Nope, I gave up on trying to run it.

laurent · 2022-02-04T11:15:01.347Z

Maybe try to delete the uslug entry from yarn.lock, then run yarn install again? That should get rid of any old reference and hopefully fix the issue.

roman_r_m · 2022-02-04T11:16:04.368Z

I think I did this and it didn't help. I may be wrong though as it's been a few weeks ago.

Daeraxa · 2022-02-04T11:35:51.490Z

Removed all the entries referencing uslug (assuming that was the correct thing to do) and it still failed in the same way.

"@joplin/fork-uslug@^1.0.4, @joplin/fork-uslug@workspace:packages/fork-uslug":
  version: 0.0.0-use.local
  resolution: "@joplin/fork-uslug@workspace:packages/fork-uslug"
  dependencies:
    node-emoji: 1.11.0
    should: ">= 0.2.1"
    unorm: ">= 1.0.0"
  languageName: unknown
  linkType: soft

"@joplin/lib@^2.6.3, @joplin/lib@workspace:packages/lib, @joplin/lib@~2.7":
  version: 0.0.0-use.local
  resolution: "@joplin/lib@workspace:packages/lib"
  dependencies:
...
    "@joplin/fork-uslug": ^1.0.4

"@joplin/renderer@^2.6.3, @joplin/renderer@workspace:packages/renderer, @joplin/renderer@~2.7":
  version: 0.0.0-use.local
  resolution: "@joplin/renderer@workspace:packages/renderer"
  dependencies:
...
    "@joplin/fork-uslug": ^1.0.4

Also removed the ones referencing https://github.com/laurent22/uslug, those ones didn't return to the lock file after I removed them and re-ran yarn install

laurent · 2022-02-06T16:10:50.801Z

For mobile I've just remembered what needs to be done: In "metro.config.js", the path to any external module (outside of /app-mobile) needs to be added so that the Metro bundler can find them. I'll make the change soon as part of another feature.

Basically Metro can only find files under app-mobiles/node_modules by default, so any other source needs to be added to metro.config.js.