Error running `yarn install`

roman_r_m · 2024-02-29T21:01:40.420Z

I'm getting this error when running yarn install from clean repo

sharp: Detected globally-installed libvips v8.15.1
sharp: Attempting to build from source via node-gyp
sharp: Found node-addon-api
sharp: Found node-gyp version 9.4.0
sharp: See https://sharp.pixelplumbing.com/install#building-from-source
Usage Error: The nearest package directory (/home/roman/dev/joplin/packages/app-cli/node_modules/sharp) doesn't seem to be part of the project declared in /home/roman/dev/joplin.

- If /home/roman/dev/joplin isn't intended to be a project, remove any yarn.lock and/or package.json file there.
- If /home/roman/dev/joplin is intended to be a project, it might be that you forgot to list packages/app-cli/node_modules/sharp in its workspace configuration.
- Finally, if /home/roman/dev/joplin is fine and you intend packages/app-cli/node_modules/sharp to be treated as a completely separate project (not even a workspace), create an empty yarn.lock file in it.

Anyone seen this before?

I'm using node 18, yarn 3.6.4 on manjaro linux.

personalizedrefriger · 2024-03-01T04:30:43.668Z

Does it build if sharp is removed from packages/app-cli/package.json (this will break the CLI app)?

Note that sharp is also present as a dependency of lib and tools.

roman_r_m · 2024-03-01T20:29:00.692Z

personalizedrefriger:

Does it build if sharp is removed from packages/app-cli/package.json (this will break the CLI app)?

I'll give it a try but I doubt it. I didn't mention it but the output yarn install has 3 sharp-related errors:

➤ YN0009: │ sharp@npm:0.33.2 couldn't be built successfully (exit code 1, logs can be found here: /tmp/xfs-7b628c65/build.log)
➤ YN0009: │ sharp@npm:0.33.2 couldn't be built successfully (exit code 1, logs can be found here: /tmp/xfs-ce5d8c5b/build.log)
➤ YN0009: │ sharp@npm:0.33.2 couldn't be built successfully (exit code 1, logs can be found here: /tmp/xfs-8720a0a7/build.log)

The first 2 logs is the error I posted above but one for packages/app-cli and the other for packages/lib.
Interestingly, the 3rd one is completely different, some c++ compilation error

make: Entering directory '/home/roman/dev/joplin/packages/tools/node_modules/sharp/src/build'
  CC(target) Release/obj.target/nothing/../../node-addon-api/nothing.o
rm -f Release/obj.target/../../node-addon-api/nothing.a Release/obj.target/../../node-addon-api/nothing.a.ar-file-list; mkdir -p `dirname Release/obj.target/../../node-addon-api/nothing.a`
ar crs Release/obj.target/../../node-addon-api/nothing.a @Release/obj.target/../../node-addon-api/nothing.a.ar-file-list
  COPY Release/nothing.a
  TOUCH Release/obj.target/libvips-cpp.stamp
  CXX(target) Release/obj.target/sharp-linux-x64/common.o
  CXX(target) Release/obj.target/sharp-linux-x64/metadata.o
../metadata.cc: In member function ‘virtual void MetadataWorker::OnOK()’:
../metadata.cc:233:46: error: ‘NewOrCopy’ is not a member of ‘Napi::Buffer<char>’
  233 |         info.Set("exif", Napi::Buffer<char>::NewOrCopy(env, baton->exif, baton->exifLength, sharp::FreeCallback));
      |                                              ^~~~~~~~~
../metadata.cc:236:45: error: ‘NewOrCopy’ is not a member of ‘Napi::Buffer<char>’
  236 |         info.Set("icc", Napi::Buffer<char>::NewOrCopy(env, baton->icc, baton->iccLength, sharp::FreeCallback));
      |                                             ^~~~~~~~~
../metadata.cc:239:46: error: ‘NewOrCopy’ is not a member of ‘Napi::Buffer<char>’
  239 |         info.Set("iptc", Napi::Buffer<char>::NewOrCopy(env, baton->iptc, baton->iptcLength, sharp::FreeCallback));
      |                                              ^~~~~~~~~
../metadata.cc:242:45: error: ‘NewOrCopy’ is not a member of ‘Napi::Buffer<char>’
  242 |         info.Set("xmp", Napi::Buffer<char>::NewOrCopy(env, baton->xmp, baton->xmpLength, sharp::FreeCallback));
      |                                             ^~~~~~~~~
../metadata.cc:246:31: error: ‘NewOrCopy’ is not a member of ‘Napi::Buffer<char>’
  246 |           Napi::Buffer<char>::NewOrCopy(env, baton->tifftagPhotoshop,
      |                               ^~~~~~~~~
make: *** [sharp-linux-x64.target.mk:174: Release/obj.target/sharp-linux-x64/metadata.o] Error 1
make: Leaving directory '/home/roman/dev/joplin/packages/tools/node_modules/sharp/src/build'

roman_r_m · 2024-03-01T20:39:09.422Z

As expected, removing the dependency didn't help.

I'll try git bisect to see if I can pinpoint the commit that broke it for me.

roman_r_m · 2024-03-01T21:36:18.200Z

Here's the commit

2924bb5437c8b06db17e7e7c670b2cefacb6a3f5 is the first bad commit
commit 2924bb5437c8b06db17e7e7c670b2cefacb6a3f5
Author: Laurent Cozic <laurent22@users.noreply.github.com>
Date:   Wed Nov 29 15:13:57 2023 +0100

    Chore: Upgrade to sharp 0.33.0 to fix random CI failures

 packages/app-cli/package.json |   2 +-
 packages/lib/package.json     |   2 +-
 packages/tools/package.json   |   2 +-
 yarn.lock                     | 299 +++++++++++++++++++++++++++++++++++-------

not sure what to make of it

roman_r_m · 2024-03-01T21:50:24.939Z

Probably caused by this "enhancement": Enhancement: Major overhaul of installation · Issue #3750 · lovell/sharp · GitHub
No idea why I'm the only one affected apparently.

personalizedrefriger · 2024-03-01T22:00:56.193Z

Related issue:

github.com/lovell/sharp

yarn-managed monorepo, building from source, nodeLinker=node-modules: The nearest package directory doesn't seem to be part of the project

opened 10:09PM - 02 Feb 24 UTC

XilinJia

installation

in my monorepo project: ✗ yarn install YN0007: │ sharp@npm:0.33.2 must be bu…ilt because it never has been before or the last one failed YN0009: │ sharp@npm:0.33.2 couldn't be built successfully (exit code 1, logs can be found here: /tmp/xfs-4494d0f3/build.log) /tmp/xfs-4494d0f3/build.log: --------- # This file contains the result of Yarn building a package (sharp@npm:0.33.2) # Script name: install sharp: Detected globally-installed libvips v8.15.1 sharp: Attempting to build from source via node-gyp sharp: Found node-addon-api sharp: Found node-gyp version 10.0.1 sharp: See https://sharp.pixelplumbing.com/install#building-from-source [31m[1mUsage Error[22m[39m: The nearest package directory (/myProject/packages/lib/node_modules/sharp) doesn't seem to be part of the project declared in /myProject. - If /myProject isn't intended to be a project, remove any yarn.lock and/or package.json file there. - If /myProject is intended to be a project, it might be that you forgot to list packages/lib/node_modules/sharp in its workspace configuration. - Finally, if /myProject is fine and you intend packages/lib/node_modules/sharp to be treated as a completely separate project (not even a workspace), create an empty yarn.lock file in it. [1m$ [22myarn run [--inspect] [--inspect-brk] [-T,--top-level] [-B,--binaries-only] [--require #0] <scriptName> ...

roman_r_m · 2024-03-01T22:03:55.445Z

Oh, good catch! I didn't find this. And the project there is rebranded Joplin! Well, at least I'm not the only one with this problem after all.

I try the solution suggested there tomorrow.

personalizedrefriger · 2024-03-01T22:06:59.316Z

Setting the SHARP_IGNORE_GLOBAL_LIBVIPS environment variable to 1 might also be a workaround. (See sharp's build instructions).

roman_r_m · 2024-03-01T22:20:35.021Z

Thanks! This does the trick. Sort of. Not seeing any more errors from sharp.

The build stil fails though but it looks completely different:

➤ YN0000: [@joplin/app-desktop]: Error: Command failed with exit code 1: git checkout 2d814a5466604daced108331d14aedf8e8414d62
➤ YN0000: [@joplin/app-desktop]: error: Your local changes to the following files would be overwritten by checkout:
➤ YN0000: [@joplin/app-desktop]:        CHANGELOG.md
➤ YN0000: [@joplin/app-desktop]:        GENERATOR_DOC.md
➤ YN0000: [@joplin/app-desktop]:        README.md
➤ YN0000: [@joplin/app-desktop]:        package-lock.json
➤ YN0000: [@joplin/app-desktop]:        package.json
➤ YN0000: [@joplin/app-desktop]:        src/locales/de_DE.json
➤ YN0000: [@joplin/app-desktop]:        src/locales/en_US.json
➤ YN0000: [@joplin/app-desktop]:        src/manifest.json
➤ YN0000: [@joplin/app-desktop]:        src/settings.ts
➤ YN0000: [@joplin/app-desktop]:        webpack.config.js
➤ YN0000: [@joplin/app-desktop]: Please commit your changes or stash them before you switch branches.
➤ YN0000: [@joplin/app-desktop]: Aborting
➤ YN0000: [@joplin/app-desktop]:     at makeError (/home/roman/dev/joplin/packages/utils/node_modules/execa/lib/error.js:60:11)
➤ YN0000: [@joplin/app-desktop]:     at handlePromise (/home/roman/dev/joplin/packages/utils/node_modules/execa/index.js:118:26)
➤ YN0000: [@joplin/app-desktop]:     at processTicksAndRejections (node:internal/process/task_queues:95:5) {
➤ YN0000: [@joplin/app-desktop]:   shortMessage: 'Command failed with exit code 1: git checkout 2d814a5466604daced108331d14aedf8e8414d62',

laurent · 2024-03-01T22:44:36.017Z

Is that something we can integrate to our build?

personalizedrefriger · 2024-03-02T00:40:13.313Z

laurent:

Is that something we can integrate to our build?

I think so.

I tried adding node-addon-api and node-gyp as dependencies wherever we have sharp as a dependency (as recommended by the upstream issue). This helped somewhat — rather than three build failures, I get two build failures. This seems to be the behavior experienced by the sharp maintainer in the upstream issue.

I haven't tried the other suggested solution yet, which is to change nmHoistingLimits to dependencies.

roman_r_m:

The build stil fails though but it looks completely different:

That looks like the default plugins build is failing (probably here). A few lines of the logs immediately above the ones posted above could be helpful!

roman_r_m · 2024-03-02T21:13:26.873Z

personalizedrefriger:

That looks like the default plugins build is failing (probably here). A few lines of the logs immediately above the ones posted above could be helpful!

Turned out the directory with the backup plugin sources was in some invalid state (maybe as a result of running git bisect and trying to build many times from various commits). After running git reset from there I can now build the app.

Thanks for your help.