Building Application Individually

Support
1

Hello,

I am current package maintainer for Joplin on archlinux, and I am struggling to update the package.
Is it possible to install and prepare the packages in a way that they can be used?
I noticed you switched to Lerna. But id like a way to package the system and be build.

At the monment I can even build the app-cli I get this:
-> % npm install
npm ERR! code E404
npm ERR! 404 Not Found - GET https://registry.npmjs.org/@joplin%2Flib - Not found
npm ERR! 404
npm ERR! 404 '@joplin/lib@^1.0.9' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
npm ERR! 404 It was specified as a dependency of 'app-desktop'
npm ERR! 404
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/masterkorp/.npm/_logs/2020-12-06T22_01_30_673Z-debug.log

Can you guys shed any light in here ?

2

It's all in the travis file: https://github.com/laurent22/joplin/blob/dev/.travis.yml

Basically running npm i from the root and then npm run dist from the desktop package should get you there.

3

Please stop bundling cli and desktop apps. This is not a good idea. It never was. (They seldomly change version numbers at the same time.)

There should be one package for the desktop app and another for the cli app. Then you can use a meta package or group that includes both.

4

That is exactly what I am trying to do, but I can seem to build them anymore.

5

If you're getting any error, let us know. The one you got above is because you are trying to run npm i from a sub-package. Instead you should do that from the root.

6

You are right, I did a npm install from the root.

masterkorp@xps13 [02:41:24 PM] [/tmp/joplin] [dev *]
-> % ls
appveyor.yml  BUILD.md     CONTRIBUTING.md  gulpfile.js            Joplin_install_and_update.sh  LICENSE                node_modules  package-lock.json  readme     SECURITY.md           tsconfig.json
Assets        _config.yml  docs             joplin.code-workspace  lerna.json                    lint-staged.config.js  package.json  packages           README.md  tsconfig.eslint.json
masterkorp@xps13 [02:41:27 PM] [/tmp/joplin] [dev *]
-> % npm install

> root@ postinstall /tmp/joplin
> lerna bootstrap --no-ci && npm run tsc

lerna notice cli v3.22.1
lerna info versioning independent
lerna info Bootstrapping 12 packages
lerna info Installing external dependencies
lerna info Symlinking packages and binaries
lerna info lifecycle @joplin/app-clipper@1.0.8~postinstall: @joplin/app-clipper@1.0.8
lerna info lifecycle @joplin/turndown-plugin-gfm@1.0.12~postinstall: @joplin/turndown-plugin-gfm@1.0.12
lerna info lifecycle @joplin/turndown@4.0.30~postinstall: @joplin/turndown@4.0.30

> @joplin/app-clipper@1.0.8 postinstall /tmp/joplin/packages/app-clipper
> cd popup && npm install


> joplin-webclipper-popup@0.1.0 postinstall /tmp/joplin/packages/app-clipper/popup
> node postinstall.js && npm run build


> joplin-webclipper-popup@0.1.0 build /tmp/joplin/packages/app-clipper/popup
> node scripts/build.js SKIP_PREFLIGHT_CHECK

Creating an optimized production build...
Browserslist: caniuse-lite is outdated. Please run the following command: `npm update`
Compiled successfully.

File sizes after gzip:

  48.12 KB  build/static/js/2.chunk.js
  7.52 KB   build/static/js/main.chunk.js
  775 B     build/static/js/runtime-main.js
  746 B     build/static/css/main.chunk.css

The project was built assuming it is hosted at ./.
You can control this with the homepage field in your package.json.

The build folder is ready to be deployed.

Find out more about deployment here:

  bit.ly/CRA-deploy

npm WARN tsutils@3.17.1 requires a peer of typescript@>=2.8.0 || >= 3.2.0-dev || >= 3.3.0-dev || >= 3.4.0-dev || >= 3.5.0-dev || >= 3.6.0-dev || >= 3.6.0-beta || >= 3.7.0-dev || >= 3.7.0-beta but none is installed. You must install peer dependencies yourself.
npm WARN The package fs-extra is included as both a dev and production dependency.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@2.1.2 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@2.1.2: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.11 (node_modules/webpack-dev-server/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.11: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.11 (node_modules/watchpack/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.11: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.12 (node_modules/react-scripts/node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.12: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.11 (node_modules/jest-haste-map/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.11: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

audited 2131 packages in 8.648s

60 packages are looking for funding
  run `npm fund` for details

found 8629 vulnerabilities (8576 low, 33 moderate, 20 high)
  run `npm audit fix` to fix them, or `npm audit` for details

> @joplin/turndown-plugin-gfm@1.0.12 postinstall /tmp/joplin/packages/turndown-plugin-gfm
> npm run build


> @joplin/turndown-plugin-gfm@1.0.12 build /tmp/joplin/packages/turndown-plugin-gfm
> rollup -c config/rollup.config.cjs.js


src/gfm.js → lib/turndown-plugin-gfm.cjs.js...
created lib/turndown-plugin-gfm.cjs.js in 48ms

> @joplin/turndown@4.0.30 postinstall /tmp/joplin/packages/turndown
> npm run build


> @joplin/turndown@4.0.30 build /tmp/joplin/packages/turndown
> rollup -c config/rollup.config.cjs.js


src/turndown.js → lib/turndown.cjs.js...
created lib/turndown.cjs.js in 200ms
lerna info lifecycle @joplin/app-desktop@1.5.4~postinstall: @joplin/app-desktop@1.5.4
lerna info lifecycle @joplin/app-mobile@0.8.9~postinstall: @joplin/app-mobile@0.8.9

> @joplin/app-desktop@1.5.4 postinstall /tmp/joplin/packages/app-desktop
> npm run build


> @joplin/app-desktop@1.5.4 build /tmp/joplin/packages/app-desktop
> gulp build

/tmp/joplin
[14:42:22] Using gulpfile /tmp/joplin/packages/app-desktop/gulpfile.js
[14:42:22] Starting 'build'...
[14:42:22] Starting 'compileScripts'...
[14:42:22] Starting 'compilePackageInfo'...
[14:42:22] Starting 'copyPluginAssets'...
[14:42:22] Starting 'copyTinyMceLangs'...
[14:42:22] Starting 'updateIgnoredTypeScriptBuild'...
Copying to /tmp/joplin/packages/app-desktop/tools/../gui/note-viewer/pluginAssets
Copying /tmp/joplin/packages/app-desktop/tools/../../../Assets/TinyMCE/langs => /tmp/joplin/packages/app-desktop/tools/../node_modules/tinymce/langs
[14:42:22] Finished 'compileScripts' after 248 ms
[14:42:22] Finished 'compilePackageInfo' after 248 ms
[14:42:22] Finished 'updateIgnoredTypeScriptBuild' after 259 ms
Copying to /tmp/joplin/packages/app-desktop/tools/../pluginAssets
[14:42:22] Finished 'copyTinyMceLangs' after 325 ms
[14:42:22] Finished 'copyPluginAssets' after 374 ms
[14:42:22] Finished 'build' after 376 ms

> @joplin/app-mobile@0.8.9 postinstall /tmp/joplin/packages/app-mobile
> jetify && npm run build

Jetifier found 1332 file(s) to forward-jetify. Using 8 workers...

> @joplin/app-mobile@0.8.9 build /tmp/joplin/packages/app-mobile
> gulp build

[14:42:24] Using gulpfile /tmp/joplin/packages/app-mobile/gulpfile.js
[14:42:24] Starting 'build'...
[14:42:24] Starting 'buildReactNativeInjectedJs'...
[14:42:24] Finished 'buildReactNativeInjectedJs' after 3.1 ms
[14:42:24] Starting 'encodeAssets'...
[14:42:24] Finished 'encodeAssets' after 127 ms
[14:42:24] Starting 'podInstall'...
[14:42:24] Finished 'podInstall' after 385 μs
[14:42:24] Finished 'build' after 132 ms
lerna info lifecycle @joplin/fork-htmlparser2@4.1.8~prepare: @joplin/fork-htmlparser2@4.1.8

> @joplin/fork-htmlparser2@4.1.8 prepare /tmp/joplin/packages/fork-htmlparser2
> npm run build


> @joplin/fork-htmlparser2@4.1.8 build /tmp/joplin/packages/fork-htmlparser2
> tsc

lerna success Bootstrapped 12 packages

> root@ tsc /tmp/joplin
> lerna run tsc --stream --parallel

lerna notice cli v3.22.1
lerna info versioning independent
lerna info Executing command in 5 packages: "npm run tsc"
@joplin/app-mobile: > @joplin/app-mobile@0.8.9 tsc /tmp/joplin/packages/app-mobile
@joplin/app-mobile: > node node_modules/typescript/bin/tsc --project tsconfig.json
@joplin/app-desktop: > @joplin/app-desktop@1.5.4 tsc /tmp/joplin/packages/app-desktop
@joplin/app-desktop: > node node_modules/typescript/bin/tsc --project tsconfig.json
@joplin/lib: > @joplin/lib@1.0.9 tsc /tmp/joplin/packages/lib
@joplin/lib: > node node_modules/typescript/bin/tsc --project tsconfig.json
joplin: > joplin@1.5.0 tsc /tmp/joplin/packages/app-cli
joplin: > node node_modules/typescript/bin/tsc --project tsconfig.json
@joplin/renderer: > @joplin/renderer@1.0.17 tsc /tmp/joplin/packages/renderer
@joplin/renderer: > node node_modules/typescript/bin/tsc --project tsconfig.json
lerna success run Ran npm script 'tsc' in 5 packages in 12.5s:
lerna success - joplin
lerna success - @joplin/app-desktop
lerna success - @joplin/app-mobile
lerna success - @joplin/lib
lerna success - @joplin/renderer
npm WARN @octokit/plugin-request-log@1.0.2 requires a peer of @octokit/core@>=3 but none is installed. You must install peer dependencies yourself.
npm WARN eslint-plugin-react-hooks@2.5.1 requires a peer of eslint@^3.0.0 || ^4.0.0 || ^5.0.0 || ^6.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.13 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.13: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

audited 1152 packages in 44.204s

59 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

I see to gulp commands running, but can I use the code directly ? Do I need to run more commands ?
Regards,

7

To build the desktop app, you then need to run npm run dist from packages/app-desktop and it should create the app in the /dist directory.

8

Ok, I got it, got it going on the desktop app for now.

But on the cli app, there is no dist script, but from reading the source at packages/tools/releaseCli.js at you have to run npm run build.
After that trying to run main.js

-> % ./main.js
node:internal/modules/cjs/loader:922
  throw err;
  ^

Error: Cannot find module '/tmp/joplin/packages/lib/node_modules/sqlite3/lib/binding/napi-v3-linux-x64/node_sqlite3.node'
Require stack:
- /tmp/joplin/packages/lib/node_modules/sqlite3/lib/sqlite3-binding.js
- /tmp/joplin/packages/lib/node_modules/sqlite3/lib/sqlite3.js
- /tmp/joplin/packages/lib/database-driver-node.js
- /tmp/joplin/packages/lib/BaseApplication.js
- /tmp/joplin/packages/app-cli/build/app.js
- /tmp/joplin/packages/app-cli/build/main.js
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:919:15)
    at Function.Module._load (node:internal/modules/cjs/loader:763:27)
    at Module.require (node:internal/modules/cjs/loader:991:19)
    at require (node:internal/modules/cjs/helpers:92:18)
    at Object.<anonymous> (/tmp/joplin/packages/lib/node_modules/sqlite3/lib/sqlite3-binding.js:4:15)
    at Module._compile (node:internal/modules/cjs/loader:1102:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1131:10)
    at Module.load (node:internal/modules/cjs/loader:967:32)
    at Function.Module._load (node:internal/modules/cjs/loader:807:14)
    at Module.require (node:internal/modules/cjs/loader:991:19) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/tmp/joplin/packages/lib/node_modules/sqlite3/lib/sqlite3-binding.js',
    '/tmp/joplin/packages/lib/node_modules/sqlite3/lib/sqlite3.js',
    '/tmp/joplin/packages/lib/database-driver-node.js',
    '/tmp/joplin/packages/lib/BaseApplication.js',
    '/tmp/joplin/packages/app-cli/build/app.js',
    '/tmp/joplin/packages/app-cli/build/main.js'
  ]
}

I did installed all the libs and no failure seems to exist. during that process.

Regards,

9

There's no script to build the cli app directly. There's a script to package the cli app for release and upload to npm.

The sqlite3 error has come up a few times in this forum. I'll have to look for it, because I can't recall what the solution was.

10

Hello,

Just wanted to share I managed to build the application.

I created a split package, that from the same source, the source is at https://github.com/alfredopalhares/arch-pkgbuilds/blob/master/joplin/PKGBUILD

I am now looking into reducing the cli app size, its almost 700mb for now, do you guys have any recomendations?

11

Indeed it's quite large. What directory (probably in node_modules) is taking so much space?

12

Or are you accidentally packaging the desktop and mobile app too with the cli app?

13

There are the tools I package on the app-cli

-> % du -sh *
231M    app-cli
149M    fork-htmlparser2
82M     fork-sax
205M    lib
179M    renderer
123M    tools
14

IMO lerna should deduplicate the packages, so I'm not sure why these directories use so much space.

Furthermore, the cli app should only be one directory.

15

You could try to hoist the packages using lerna bootstrap --hoist but I don't know if it will make a big difference.