Integration testing for joplin

Hi,
I was thinking about creating an integration testing plan for Joplin as in the GSoC idealist. Here is my original thought:

Framework

I choose Puppeteer and Jest as the testing framework for several reasons:

  1. Compared to other end-to-end testing frameworks
    a. Spectron: is now deprecated, and there are no active maintainers.
    b. Playwright: support electron as an experimental feature, tested not working for joplin now as it cannot launch Joplin properly (something is forbidding Joplin to render its content).
  2. Jest is currently the unit test runner framework for Joplin, developers in this community are more familiar with it.

Testing Workflow

  1. Build packages/app-desktop by running yarn run dist.
  2. Run the built binary file by spawn() in node.js modules child_process with flags --remote-debugging-port.
  3. Connect puppeteer-core to the --remote-debugging-port and get the app and page instance.
  4. Run tests written in jest and get results.
  5. Close app by page.close().

I'm also going to write a script to automatically run tests in CI.

Testing Plan

I plan to test the features in the Markdown editor, including all the features presents on the toolbar. Using a predefined markdown text to see if all the text is correctly rendered.

Here is an example for testing adding a new notebook by pressing the left sidebar add icon:

describe('App', () => {
	test('Create new notebook', async () => {
		const createNoteBtn = await page.waitForSelector('.sc-AxirZ.sc-AxmLO.jqgJbV.sc-fzozJi.sc-fzqBkg.jIUOKF');
		await createNoteBtn.click();
		await page.waitForSelector('.sc-fzqMAW.dPloZt');
		await (await page.$('.sc-fzqMAW.dPloZt')).type('Hello World');
		await page.click('.dialog-content+[style="text-align: right; margin-top: 10px;"]>button');
		const expected = ['Hello World'];
		const titleList = [];
		await sleep(3000);
		const titleArr = await page.$$('.title');
        for (let i = 0; i < titleArr.length; i++) {
                titleArr[i].getProperty('innerText').then((text) => {
				titleList.push(text._remoteObject.value);
		        });
        }
		expect(titleList).toEqual(expect.arrayContaining(expected));

	});
});

Test result:
image

Looking forward to your suggestions!

1 Like

That's a very cool example! One thing that will need to be taken care of for this is the element naming. Currently element names (like .sc-AxirZ...) are not stable and can't be used for this level of testing (or styling either).

Thank you CalebJohn! I guess I can switch to using combinators to select elements since many of the elements don't have a unique attribute or the class name is unstable?

I think we would give them stable class names (or IDs) for this to work well.

1 Like