Get help with testing, discuss unit testing strategies etc.


Post by stenjo »

Curious if anyone can point me to good examples of github actions scripts for running unit tests using siesta?
We have a monolithic repository with several sencha-based applications with unit tests that we would like to have run upon push and merge. Difficult to find good examples of that around.


Post by nickolay »

At Bryntum we use TeamCity plugin for testing the pull requests, but any other option will work too. For example this action: https://github.com/marketplace/actions/github-action-tester

There are also git hooks, which are the simplest option I think.


Post by stenjo »

Thanks for the reply. We currently are using TeamCity as well - works great.
We're looking at ways of getting possible errors even more "in our face" through github actions as that will allow us to run the tests also when we create a PR - not waiting for merge.
GitHub Actions is a fairly new technology so I realize this may be some groundbreaking work on our part.
Our current approach is to use what is running on TC and modify it to run on GitHub Actions as well.
I'll look into your suggestion


Post by stenjo »

To let you know where we are at the moment: We're trying to run the following script as a github action:

name: Siesta tests

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  build:

runs-on: windows-latest
defaults: 
  run:
    working-directory: ./ProjectPlannerUI
    shell: pwsh

strategy:
  matrix:
    node-version: [10.x]
    # See supported Node.js release schedule at https://nodejs.org/en/about/releases/

steps:
- uses: actions/checkout@v2
- name: Use Node.js ${{ matrix.node-version }}
  uses: actions/setup-node@v1
  with:
    node-version: ${{ matrix.node-version }}
    
- name: Install npm
  run: npm install
  
- name: Run tests
  run: ../sdk/siesta/bin/webdriver http://127.0.0.1/tests/unittests-no-ui.html?onserver=1 --browser chrome --pause 0 --headless
  working-directory: ./ProjectPlannerUI/tests

At some point in the steps, I (probably) need to get a webserver up and running with the root in the tests subdir. I am not able to find out how to make that happen.


Post by nickolay »

Where this script is physically executed? On your CI machine?

You don't need webserver with the root in the tests subdir, the web root can be any directory. You only need to know what is the "web" path to the tests dir and you point the Siesta launcher to that path.


Post by stenjo »

Thanks again. Getting further - babysteps.
GitHub Actions sets up a container - in my case running the latest version of ubuntu.
I've managed to copy files from my test folder onto the web-root (document-root) of my localhost on this container, and verify that I have access to and can run the web-files when needed.
Trying to install sencha-cmd via npm on the same container, getting:

##[debug]Loading env
Run npm install -g sencha-cmd
  npm install -g sencha-cmd
  shell: /usr/bin/bash -e {0}
  env:
    TEST_BASE_URL: http://127.0.0.1:80
    JAVA_HOME: /opt/hostedtoolcache/Java_Zulu_jdk/12.0.2-3/x64
##[debug]/usr/bin/bash -e /home/runner/work/_temp/ad0d563c-d6e8-4977-8b34-01a36cd25132.sh
+ sencha-cmd@0.1.4
added 2 packages from 2 contributors in 0.281s
##[debug]Finishing: Install sencha

which looks good. Next step I'm trying to run sencha command for building my app:

##[debug]Loading env
Run sencha app build
  sencha app build
  shell: /usr/bin/bash -e {0}
  env:
    TEST_BASE_URL: http://127.0.0.1:80
    JAVA_HOME: /opt/hostedtoolcache/Java_Zulu_jdk/12.0.2-3/x64
##[debug]/usr/bin/bash -e /home/runner/work/_temp/034d418f-8e00-4aea-88c9-bd14eede5bd1.sh
/home/runner/work/_temp/034d418f-8e00-4aea-88c9-bd14eede5bd1.sh: line 1: sencha: command not found
Error: Process completed with exit code 127.
##[debug]Finishing: Build

This is obviously failing out of not finding the sencha command somewhere. Am I missing a path?

Also when running the webrunner test, it seems not to be able to find the path of java, even though Java is successfully installed:

##[debug]Loading env
Run java -version
  java -version
  shell: /usr/bin/bash -e {0}
  env:
    TEST_BASE_URL: http://127.0.0.1:80
    _JAVA_OPTIONS: -Xms512m -Xmx1512m -XX:-UseGCOverheadLimit
    JAVA_HOME: /opt/hostedtoolcache/Java_Zulu_jdk/12.0.2-3/x64
##[debug]/usr/bin/bash -e /home/runner/work/_temp/771f3c7a-36f7-4731-90d2-3f2f876e5312.sh
Picked up _JAVA_OPTIONS: -Xms512m -Xmx1512m -XX:-UseGCOverheadLimit
openjdk version "12.0.2" 2019-07-16
OpenJDK Runtime Environment Zulu12.3+11-CA (build 12.0.2+3)
OpenJDK 64-Bit Server VM Zulu12.3+11-CA (build 12.0.2+3, mixed mode, sharing)
##[debug]Finishing: Run java -version

Failing test run:

##[debug]Loading env
Run ../sdk/siesta/bin/webdriver http://127.0.0.1:80/tests/unittests-no-ui.html?onserver=1 --browser chrome --pause 0 --headless
  ../sdk/siesta/bin/webdriver http://127.0.0.1:80/tests/unittests-no-ui.html?onserver=1 --browser chrome --pause 0 --headless
  shell: /usr/bin/bash -e {0}
  env:
    TEST_BASE_URL: http://127.0.0.1:80
    _JAVA_OPTIONS: -Xms512m -Xmx1512m -XX:-UseGCOverheadLimit
    JAVA_HOME: /opt/hostedtoolcache/Java_Zulu_jdk/12.0.2-3/x64
##[debug]/usr/bin/bash -e /home/runner/work/_temp/dc6d6fef-97a0-464d-b01a-22fb6034e99a.sh
#!/bin/bash -xv

# get the directory in which the script reside
DIR="$( cd "$( dirname "$0" )" && pwd )"
+++ dirname ../sdk/siesta/bin/webdriver
++ cd ../sdk/siesta/bin
++ pwd
+ DIR=/home/runner/work/wellcom-mono/wellcom-mono/ProjectPlannerUI/sdk/siesta/bin

which java > /dev/null 2>&1
+ which java

if [ "$?" -eq "1" ]; then
Error: o '[ERROR]: Java executable (`java`) is not installed or not available in the PATH'
    exit 9
fi
+ '[' 0 -eq 1 ']'

if [[ $OSTYPE =~ "darwin" ]]; then
    NODE_BIN_PATH="macos"
elif [[ `expr match $(uname -m) ".*64"` == "0" ]]; then
    NODE_BIN_PATH="linux32"
else
    NODE_BIN_PATH="linux64"
fi
+ [[ linux-gnu =~ darwin ]]
+++ uname -m
++ expr match x86_64 '.*64'
+ [[ 6 == \0 ]]
+ NODE_BIN_PATH=linux64

if [[ -z "$STDERRLOG" ]]; then
    STDERRLOG="$DIR/webdriver.log"
fi
+ [[ -z '' ]]
+ STDERRLOG=/home/runner/work/wellcom-mono/wellcom-mono/ProjectPlannerUI/sdk/siesta/bin/webdriver.log

"$DIR/binary/nodejs/$NODE_BIN_PATH/node" "$DIR/webdriver-launcher.js" "$DIR" "NOT_USED" "$@" 2>"$STDERRLOG"
+ /home/runner/work/wellcom-mono/wellcom-mono/ProjectPlannerUI/sdk/siesta/bin/binary/nodejs/linux64/node /home/runner/work/wellcom-mono/wellcom-mono/ProjectPlannerUI/sdk/siesta/bin/webdriver-launcher.js /home/runner/work/wellcom-mono/wellcom-mono/ProjectPlannerUI/sdk/siesta/bin NOT_USED 'http://127.0.0.1:80/tests/unittests-no-ui.html?onserver=1' --browser chrome --pause 0 --headless
Error: Process completed with exit code 127.
##[debug]Finishing: Run tests on ProjectPlannerUI

Post by nickolay »

Hard to say what happens, seems something with the PATH environment variable. The best way to debug is to open a shell into the container, using the same user, and see how it is set there.


Post Reply