Get help with testing, discuss unit testing strategies etc.


Post by steve.schreiner »

Please add MacOS support for the xvfb command line switch :)

Post by nickolay »

We hear you :)

Post by steve.schreiner »

Is there a way to do this manually on my macbook before it is supported by you guys?

p.s. Will you be at SenchaCon?

Post by nickolay »

It should work already, if you have a functional "xfvb-run" in your system. "xvfb-run" is a wrapper around "xvfb" which does some additional work, like finding a free desktop number etc. We've actually forked the "xvfb-run", to fix some issues in it, the remaining piece is to figure out for what reason it uses "xauth" and "mcookie" commands. Probably those are for some special cases, which we don't care of. You can check the "bin/xvfb/xvfb-run-my"..

Post by nickolay »

Yes, me and Mats will be at Sencha Con. You going too?

Post by steve.schreiner »

Yes, I will make sure to stop by your booth and introduce myself.

Post by nickolay »

Great, see you there then!

Post by steve.schreiner »

So I downloaded and installed https://www.xquartz.org/index.html and I can see when I run "which xvfb" it outputs: /opt/X11/bin/xvfb

Do I need to start the server or do anything to get it to work with the switch below?
siesta-4.2.0/bin/webdriver https://10.10.X.X/siesta-tests.html?test=login --report-file=siesta-results --report-format=HTML --width=1600 --xvfb --debug

Post by nickolay »

You need to modify the "bin/xvfb/xvfb-run-my" file

Replace the
MCOOKIE=$(mcookie)
with the
# Get the cookie to use.
set +e
MCOOKIE=$(mcookie 2>/dev/null)

# If the mcookie utility is not installed, simulate it.

if [ $? -ne 0 ]; then
   #
   # Set the random device to /dev/random if you need very secure 
   # random numbers. Otherwise, /dev/urandom will be fine.
   # 

   RANDOM_DEVICE=/dev/urandom

   MCOOKIE=$(od -X -A n -N 16 $RANDOM_DEVICE | tr -d '\011\040')
fi
set -e
Then you just launch the command with --xvfb switch. If it starts ok and you see the normal test output, try adding --max-workers :) Please let me know how it works.

Post by steve.schreiner »

Thanks. I tried that but it still launches the browser:
siesta-4.2.0/bin/webdriver https://10.10.X.X/siesta-tests.html?test=login --report-file=siesta-results --report-format=HTML --width=1600 --xvfb --debug
[DEBUG] Dispatcher start
[DEBUG] Runner setup: a Siesta.Launcher.Runner.WebDriverNodeJS
[DEBUG] Dispatcher setup starting
[DEBUG] Trying to create a WebDriver instance for browser: chrome
[DEBUG] WebDriver instantiated successfully
[DEBUG] PageId: 1, method : getConfigInfo starting, args size: 16
[DEBUG] PageId: 1, method : getConfigInfo, took 0.008s, result size : 500
[DEBUG] Dispatcher setup completed, launching the suite
Launching test suite, OS: MacOS, browser: Chrome 54.0.2840.71
[DEBUG] Launch runner, pagesCount: 1, max : 1, pages left: 1
[DEBUG] PageId: 1, method : launchAutomatedTests starting, args size: 154
[DEBUG] PageId: 1, method : launchAutomatedTests, took 0.011s, result size : 4
[DEBUG] Page polling has started
[DEBUG] PageId: 1, method : getAutomationState starting, args size: 2
[DEBUG] PageId: 1, method : getAutomationState, took 0.011s, result size : 118
[DEBUG] PageId: 1, method : getAutomationState starting, args size: 2
[DEBUG] PageId: 1, method : getAutomationState, took 5.823s, result size : 1050
[DEBUG] PageId: 1, method : getAutomationState starting, args size: 2
[DEBUG] PageId: 1, method : getAutomationState, took 5.425s, result size : 273
[DEBUG] PageId: 1, method : getAutomationState starting, args size: 2
[DEBUG] PageId: 1, method : getAutomationState, took 0.008s, result size : 1054
[PASS]  test/login/200_login.t.js
[DEBUG] Recevied results for all tests in the suite, proceeding to finalization
2 passed, 0 failed assertions took 19.892s to complete
[DEBUG] Generating reports (if any)
[DEBUG] Reports processed (if any)
[DEBUG] Page close started: 1
xvfb-run-my file:
#!/bin/sh

# This script starts an instance of Xvfb, the "fake" X server, runs a command
# with that server available, and kills the X server when done.  The return
# value of the command becomes the return value of this script, except in cases
# where this script encounters an error.
#
# If anyone is using this to build a Debian package, make sure the package
# Build-Depends on xvfb and xauth.

set -e

# allow settings to be updated via environment
: "${xvfb_lockdir:=$HOME/.xvfb-locks}"
: "${xvfb_display_min:=99}"
: "${xvfb_display_max:=599}"

# assuming only one user will use this, let's put the locks in our own home directory
# avoids vulnerability to symlink attacks.
mkdir -p -- "$xvfb_lockdir" || exit

PROGNAME=xvfb-run
SERVERNUM=99
AUTHFILE=
ERRORFILE=/dev/null
XVFBARGS="-screen 0 1280x960x16"
LISTENTCP="-nolisten tcp"
XAUTHPROTO=.

# Query the terminal to establish a default number of columns to use for
# displaying messages to the user.  This is used only as a fallback in the event
# the COLUMNS variable is not set.  ($COLUMNS can react to SIGWINCH while the
# script is running, and this cannot, only being calculated once.)
DEFCOLUMNS=$(stty size 2>/dev/null | awk '{print $2}') || true
if ! expr "$DEFCOLUMNS" : "[[:digit:]]\+$" >/dev/null 2>&1; then
    DEFCOLUMNS=80
fi

# Display a message, wrapping lines at the terminal width.
message () {
    echo "$PROGNAME: $*" | fmt -t -w ${COLUMNS:-$DEFCOLUMNS}
}

# Display an error message.
error () {
    message "error: $*" >&2
}

# Display a usage message.
usage () {
    if [ -n "$*" ]; then
        message "usage error: $*"
    fi
    cat <<EOF
Usage: $PROGNAME [OPTION ...] COMMAND
Run COMMAND (usually an X client) in a virtual X server environment.
Options:
-a        --auto-servernum          try to get a free server number, starting at
                                    --server-num
-e FILE   --error-file=FILE         file used to store xauth errors and Xvfb
                                    output (default: $ERRORFILE)
-f FILE   --auth-file=FILE          file used to store auth cookie
                                    (default: ./.Xauthority)
-h        --help                    display this usage message and exit
-n NUM    --server-num=NUM          server number to use (default: $SERVERNUM)
-l        --listen-tcp              enable TCP port listening in the X server
-p PROTO  --xauth-protocol=PROTO    X authority protocol name to use
                                    (default: xauth command's default)
-s ARGS   --server-args=ARGS        arguments (other than server number and
                                    "-nolisten tcp") to pass to the Xvfb server
                                    (default: "$XVFBARGS")
EOF
}

# Find a free server number by looking at .X*-lock files in /tmp.
find_free_servernum() {
    i=$xvfb_display_min     # minimum display number
    while [ $i -lt $xvfb_display_max ]; do
        if [ -f "/tmp/.X$i-lock" ]; then                # still avoid an obvious open display
            i=$(($i + 1))
            continue
        fi
        exec 5>"$xvfb_lockdir/$i" || continue           # open a lockfile
        if flock -x -n 5; then                          # try to lock it
            SERVERNUM=$i
            break
        fi
        i=$(($i + 1))
    done
}

# Clean up files
clean_up() {
    if [ -n "$SERVERNUM" ]; then
        rm "$xvfb_lockdir/$SERVERNUM"
    fi

    if [ -e "$AUTHFILE" ]; then
        XAUTHORITY=$AUTHFILE xauth remove ":$SERVERNUM" >>"$ERRORFILE" 2>&1
    fi
    if [ -n "$XVFB_RUN_TMPDIR" ]; then
        if ! rm -r "$XVFB_RUN_TMPDIR"; then
            error "problem while cleaning up temporary directory"
            exit 5
        fi
    fi
    if [ -n "$XVFBPID" ]; then
        kill "$XVFBPID" >>"$ERRORFILE" 2>&1
    fi
}

# tidy up after ourselves
trap clean_up EXIT

# Parse the command line.
ARGS=$(getopt --options +ae:f:hn:lp:s:w: \
       --long auto-servernum,error-file:,auth-file:,help,server-num:,listen-tcp,xauth-protocol:,server-args:,wait: \
       --name "$PROGNAME" -- "$@")
GETOPT_STATUS=$?

if [ $GETOPT_STATUS -ne 0 ]; then
    error "internal error; getopt exited with status $GETOPT_STATUS"
    exit 6
fi

eval set -- "$ARGS"

while :; do
    case "$1" in
        -a|--auto-servernum) find_free_servernum; AUTONUM="yes" ;;
        -e|--error-file) ERRORFILE="$2"; shift ;;
        -f|--auth-file) AUTHFILE="$2"; shift ;;
        -h|--help) SHOWHELP="yes" ;;
        -n|--server-num) SERVERNUM="$2"; shift ;;
        -l|--listen-tcp) LISTENTCP="" ;;
        -p|--xauth-protocol) XAUTHPROTO="$2"; shift ;;
        -s|--server-args) XVFBARGS="$2"; shift ;;
        -w|--wait) shift ;;
        --) shift; break ;;
        *) error "internal error; getopt permitted \"$1\" unexpectedly"
           exit 6
           ;;
    esac
    shift
done

if [ "$SHOWHELP" ]; then
    usage
    exit 0
fi

if [ -z "$*" ]; then
    usage "need a command to run" >&2
    exit 2
fi

if ! which xauth >/dev/null; then
    error "xauth command not found"
    exit 3
fi

# If the user did not specify an X authorization file to use, set up a temporary
# directory to house one.
if [ -z "$AUTHFILE" ]; then
    XVFB_RUN_TMPDIR="$(mktemp -d -t $PROGNAME.XXXXXX)"
    # Create empty file to avoid xauth warning
    AUTHFILE=$(tempfile -n "$XVFB_RUN_TMPDIR/Xauthority")
fi

# Start Xvfb.
# Get the cookie to use.
set +e
MCOOKIE=$(mcookie 2>/dev/null)

# If the mcookie utility is not installed, simulate it.

if [ $? -ne 0 ]; then
   #
   # Set the random device to /dev/random if you need very secure
   # random numbers. Otherwise, /dev/urandom will be fine.
   #

   RANDOM_DEVICE=/dev/urandom

   MCOOKIE=$(od -X -A n -N 16 $RANDOM_DEVICE | tr -d '\011\040')
fi
set -e
tries=10
while [ $tries -gt 0 ]; do
    tries=$(( $tries - 1 ))
    XAUTHORITY=$AUTHFILE xauth source - << EOF >>"$ERRORFILE" 2>&1
add :$SERVERNUM $XAUTHPROTO $MCOOKIE
EOF
    # handle SIGUSR1 so Xvfb knows to send a signal when it's ready to accept
    # connections
    trap : USR1
    (trap '' USR1; exec Xvfb ":$SERVERNUM" $XVFBARGS $LISTENTCP -auth $AUTHFILE >>"$ERRORFILE" 2>&1) &
    XVFBPID=$!

    wait || :
    if kill -0 $XVFBPID 2>/dev/null; then
        break
    elif [ -n "$AUTONUM" ]; then
        # The display is in use so try another one (if '-a' was specified).
        find_free_servernum
        continue
    fi
    error "Xvfb failed to start" >&2
    XVFBPID=
    exit 1
done

PID=$$
PARENT_PID=$(ps -f|awk '$2=='$PID'{print $3 }')

# Start the command and save its exit status.
set +e
DISPLAY=:$SERVERNUM XAUTHORITY=$AUTHFILE "$@" 2>&1 &
PROCESS_PID=$!

while [ true ]; do
    if ! kill -0 $PARENT_PID > /dev/null 2>&1; then
        kill -9 $PROCESS_PID        
        exit 0
    fi
    
    sleep 3
done

set -e

# Return the executed command's exit status.
exit 0

# vim:set ai et sts=4 sw=4 tw=80:

Post Reply