browsertests: profile for default settings

Some scenarii are meant to be tested on specific wiki instances such as
commmons.beta.wmflabs.org or sandbox.translatewiki.net. For the Gerrit
integration, we are using a fresh wiki having the default ULS settings,
hence the specific tests will definitely fail.

This patch introduces a new cucumber tag `@needs-custom-setup`, it is
meant to flag tests which require further configuration tweaking, for
example tests explicitly targetting an already existing wiki such as
beta or translatewiki

I am introducing a new cucumber profile 'ciwmf' which exclude any test
tagged '@needs-custom-setup' or '@phantomjs-bug'.  That profile is going
to be used by the Jenkins/Gerrit install using something like:

  cucumber -p ciwmf

equivalent to:

  cucumber -t '~@phantomjs-bug' -t '~@needs-custom-setup'

Marked some tests as being bugged on phantomjs:
* autonym ones because of bug 57101 "browser tests must normalize
  font-family".
* accept_language since we dont have yet a way to pass the accept
  language header via phantomjs.
* Commented out an 'input method is visible' test with Vector + Hebrew
  since it does not pass.
* Malayalam link is looked by title which is translated by default.

This patch is passing on my local wiki, albeit skipping a lot of steps.

Change-Id: Ib055f5470886e1b818d4c53cb4c61a751d81e154
This commit is contained in:
Antoine Musso
2013-11-25 10:35:13 +01:00
parent d7d43e1dbb
commit 9fa4b32c08
9 changed files with 23 additions and 7 deletions

View File

@@ -1,2 +1,7 @@
# Cloudbee/Saucelab configuration:
ci: --format Cucumber::Formatter::Sauce --out reports/junit
# WMF Jenkins/Gerrit
ciwmf: -t ~@needs-custom-setup -t ~@phantomjs-bug
default: --format progress

View File

@@ -1,4 +1,5 @@
@language @sandbox.translatewiki.net
# On phantomjs, we need to figure out a way to pass the accept language header.
@language @phantomjs-bug @sandbox.translatewiki.net
Feature: Accept-Language
Scenario Outline: Accept-Language

View File

@@ -1,3 +1,8 @@
# The tests do not normalize the font-family passed back by the browser
# Firefox/Chrome/Phantomjs handle the normalization differently.
#
# https://bugzilla.wikimedia.org/show_bug.cgi?id=57101
@phantomjs-bug
Feature: Autonym font
* Web font should always be applied to the ULS language selector's language

View File

@@ -1,4 +1,4 @@
@commons.wikimedia.beta.wmflabs.org @login
@commons.wikimedia.beta.wmflabs.org @login @needs-custom-setup
Feature: Font selection
In order to have better using experience,

View File

@@ -14,7 +14,7 @@ input method.
When I click on an input box
Then I should see the input method indicator
@commons.wikimedia.beta.wmflabs.org
@commons.wikimedia.beta.wmflabs.org @needs-custom-setup
Scenario: Input method menu
Input method menu is shown when user clicks the input method indicator.
@@ -26,6 +26,7 @@ input method.
And I should see a list of available input methods
And I should see a list of suggested languages
@needs-custom-setup
Scenario: Unsupported input language
User visits a wiki with content language that does not have have an input
@@ -35,7 +36,7 @@ input method.
When I open the input method menu
Then I should see input methods for Kotava
@commons.wikimedia.beta.wmflabs.org
@needs-custom-setup @commons.wikimedia.beta.wmflabs.org
Scenario: Sticky input methods
Chosen input method selection persists across page loads.

View File

@@ -5,7 +5,7 @@ Feature: Live preview of display language changes
Given I am logged in
And I am at random page
@commons.wikimedia.beta.wmflabs.org
@needs-custom-setup @commons.wikimedia.beta.wmflabs.org
Scenario: Display language change is previewed immediately
Given I open the Universal Language Selector
And I open Display panel of language settings

View File

@@ -1,4 +1,5 @@
@commons.wikimedia.beta.wmflabs.org @login
# First scenario seems to fail whenever OpenDyslexic is missing
@commons.wikimedia.beta.wmflabs.org @login @needs-custom-setup
Feature: Persistent settings
This assumes wiki content language is English.

View File

@@ -18,7 +18,8 @@ Feature: Font preferences respected in different languages
And I start editing a page
Then I should see the edit area text being displayed using "monospace" font
@commons.wikimedia.beta.wmflabs.org
# Needs OpenDyslexic font to have been configured on the target wiki
@needs-custom-setup @commons.wikimedia.beta.wmflabs.org
Scenario: Edit area should use the fonts selected by the user from ULS for a language
When I select OpenDyslexic font for the content language
And I start editing a page
@@ -42,6 +43,7 @@ Feature: Font preferences respected in different languages
And I start editing a page
Then I should see the edit area text being displayed using "monospace" font
@phantomjs-bug
Scenario: Edit area should use browser's default Monospace font for languages that have a default monospace font (Latin, Cyrillic, Hebrew etc.)
When I start editing a page
Then I should see the edit area text being displayed using "monospace" font

View File

@@ -1,3 +1,4 @@
@needs-custom-setup
Feature: Trigger in personal toolbar
@commons.wikimedia.beta.wmflabs.org @login