From 9fa4b32c087aab8cd08578aa574924fabae946bb Mon Sep 17 00:00:00 2001 From: Antoine Musso Date: Mon, 25 Nov 2013 10:35:13 +0100 Subject: [PATCH] browsertests: profile for default settings Some scenarii are meant to be tested on specific wiki instances such as commmons.beta.wmflabs.org or sandbox.translatewiki.net. For the Gerrit integration, we are using a fresh wiki having the default ULS settings, hence the specific tests will definitely fail. This patch introduces a new cucumber tag `@needs-custom-setup`, it is meant to flag tests which require further configuration tweaking, for example tests explicitly targetting an already existing wiki such as beta or translatewiki I am introducing a new cucumber profile 'ciwmf' which exclude any test tagged '@needs-custom-setup' or '@phantomjs-bug'. That profile is going to be used by the Jenkins/Gerrit install using something like: cucumber -p ciwmf equivalent to: cucumber -t '~@phantomjs-bug' -t '~@needs-custom-setup' Marked some tests as being bugged on phantomjs: * autonym ones because of bug 57101 "browser tests must normalize font-family". * accept_language since we dont have yet a way to pass the accept language header via phantomjs. * Commented out an 'input method is visible' test with Vector + Hebrew since it does not pass. * Malayalam link is looked by title which is translated by default. This patch is passing on my local wiki, albeit skipping a lot of steps. Change-Id: Ib055f5470886e1b818d4c53cb4c61a751d81e154 --- tests/browser/config/cucumber.yml | 5 +++++ tests/browser/features/accept_language.feature | 3 ++- tests/browser/features/autonym.feature | 5 +++++ tests/browser/features/font_selection.feature | 2 +- tests/browser/features/ime.feature | 5 +++-- .../features/live_preview_of_display_language.feature | 2 +- tests/browser/features/persistent_settings.feature | 3 ++- tests/browser/features/textarea_font.feature | 4 +++- tests/browser/features/triggers.feature | 1 + 9 files changed, 23 insertions(+), 7 deletions(-) diff --git a/tests/browser/config/cucumber.yml b/tests/browser/config/cucumber.yml index 150c2840..fa27e44b 100644 --- a/tests/browser/config/cucumber.yml +++ b/tests/browser/config/cucumber.yml @@ -1,2 +1,7 @@ +# Cloudbee/Saucelab configuration: ci: --format Cucumber::Formatter::Sauce --out reports/junit + +# WMF Jenkins/Gerrit +ciwmf: -t ~@needs-custom-setup -t ~@phantomjs-bug + default: --format progress diff --git a/tests/browser/features/accept_language.feature b/tests/browser/features/accept_language.feature index 5abc2ae7..7e7faaa5 100644 --- a/tests/browser/features/accept_language.feature +++ b/tests/browser/features/accept_language.feature @@ -1,4 +1,5 @@ -@language @sandbox.translatewiki.net +# On phantomjs, we need to figure out a way to pass the accept language header. +@language @phantomjs-bug @sandbox.translatewiki.net Feature: Accept-Language Scenario Outline: Accept-Language diff --git a/tests/browser/features/autonym.feature b/tests/browser/features/autonym.feature index a01ea3b1..fe6f7bbc 100644 --- a/tests/browser/features/autonym.feature +++ b/tests/browser/features/autonym.feature @@ -1,3 +1,8 @@ +# The tests do not normalize the font-family passed back by the browser +# Firefox/Chrome/Phantomjs handle the normalization differently. +# +# https://bugzilla.wikimedia.org/show_bug.cgi?id=57101 +@phantomjs-bug Feature: Autonym font * Web font should always be applied to the ULS language selector's language diff --git a/tests/browser/features/font_selection.feature b/tests/browser/features/font_selection.feature index 6fcbeaae..51e10e6f 100644 --- a/tests/browser/features/font_selection.feature +++ b/tests/browser/features/font_selection.feature @@ -1,4 +1,4 @@ -@commons.wikimedia.beta.wmflabs.org @login +@commons.wikimedia.beta.wmflabs.org @login @needs-custom-setup Feature: Font selection In order to have better using experience, diff --git a/tests/browser/features/ime.feature b/tests/browser/features/ime.feature index ad35c887..2590fb95 100644 --- a/tests/browser/features/ime.feature +++ b/tests/browser/features/ime.feature @@ -14,7 +14,7 @@ input method. When I click on an input box Then I should see the input method indicator - @commons.wikimedia.beta.wmflabs.org + @commons.wikimedia.beta.wmflabs.org @needs-custom-setup Scenario: Input method menu Input method menu is shown when user clicks the input method indicator. @@ -26,6 +26,7 @@ input method. And I should see a list of available input methods And I should see a list of suggested languages + @needs-custom-setup Scenario: Unsupported input language User visits a wiki with content language that does not have have an input @@ -35,7 +36,7 @@ input method. When I open the input method menu Then I should see input methods for Kotava - @commons.wikimedia.beta.wmflabs.org + @needs-custom-setup @commons.wikimedia.beta.wmflabs.org Scenario: Sticky input methods Chosen input method selection persists across page loads. diff --git a/tests/browser/features/live_preview_of_display_language.feature b/tests/browser/features/live_preview_of_display_language.feature index cc971004..aa4bba3a 100644 --- a/tests/browser/features/live_preview_of_display_language.feature +++ b/tests/browser/features/live_preview_of_display_language.feature @@ -5,7 +5,7 @@ Feature: Live preview of display language changes Given I am logged in And I am at random page - @commons.wikimedia.beta.wmflabs.org + @needs-custom-setup @commons.wikimedia.beta.wmflabs.org Scenario: Display language change is previewed immediately Given I open the Universal Language Selector And I open Display panel of language settings diff --git a/tests/browser/features/persistent_settings.feature b/tests/browser/features/persistent_settings.feature index 09ab3099..697a75be 100644 --- a/tests/browser/features/persistent_settings.feature +++ b/tests/browser/features/persistent_settings.feature @@ -1,4 +1,5 @@ -@commons.wikimedia.beta.wmflabs.org @login +# First scenario seems to fail whenever OpenDyslexic is missing +@commons.wikimedia.beta.wmflabs.org @login @needs-custom-setup Feature: Persistent settings This assumes wiki content language is English. diff --git a/tests/browser/features/textarea_font.feature b/tests/browser/features/textarea_font.feature index ea76d728..978b45e6 100644 --- a/tests/browser/features/textarea_font.feature +++ b/tests/browser/features/textarea_font.feature @@ -18,7 +18,8 @@ Feature: Font preferences respected in different languages And I start editing a page Then I should see the edit area text being displayed using "monospace" font - @commons.wikimedia.beta.wmflabs.org + # Needs OpenDyslexic font to have been configured on the target wiki + @needs-custom-setup @commons.wikimedia.beta.wmflabs.org Scenario: Edit area should use the fonts selected by the user from ULS for a language When I select OpenDyslexic font for the content language And I start editing a page @@ -42,6 +43,7 @@ Feature: Font preferences respected in different languages And I start editing a page Then I should see the edit area text being displayed using "monospace" font + @phantomjs-bug Scenario: Edit area should use browser's default Monospace font for languages that have a default monospace font (Latin, Cyrillic, Hebrew etc.) When I start editing a page Then I should see the edit area text being displayed using "monospace" font diff --git a/tests/browser/features/triggers.feature b/tests/browser/features/triggers.feature index c9a2834b..3f35ee8e 100644 --- a/tests/browser/features/triggers.feature +++ b/tests/browser/features/triggers.feature @@ -1,3 +1,4 @@ +@needs-custom-setup Feature: Trigger in personal toolbar @commons.wikimedia.beta.wmflabs.org @login