Move documentation to Read the Docs

Read the docs is used to host the documentation.
https://language-data.readthedocs.io/en/latest/

The following documentation has been moved,

* Introduction
* Using the PHP / Node.js libraries
* Adding new languages
* PHP API documentation

Updated the README.md to point to the new documentation.

Doxygen is used to pull out the PHPDoc comments into XML.
This is parsed via doxyphp2sphinx into Sphinx which is
then used by Read the docs to generate the documentation.

Read the docs has been configured to update the code documentation
under the docs/api folder automatically whenever a commit is made
so no manual work is needed.

Bug: T218639

docs/Makefile

@@ -0,0 +1,19 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line.
+SPHINXOPTS    =
+SPHINXBUILD   = sphinx-build
+SOURCEDIR     = .
+BUILDDIR      = _build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/_templates/breadcrumbs.html

@@ -0,0 +1,5 @@
+<!-- Remove the Edit on Github link -->
+{%- extends "sphinx_rtd_theme/breadcrumbs.html" %}
+
+{% block breadcrumbs_aside %}
+{% endblock %}

docs/api.rst

@@ -0,0 +1,10 @@
+API documentation
+=================
+
+.. toctree::
+   :glob:
+
+   api/*
+
+
+    

docs/api/languagedata.rst

@@ -0,0 +1,172 @@
+LanguageData
+============
+
+A singleton utility class to query the language data.
+
+:Qualified name: ``Wikimedia\LanguageData``
+
+.. php:class:: LanguageData
+
+  .. php:method:: addLanguage (string $languageCode, array $options)
+
+    Adds a language in run time and sets its options as provided. If the target option is provided, the language is defined as a redirect. Other possible options are script (string), regions (array) and autonym (string).
+
+    :param string $languageCode:
+      New language code.
+    :param array $options:
+      Language properties.
+
+  .. php:method:: getAutonym (string $languageCode)
+
+    Returns the autonym of the language
+
+    :param string $languageCode:
+      Language code
+    :returns: string|bool Autonym of the language or false if the language is unknown
+
+  .. php:method:: getAutonyms () -> array
+
+    Returns all language codes and corresponding autonyms
+
+    :returns: array -- The key is the language code, and the values are corresponding autonym
+
+  .. php:method:: getDir (string $languageCode)
+
+    Return the direction of the language
+
+    :param string $languageCode:
+      Language code
+    :returns: string|bool Returns 'rtl' or 'ltr'. If the language code is unknown, returns false.
+
+  .. php:method:: getGroupOfScript (string $script) -> string
+
+    Returns the script group of a script or "Other" if it doesn't belong to any group
+
+    :param string $script:
+      Name of the script
+    :returns: string -- Script group name or "Other" if the script doesn't belong to any group
+
+  .. php:method:: getLanguages ()
+
+    Get all the languages. The properties in the returned object are ISO 639 language codes The value of each property is an array that has, [writing system code, [regions list], autonym]
+
+    :returns: object
+
+  .. php:method:: getLanguagesByScriptGroup (array $languageCodes) -> array
+
+    Return the list of languages passed, grouped by their script group
+
+    :param array $languageCodes:
+      List of language codes to group
+    :returns: array -- List of language codes grouped by script group
+
+  .. php:method:: getLanguagesByScriptGroupInRegion (string $region) -> LanguageData::getLanguagesByScriptGroupInRegions
+
+    Returns an associative array of languages in a region, grouped by their script
+
+    :param string $region:
+      Region code
+    :returns: :class:`LanguageData::getLanguagesByScriptGroupInRegions` -- 
+
+  .. php:method:: getLanguagesByScriptGroupInRegions (array $regions) -> array
+
+    Returns an associative array of languages in several regions, grouped by script group
+
+    :param array $regions:
+      List of strings representing region codes
+    :returns: array -- Returns an associative array. They key is the script group name, and the value is a list of language codes in that region.
+
+  .. php:method:: getLanguagesInScript (string $script) -> array
+
+    Returns all languages written in the given script
+
+    :param string $script:
+      Name of the script
+    :returns: array -- 
+
+  .. php:method:: getLanguagesInScripts (array $scripts) -> array
+
+    Returns all languages written in the given scripts
+
+    :param array $scripts:
+      List of strings, each being the name of a script
+    :returns: array -- 
+
+  .. php:method:: getLanguagesInTerritory (string $territory)
+
+    Returns the languages spoken in a territory
+
+    :param string $territory:
+      Territory code
+    :returns: array|bool List of language codes in the territory, or else false if invalid territory is passed
+
+  .. php:method:: getRegions (string $languageCode)
+
+    Returns the regions in which a language is spoken
+
+    :param string $languageCode:
+      Language code
+    :returns: array|bool List of regions or false if language is unknown
+
+  .. php:method:: getScript (string $languageCode)
+
+    Returns the script of the language
+
+    :param string $languageCode:
+      Language code
+    :returns: string|bool Language script or false if the language is unknown
+
+  .. php:method:: getScriptGroupOfLanguage (string $languageCode) -> string
+
+    Returns the script group of a language. Language belongs to a script, and the script belongs to a script group
+
+    :param string $languageCode:
+      Language code
+    :returns: string -- script group name
+
+  .. php:method:: isKnown (string $languageCode) -> bool
+
+    Checks if a language code is valid
+
+    :param string $languageCode:
+      Language code
+    :returns: bool -- 
+
+  .. php:method:: isRedirect (string $languageCode)
+
+    Checks if the language is a redirect and returns the target language code
+
+    :param string $languageCode:
+      Language code
+    :returns: string|bool Target language code if it's a redirect or false if it's not
+
+  .. php:method:: isRtl (string $languageCode) -> bool
+
+    Check if a language is right-to-left
+
+    :param string $languageCode:
+      Language code
+    :returns: bool -- true if it is an RTL language, else false. Returns false if an unknown language code is passed.
+
+  .. php:method:: sortByAutonym (array $languageCodes) -> array
+
+    Sort languages by their autonym
+
+    :param array $languageCodes:
+      List of language codes to sort
+    :returns: array -- List of sorted language codes returned by their autonym
+
+  .. php:method:: sortByScriptGroup (array $languageCodes) -> array
+
+    Return the list of languages sorted by their script groups
+
+    :param array $languageCodes:
+      List of language codes to sort
+    :returns: array -- Sorted list of strings containing language codes
+
+  .. php:staticmethod:: get () -> LanguageData
+
+    Returns an instance of the class that can be used to then call the other methods in the class.
+
+    :returns: :class:`LanguageData` -- 
+

docs/conf.py

@@ -0,0 +1,190 @@
+# -*- coding: utf-8 -*-
+#
+# Configuration file for the Sphinx documentation builder.
+#
+# This file does only contain a selection of the most common options. For a
+# full list see the documentation:
+# http://www.sphinx-doc.org/en/master/config
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+# import os
+# import sys
+# sys.path.insert(0, os.path.abspath('.'))
+
+
+# -- Project information -----------------------------------------------------
+
+project = u'LanguageData'
+copyright = u'2020, Wikimedia Foundation'
+author = u'Wikimedia Foundation'
+
+# The short X.Y version
+version = u''
+# The full version, including alpha/beta/rc tags
+release = u''
+
+
+# -- General configuration ---------------------------------------------------
+
+# If your documentation needs a minimal Sphinx version, state it here.
+#
+# needs_sphinx = '1.0'
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+	"sphinxcontrib.phpdomain"
+]
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# The suffix(es) of source filenames.
+# You can specify multiple suffix as a list of string:
+#
+# source_suffix = ['.rst', '.md']
+source_suffix = '.rst'
+
+# The master toctree document.
+master_doc = 'index'
+
+# The language for content autogenerated by Sphinx. Refer to documentation
+# for a list of supported languages.
+#
+# This is also used if you do content translation via gettext catalogs.
+# Usually you set "language" from the command line for these cases.
+language = None
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = [u'_build', 'Thumbs.db', '.DS_Store']
+
+# The name of the Pygments (syntax highlighting) style to use.
+pygments_style = None
+
+
+# -- Options for HTML output -------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+#
+html_theme = 'sphinx_rtd_theme'
+
+# Theme options are theme-specific and customize the look and feel of a theme
+# further.  For a list of options available for each theme, see the
+# documentation.
+#
+# html_theme_options = {}
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+
+# Custom sidebar templates, must be a dictionary that maps document names
+# to template names.
+#
+# The default sidebars (for documents that don't match any pattern) are
+# defined by theme itself.  Builtin themes are using these templates by
+# default: ``['localtoc.html', 'relations.html', 'sourcelink.html',
+# 'searchbox.html']``.
+#
+# html_sidebars = {}
+
+
+# -- Options for HTMLHelp output ---------------------------------------------
+
+# Output file base name for HTML help builder.
+htmlhelp_basename = 'LanguageDatadoc'
+
+
+# -- Options for LaTeX output ------------------------------------------------
+
+latex_elements = {
+    # The paper size ('letterpaper' or 'a4paper').
+    #
+    # 'papersize': 'letterpaper',
+
+    # The font size ('10pt', '11pt' or '12pt').
+    #
+    # 'pointsize': '10pt',
+
+    # Additional stuff for the LaTeX preamble.
+    #
+    # 'preamble': '',
+
+    # Latex figure (float) alignment
+    #
+    # 'figure_align': 'htbp',
+}
+
+# Grouping the document tree into LaTeX files. List of tuples
+# (source start file, target name, title,
+#  author, documentclass [howto, manual, or own class]).
+latex_documents = [
+    (master_doc, 'LanguageData.tex', u'LanguageData Documentation',
+     u'Wikimedia Foundation', 'manual'),
+]
+
+
+# -- Options for manual page output ------------------------------------------
+
+# One entry per manual page. List of tuples
+# (source start file, name, description, authors, manual section).
+man_pages = [
+    (master_doc, 'languagedata', u'LanguageData Documentation',
+     [author], 1)
+]
+
+
+# -- Options for Texinfo output ----------------------------------------------
+
+# Grouping the document tree into Texinfo files. List of tuples
+# (source start file, target name, title, author,
+#  dir menu entry, description, category)
+texinfo_documents = [
+    (master_doc, 'LanguageData', u'LanguageData Documentation',
+     author, 'LanguageData', 'One line description of project.',
+     'Miscellaneous'),
+]
+
+
+# -- Options for Epub output -------------------------------------------------
+
+# Bibliographic Dublin Core info.
+epub_title = project
+
+# The unique identifier of the text. This can be a ISBN number
+# or the project homepage.
+#
+# epub_identifier = ''
+
+# A unique identification for the text.
+#
+# epub_uid = ''
+
+# A list of files that should not be packed into the epub file.
+epub_exclude_files = ['search.html']
+
+# PHP Syntax
+from sphinx.highlighting import lexers
+from pygments.lexers.web import PhpLexer
+lexers["php"] = PhpLexer(startinline=True, linenos=1)
+lexers["php-annotations"] = PhpLexer(startinline=True, linenos=1)
+
+# Set domain
+primary_domain = "php"
+
+# Regenerate API docs via doxygen + doxyphp2sphinx
+import subprocess, os
+read_the_docs_build = os.environ.get('READTHEDOCS', None) == 'True'
+if read_the_docs_build:
+    subprocess.call(['doxygen', 'Doxyfile'])
+    subprocess.call(['doxyphp2sphinx', 'Wikimedia'])

docs/index.rst

@@ -0,0 +1,113 @@
+.. LanguageData documentation master file, created by
+   sphinx-quickstart on Thu Jan 30 13:47:31 2020.
+   You can adapt this file completely to your liking, but it should at least
+   contain the root `toctree` directive.
+
+CLDR based language data and utilities
+========================================
+
+This library contains language related data, and utility libraries written in PHP and Node.js to
+interact with that data.
+
+The language related data comprises of the following,
+
+1. The script in which a language is written
+2. The script code
+3. The language code
+4. The regions in which the language is spoken
+5. The autonym - language name written in its own script
+6. The directionality of the text
+
+This data is populated from the current version of
+`CLDR supplemental data <http://unicode.org/repos/cldr/trunk/common/supplemental/supplementalData.xml>`_.
+
+Using the PHP library
+----------------------------
+|php-build|
+
+.. |php-build| image:: https://github.com/Abijeet/language-data/workflows/PHP%20build/badge.svg
+		 :target: https://github.com/Abijeet/language-data/actions?query=workflow%3A%22PHP+build%22
+
+Installation
+^^^^^^^^^^^^^
+You can add this library to your project by running:
+
+.. code-block:: bash
+
+		composer install wikimedia/language-data
+
+Basic usage
+^^^^^^^^^^^^^
+The basic usage is like this:
+
+.. code-block:: php
+
+		<?php
+		use Wikimedia\LanguageData;
+
+		$languageData = LanguageData::get();
+		// Returns English
+		$languageData->getAutonym( 'en' );
+
+For a full list of methods see the documentation for the `LanguageData <api/languagedata.html>`_ class.
+
+Using the Node.js library
+------------------------------
+
+|npm| |npm-build|
+
+.. |npm| image:: https://img.shields.io/npm/v/@wikimedia/language-data.svg
+      :target: https://npmjs.com/package/@wikimedia/language-data
+
+
+.. |npm-build| image:: https://github.com/Abijeet/language-data/workflows/Node.js%20build/badge.svg
+      :target: https://github.com/Abijeet/language-data/actions?query=workflow%3A%22Node.js+build%22
+
+Installation
+^^^^^^^^^^^^^
+You can add this library to your project by running,
+
+.. code-block:: bash
+
+		npm i @wikimedia/language-data
+
+Basic usage
+^^^^^^^^^^^^^
+The basic usage is like this:
+
+.. code-block:: js
+
+		const languageData = require('@wikimedia/language-data');
+
+		// Returns English
+		languageData.getAutonym( 'en');
+
+The exposed methods are similar to the methods present in the PHP `LanguageData <api/languagedata.html>`_ class.
+
+Contribute
+----------
+
+- Issue Tracker: https://github.com/wikimedia/language-data/issues
+- Source Code: https://github.com/wikimedia/language-data
+
+Navigation
+==========
+
+.. toctree::
+    :maxdepth: 1
+    :caption: User Documentation
+
+		Adding new languages <user/adding_new_language.rst>
+
+.. toctree::
+    :maxdepth: 2
+    :caption: PHP API Documentation
+
+  	LanguageData class <api/languagedata.rst>
+
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`search`

docs/make.bat

@@ -0,0 +1,35 @@
+@ECHO OFF
+
+pushd %~dp0
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+	set SPHINXBUILD=sphinx-build
+)
+set SOURCEDIR=.
+set BUILDDIR=_build
+
+if "%1" == "" goto help
+
+%SPHINXBUILD% >NUL 2>NUL
+if errorlevel 9009 (
+	echo.
+	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+	echo.installed, then set the SPHINXBUILD environment variable to point
+	echo.to the full path of the 'sphinx-build' executable. Alternatively you
+	echo.may add the Sphinx directory to PATH.
+	echo.
+	echo.If you don't have Sphinx installed, grab it from
+	echo.http://sphinx-doc.org/
+	exit /b 1
+)
+
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%
+goto end
+
+:help
+%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%
+
+:end
+popd

docs/requirements.txt

@@ -0,0 +1,4 @@
+Sphinx==1.8.4
+sphinx-rtd-theme==0.4.2
+sphinxcontrib-phpdomain==0.6.3
+doxyphp2sphinx>=1.0.1

docs/user/adding_new_language.rst

@@ -0,0 +1,31 @@
+Adding new languages
+=========================
+
+New languages must be added to the ``data/langdb.yaml`` file.
+
+The file format is: `ISO 639 <https://en.wikipedia.org/wiki/ISO_639>`_ code:
+``[writing system code, [regions list], autonym]``
+
+The writing system is indicated using `ISO 15924 <https://en.wikipedia.org/wiki/ISO_15924>`_
+codes. Make sure that the code appears in the ``scriptgroups`` section towards the end of
+the file, and add it, if it doesn't.
+
+The list of region codes appears at the end of ``data/langdb.yaml``.
+
+The autonym is the name of the language in the language itself. In some cases, for example for
+extinct languages such as Jewish Babylonian Aramaic (tmr), the name can be something that is
+useful for modern users, but in most cases it should be the natural name in the language itself.
+Please do your best to verify that it's spelled correctly in reliable sources.
+
+Example: ``myv: [Cyrl, [EU], эрзянь]``
+
+This is the `Erzya language <https://en.wikipedia.org/wiki/Erzya_language>`_. Its writing system
+is Cyrillic (ISO 15924: Cyrl). It's spoken in Europe (EU). Its autonym is "эрзянь".
+
+Some languages are listed as redirects. In this case, the only value in the square brackets is
+the target language code. For example: fil: [tl]
+
+This is the Filipino language, which is a redirect to Tagalog (tl).
+
+After adding a language to ``data/langdb.yaml``, run ``php src/util/ulsdata2json.php`` in the
+base directory to generate the ``language-data.json`` file. Don't edit ``language-data.json`` manually.