diff options
Diffstat (limited to 'docs/docsite/rst/dev_guide/testing/sanity')
57 files changed, 0 insertions, 1239 deletions
diff --git a/docs/docsite/rst/dev_guide/testing/sanity/action-plugin-docs.rst b/docs/docsite/rst/dev_guide/testing/sanity/action-plugin-docs.rst deleted file mode 100644 index e3a5d8b838..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/action-plugin-docs.rst +++ /dev/null @@ -1,4 +0,0 @@ -action-plugin-docs -================== - -Each action plugin should have a matching module of the same name to provide documentation. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/ansible-doc.rst b/docs/docsite/rst/dev_guide/testing/sanity/ansible-doc.rst deleted file mode 100644 index 9f2c4f5f45..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/ansible-doc.rst +++ /dev/null @@ -1,4 +0,0 @@ -ansible-doc -=========== - -Verifies that ``ansible-doc`` can parse module documentation on all supported Python versions. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/ansible-requirements.rst b/docs/docsite/rst/dev_guide/testing/sanity/ansible-requirements.rst deleted file mode 100644 index f348b07f00..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/ansible-requirements.rst +++ /dev/null @@ -1,4 +0,0 @@ -ansible-requirements -==================== - -``test/lib/ansible_test/_data/requirements/sanity.import-plugins.txt`` must be an identical copy of ``requirements.txt`` found in the project's root. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/ansible-test-future-boilerplate.rst b/docs/docsite/rst/dev_guide/testing/sanity/ansible-test-future-boilerplate.rst deleted file mode 100644 index 43dfe32465..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/ansible-test-future-boilerplate.rst +++ /dev/null @@ -1,8 +0,0 @@ -ansible-test-future-boilerplate -=============================== - -The ``_internal`` code for ``ansible-test`` requires the following ``__future__`` import: - -.. code-block:: python - - from __future__ import annotations diff --git a/docs/docsite/rst/dev_guide/testing/sanity/ansible-var-precedence-check.rst b/docs/docsite/rst/dev_guide/testing/sanity/ansible-var-precedence-check.rst deleted file mode 100644 index 1906886f4f..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/ansible-var-precedence-check.rst +++ /dev/null @@ -1,6 +0,0 @@ -:orphan: - -ansible-var-precedence-check -============================ - -Check the order of precedence for Ansible variables against :ref:`ansible_variable_precedence`. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/azure-requirements.rst b/docs/docsite/rst/dev_guide/testing/sanity/azure-requirements.rst deleted file mode 100644 index 5e0cc04444..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/azure-requirements.rst +++ /dev/null @@ -1,10 +0,0 @@ -:orphan: - -azure-requirements -================== - -Update the Azure integration test requirements file when changes are made to the Azure packaging requirements file: - -.. code-block:: bash - - cp packaging/requirements/requirements-azure.txt test/lib/ansible_test/_data/requirements/integration.cloud.azure.txt diff --git a/docs/docsite/rst/dev_guide/testing/sanity/bin-symlinks.rst b/docs/docsite/rst/dev_guide/testing/sanity/bin-symlinks.rst deleted file mode 100644 index a8e33a7f44..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/bin-symlinks.rst +++ /dev/null @@ -1,11 +0,0 @@ -bin-symlinks -============ - -The ``bin/`` directory in Ansible must contain only symbolic links to executable files. -These files must reside in the ``lib/ansible/`` or ``test/lib/ansible_test/`` directories. - -This is required to allow ``ansible-test`` to work with containers and remote hosts when running from an installed version of Ansible. - -Symlinks for each entry point in ``bin/`` must also be present in ``test/lib/ansible_test/_util/target/injector/``. -Each symlink should point to the ``python.py`` script in the same directory. -This facilitates running with the correct Python interpreter and enabling code coverage. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/boilerplate.rst b/docs/docsite/rst/dev_guide/testing/sanity/boilerplate.rst deleted file mode 100644 index 51c0c08942..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/boilerplate.rst +++ /dev/null @@ -1,11 +0,0 @@ -:orphan: - -boilerplate -=========== - -Most Python files should include the following boilerplate: - -.. code-block:: python - - from __future__ import (absolute_import, division, print_function) - __metaclass__ = type diff --git a/docs/docsite/rst/dev_guide/testing/sanity/botmeta.rst b/docs/docsite/rst/dev_guide/testing/sanity/botmeta.rst deleted file mode 100644 index 639bb0bf01..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/botmeta.rst +++ /dev/null @@ -1,4 +0,0 @@ -botmeta -======= - -Verifies that ``./github/BOTMETA.yml`` is valid. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/changelog.rst b/docs/docsite/rst/dev_guide/testing/sanity/changelog.rst deleted file mode 100644 index 2d557c72d5..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/changelog.rst +++ /dev/null @@ -1,19 +0,0 @@ -changelog -========= - -Basic linting of changelog fragments with `antsibull-changelog lint <https://pypi.org/project/antsibull-changelog/>`_. - -One or more of the following sections are required: - -- major_changes -- minor_changes -- breaking_changes -- deprecated_features -- removed_features -- security_fixes -- bugfixes -- known_issues - -New modules and plugins must not be included in changelog fragments. - -See :ref:`collection_changelogs` for details. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/compile.rst b/docs/docsite/rst/dev_guide/testing/sanity/compile.rst deleted file mode 100644 index 40367218cd..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/compile.rst +++ /dev/null @@ -1,32 +0,0 @@ -.. _testing_compile: - -compile -======= - -All Python source files must successfully compile using all supported Python versions. - -.. note:: - - The list of supported Python versions is dependent on the version of ``ansible-core`` that you are using. - Make sure you consult the version of the documentation which matches your ``ansible-core`` version. - -Controller code, including plugins in Ansible Collections, must support the following Python versions: - -- 3.11 -- 3.10 -- 3.9 - -Code which runs on targets (``modules`` and ``module_utils``) must support all controller supported Python versions, -as well as the additional Python versions supported only on targets: - -- 3.8 -- 3.7 -- 3.6 -- 3.5 -- 2.7 - -.. note:: - - Ansible Collections can be - `configured <https://github.com/ansible/ansible/blob/devel/test/lib/ansible_test/config/config.yml>`_ - to support a subset of the target-only Python versions. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/configure-remoting-ps1.rst b/docs/docsite/rst/dev_guide/testing/sanity/configure-remoting-ps1.rst deleted file mode 100644 index e83bc78d89..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/configure-remoting-ps1.rst +++ /dev/null @@ -1,5 +0,0 @@ -configure-remoting-ps1 -====================== - -The file ``examples/scripts/ConfigureRemotingForAnsible.ps1`` is required and must be a regular file. -It is used by external automated processes and cannot be moved, renamed or replaced with a symbolic link. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/deprecated-config.rst b/docs/docsite/rst/dev_guide/testing/sanity/deprecated-config.rst deleted file mode 100644 index 950805a282..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/deprecated-config.rst +++ /dev/null @@ -1,6 +0,0 @@ -:orphan: - -deprecated-config -================= - -``DOCUMENTATION`` config is scheduled for removal diff --git a/docs/docsite/rst/dev_guide/testing/sanity/docs-build.rst b/docs/docsite/rst/dev_guide/testing/sanity/docs-build.rst deleted file mode 100644 index 23f3c5524b..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/docs-build.rst +++ /dev/null @@ -1,4 +0,0 @@ -docs-build -========== - -Verifies that ``make singlehtmldocs`` in ``docs/docsite/`` completes without errors. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/empty-init.rst b/docs/docsite/rst/dev_guide/testing/sanity/empty-init.rst deleted file mode 100644 index e87bb71ed6..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/empty-init.rst +++ /dev/null @@ -1,10 +0,0 @@ -empty-init -========== - -The ``__init__.py`` files under the following directories must be empty. For some of these (modules -and tests), ``__init__.py`` files with code won't be used. For others (module_utils), we want the -possibility of using Python namespaces which an empty ``__init__.py`` will allow for. - -- ``lib/ansible/modules/`` -- ``lib/ansible/module_utils/`` -- ``test/units/`` diff --git a/docs/docsite/rst/dev_guide/testing/sanity/future-import-boilerplate.rst b/docs/docsite/rst/dev_guide/testing/sanity/future-import-boilerplate.rst deleted file mode 100644 index 658ef06444..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/future-import-boilerplate.rst +++ /dev/null @@ -1,51 +0,0 @@ -future-import-boilerplate -========================= - -Most Python files should include the following boilerplate at the top of the file, right after the -comment header: - -.. code-block:: python - - from __future__ import (absolute_import, division, print_function) - -This uses Python 3 semantics for absolute versus relative imports, division, and print. By doing this, -we can write code which is portable between Python 2 and Python 3 by following the Python 3 semantics. - - -absolute_import ---------------- - -When Python 2 encounters an import of a name in a file like ``import copy`` it attempts to load -``copy.py`` from the same directory as the file is in. This can cause problems if there is a python -file of that name in the directory and also a python module in ``sys.path`` with that same name. In -that case, Python 2 would load the one in the same directory and there would be no way to load the -one on ``sys.path``. Python 3 fixes this by making imports absolute by default. ``import copy`` -will find ``copy.py`` from ``sys.path``. If you want to import ``copy.py`` from the same directory, -the code needs to be changed to perform a relative import: ``from . import copy``. - -.. seealso:: - - * `Absolute and relative imports <https://www.python.org/dev/peps/pep-0328>`_ - -division --------- - -In Python 2, the division operator (``/``) returns integer values when used with integers. If there -was a remainder, this part would be left off (aka, `floor division`). In Python 3, the division -operator (``/``) always returns a floating point number. Code that needs to calculate the integer -portion of the quotient needs to switch to using the floor division operator (`//`) instead. - -.. seealso:: - - * `Changing the division operator <https://www.python.org/dev/peps/pep-0238>`_ - -print_function --------------- - -In Python 2, :func:`python:print` is a keyword. In Python 3, :func:`python3:print` is a function with different -parameters. Using this ``__future__`` allows using the Python 3 print semantics everywhere. - -.. seealso:: - - * `Make print a function <https://www.python.org/dev/peps/pep-3105>`_ - diff --git a/docs/docsite/rst/dev_guide/testing/sanity/ignores.rst b/docs/docsite/rst/dev_guide/testing/sanity/ignores.rst deleted file mode 100644 index 69190c8d73..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/ignores.rst +++ /dev/null @@ -1,105 +0,0 @@ -ignores -======= - -Sanity tests for individual files can be skipped, and specific errors can be ignored. - -When to Ignore Errors ---------------------- - -Sanity tests are designed to improve code quality and identify common issues with content. -When issues are identified during development, those issues should be corrected. - -As development of Ansible continues, sanity tests are expanded to detect issues that previous releases could not. -To allow time for existing content to be updated to pass newer tests, ignore entries can be added. -New content should not use ignores for existing sanity tests. - -When code is fixed to resolve sanity test errors, any relevant ignores must also be removed. -If the ignores are not removed, this will be reported as an unnecessary ignore error. -This is intended to prevent future regressions due to the same error recurring after being fixed. - -When to Skip Tests ------------------- - -Although rare, there are reasons for skipping a sanity test instead of ignoring the errors it reports. - -If a sanity test results in a traceback when processing content, that error cannot be ignored. -If this occurs, open a new `bug report <https://github.com/ansible/ansible/issues/new?template=bug_report.md>`_ for the issue so it can be fixed. -If the traceback occurs due to an issue with the content, that issue should be fixed. -If the content is correct, the test will need to be skipped until the bug in the sanity test is fixed. - - Caution should be used when skipping sanity tests instead of ignoring them. - Since the test is skipped entirely, resolution of the issue will not be automatically detected. - This will prevent prevent regression detection from working once the issue has been resolved. - For this reason it is a good idea to periodically review skipped entries manually to verify they are required. - -Ignore File Location --------------------- - -The location of the ignore file depends on the type of content being tested. - -Ansible Collections -^^^^^^^^^^^^^^^^^^^ - -Since sanity tests change between Ansible releases, a separate ignore file is needed for each Ansible major release. - -The filename is ``tests/sanity/ignore-X.Y.txt`` where ``X.Y`` is the Ansible release being used to test the collection. - -Maintaining a separate file for each Ansible release allows a collection to pass tests for multiple versions of Ansible. - -Ansible -^^^^^^^ - -When testing Ansible, all ignores are placed in the ``test/sanity/ignore.txt`` file. - -Only a single file is needed because ``ansible-test`` is developed and released as a part of Ansible itself. - -Ignore File Format ------------------- - -The ignore file contains one entry per line. -Each line consists of two columns, separated by a single space. -Comments may be added at the end of an entry, started with a hash (``#``) character, which can be proceeded by zero or more spaces. -Blank and comment only lines are not allowed. - -The first column specifies the file path that the entry applies to. -File paths must be relative to the root of the content being tested. -This is either the Ansible source or an Ansible collection. -File paths cannot contain a space or the hash (``#``) character. - -The second column specifies the sanity test that the entry applies to. -This will be the name of the sanity test. -If the sanity test is specific to a version of Python, the name will include a dash (``-``) and the relevant Python version. -If the named test uses error codes then the error code to ignore must be appended to the name of the test, separated by a colon (``:``). - -Below are some example ignore entries for an Ansible collection: - -.. code-block:: text - - roles/my_role/files/my_script.sh shellcheck:SC2154 # ignore undefined variable - plugins/modules/my_module.py validate-modules:missing-gplv3-license # ignore license check - plugins/modules/my_module.py import-3.8 # needs update to support collections.abc on Python 3.8+ - -It is also possible to skip a sanity test for a specific file. -This is done by adding ``!skip`` after the sanity test name in the second column. -When this is done, no error code is included, even if the sanity test uses error codes. - -Below are some example skip entries for an Ansible collection: - -.. code-block:: text - - plugins/module_utils/my_util.py validate-modules!skip # waiting for bug fix in module validator - plugins/lookup/my_plugin.py compile-2.6!skip # Python 2.6 is not supported on the controller - -See the full list of :ref:`sanity tests <all_sanity_tests>`, which details the various tests and details how to fix identified issues. - -Ignore File Errors ------------------- - -There are various errors that can be reported for the ignore file itself: - -- syntax errors parsing the ignore file -- references a file path that does not exist -- references to a sanity test that does not exist -- ignoring an error that does not occur -- ignoring a file which is skipped -- duplicate entries diff --git a/docs/docsite/rst/dev_guide/testing/sanity/import.rst b/docs/docsite/rst/dev_guide/testing/sanity/import.rst deleted file mode 100644 index ddfe1217f0..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/import.rst +++ /dev/null @@ -1,137 +0,0 @@ -import -====== - -Ansible :ref:`allows unchecked imports<allowed_unchecked_imports>` of some libraries from specific directories. -Importing any other Python library requires :ref:`handling import errors<handling_import_errors>`. -This enables support for sanity tests such as :ref:`testing_validate-modules` and provides better error messages to the user. - -.. _handling_import_errors: - -Handling import errors ----------------------- - -Ansible executes across many hosts and can use multiple Python interpreters at the same time, which may even have different versions. -To ensure users get an actionable and easy to understand error we try to ensure any non-core imports in modules/plugins are guarded to avoid a traceback, -which most users won't be able to understand, much less use to solve the issue. - -Another reason Ansible does this is to import the code for inspection. This allows Ansible to easily test, document, configure, etc based on the code without having to install -any and all requirements everywhere, especially when that is not the context in which you execute the code. - -The code below shows examples of how to avoid errors on import and then use the provided ``missing_required_lib`` to ensure the user knows which LIBRARY is missing, -on which HOST it is missing and the specific INTERPRETER that requires it. - - -In modules -^^^^^^^^^^ - -Instead of using ``import another_library``: - -.. code-block:: python - - import traceback - - from ansible.module_utils.basic import missing_required_lib - - try: - import another_library - except ImportError: - HAS_ANOTHER_LIBRARY = False - ANOTHER_LIBRARY_IMPORT_ERROR = traceback.format_exc() - else: - HAS_ANOTHER_LIBRARY = True - ANOTHER_LIBRARY_IMPORT_ERROR = None - -.. note:: - - The ``missing_required_lib`` import above will be used below. - -Then in the module code, normally inside the ``main`` method: - -.. code-block:: python - - module = AnsibleModule(...) - - if not HAS_ANOTHER_LIBRARY: - module.fail_json( - msg=missing_required_lib('another_library'), - exception=ANOTHER_LIBRARY_IMPORT_ERROR) - -In plugins -^^^^^^^^^^ - -Instead of using ``import another_library``: - -.. code-block:: python - - try: - import another_library - except ImportError as imp_exc: - ANOTHER_LIBRARY_IMPORT_ERROR = imp_exc - else: - ANOTHER_LIBRARY_IMPORT_ERROR = None - -Then in the plugin code, for example in the ``run`` method of the plugin (some plugins don't have a ``run`` method and will require it in the ``__init__`` method instead): - -.. code-block:: python - - if ANOTHER_LIBRARY_IMPORT_ERROR: - raise AnsibleError('another_library must be installed to use this plugin') from ANOTHER_LIBRARY_IMPORT_ERROR - -When used as base classes -^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. important:: - - This solution builds on the previous two examples. - Make sure to pick the appropriate one before continuing with this solution. - -Sometimes an import is used in a base class, for example: - -.. code-block:: python - - from another_library import UsefulThing - - class CustomThing(UsefulThing): - pass - -One option is make the entire class definition conditional: - -.. code-block:: python - - if not ANOTHER_LIBRARY_IMPORT_ERROR: - class CustomThing(UsefulThing): - pass - -Another option is to define a substitute base class by modifying the exception handler: - -.. code-block:: python - - try: - from another_library import UsefulThing - except ImportError: - class UsefulThing: - pass - ... - -.. _allowed_unchecked_imports: - -Allowed unchecked imports -------------------------- - -Ansible allows the following unchecked imports from these specific directories: - -* ansible-core: - - * For ``lib/ansible/modules/`` and ``lib/ansible/module_utils/``, unchecked imports are only allowed from the Python standard library; - * For ``lib/ansible/plugins/``, unchecked imports are only allowed from the Python standard library, from public dependencies of ansible-core, and from ansible-core itself; - -* collections: - - * For ``plugins/modules/`` and ``plugins/module_utils/``, unchecked imports are only allowed from the Python standard library; - * For other directories in ``plugins/`` (see `the community collection requirements <https://github.com/ansible-collections/overview/blob/main/collection_requirements.rst#modules-plugins>`_ for a list), unchecked imports are only allowed from the Python standard library, from public dependencies of ansible-core, and from ansible-core itself. - -Public dependencies of ansible-core are: - - * Jinja2 - * PyYAML - * MarkupSafe (as a dependency of Jinja2) diff --git a/docs/docsite/rst/dev_guide/testing/sanity/integration-aliases.rst b/docs/docsite/rst/dev_guide/testing/sanity/integration-aliases.rst deleted file mode 100644 index 1d70dbc78e..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/integration-aliases.rst +++ /dev/null @@ -1,183 +0,0 @@ -integration-aliases -=================== - -Integration tests are executed by ``ansible-test`` and reside in directories under ``test/integration/targets/``. -Each test MUST have an ``aliases`` file to control test execution. - -Aliases are explained in the following sections. Each alias must be on a separate line in an ``aliases`` file. - -Groups ------- - -Tests must be configured to run in exactly one group. This is done by adding the appropriate group to the ``aliases`` file. - -The following are examples of some of the available groups: - -- ``shippable/posix/group1`` -- ``shippable/windows/group2`` -- ``shippable/azure/group3`` -- ``shippable/aws/group1`` -- ``shippable/cloud/group1`` - -Groups are used to balance tests across multiple CI jobs to minimize test run time. -They also improve efficiency by keeping tests with similar requirements running together. - -When selecting a group for a new test, use the same group as existing tests similar to the one being added. -If more than one group is available, select one randomly. - -Setup ------ - -Aliases can be used to execute setup targets before running tests: - -- ``setup/once/TARGET`` - Run the target ``TARGET`` before the first target that requires it. -- ``setup/always/TARGET`` - Run the target ``TARGET`` before each target that requires it. - -Requirements ------------- - -Aliases can be used to express some test requirements: - -- ``needs/privileged`` - Requires ``--docker-privileged`` when running tests with ``--docker``. -- ``needs/root`` - Requires running tests as ``root`` or with ``--docker``. -- ``needs/ssh`` - Requires SSH connections to localhost (or the test container with ``--docker``) without a password. -- ``needs/httptester`` - Requires use of the http-test-container to run tests. - -Dependencies ------------- - -Some test dependencies are automatically discovered: - -- Ansible role dependencies defined in ``meta/main.yml`` files. -- Setup targets defined with ``setup/*`` aliases. -- Symbolic links from one target to a file in another target. - -Aliases can be used to declare dependencies that are not handled automatically: - -- ``needs/target/TARGET`` - Requires use of the test target ``TARGET``. -- ``needs/file/PATH`` - Requires use of the file ``PATH`` relative to the git root. - -Skipping --------- - -Aliases can be used to skip platforms using one of the following: - -- ``skip/freebsd`` - Skip tests on FreeBSD. -- ``skip/macos`` - Skip tests on macOS. -- ``skip/rhel`` - Skip tests on RHEL. -- ``skip/docker`` - Skip tests when running in a Docker container. - -Platform versions, as specified using the ``--remote`` option with ``/`` removed, can also be skipped: - -- ``skip/freebsd11.1`` - Skip tests on FreeBSD 11.1. -- ``skip/rhel7.6`` - Skip tests on RHEL 7.6. - -Windows versions, as specified using the ``--windows`` option can also be skipped: - -- ``skip/windows/2016`` - Skip tests on Windows Server 2016. -- ``skip/windows/2019`` - Skip tests on Windows Server 2019. -- ``skip/windows/2022`` - Skip tests on Windows Server 2022. - -Aliases can be used to skip Python major versions using one of the following: - -- ``skip/python2`` - Skip tests on Python 2.x. -- ``skip/python3`` - Skip tests on Python 3.x. - -For more fine grained skipping, use conditionals in integration test playbooks, such as: - -.. code-block:: yaml - - when: ansible_distribution in ('Ubuntu') - - -Miscellaneous -------------- - -There are several other aliases available as well: - -- ``destructive`` - Requires ``--allow-destructive`` to run without ``--docker`` or ``--remote``. -- ``hidden`` - Target is ignored. Usable as a dependency. Automatic for ``setup_`` and ``prepare_`` prefixed targets. -- ``retry/never`` - Target is excluded from retries enabled by the ``--retry-on-error`` option. - -Unstable --------- - -Tests which fail sometimes should be marked with the ``unstable`` alias until the instability has been fixed. -These tests will continue to run for pull requests which modify the test or the module under test. - -This avoids unnecessary test failures for other pull requests, as well as tests on merge runs and nightly CI jobs. - -There are two ways to run unstable tests manually: - -- Use the ``--allow-unstable`` option for ``ansible-test`` -- Prefix the test name with ``unstable/`` when passing it to ``ansible-test``. - -Tests will be marked as unstable by a member of the Ansible Core Team. -GitHub issues_ will be created to track each unstable test. - -Disabled --------- - -Tests which always fail should be marked with the ``disabled`` alias until they can be fixed. - -Disabled tests are automatically skipped. - -There are two ways to run disabled tests manually: - -- Use the ``--allow-disabled`` option for ``ansible-test`` -- Prefix the test name with ``disabled/`` when passing it to ``ansible-test``. - -Tests will be marked as disabled by a member of the Ansible Core Team. -GitHub issues_ will be created to track each disabled test. - -Unsupported ------------ - -Tests which cannot be run in CI should be marked with the ``unsupported`` alias. -Most tests can be supported through the use of simulators and/or cloud plugins. - -However, if that is not possible then marking a test as unsupported will prevent it from running in CI. - -There are two ways to run unsupported tests manually: - -* Use the ``--allow-unsupported`` option for ``ansible-test`` -* Prefix the test name with ``unsupported/`` when passing it to ``ansible-test``. - -Tests will be marked as unsupported by the contributor of the test. - -Cloud ------ - -Tests for cloud services and other modules that require access to external APIs usually require special support for testing in CI. - -These require an additional alias to indicate the required test plugin. - -Some of the available aliases are: - -- ``cloud/aws`` -- ``cloud/azure`` -- ``cloud/cs`` -- ``cloud/digitalocean`` -- ``cloud/openshift`` -- ``cloud/vcenter`` - -Untested --------- - -Every module and plugin should have integration tests, even if the tests cannot be run in CI. - -Issues ------- - -Tests that are marked as unstable_ or disabled_ will have an issue created to track the status of the test. -Each issue will be assigned to one of the following projects: - -- `AWS <https://github.com/ansible/ansible/projects/21>`_ -- `Azure <https://github.com/ansible/ansible/projects/22>`_ -- `Windows <https://github.com/ansible/ansible/projects/23>`_ -- `General <https://github.com/ansible/ansible/projects/25>`_ - -Questions ---------- - -For questions about integration tests reach out to @mattclay or @gundalow on GitHub or the ``#ansible-devel`` chat channel (using Matrix at ansible.im or using IRC at `irc.libera.chat <https://libera.chat/>`_). diff --git a/docs/docsite/rst/dev_guide/testing/sanity/line-endings.rst b/docs/docsite/rst/dev_guide/testing/sanity/line-endings.rst deleted file mode 100644 index d56cfc12e5..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/line-endings.rst +++ /dev/null @@ -1,4 +0,0 @@ -line-endings -============ - -All files must use ``\n`` for line endings instead of ``\r\n``. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/metaclass-boilerplate.rst b/docs/docsite/rst/dev_guide/testing/sanity/metaclass-boilerplate.rst deleted file mode 100644 index c7327b39fa..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/metaclass-boilerplate.rst +++ /dev/null @@ -1,23 +0,0 @@ -metaclass-boilerplate -===================== - -Most Python files should include the following boilerplate at the top of the file, right after the -comment header and ``from __future__ import``: - -.. code-block:: python - - __metaclass__ = type - - -Python 2 has "new-style classes" and "old-style classes" whereas Python 3 only has new-style classes. -Adding the ``__metaclass__ = type`` boilerplate makes every class defined in that file into -a new-style class as well. - -.. code-block:: python - - from __future__ import absolute_import, division, print_function - __metaclass__ = type - - class Foo: - # This is a new-style class even on Python 2 because of the __metaclass__ - pass diff --git a/docs/docsite/rst/dev_guide/testing/sanity/mypy.rst b/docs/docsite/rst/dev_guide/testing/sanity/mypy.rst deleted file mode 100644 index 9eb46bafca..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/mypy.rst +++ /dev/null @@ -1,14 +0,0 @@ -mypy -==== - -The ``mypy`` static type checker is used to check the following code against each Python version supported by the controller: - - * ``lib/ansible/`` - * ``test/lib/ansible_test/_internal/`` - -Additionally, the following code is checked against Python versions supported only on managed nodes: - - * ``lib/ansible/modules/`` - * ``lib/ansible/module_utils/`` - -See `the mypy documentation <https://mypy.readthedocs.io/en/stable/>`_ diff --git a/docs/docsite/rst/dev_guide/testing/sanity/no-assert.rst b/docs/docsite/rst/dev_guide/testing/sanity/no-assert.rst deleted file mode 100644 index 489f917f62..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/no-assert.rst +++ /dev/null @@ -1,16 +0,0 @@ -no-assert -========= - -Do not use ``assert`` in production Ansible python code. When running Python -with optimizations, Python will remove ``assert`` statements, potentially -allowing for unexpected behavior throughout the Ansible code base. - -Instead of using ``assert`` you should utilize simple ``if`` statements, -that result in raising an exception. There is a new exception called -``AnsibleAssertionError`` that inherits from ``AnsibleError`` and -``AssertionError``. When possible, utilize a more specific exception -than ``AnsibleAssertionError``. - -Modules will not have access to ``AnsibleAssertionError`` and should instead -raise ``AssertionError``, a more specific exception, or just use -``module.fail_json`` at the failure point. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/no-basestring.rst b/docs/docsite/rst/dev_guide/testing/sanity/no-basestring.rst deleted file mode 100644 index f2fea137eb..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/no-basestring.rst +++ /dev/null @@ -1,11 +0,0 @@ -no-basestring -============= - -Do not use ``isinstance(s, basestring)`` as basestring has been removed in -Python3. You can import ``string_types``, ``binary_type``, or ``text_type`` -from ``ansible.module_utils.six`` and then use ``isinstance(s, string_types)`` -or ``isinstance(s, (binary_type, text_type))`` instead. - -If this is part of code to convert a string to a particular type, -``ansible.module_utils.common.text.converters`` contains several functions -that may be even better for you: ``to_text``, ``to_bytes``, and ``to_native``. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/no-dict-iteritems.rst b/docs/docsite/rst/dev_guide/testing/sanity/no-dict-iteritems.rst deleted file mode 100644 index e231c796c4..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/no-dict-iteritems.rst +++ /dev/null @@ -1,16 +0,0 @@ -no-dict-iteritems -================= - -The ``dict.iteritems`` method has been removed in Python 3. There are two recommended alternatives: - -.. code-block:: python - - for KEY, VALUE in DICT.items(): - pass - -.. code-block:: python - - from ansible.module_utils.six import iteritems - - for KEY, VALUE in iteritems(DICT): - pass diff --git a/docs/docsite/rst/dev_guide/testing/sanity/no-dict-iterkeys.rst b/docs/docsite/rst/dev_guide/testing/sanity/no-dict-iterkeys.rst deleted file mode 100644 index 9dc4a97855..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/no-dict-iterkeys.rst +++ /dev/null @@ -1,9 +0,0 @@ -no-dict-iterkeys -================ - -The ``dict.iterkeys`` method has been removed in Python 3. Use the following instead: - -.. code-block:: python - - for KEY in DICT: - pass diff --git a/docs/docsite/rst/dev_guide/testing/sanity/no-dict-itervalues.rst b/docs/docsite/rst/dev_guide/testing/sanity/no-dict-itervalues.rst deleted file mode 100644 index 979450e418..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/no-dict-itervalues.rst +++ /dev/null @@ -1,16 +0,0 @@ -no-dict-itervalues -================== - -The ``dict.itervalues`` method has been removed in Python 3. There are two recommended alternatives: - -.. code-block:: python - - for VALUE in DICT.values(): - pass - -.. code-block:: python - - from ansible.module_utils.six import itervalues - - for VALUE in itervalues(DICT): - pass diff --git a/docs/docsite/rst/dev_guide/testing/sanity/no-get-exception.rst b/docs/docsite/rst/dev_guide/testing/sanity/no-get-exception.rst deleted file mode 100644 index 67f1646f21..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/no-get-exception.rst +++ /dev/null @@ -1,28 +0,0 @@ -no-get-exception -================ - -We created a function, ``ansible.module_utils.pycompat24.get_exception`` to -help retrieve exceptions in a manner compatible with Python 2.4 through -Python 3.6. We no longer support Python 2.4 and Python 2.5 so this is -extraneous and we want to deprecate the function. Porting code should look -something like this: - -.. code-block:: python - - # Unfixed code: - try: - raise IOError('test') - except IOError: - e = get_exception() - do_something(e) - except: - e = get_exception() - do_something_else(e) - - # After fixing: - try: - raise IOError('test') - except IOErrors as e: - do_something(e) - except Exception as e: - do_something_else(e) diff --git a/docs/docsite/rst/dev_guide/testing/sanity/no-illegal-filenames.rst b/docs/docsite/rst/dev_guide/testing/sanity/no-illegal-filenames.rst deleted file mode 100644 index 6e6f565eed..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/no-illegal-filenames.rst +++ /dev/null @@ -1,61 +0,0 @@ -no-illegal-filenames -==================== - -Files and directories should not contain illegal characters or names so that -Ansible can be checked out on any Operating System. - -Illegal Characters ------------------- - -The following characters are not allowed to be used in any part of the file or -directory name; - -* ``<`` -* ``>`` -* ``:`` -* ``"`` -* ``/`` -* ``\`` -* ``|`` -* ``?`` -* ``*`` -* Any characters whose integer representations are in the range from 0 through to 31 like ``\n`` - -The following characters are not allowed to be used as the last character of a -file or directory; - -* ``.`` -* ``" "`` (just the space character) - -Illegal Names -------------- - -The following names are not allowed to be used as the name of a file or -directory excluding the extension; - -* ``CON`` -* ``PRN`` -* ``AUX`` -* ``NUL`` -* ``COM1`` -* ``COM2`` -* ``COM3`` -* ``COM4`` -* ``COM5`` -* ``COM6`` -* ``COM7`` -* ``COM8`` -* ``COM9`` -* ``LPT1`` -* ``LPT2`` -* ``LPT3`` -* ``LPT4`` -* ``LPT5`` -* ``LPT6`` -* ``LPT7`` -* ``LPT8`` -* ``LPT9`` - -For example, the file ``folder/COM1``, ``folder/COM1.txt`` are illegal but -``folder/COM1-file`` or ``folder/COM1-file.txt`` is allowed. - diff --git a/docs/docsite/rst/dev_guide/testing/sanity/no-main-display.rst b/docs/docsite/rst/dev_guide/testing/sanity/no-main-display.rst deleted file mode 100644 index 271f88f188..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/no-main-display.rst +++ /dev/null @@ -1,14 +0,0 @@ -no-main-display -=============== - -As of Ansible 2.8, ``Display`` should no longer be imported from ``__main__``. - -``Display`` is now a singleton and should be utilized like the following: - -.. code-block:: python - - from ansible.utils.display import Display - display = Display() - -There is no longer a need to attempt ``from __main__ import display`` inside -a ``try/except`` block. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/no-smart-quotes.rst b/docs/docsite/rst/dev_guide/testing/sanity/no-smart-quotes.rst deleted file mode 100644 index 50dc7bafc9..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/no-smart-quotes.rst +++ /dev/null @@ -1,4 +0,0 @@ -no-smart-quotes -=============== - -Smart quotes (``”“‘’``) should not be used. Use plain ascii quotes (``"'``) instead. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/no-tests-as-filters.rst b/docs/docsite/rst/dev_guide/testing/sanity/no-tests-as-filters.rst deleted file mode 100644 index 0c1f99ac78..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/no-tests-as-filters.rst +++ /dev/null @@ -1,12 +0,0 @@ -:orphan: - -no-tests-as-filters -=================== - -Using Ansible provided Jinja2 tests as filters will be removed in Ansible 2.9. - -Prior to Ansible 2.5, Jinja2 tests included within Ansible were most often used as filters. The large difference in use is that filters are referenced as ``variable | filter_name`` while Jinja2 tests are referenced as ``variable is test_name``. - -Jinja2 tests are used for comparisons, whereas filters are used for data manipulation, and have different applications in Jinja2. This change is to help differentiate the concepts for a better understanding of Jinja2, and where each can be appropriately used. - -As of Ansible 2.5 using an Ansible provided Jinja2 test with filter syntax will display a deprecation error. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/no-underscore-variable.rst b/docs/docsite/rst/dev_guide/testing/sanity/no-underscore-variable.rst deleted file mode 100644 index 5174a43adf..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/no-underscore-variable.rst +++ /dev/null @@ -1,30 +0,0 @@ -:orphan: - -no-underscore-variable -====================== - -In the future, Ansible may use the identifier ``_`` to internationalize its -message strings. To be ready for that, we need to make sure that there are -no conflicting identifiers defined in the code base. - -In common practice, ``_`` is frequently used as a dummy variable (a variable -to receive a value from a function where the value is useless and never used). -In Ansible, we're using the identifier ``dummy`` for this purpose instead. - -Example of unfixed code: - -.. code-block:: python - - for _ in range(0, retries): - success = retry_thing() - if success: - break - -Example of fixed code: - -.. code-block:: python - - for dummy in range(0, retries): - success = retry_thing() - if success: - break diff --git a/docs/docsite/rst/dev_guide/testing/sanity/no-unicode-literals.rst b/docs/docsite/rst/dev_guide/testing/sanity/no-unicode-literals.rst deleted file mode 100644 index f8ca1d2c54..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/no-unicode-literals.rst +++ /dev/null @@ -1,16 +0,0 @@ -no-unicode_literals -=================== - -The use of :code:`from __future__ import unicode_literals` has been deemed an anti-pattern. The -problems with it are: - -* It makes it so one can't jump into the middle of a file and know whether a bare literal string is - a byte string or text string. The programmer has to first check the top of the file to see if the - import is there. -* It removes the ability to define native strings (a string which should be a byte string on python2 - and a text string on python3) by a string literal. -* It makes for more context switching. A programmer could be reading one file which has - `unicode_literals` and know that bare string literals are text strings but then switch to another - file (perhaps tracing program execution into a third party library) and have to switch their - understanding of what bare string literals are. - diff --git a/docs/docsite/rst/dev_guide/testing/sanity/no-unwanted-files.rst b/docs/docsite/rst/dev_guide/testing/sanity/no-unwanted-files.rst deleted file mode 100644 index 3d76324eea..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/no-unwanted-files.rst +++ /dev/null @@ -1,13 +0,0 @@ -no-unwanted-files -================= - -Specific file types are allowed in certain directories: - -- ``lib`` - All content must reside in the ``lib/ansible`` directory. - -- ``lib/ansible`` - Only source code with one of the following extensions is allowed: - - - ``*.cs`` - C# - - ``*.ps1`` - PowerShell - - ``*.psm1`` - PowerShell - - ``*.py`` - Python diff --git a/docs/docsite/rst/dev_guide/testing/sanity/no-wildcard-import.rst b/docs/docsite/rst/dev_guide/testing/sanity/no-wildcard-import.rst deleted file mode 100644 index fdaf07b097..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/no-wildcard-import.rst +++ /dev/null @@ -1,31 +0,0 @@ -:orphan: - -no-wildcard-import -================== - -Using :code:`import *` is a bad habit which pollutes your namespace, hinders -debugging, and interferes with static analysis of code. For those reasons, we -do want to limit the use of :code:`import *` in the ansible code. Change our -code to import the specific names that you need instead. - -Examples of unfixed code: - -.. code-block:: python - - from ansible.module_utils.six import * - if isinstance(variable, string_types): - do_something(variable) - - from ansible.module_utils.basic import * - module = AnsibleModule() - -Examples of fixed code: - -.. code-block:: python - - from ansible.module_utils import six - if isinstance(variable, six.string_types): - do_something(variable) - - from ansible.module_utils.basic import AnsibleModule - module = AnsibleModule() diff --git a/docs/docsite/rst/dev_guide/testing/sanity/obsolete-files.rst b/docs/docsite/rst/dev_guide/testing/sanity/obsolete-files.rst deleted file mode 100644 index cb23746894..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/obsolete-files.rst +++ /dev/null @@ -1,14 +0,0 @@ -obsolete-files -============== - -Directories in the Ansible source tree are sometimes made obsolete. -Files should not exist in these directories. -The new location (if any) is dependent on which directory has been made obsolete. - -Below are some of the obsolete directories and their new locations: - -- All of ``test/runner/`` is now under ``test/lib/ansible_test/`` instead. The organization of files in the new directory has changed. -- Most subdirectories of ``test/sanity/`` (with some exceptions) are now under ``test/lib/ansible_test/_util/controller/sanity/`` instead. - -This error occurs most frequently for open pull requests which add or modify files in directories which are now obsolete. -Make sure the branch you are working from is current so that changes can be made in the correct location. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/package-data.rst b/docs/docsite/rst/dev_guide/testing/sanity/package-data.rst deleted file mode 100644 index 220872dd28..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/package-data.rst +++ /dev/null @@ -1,5 +0,0 @@ -package-data -============ - -Verifies that the combination of ``MANIFEST.in`` and ``package_data`` from ``setup.py`` -properly installs data files from within ``lib/ansible`` diff --git a/docs/docsite/rst/dev_guide/testing/sanity/pep8.rst b/docs/docsite/rst/dev_guide/testing/sanity/pep8.rst deleted file mode 100644 index 9424bda8ab..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/pep8.rst +++ /dev/null @@ -1,24 +0,0 @@ -.. _testing_pep8: - -pep8 -==== - -Python static analysis for PEP 8 style guideline compliance. - -`PEP 8`_ style guidelines are enforced by `pycodestyle`_ on all python files in the repository by default. - -Running locally ------------------ - -The `PEP 8`_ check can be run locally as follows: - -.. code-block:: shell - - ansible-test sanity --test pep8 [file-or-directory-path-to-check] ... - - - -.. _PEP 8: https://www.python.org/dev/peps/pep-0008/ -.. _pycodestyle: https://pypi.org/project/pycodestyle/ - - diff --git a/docs/docsite/rst/dev_guide/testing/sanity/pslint.rst b/docs/docsite/rst/dev_guide/testing/sanity/pslint.rst deleted file mode 100644 index baa4fa034f..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/pslint.rst +++ /dev/null @@ -1,4 +0,0 @@ -pslint -====== - -PowerShell static analysis for common programming errors using `PSScriptAnalyzer <https://github.com/PowerShell/PSScriptAnalyzer/>`_. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/pylint-ansible-test.rst b/docs/docsite/rst/dev_guide/testing/sanity/pylint-ansible-test.rst deleted file mode 100644 index a80ddc1eb0..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/pylint-ansible-test.rst +++ /dev/null @@ -1,8 +0,0 @@ -:orphan: - -pylint-ansible-test -=================== - -Python static analysis for common programming errors. - -A more strict set of rules applied to ``ansible-test``. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/pylint.rst b/docs/docsite/rst/dev_guide/testing/sanity/pylint.rst deleted file mode 100644 index 2b2ef9e53b..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/pylint.rst +++ /dev/null @@ -1,4 +0,0 @@ -pylint -====== - -Python static analysis for common programming errors. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/release-names.rst b/docs/docsite/rst/dev_guide/testing/sanity/release-names.rst deleted file mode 100644 index 359f7ecb54..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/release-names.rst +++ /dev/null @@ -1,4 +0,0 @@ -Release names -============= - -Verifies that the most recent release name has been added to ``./github/RELEASE_NAMES.yml`` diff --git a/docs/docsite/rst/dev_guide/testing/sanity/replace-urlopen.rst b/docs/docsite/rst/dev_guide/testing/sanity/replace-urlopen.rst deleted file mode 100644 index 705195c942..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/replace-urlopen.rst +++ /dev/null @@ -1,4 +0,0 @@ -replace-urlopen -=============== - -Use ``open_url`` from ``module_utils`` instead of ``urlopen``. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/required-and-default-attributes.rst b/docs/docsite/rst/dev_guide/testing/sanity/required-and-default-attributes.rst deleted file mode 100644 index 573c361500..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/required-and-default-attributes.rst +++ /dev/null @@ -1,5 +0,0 @@ -required-and-default-attributes -=============================== - -Use only one of ``default`` or ``required`` with ``FieldAttribute``. - diff --git a/docs/docsite/rst/dev_guide/testing/sanity/rstcheck.rst b/docs/docsite/rst/dev_guide/testing/sanity/rstcheck.rst deleted file mode 100644 index 8fcbbce343..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/rstcheck.rst +++ /dev/null @@ -1,4 +0,0 @@ -rstcheck -======== - -Check reStructuredText files for syntax and formatting issues. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/runtime-metadata.rst b/docs/docsite/rst/dev_guide/testing/sanity/runtime-metadata.rst deleted file mode 100644 index 1f3c32ad77..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/runtime-metadata.rst +++ /dev/null @@ -1,7 +0,0 @@ -runtime-metadata.yml -==================== - -Validates the schema for: - -* ansible-core's ``lib/ansible/config/ansible_builtin_runtime.yml`` -* collection's ``meta/runtime.yml`` diff --git a/docs/docsite/rst/dev_guide/testing/sanity/sanity-docs.rst b/docs/docsite/rst/dev_guide/testing/sanity/sanity-docs.rst deleted file mode 100644 index 34265c34be..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/sanity-docs.rst +++ /dev/null @@ -1,4 +0,0 @@ -sanity-docs -=========== - -Documentation for each ``ansible-test sanity`` test is required. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/shebang.rst b/docs/docsite/rst/dev_guide/testing/sanity/shebang.rst deleted file mode 100644 index cff2aa0914..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/shebang.rst +++ /dev/null @@ -1,16 +0,0 @@ -shebang -======= - -Most executable files should only use one of the following shebangs: - -- ``#!/bin/sh`` -- ``#!/bin/bash`` -- ``#!/usr/bin/make`` -- ``#!/usr/bin/env python`` -- ``#!/usr/bin/env bash`` - -NOTE: For ``#!/bin/bash``, any of the options ``eux`` may also be used, such as ``#!/bin/bash -eux``. - -This does not apply to Ansible modules, which should not be executable and must always use ``#!/usr/bin/python``. - -Some exceptions are permitted. Ask if you have questions. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/shellcheck.rst b/docs/docsite/rst/dev_guide/testing/sanity/shellcheck.rst deleted file mode 100644 index 446ee1ee78..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/shellcheck.rst +++ /dev/null @@ -1,4 +0,0 @@ -shellcheck -========== - -Static code analysis for shell scripts using the excellent `shellcheck <https://www.shellcheck.net/>`_ tool. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/symlinks.rst b/docs/docsite/rst/dev_guide/testing/sanity/symlinks.rst deleted file mode 100644 index 017209bdd3..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/symlinks.rst +++ /dev/null @@ -1,6 +0,0 @@ -symlinks -======== - -Symbolic links are only permitted for files that exist to ensure proper tarball generation during a release. - -If other types of symlinks are needed for tests they must be created as part of the test. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/test-constraints.rst b/docs/docsite/rst/dev_guide/testing/sanity/test-constraints.rst deleted file mode 100644 index 36ceb36130..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/test-constraints.rst +++ /dev/null @@ -1,4 +0,0 @@ -test-constraints -================ - -Constraints for test requirements should be in ``test/lib/ansible_test/_data/requirements/constraints.txt``. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/update-bundled.rst b/docs/docsite/rst/dev_guide/testing/sanity/update-bundled.rst deleted file mode 100644 index d8f19385b3..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/update-bundled.rst +++ /dev/null @@ -1,31 +0,0 @@ -:orphan: - -update-bundled -============== - -Check whether any of our known bundled code needs to be updated for a new upstream release. - -This test can error in the following ways: - -* The bundled code is out of date with regard to the latest release on pypi. Update the code - to the new version and update the version in _BUNDLED_METADATA to solve this. - -* The code is lacking a _BUNDLED_METADATA variable. This typically happens when a bundled version - is updated and we forget to add a _BUNDLED_METADATA variable to the updated file. Once that is - added, this error should go away. - -* A file has a _BUNDLED_METADATA variable but the file isn't specified in - :file:`test/sanity/code-smell/update-bundled.py`. This typically happens when a new bundled - library is added. Add the file to the `get_bundled_libs()` function in the `update-bundled.py` - test script to solve this error. - -_BUNDLED_METADATA has the following fields: - -:pypi_name: Name of the bundled package on pypi - -:version: Version of the package that we are including here - -:version_constraints: Optional PEP440 specifier for the version range that we are bundling. - Currently, the only valid use of this is to follow a version that is - compatible with the Python stdlib when newer versions of the pypi package - implement a new API. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/use-argspec-type-path.rst b/docs/docsite/rst/dev_guide/testing/sanity/use-argspec-type-path.rst deleted file mode 100644 index e06d83dd1a..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/use-argspec-type-path.rst +++ /dev/null @@ -1,10 +0,0 @@ -use-argspec-type-path -===================== - -The AnsibleModule argument_spec knows of several types beyond the standard python types. One of -these is ``path``. When used, type ``path`` ensures that an argument is a string and expands any -shell variables and tilde characters. - -This test looks for use of :func:`os.path.expanduser <python:os.path.expanduser>` in modules. When found, it tells the user to -replace it with ``type='path'`` in the module's argument_spec or list it as a false positive in the -test. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/use-compat-six.rst b/docs/docsite/rst/dev_guide/testing/sanity/use-compat-six.rst deleted file mode 100644 index 1f4150056d..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/use-compat-six.rst +++ /dev/null @@ -1,4 +0,0 @@ -use-compat-six -============== - -Use ``six`` from ``module_utils`` instead of ``six``. diff --git a/docs/docsite/rst/dev_guide/testing/sanity/validate-modules.rst b/docs/docsite/rst/dev_guide/testing/sanity/validate-modules.rst deleted file mode 100644 index e2c20f4ddd..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/validate-modules.rst +++ /dev/null @@ -1,141 +0,0 @@ -.. _testing_validate-modules: - -validate-modules -================ - -Analyze modules for common issues in code and documentation. - -.. contents:: - :local: - -Usage ------- - -.. code:: shell - - cd /path/to/ansible/source - source hacking/env-setup - ansible-test sanity --test validate-modules - -Help ------ - -Type ``ansible-test sanity validate-modules -h`` to display help for using this sanity test. - - - -Extending validate-modules ---------------------------- - -The ``validate-modules`` tool has a `schema.py <https://github.com/ansible/ansible/blob/devel/test/lib/ansible_test/_util/controller/sanity/validate-modules/validate_modules/schema.py>`_ that is used to validate the YAML blocks, such as ``DOCUMENTATION`` and ``RETURNS``. - - -Codes ------- - -============================================================ ================== ==================== ========================================================================================= - **Error Code** **Type** **Level** **Sample Message** ------------------------------------------------------------- ------------------ -------------------- ----------------------------------------------------------------------------------------- - ansible-deprecated-module Documentation Error A module is deprecated and supposed to be removed in the current or an earlier Ansible version - collection-deprecated-module Documentation Error A module is deprecated and supposed to be removed in the current or an earlier collection version - ansible-deprecated-version Documentation Error A feature is deprecated and supposed to be removed in the current or an earlier Ansible version - ansible-module-not-initialized Syntax Error Execution of the module did not result in initialization of AnsibleModule - collection-deprecated-version Documentation Error A feature is deprecated and supposed to be removed in the current or an earlier collection version - deprecated-date Documentation Error A date before today appears as ``removed_at_date`` or in ``deprecated_aliases`` - deprecation-mismatch Documentation Error Module marked as deprecated or removed in at least one of the filename, its metadata, or in DOCUMENTATION (setting DOCUMENTATION.deprecated for deprecation or removing all Documentation for removed) but not in all three places. - doc-choices-do-not-match-spec Documentation Error Value for "choices" from the argument_spec does not match the documentation - doc-choices-incompatible-type Documentation Error Choices value from the documentation is not compatible with type defined in the argument_spec - doc-default-does-not-match-spec Documentation Error Value for "default" from the argument_spec does not match the documentation - doc-default-incompatible-type Documentation Error Default value from the documentation is not compatible with type defined in the argument_spec - doc-elements-invalid Documentation Error Documentation specifies elements for argument, when "type" is not ``list``. - doc-elements-mismatch Documentation Error Argument_spec defines elements different than documentation does - doc-missing-type Documentation Error Documentation doesn't specify a type but argument in ``argument_spec`` use default type (``str``) - doc-required-mismatch Documentation Error argument in argument_spec is required but documentation says it is not, or vice versa - doc-type-does-not-match-spec Documentation Error Argument_spec defines type different than documentation does - documentation-error Documentation Error Unknown ``DOCUMENTATION`` error - documentation-syntax-error Documentation Error Invalid ``DOCUMENTATION`` schema - import-before-documentation Imports Error Import found before documentation variables. All imports must appear below ``DOCUMENTATION``/``EXAMPLES``/``RETURN`` - import-error Documentation Error ``Exception`` attempting to import module for ``argument_spec`` introspection - attributes-check-mode Documentation Error If a module documents the ``check_mode`` attribute, its ``support`` value must be compatible with the ``supports_check_mode`` parameter of ``AnsibleModule`` - attributes-check-mode-details Documentation Error If a module documents the ``check_mode`` attribute with support values ``partial`` or ``N/A``, it must provide ``details`` - import-placement Locations Warning Imports should be directly below ``DOCUMENTATION``/``EXAMPLES``/``RETURN`` - imports-improper-location Imports Error Imports should be directly below ``DOCUMENTATION``/``EXAMPLES``/``RETURN`` - incompatible-choices Documentation Error Choices value from the argument_spec is not compatible with type defined in the argument_spec - incompatible-default-type Documentation Error Default value from the argument_spec is not compatible with type defined in the argument_spec - invalid-argument-name Documentation Error Argument in argument_spec must not be one of 'message', 'syslog_facility' as it is used internally by Ansible Core Engine - invalid-argument-spec Documentation Error Argument in argument_spec must be a dictionary/hash when used - invalid-argument-spec-options Documentation Error Suboptions in argument_spec are invalid - invalid-documentation Documentation Error ``DOCUMENTATION`` is not valid YAML - invalid-documentation-markup Documentation Error ``DOCUMENTATION`` or ``RETURN`` contains invalid markup - invalid-documentation-options Documentation Error ``DOCUMENTATION.options`` must be a dictionary/hash when used - invalid-examples Documentation Error ``EXAMPLES`` is not valid YAML - invalid-extension Naming Error Official Ansible modules must have a ``.py`` extension for python modules or a ``.ps1`` for powershell modules - invalid-module-schema Documentation Error ``AnsibleModule`` schema validation error - invalid-removal-version Documentation Error The version at which a feature is supposed to be removed cannot be parsed (for collections, it must be a `semantic version <https://semver.org/>`_) - invalid-requires-extension Naming Error Module ``#AnsibleRequires -CSharpUtil`` should not end in .cs, Module ``#Requires`` should not end in .psm1 - missing-doc-fragment Documentation Error ``DOCUMENTATION`` fragment missing - missing-existing-doc-fragment Documentation Warning Pre-existing ``DOCUMENTATION`` fragment missing - missing-documentation Documentation Error No ``DOCUMENTATION`` provided - missing-examples Documentation Error No ``EXAMPLES`` provided - missing-gplv3-license Documentation Error GPLv3 license header not found - missing-module-utils-basic-import Imports Warning Did not find ``ansible.module_utils.basic`` import - missing-module-utils-import-csharp-requirements Imports Error No ``Ansible.ModuleUtils`` or C# Ansible util requirements/imports found - missing-powershell-interpreter Syntax Error Interpreter line is not ``#!powershell`` - missing-python-interpreter Syntax Error Interpreter line is not ``#!/usr/bin/python`` - missing-return Documentation Error No ``RETURN`` documentation provided - missing-return-legacy Documentation Warning No ``RETURN`` documentation provided for legacy module - missing-suboption-docs Documentation Error Argument in argument_spec has sub-options but documentation does not define sub-options - module-incorrect-version-added Documentation Error Module level ``version_added`` is incorrect - module-invalid-version-added Documentation Error Module level ``version_added`` is not a valid version number - module-utils-specific-import Imports Error ``module_utils`` imports should import specific components, not ``*`` - multiple-utils-per-requires Imports Error ``Ansible.ModuleUtils`` requirements do not support multiple modules per statement - multiple-csharp-utils-per-requires Imports Error Ansible C# util requirements do not support multiple utils per statement - no-default-for-required-parameter Documentation Error Option is marked as required but specifies a default. Arguments with a default should not be marked as required - no-log-needed Parameters Error Option name suggests that the option contains a secret value, while ``no_log`` is not specified for this option in the argument spec. If this is a false positive, explicitly set ``no_log=False`` - nonexistent-parameter-documented Documentation Error Argument is listed in DOCUMENTATION.options, but not accepted by the module - option-incorrect-version-added Documentation Error ``version_added`` for new option is incorrect - option-invalid-version-added Documentation Error ``version_added`` for option is not a valid version number - parameter-invalid Documentation Error Argument in argument_spec is not a valid python identifier - parameter-invalid-elements Documentation Error Value for "elements" is valid only when value of "type" is ``list`` - implied-parameter-type-mismatch Documentation Error Argument_spec implies ``type="str"`` but documentation defines it as different data type - parameter-type-not-in-doc Documentation Error Type value is defined in ``argument_spec`` but documentation doesn't specify a type - parameter-alias-repeated Parameters Error argument in argument_spec has at least one alias specified multiple times in aliases - parameter-alias-self Parameters Error argument in argument_spec is specified as its own alias - parameter-documented-multiple-times Documentation Error argument in argument_spec with aliases is documented multiple times - parameter-list-no-elements Parameters Error argument in argument_spec "type" is specified as ``list`` without defining "elements" - parameter-state-invalid-choice Parameters Error Argument ``state`` includes ``get``, ``list`` or ``info`` as a choice. Functionality should be in an ``_info`` or (if further conditions apply) ``_facts`` module. - python-syntax-error Syntax Error Python ``SyntaxError`` while parsing module - removal-version-must-be-major Documentation Error According to the semantic versioning specification (https://semver.org/), the only versions in which features are allowed to be removed are major versions (x.0.0) - return-syntax-error Documentation Error ``RETURN`` is not valid YAML, ``RETURN`` fragments missing or invalid - return-invalid-version-added Documentation Error ``version_added`` for return value is not a valid version number - subdirectory-missing-init Naming Error Ansible module subdirectories must contain an ``__init__.py`` - try-except-missing-has Imports Warning Try/Except ``HAS_`` expression missing - undocumented-parameter Documentation Error Argument is listed in the argument_spec, but not documented in the module - unidiomatic-typecheck Syntax Error Type comparison using ``type()`` found. Use ``isinstance()`` instead - unknown-doc-fragment Documentation Warning Unknown pre-existing ``DOCUMENTATION`` error - use-boto3 Imports Error ``boto`` import found, new modules should use ``boto3`` - use-fail-json-not-sys-exit Imports Error ``sys.exit()`` call found. Should be ``exit_json``/``fail_json`` - use-module-utils-urls Imports Error ``requests`` import found, should use ``ansible.module_utils.urls`` instead - use-run-command-not-os-call Imports Error ``os.call`` used instead of ``module.run_command`` - use-run-command-not-popen Imports Error ``subprocess.Popen`` used instead of ``module.run_command`` - use-short-gplv3-license Documentation Error GPLv3 license header should be the :ref:`short form <copyright>` for new modules - mutually_exclusive-type Documentation Error mutually_exclusive entry contains non-string value - mutually_exclusive-collision Documentation Error mutually_exclusive entry has repeated terms - mutually_exclusive-unknown Documentation Error mutually_exclusive entry contains option which does not appear in argument_spec (potentially an alias of an option?) - required_one_of-type Documentation Error required_one_of entry contains non-string value - required_one_of-collision Documentation Error required_one_of entry has repeated terms - required_one_of-unknown Documentation Error required_one_of entry contains option which does not appear in argument_spec (potentially an alias of an option?) - required_together-type Documentation Error required_together entry contains non-string value - required_together-collision Documentation Error required_together entry has repeated terms - required_together-unknown Documentation Error required_together entry contains option which does not appear in argument_spec (potentially an alias of an option?) - required_if-is_one_of-type Documentation Error required_if entry has a fourth value which is not a bool - required_if-requirements-type Documentation Error required_if entry has a third value (requirements) which is not a list or tuple - required_if-requirements-collision Documentation Error required_if entry has repeated terms in requirements - required_if-requirements-unknown Documentation Error required_if entry's requirements contains option which does not appear in argument_spec (potentially an alias of an option?) - required_if-unknown-key Documentation Error required_if entry's key does not appear in argument_spec (potentially an alias of an option?) - required_if-key-in-requirements Documentation Error required_if entry contains its key in requirements list/tuple - required_if-value-type Documentation Error required_if entry's value is not of the type specified for its key - required_by-collision Documentation Error required_by entry has repeated terms - required_by-unknown Documentation Error required_by entry contains option which does not appear in argument_spec (potentially an alias of an option?) - version-added-must-be-major-or-minor Documentation Error According to the semantic versioning specification (https://semver.org/), the only versions in which features are allowed to be added are major and minor versions (x.y.0) -============================================================ ================== ==================== ========================================================================================= diff --git a/docs/docsite/rst/dev_guide/testing/sanity/yamllint.rst b/docs/docsite/rst/dev_guide/testing/sanity/yamllint.rst deleted file mode 100644 index 5822bb7c1e..0000000000 --- a/docs/docsite/rst/dev_guide/testing/sanity/yamllint.rst +++ /dev/null @@ -1,4 +0,0 @@ -yamllint -======== - -Check YAML files for syntax and formatting issues. |