Reviewed copy of Planning and releasing API versions

DRAFT FOR REVIEW (please use comments on the page - when selecting text the comment button will appear)

WARNING: PAGE is currently inconsistent - UPDATE ongoing to decouple release name/number from contained API version(s)

This section gives the overview of the handling of API versions throughout the release management process.

The Release Management process is based on creating releases for API versions, aligned with given Commonalities and ICM release in a given meta-release.

The release of an API version consists in:

• Putting a release tag on the main or on a release branch (the latter in case of changes to a public-released API) holding the API version 

• Creating a release package with the required API release assets for the API version. 

The following types of API versions are used throughout the API's life cycle:

• initial (0.y.z) or alpha (x.y.z-alpha.m) API versions (with version extensions) for CAMARA internal API rapid development purposes

• release-candidate (x.y.z-rc.n) API versions (with version extensions) for CAMARA internal API release bug fixing purposes

• public-release (x.y.z) API versions for API versions that are part of a meta-release. These API versions only have API version number x.y.z (semver 2.0), no version extension.

• work-in-progress (wip) API versions used during the development of an API in between pre-releases. Such API versions cannot be released and is not usable by API consumers.

Note 1: People outside CAMARA are free to use any of the API version pre-releases (initial, alpha or release-candidate) made available in the CAMARA GitHub, BUT AT THEIR OWN RISK.

Note 2: API families are not versioned, nor release-managed. Only API versions are release-managed.

The use of API version types throughout the API life cycle is illustrated in the following figure.

Open

The next sections provide the definitions and the description of the release management process for APIs, and illustrates by examples.

The rest of the text in this section can be removed once the proposal is agreed.

It consolidates inputs from 

• Rafal's proposal for the CAMARA API Maturity Levels - proposal - CAMARA Project - Confluence and Herbert's comments

• the inputs from Consolidation Issue for open points release management · Issue #9 · camaraproject/ReleaseManagement (github.com).

• Proposal to establish API Families as Working Groups across API Sub Projects · Issue #7 · camaraproject/ReleaseManagement (github.com)

• Open Issues Consolidated - CAMARA Project - Confluence

• Multiple APIs in an API Sub-project (dedicated meeting)

Table of Contents

Definitions

API version

The API version is defined in the "version" field of the OAS definition of an API. Its content must follow Semantic Versioning 2.0.0 | Semantic Versioning (semver.org).

An API version has the format:  x.y.z  where x, y and z are numbers corresponding to MAJOR, MINOR and PATCH releases.

The following bullets are extracted of the semver 2.0 specification (and links to the corresponding spec item):

• Major version zero (0.y.z) is for initial development. Anything MAY change at any time. The public API SHOULD NOT be considered stable. https://semver.org/#spec-item-4

• Version 1.0.0 defines the public API. The way in which the version number is incremented after this release is dependent on this public API and how it changes. https://semver.org/#spec-item-5

  • Patch version Z (x.y.Z | x > 0) MUST be incremented if only backward compatible bug fixes are introduced. A bug fix is defined as an internal change that fixes incorrect behavior. https://semver.org/#spec-item-6

  • Minor version Y (x.Y.z | x > 0) MUST be incremented if new, backward compatible functionality is introduced to the public API. It MUST be incremented if any public API functionality is marked as deprecated. It MAY be incremented if substantial new functionality or improvements are introduced within the private code. It MAY include patch level changes. Patch version MUST be reset to 0 when minor version is incremented. 

  • Major version X (X.y.z | X > 0) MUST be incremented if any backward incompatible changes are introduced to the public API. It MAY also include minor and patch level changes. Patch and minor versions MUST be reset to 0 when major version is incremented.

Precedence example:  1.0.0 < 2.0.0 < 2.1.0 < 2.1.1 < 3.0.0.

The CAMARA API guidelines describe the applicable versioning strategy. NOTE: if this proposal is accepted, some updates of the guidelines may be needed.

API version extensions 

During the API development and release preparation, alpha (-alpha) or release-candidate (-rc) extensions shall be used to identify intermediate stable versions of an API.  

The notions of alpha and release-candidate API versions, and their related API version extensions, are INTERNAL to the CAMARA project and used only for the API development and release management process.

The API version extension is added to the "version" field of the OAS definition of an API, as well as to the URL (see below table). Its content must follow the here described syntax and semantics.

The following API version extensions shall be used in the API version:

• alpha API versions: -alpha.m

• release-candidate API versions: -rc.n

• no extension - this is allowed only for initial API versions 0.y.z (with x=0).

Extensions are post-fixed to the API version numbers and separated from it by a hyphen "-". m and n are numbers starting at 1.

All extensions must have a number (so just "-alpha" or "-rc" is not allowed) and should be minimally "-alpha.1" or "-rc.1".

Precedence example: 1.0.0 < 1.1.0-alpha.1 < 1.1.0-alpha.2 < 1.1.0-rc.1 < 1.1.0-rc.2 < 1.1.0 (public-release)

NOTE: a public-release API version shall have no extension in the API version field of the OAS definition file, only the major.minor.patch version number as described in the previous section.

API version in URL

The API guidelines contain the following approach for defining the API endpoint URL:

^{servers: - url: "{apiRoot}/qod/v2" variables: apiRoot: default: http://localhost:9091 description: API root}

The apiRoot is to be adapted by the implementer of the API endpoint (the API provider).

Once a public-release API version is created and published as part of a meta-release, the API version in the URL of the public-release API version shall ONLY contain the MAJOR digit of the API version, e.g. "v2" in the above example.

For alpha and release-candidate API versions, the API version extension shall be included in the URL, but without any hyphens or dots.

Exception: For initial API versions:

• 0.y.z, the second version number (y) may be include in the URL, e.g. qod/v0.y, to simplify testing of API implementations during initial API development.

• In this exception case, the y is used for major (breaking) changes, and the z is used for minor (non-breaking) changes. 

• The dot (".") is kept for readability reasons. See initial API version section below for exception examples.

API name

The API name used in this document is the segment in the url field in the OAS definition file before the segment holding the API version.

initial API version

• Initial API versions only exist for new APIs. They are API versions with x = 0 (API versions 0.y.z, with or without version extension)

• An initial API version may exist for several minor version numbers without extensions, but should at some point mature to become an initial alpha API version: api-0.y.z-alpha.1

• During initial API version development, the API version in the URL shall also include the y version number v0.y (with dots). This is to simplify the testing of different initial API versions..

• In initial API versions, the y number is increased with breaking changes, the z number is increased with non-breaking changes (see also the exception below).

• Exception: for initial API versions:  

  • A breaking change results in an API version change from 0.y.z-extension.n to an API version: 0.y+1.0. Example: 0.9.0-alpha.m or 0.9.0-rc.n → 0.10.0

  • A non-breaking change results in an API version change from 0.y.z-extension.n to an API version. 0.y.z+1. Example: 0.9.0-alpha.m or 0.9.0-rc.n → 0.9.1

  • After this, -alpha.m and -rc.n extensions can again be applied throughout the initial API version development.

• Once a first public-release API version x.y.z has been created (x > 0), normally no further initial API versions (x = 0) shall be created.

alpha (alpha) API version

• An alpha API version (extension  alpha.m) is an intermediate API version with changes with respect to a previous API version.

• The purpose of an -alpha API version is to support rapid development of an API and testing of its implementation for feedback purposes.

• An alpha API version provides the relevant alpha items of the readiness checklist. A release package is optional.

• An alpha API version cannot be part of a CAMARA meta-release. 

• It needs to go through the below API pre-release and release steps first.

release-candidate (rc) API version

• A release-candidate API version (extension -rc.n) is a pre-release of an API version which is stabilized and intended to become the next public-release API version.

• The purpose of a release-candidate API version is to allow for (only) bug fixing encountered during further API implementation testing. It shall be used to create the API test results and all required release assets. - moved to next bullet - remove this bullet - questing on test results is still valid.

• A release-candidate API version shall provides all relevant rc items of the release checklist, such as API test results.

• A release-candidate API version proposed for a meta-release needs to provide all the public-release items on the release checklist.

• The final release-candidate API version needs to go through TSC approval such that its corresponding public-release API version can be prepared and included in the meta-release.

public-release API version

• A public-release API version (x.y.z with x>0) is a major/minor/patch stable version of an API. It is referred to as an incubated API version.

• A public-release API version with x=0 may exist. It is referred to as a sandbox API version. 

• A public-release API version is published as part of a CAMARA meta-release.

• A public-release API version provides the relevant public items of the release checklist.

• A next public-release API version is introduced if there are updates to the API definition (major/minor/patch).

• The update of a public-release API version needs to be carefully announced and discussed with API consumers.

• Minor updates or patches to a released API results in the creation of a maintenance branch with its own releases as needed.

• Cumulative minor changes shall be released in the next meta-release, following the normal release process.

• A patch release replaces the released API version in the existing meta-release.

sandbox API version

An API that has not yet reached full maturity, but is considered stable enough to have a public-release as part of a meta-release. It is expected that in a next meta-release the API version become incubated.

incubated API version

A stable public-release API version which has been deployed by at least 2 operators.

work in progress API version

• A work in progress API version, with "wip" instead of x.y.z as version number, may exist for an API during the time that one or more Pull Requests (PRs) are committed after a release tag was created.

• The purpose of a work-in-progress API version is to indicate that the API is unstable due to one or more PRs being committed, possibly resulting in temporary inconsistencies.

• Between different PR commits, the API version remains "wip".

• After or with the last PR commit, the next API version pre-release is prepared.

• The maintainers of the API decide on the next API version and the corresponding release assets are created for the next API version.

pre-release

The term pre-release can be used to refer to any of the initial, alpha or release-candidate releases of an API version.

NOTE: all pre-release API versions are publicly available in the CAMARA GitHub and can be used AT THE USER'S OWN RISK, as changes may happen to such API versions without notice.

release

The term release can be used to refer to either a pre-release or a public-release of an API version.

meta-release

A meta-release is a set of public-releases of API versions made available at a given point in time (September and March).

All API versions of a given meta-release shall be aligned to the Commonalities and Identity and Consent Management releases that are part of that same meta-release.

release tag

A release tag is a GitHub tag placed on the main or a maintenance branch to allow locating the corresponding API version in the repository.

release package

A release package is a zip file created using the GitHub release mechanism together with the release tag. It contains a snapshot of the API Sub-project repository with the content indicated by the release tag.

Releasing API versions

API versioning throughout the release process

During the development, release preparation and release of an API version,

• multiple intermediate pre-releases for initial, alpha or release-candidate API versions may be created, and

• a final public-release is created for the API version for inclusion in the meta-release

In the below table, the column headers mean the following:

• the API version (OAS file), without or with an extension as specified, is what is put in the version field of the API OAS definition file

• the API version in URL (OAS file) is a lowercase "v" followed by the API major version number which is what is put in the URL field of the API OAS definition file

For a given API version, its versioning is done as follows:

Release numbering 

For a given API version, a release consists in creating the GitHub release tag and release package which should contain the relevant set of release items as in the release asset table table.

The release package shall contain all assets as defined in the 

In the above API release asset table, the column headers mean the following:

• the release tag is the GitHub tag put on the main or update branch allowing to retrieve the released API version

• the release package, tagged "pre-release" or "latest", containing the zipped version of the whole API Sub-project repository.

API version changes

To create an API version (pre-)release, the API (pre-)release assets need to be created with naming rules as indicated in the above table

The following rules shall apply: 

• With the exception of initial API versions, all pre-release API versions must have an API extension.

• An initial or alpha API version may have an API version release package, which must have the "pre-release" tag

• A release-candidate and public-release API version must have an API version release package

• Only the latest release package of an API version may have the tag "latest".

Version of new APIs (wip)

A newly introduced API is in the main branch and has API version in the API URL set to "vwip" (see table above),

It stay like that until the first initial API version (with x=0), e.g. api-0.1.0 or api-0.1.0-alpha.1, is decided to be created by the API Sub-project team.

Special case: initial API versions (0.y.z)

While public-release API versions follow the semver standard, initial API versioning uses the version numbers exceptionally as follows:

• the major version (x) number is 0 in all cases

• the minor version (y) number is used for breaking changes

• the patch version (z) number is used for non-breaking changes

• version extensions can be used but a release candidate API version may go back to an alpha API version or to an initial API version without any extension. 

The below table provides guidelines for the initial API versioning. The semver numbering sequencing does apply for initial API versions. 

Examples: 0.1.0 < 0.1.1< 0.2.0 < 0.2.0-alpha.m < 0.2.0-rc.n < 0.2.1 < 0.3.0, etc.

In the above table, the choice between 2 proposed next API version options is done by the API maintainers based on the estimated stability of the API with respect to the introduced changes. 

In addition, the normal API version evolution through -alpha.m and -rc.n extensions is applicable throughout the API version development.

Once an initial API version 0.y.z is deemed sufficiently stable, a first alpha API version api-0.y.z-alpha.1 shall be created. The creation of a release package is optional for initial and alpha releases.

From this point onward, the below API (pre-)release steps apply and further alpha and release-candidate API versions with their corresponding API release assets as per the above table shall be created as needed.

The first public-release API version 1.0.0 is created from the last agreed and accepted release-candidate API version api-0.x.y-rc.n

Developing an API (wip)

During the first development of an API, and in between API releases, several contributions can be made by different people from their private forks or by maintainers of the API on their development branch.

The contributions are proposed through Pull Request (PRs) and committed to the target branch: main 

While multiple PRs are being committed to the target branch, unstable API versions may exist. To avoid that people use such unstable versions, the following guideline shall be applied:

• The first PR commit will ensure that the API will get the version "wip" instead of x.y.z. 

• All subsequent PR commits shall maintain that same API version as "wip"

• Once all PR commits are done and stability, usability and consistency of the updated API is agreed on, the maintainers create the next API version tag on the ytaret branch.

• This new version can now be used by API consumers who want to test the new pre-release version.

The following figure illustrates the work-in-progress version of the API created in this process.

Open

Here are explanations on the above figure:

• rx.y:  the release of any API version (with or without extension) to which changes are to be applied through PR(s).

• rnext: the tag of the next release of the API version with respect to rx.y after the PRs have been committed and API stability is reached.

This work-in-progress process can be applied to any release as needed for patches, updates or other evolutions of APIs

• The first/all PRs shall set the version field of the API OAS definition file(s) to “wip”.

• This wip versions shall remain until the project maintainers decide to release the API version (rnext)

• If the rx.y has an associated release package, a release packages shall be created for rnext.

• Only maintainers can create development branches.

• Other developers should fork the repo and do PRs from there.

• All PRs shall be approved by maintainers before being committed.

• Once all PRs are merged, the last PR shall updated the version field of the API OAS definition file(s) to the new API version

• The API version shall be released under the rnext release tag applied to the branch.

Preparing an API (pre-)release

An API Sub-project can create as many initial or alpha API versions as they want during API development in preparation for the release.

For API versions with x>0, the next API version number is defined and does NOT change during API development and release preparation. Only the API version extensions and their numbers may change. 

To prepare the release of an API version, one or more release-candidate API versions MUST be created as follows:

• At the M3 release milestone: when the API is agreed stable for release by the API Sub-project.

• At the M4 release milestone: the final release-candidate API version is delivered for meta-release approval.

Releasing an API

A public-release API version is created when the release-candidate API version is approved for a meta-release.

• At the M5 release milestone, the final API version (without any extension) is created with its release package, and included in the meta-release.

• Inclusion in a meta release is done by adding the required information for the API version on the meta-release page.

Evolution of a public-release API

Evolution of a public-release API version need to be carefully planned. this is especially true for breaking changes, but also for non-breaking changes such as new optional functional extensions.

To manage evolution of a public-release API version, the semver 2.0 versioning applies to the API version number x.y.z as follows:

• changes in the major (x) number when creating non-backward compatible / breaking changes to the API

• changes in the minor (y) number when creating backward compatible changes to the API

• changes in the patch (z) number when correcting an error of the API

During the development of an API version update, all pre-release API versions need to get a version extension so its release can be managed properly.

Example for a backward-compatible update of a public-release API version 1.0.0, resulting in API version 1.1.0:

• Develop the 1.1.0 update on the main branch

• Once sufficiently stable, create a new API version api-1.1.0-alpha.1 by tagging the main branch

• Several intermediate alpha API version releases may be created, with or without release packages

• Then one or more intermediate release-candidate API versions (rc branches + release packages) may be created (see example table below).

• The last release-candidate API version will be proposed for meta-release. 

• After meta-release approval, create the new public-release API version 1.1.0 for public-release (tag and release package ("latest")).

Maintenance of a public-release API

Once an API has been published as part of a meta-release, situations may occur where minor or patch changes to this API need to be made.

• In this case a dedicated branch will be opened for the maintenance of the API.

• The changes are proposed through Pull Request (PRs) and committed to the maintenance branch.

• In this case, a maintenance branch is opened to which contributions through PRs will be committed. The version of the resulting updated API shall follow the semver rules. A new public-release may be created immediately without waiting for the next meta-release.

• The availability of the new release should be reflected through the API release tracker page.

Meta-release approval for an API version

• To be approved for meta release, the API version must provide all the public items of the release checklist and meet the meta release acceptance criteria.

• The API Sub-project shall announce their intention for public-release API version by adding the relevant release-candidate API version on their API release tracker page.

• After check by the release management team, approval is documents there.

• Once approved the API Sub-project shall create the public-release API version and its API version assets.

• Finally the API sub-project shall update the meta-release page with the reference to the public-release API version.

API family

The notion of API family will not be managed as part of release management. CAMARA releases only consider API versions.

It is recommended to have one API per API sub-project to facilitate the release management and versioning of APIs.

If multiple API are managed in one API Sub-project, each API file shall have its own folder, such that release packages can be created with a focus on a specific API. The API name and the API version of this API shall be used in the naming of the API version tag and API version release package. However, as the release package will also include all other APIs of the Sub-project, this may lead to confusion.

An API Sub-project may also decided to release all APIs in the same API family always at the same time with the same API version. In this case,

• all API files in the release must have the same API version.

• instead of the API name, the API Sub-project name shall be used in the API version tag and the API version release package name.

If later on one API of the API family requires a separate evolution with its own version number, then this API shall be put in its own API Sub-project.

The grouping of APIs into families or other groupings, e.g. by related working groups, can be done easily using a CAMARA wiki page.

This will reduce the number of meetings and reporting load with the increasing number of APIs, and facilitate moving APIs from one family or group to another, or combining APIs from multiple families into into into family.

See also Herbert's proposal here: Proposal to establish API Families as Working Groups across API Sub Projects · Issue #7 · camaraproject/ReleaseManagement (github.com)

API release checklist

NOTE: This proposal can be fed back into Commonalities project as needed or moved here.

The following table identifies the assets that need to be provided for an API version for a particular release.

It is intended as a checklist for the API sub-projects to prepare an API version release.

Examples

The following are examples of the API release asset and release naming for the various API versions throughout the API version's lifecycle. 

API version 0.y.z (initial API development and first release)

API version x.y.z (x>0) (public-release API updates)

The following table illustrates the usage of versions and version extensions throughout the API lifecycle for the 3 different types of update: minor, major, patch.

Minor update of API version

Patch update of API version

Major update of API version