The following sections provide guidance on how to review an API release PR.
| Table of Contents | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Please add questions in the Q&A section as you come across them in the respective sections and add comments/text for answering them.
...
The Release Management team will create a review issue in which all check points are listed as per the template described in the next section.
For APIs that are targeting to become a stable public API, for M3, the Release Management team may request an additional review by Commonalities and ICM teams.
As a reviewer, if you see a release PR that is nearly ready, add a comment as follows: "Please add @release-management_maintainers to this PR once the PR is ready"
...
Create the review issue
Assign the issue to a Release Management team member who will
Check each task and tick it when done.
Check if further review by TSC / Commonalities / ICM is needed (e.g. for targeted stable APIs), and leave issue open until those reviews are marked as done and OK in the review issue
For final approval of a release PR, use a comment like: "LGTM from ReleaseManagement."
When all tasks and complementary reviews are completed, close the review issue with a comment (in the review issue) on the overall status of the API.
...
.
...
Review issue - usage
A review issue is created by the Release Management team when the review of an API release PR is requested by the API Sub Project for M3 or M4 approval.
...
API definition files (YAML) (check version in info & servers objects)
test file(s) availability
changelog updated
readme updated (enforce correct release number / API version naming)
API readiness checklist(s)
Release actions
API release tracker available on wiki
Review comments addressed (by Sub Project)
Release PR approved (on behalf of Release Management)
PR merged (by Sub Project codeowner)
Release created within GitHub (by Sub Project codeowner)
Release Tracker updated (with creation date of the release and the release tag link)
In case of maintenance release only: add a PR on https://github.com/camaraproject/.github/blob/main/profile/README.md created (by Release Management) and merged (by CAMARA admin)
The following sections provide more details for the reviewers of API release PRs.
Review actions - details
API definition files (YAML) (check version in info & servers objects)
Check Info object
version: aligned with the released API version and formatted correctly (per Commonalities)
description: check that it
contains sufficient in-line API documentation
does not use Telco Jargon / abbreviations (non blocking for release if in documentation), e.g. avoids use of
"customer", unless explicitly explained what is meant. Instead use as applicable “API consumer”, (application) “end-user”, or whatever else is applicable.
"Telco/operator/CSP", as also Aggregators, etc. may expose the API. Use a more general term e.g. “API exposure platform owner” as applicable.
"subscriber". Use “Device” or “End-user” (defined as the application user) as applicable
“API customer”: use “API consumer” or (application) “end-user”, as applicable
contains the mandatory text from the section on authentication and authorization from the ICM CAMARA-API-access-and-user-consent.md.
contains the x-camara-commonalities version from the Commonalities release referenced in the API readiness checklist.
Check servers object has the correct version format e.g. v0.xrc1, v1rc1, v0.x or v1.
Check security schemes: check presence and format (OIDC) and scope name format.
In externalDocs: this field is optional, but recommend to point to the CAMARA , repoAPI repository.
description: Project documentation at CAMARA repository
url: https://github.com/camaraproject/<repo>Tanja de Groot to create a Commonalities backlog issue
Shall error cases be <api-repository>
Error cases are sufficiently explained in in-line documentation ? (in info.description ) It seems yes as per Commonalities/documentation/API-DocumentationTemplate.md - Deprecate the doc and doc shall be inline in OAS spec. @Rafal to updater in Commonalitiesfield.
test file(s) availability
should the test file contain the version number ? e.g. Feature: CAMARA Call Fowarwing Signal API, v0.1.0 - Operation call-forwardings - is it required to include the version (as people forget to update it). As the tests are in the same release package, can the version number be avoided in the test features ?
all error codes shall be covered by final test scenarios. what do we consider the
main error cases for rc ? can we automate checking that all API spec errors/cases are covered by the tests ?release-candidates / initial versions shall respect Commonalities / documentation / API-Testing-Guidelines.md
changelog updated
Comment: For capturing changes, its is recommended to use the GitHub release feature to get the list of all the changes which can then be selected as appropriate and put in the right categorie (added, changed etc) - ask a GitHub expert how to do this by creating a draft release (guidelines are WIP).
...
API definition files
commonalities
availability of linting report in release issue (tbc)
ICM
versioning (checked in yaml)
documentation (beyond in-line)
check for Telco jargon / abbeviations
user stories
basic test files (for M3)
is it necessary for a Feature to reference the API version ? as it is anyway in the same release package ?
cover 200 response and all error responses (tbc)
enhanced test files (for M4)
all all error responses shall be covered by M4
add guideline to use error codes in scenario names ?
release numbering
changelog
certification - for stable public APIs
Release actions - details
API release tracker available on wiki
Go to the API Sub Project’s home page on the wiki and check the availability of the API release tracker and its contents.
Check that the API is visible on the targeted meta-release plan indicated in the tracker. If not, indicate to the API Sub Project to update the tracker page with the appropriate meta-release label.
Review comments addressed (by Sub Project)
Check the tick box when the review comments provided by the Release Management team have been addressed by the API Sub Project.
Release PR approved (on behalf of Release Management)
Approve the release PR by putting a comment in the release PR like: “LGTM from ReleaseManagement”.
PR merged (by Sub Project codeowner)
Check the tick box when the release PR has been merged.
Release created within GitHub (by Sub Project codeowner)
Check the tick box when the release has been created.
Release Tracker updated (with creation date of the release and the release tag link)
Check the tick box when the release tracker has been updated with the new release and the date
In case of a maintenance release, update the CAMARA home page
In case of maintenance release only: add a PR on https://github.com/camaraproject/.github/blob/main/profile/README.md created (by Release Management) and merged (by CAMARA admin).
Commonalities and ICM review checks
For APIs that are targeting to become a stable public API, for M3, the Release Management team shall request a review by Commonalities and ICM teams.
Commonalities team review
...
- details
Most checks are covered by the Release Management review.
ICM team review - details
THese points are taken from: https://github.com/camaraproject/IdentityAndConsentManagement/issues/189#issuecomment-2315026741 )
Ref fro for examples: https://github.com/camaraproject/SimpleEdgeDiscovery/blob/r1.2/code/API_definitions/simple-edge-discovery.yaml
...
Who can approve the M3 release PR ? Code Owners after RM approval is noted in the release PR or its issue.
When to create a RM review issue ? when the release-management_maintainers list is included as a reviewer in a release PR.
When do I tick a review issue task ?
Review actions: tick after checking the related file(s) and putting the review comment(s) in the PR/PR issue about the(se) files. Also put just OK for the file if no comments.
Also put a comment in the review issue with the major review comments and the overall state of the API.
Release actions: tick when task is done.
When do we close the review issue ? When all tasks in the review issue are ticked as done. Close as completed.
Question: should we add a milestone label so we can filter on it later ?
How to know if an APIs is covered by RM ? Use a comment in the comment fields of the API release tracker, visible on the Meta-release page.
Is there a way to know if an API requires consent request ? No, this is local regulation. It is covered when using the security scheme and the scope rules.
Open questions
should the test file contain the version number ? e.g. Feature: CAMARA Call Fowarding Signal API, v0.1.0 - Operation call-forwardings - is it required to include the version (as people forget to update it) As the tests are in the same release package, can the version number be avoided in the test features ?
all error codes shall be covered by final test scenarios. what do we consider the main error cases for rc ?
can we automate checking that all API spec errors/cases are covered by the tests ?